Scholarly article on topic 'Definition of Water Meter Substitution Plans based on a Composite Indicator'

Definition of Water Meter Substitution Plans based on a Composite Indicator Academic research paper on "Civil engineering"

Share paper
Academic journal
Procedia Engineering
OECD Field of science
{"Apparent losses" / "metering error" / "replacement strategy" / "water distribution systems" / "water meters"}

Abstract of research paper on Civil engineering, author of scientific article — V. Puleo, C.M. Fontanazza, V. Notaro, M. De Marchis, G. La Loggia, et al.

Abstract This paper presents a water meter substitution plan based on a composite “Replacement indicator” which was defined and compared with common substitution strategies based on meter age and on run-to-fail approaches. The methodology was applied to one of the 17 sub-networks in which the Palermo city water distribution network (Italy) is divided. The analysis was carried out considering a substitution budget limitation and the results showed that the use of “Replacement indicator” outperform the classical substitution strategies based on meter age because it takes into account some other variables that may affect meter precision and wearing.

Academic research paper on topic "Definition of Water Meter Substitution Plans based on a Composite Indicator"

Available online at


Procedía Engineering


Procedía Engineering 70 (2014) 1369 - 1377

12th International Conference on Computing and Control for the Water Industry, CCWI2013

Definition of water meter substitution plans based on a composite indicator

V. Puleoa, C.M. Fontanazzab, V. Notarob, M. De Marchisb, G. La Loggiaa, G. Frenib*

a Dipartimento di Ingegneria Civile, Ambientale, Aerospaziale e dei Materiali, University of Palermo, Viale delle Scienze, 90128 Palermo (Italy) b Facolta di Ingegneria ed Architettura, Universita "KORE" di Enna, Cittadella Universitaria, 94100 Enna (Italy)

This paper presents a water meter substitution plan based on a composite "Replacement indicator" which was defined and compared with common substitution strategies based on meter age and on run-to-fail approaches. The methodology was applied to one of the 17 sub-networks in which the Palermo city water distribution network (Italy) is divided. The analysis was carried out considering a substitution budget limitation and the results showed that the use of "Replacement indicator" outperform the classical substitution strategies based on meter age because it takes into account some other variables that may affect meter precision and wearing.

©2013TheAuthors.PublishedbyElsevier Ltd.

Selection andpeer-reviewunderresponsibilityoftheCCWI2013Committee

Keywords: Apparent losses; metering error; replacement strategy; water distribution systems; water meters.

1. Introduction

In water supply management, velocity water meter are typically used to measure users' consumption. With water meters, utilities can collect useful data for billing, assess the water balance of the system, and identify failures in the network, water theft and anomalous user behaviour. Despite their importance, these instruments are characterised by intrinsic errors that cause the apparent losses due to meter under-registration. The complexity of the physical phenomena associated with metering errors in aging water meters does not allow meter replacement to be guided by single parameters, such as the meter age or the total volume passed through the meter.

The metering error depends on the actual flow rate passing through the meter. Therefore the rate of water consumption recorded by the meter depends on the temporal pattern of end user demand (Male et al., 1985; Ferreol, 2005). For consumption at medium and high flow rates, the error can be very low. For consumption at flow rates lower than the minimum, the error will be negative and very high until reach -100% when flow is lower than the starting flow. As a result, the apparent losses due to meter under-registration depend on the percentage of user consumption occurring at low and very low flow rates, as well as the capability of the meter to accurately register the water volume consumed.

In countries suffering water shortage (such as arid and semi-arid countries) the presence of storage tanks in the water systems inside buildings (Rizzo and Cilia, 2005; Criminisi et al., 2009) affects the share of the consumption at low flow rates. The use of storage tanks is a common practice for users to cope with a water scarcity condition (Cubillo, 2004; Andey and Kelkar, 2009). During water shortage period, discontinuous water supply and water resources rationing are usually the main measures adopted by the water utility to distribute limited water resources as efficiently as possible. Users compensate for intermittent water service by collecting water during serviceable periods in the tank and

Corresponding author. Tel.: +39-0935-536439; fax: +39-0935-536710. E-mail address:


1877-7058 © 2013 The Authors. Published by Elsevier Ltd.

Selection and peer-review under responsibility of the CCWI2013 Committee


redistributing it when public water service is not available (Fontanazza et al., 2008). Private tanks are filled using a proportional float valve that opens partially or totally as a function of tank water level and network pressure. When users are subjected to intermittent supply (De Marchis et al., 2010; 2011), water flows into the tank only when network pressure at user connection is sufficient to supply the tank. Until the pressure is low, tank water levels drop to meet user needs, with tanks very often almost empty by the time the pressure increases. As a consequence, the float valve is completely open and allows for water passing at a very high flow rate. The dependency of apparent losses on the presence and behaviour of private tanks has been analysed previously (Rizzo and Cilia, 2005; Cobacho et al., 2008; Criminisi et al., 2009). The effect of the tank on consumption flow rates raises the global error of the user's meter, ranging from about -10% for a new meter to -40% for a worn-out meter. In terms of apparent losses, experimental evidences show that the average under-registration percentage of worn-out meter ranges from about 10% of the total household consumption to about 50% (Criminisi et al., 2009).

Several other factors cause water meters to lose their efficiency. Thornton and Rizzo (2002) identified meter wear and tear, incorrect installation practices, lack of maintenance or calibration, incorrect meter type and class with respect to their current application, incorrect meter sizing and demand pattern as possible reasons for meter inaccuracy. Arregui et al. (2005) presented real field and laboratory data regarding the impact of several parameters on the accuracy of both domestic and industrial water meters and on different meter technologies (single jet, multiple jet, oscillating piston, Woltman and Tangential meters). Incorrect mounting position, wear of moving parts, suspended solids and deposits, leaks and user's storage tanks, partial blockage of the inlet strainer are all said to influence the error curve of domestic water meters (although this varies with meter technology). In addition to these factors, velocity profile distortions and proper meter sizing also have an effect on industrial water meter measuring error (Arregui et al., 2006; Arregui et al., 2007).

Water utilities usually try to limit apparent losses related to metering errors by conducting meter replacement programs; water meters are relatively cheap, and water utilities generally prefer to replace rather than recalibrate them (Arregui et al., 2006). Simple rules linked to a maximum acceptable meter age or the total registered volume are often used by utilities as meter replacement criteria. The methods used for identifying an adequate replacement frequency of water meters and determining the most suitable meter type are based on lab tests of used meters, field measurements of real consumption patterns, or the use of company billing data systems (Arregui et al. 2003, 2011). A solution is generally obtained by minimising a function representing the average annual costs of the meter, which includes the cost of the equipment, its installation and the unaccounted-for water.

A meter replacement strategy based on a composite "Replacement Indicator" (RI) was previously developed that aims to reduce apparent losses (Fontanazza et al. 2012). The present paper suggests the definition of a meter substitution plan based on this indicator and on substitution budget limitations and it compares the proposed approach with common substitution strategy based on meter age and on run-to-fail approaches. The substitution cost was evaluated on the basis of current manager practice. The different strategies were applied to a real district network in Palermo (Italy) and then compared with respect to economic return. In the following a brief description of "Replacement Indicator" is provided; the case study is also presented and finally the results of the cost analysis are shown.

2. Water meter "Replacement Indicator", RI

As mentioned above the composite indicator developed in Fontanazza et al. (2012) was applied in this analysis. This section aims to provide some information about the Replacement Indicator (RI) definition.

RI was developed as a combination of three significant variables: the meter age, the network pressure at the meter, and the water volume passing through the meter. The purpose of this composite indicator is to provide a consistent ranking for the replacement of meters installed in a water supply network.

The construction of the composite indicator required individual steps in which different analytical procedures were performed. For this purpose, the OECD guidelines (OECD, 2008) were taken into account. First, individual sub-indicators were selected according to their analytical consistency, measurability and relevance to the analysed phenomenon. For each sub-indicator and for each variable used to compute the sub-indicator, the identification of a related data set, the imputation of missing data, the evaluation of intrinsic errors and normalisation of the variable were required. Then, the composite indicator, RI, was obtained once the normalised components were weighted and aggregated according to an underlying methodology.

All necessary data were provided by the water utility. Prior to the development of the composite indicator, it was necessary to determine whether data were missing. Missing data often hinder the development of robust composite indicators. In the literature, general methods are proposed for managing missing data: e.g. case deletion, single imputation or multiple imputations (Little and Rubin, 2002). Once continuous data sets were available, a normalisation step was performed to compare individual indicators with different dimensions. Three different normalisation methods were used in this study, including the Min-Max (Eq. 1), the Standardisation (Eq. 2), and the Distance to a reference methods (Eq. 3) (Freudenberg, 2003; Jacobs et al., 2004):

xqc - min(xq )

Ic =-^- q (1)

q'c max(xq) - min(xq)

xq c - mean(xq )

I q,c = , „ N ' (2)

std (xq )

Iq c = (3)

where xqc is the generic value of the qh individual sub-indicator for meter c and c=1,...M, with M being the total number of meters analysed; Iqc is the normalised value; min(xq), max(xq), mean(xq) and std(xq) are the minimum, maximum, mean and standard deviation values of xq across all data sets, respectively; and xqref is a reference value. In the present study, the maximum value for each indicator was used as a reference value.

In addition to the implicit weights assessed by the normalisation step, explicit weights were introduced during aggregation to reflect the relative importance of each component. Several weighting techniques exist in the literature. Some approaches rely on statistical models, whereas others weight components that are deemed more (or less) influential according to expert opinions (OECD, 2008). Usually, explicit weights are supposed to express a balance between components, and a surplus for one component is balanced by a deficit in another. Frequently, the following is assumed:

^ wq = 1, withO < wq < 1, forq = 1,..., Q (4)

where wq is the weight attached to each individual component, Iq, and Q is the total number of individual components. Three cases were considered in this study: equal weights, weights based on expert judgments and weights proportional to the inverse of the average value of the individual indicator for each water meter.

Regarding aggregation techniques, the literature offers both additive and other less common aggregation methods, including multiplicative (geometric) and nonlinear aggregation, such as multicriteria analysis methods (Karavanas et al., 2009). Eqs 5 and 6 express both the additive and geometric aggregation methods used in this study:

R = X wq • (5)

RI = n (6)

where RI is the composite indicator. In this manner, a replacement ranking consistent with the assessed RI values was created.

According to the previous explanation, different RI formulations are possible depending on the operator's subjective decisions, which may lead to different results (meter replacement rankings). Moreover, the variables used for computing the individual sub-indicators can be affected by measuring errors, which may cause further uncertainty in the appropriate ranking of the meters to be replaced. To investigate the consistency of the results, an uncertainty analysis was performed by means of Monte Carlo simulations (MCSs), where different random combinations of analytical formulations were performed while including measurement errors. The goal was testing the consistency of the ranking itself and/or the impact of subjective choices on the selection of the RI formulation.

3. Case study

The entire water distribution system of the city of Palermo (Italy) is made of 17 sub-networks that supply as many zones of the city. One of these sub-networks (Fig. 1) was chosen as case study because all its geometric characteristics are precisely known, as well as the number and the distribution of user connections, the water volumes delivered and measured, and the pressure and the flow values in a few important nodes (Fig. 1). This network was, in fact, totally rebuilt in the 2002, even if the renovation process regarded the sole distribution network keeping in service the old cast iron feeding pipes that connect two reservoirs at different levels to the network. The two reservoirs can store up to

40,000 m3 per day, and supply around 35,000 inhabitants corresponding to 11,619 users equipped with single and multi-jet water meters.

The entire network is made of polyethylene and it is about 40 km long. The pipes have diameters ranging between 110 and 225 mm. The network was designed to supply about 400 l/capita/d, but actual average consumption is about 260 l/capita/d. As consequence, the network was characterised by low water velocities and correspondently high pressures, which resulted in background leakage in the past.

The network is monitored by six pressure cells and two electromagnetic flow meters (Fig. 1) that have provided data on hourly basis almost continuously since 2001.

The network hydraulic model have been presented in a previous study (Fontanazza et al., 2008), its calibration is constantly updated when new data become available. This allowed to compute the complete time series of pressure and flow for each network node in which more than one meter can be installed. As mentioned above the composite indicator RI is based on pressure and flow parameters related to each water meter: the former was set equal to that simulated at the node; the latter was obtained dividing the total volume supplied at the node among all water meters considered installed to it, once the meter average annual total volume was known as ratio between the cumulated volume and the age, both displayed on the device.

The age of water meters ranged from 1 to 60 years (Fig. 2). The 1- to 10-year-old meters are included in class C and the 11- to 60-year-old meters are in class B (European Council directive 73/33/EEC, 1974).

Fig. 1 A schematic of the Palermo sub-network

Metrological class (75/33/EEC, 1974) Meter ages _____

(years) class C class B TOT

Ho 1,970 1,970

11-20 3,873 3,873

21-30 2,747 2,747

31-40 2,351 2,351

41-60 678 678

TOT 1,970 9,649 11,619

31-40 yrs 41-60 yrs


Fig. 2 Distribution of meter ages in the network

Available data regarding water meters are: installation date, total metered volume, periodic water consumption data since the installation year, meter position and technical/geometrical characteristics, characteristic of the user (including possible tank geometry and position).

Once complete data time series were obtained, normalisation was performed. The Min-Max, Standardisation and Distance to a reference methods, as expressed by Eqs. 1, 2 and 3, respectively, were applied.

Finally, three weighting procedures were investigated for aggregating the individual indicators: equal weights for all individual indicators, inverse mean value weights and expert opinion weights. For instance, the weights obtained with the expert opinion weights procedure were wage=0.3, wpressure=0.1, and waow=0.6. The expert opinion weights were determined from interviews with the operators and managers of the water utility. Then, the composite indicator for a specific meter c, RIc, was calculated using the additive (Eq. 5) and geometric (Eq. 6) aggregation methods:

The composite indicator, RIc, assigned a rank to each meter c. According to the theoretical framework, water meter replacements should be performed following decreasing values of RIc. Along with the combination of the previous choices, 18 different RI formulations were possible.

To evaluate the consistency of the meter replacement ranking and to account for the uncertainties present in the estimation of the RI, a total of 1,000 Monte Carlo simulations were run; different random combinations of the analytical formulations were used, and measurement errors were included. The meter age was assumed to be unaffected by errors because the installation date is printed directly on the meter quadrant. According to the instrument accuracy of the pressure cell, errors in the pressure data were accounted for by applying a uniform random error of ±2%. The error in the water meters was obtained using meter error curves that were presented in a previous study, where the metering error was related to the flow passing through the meter and the meter age (Fontanazza et al., 2010). Fontanazza et al. (2010) performed laboratory experiments in which several water meters, taken from the same urban distribution network, were tested using a test bench (ISO 4064:2005). The meters were divided into age classes, and several error curves were experimentally obtained for each meter. In this uncertainty analysis, a random error curve was applied to each meter in each Monte Carlo simulation depending on its metrological class and age.

Each Monte Carlo simulation was performed by randomly selecting the following aspects: the normalization approach, the weighting method and the analytical aggregation of individual indicators among the methods cited above; the pressure error (choosing a random error for each metered value); the meter error curve (randomly choosing an error curve for each meter among those presented in Fontanazza et al. (2010), for water meters of the same metrological class).

4. Results

Initially the results of uncertainty analysis were evaluated in order to investigate the impact of RI formulation on water meter ranking. In Fig. 3, all the water meters were divided in 116 groups each including 100 elements. Each group contains ranked water meters to be substituted according to the median value of RI (related to the different indicator formulation): the first group contains the first 100 meters to be substituted, the second group contains the second hundred to be substituted and so continuing till group 116, containing the last group of meters to be substituted according to the median value of the rank. Each graph in Fig. 3 contains box plots showing the uncertainty in the estimation of the rank: the box is plotted between 25th and 75th percentile of rank; the whiskers indicate the 5th and the 95th percentiles. In general, the uncertainty related to the selection of the appropriate RI formulation is relevant considering that the uncertainty bands (the distance between the 5th and 95th percentiles) can be double with respect to the median ranking, thus meaning that a water meter being in the 30th group (i.e. to be substituted after the first 3,000 in the network) can be ranked around the 1,000th place or the 6,000th place depending on the R formulation adopted.

The figure shows that the first groups and last groups, selected according to the ranking, are affected by lower uncertainty thus demonstrating that the RI is reliable in the identification of the first (and consequently the last) water meters to be substituted. The central groups are characterized by uncertainty bands (between the 5th and 95th percentile) that are four times larger than the first group to be substituted. The impact on the band between the 25th and 75th percentiles is even larger, showing ratios between the first group and the central groups in the range of 6-7 times. The high uncertainty related to the ranking of the central groups of water meters is a weak point of the methodology but this is less important in practical terms because ranking is recomputed each year and only the first groups are reliably selected for substitution considering common budget limitations. Reasonably, only 5% - 10% of the water meters will be substituted each year and this means that only the first ten groups in Fig. 3 will be of practical use. According to the above considerations, the median R-criterion was used in the following analysis.

Once the 11,619 meters were ordered according to the median ranking of each water meter during the uncertainty analysis, a comparison with the age approach was carried out with respect to ranking. Fig. 4 shows the ranking according to the age criterion and to the R-based criterion. From the graph, the following considerations can be drawn: • The R-based criterion tends to keep in place old water meters that should be first substituted according to the age criterion (the first 2,000 meters to be substituted according to the age criterion); this consideration is due to the fact

that additional local conditions (such as consumption and pressure) may suggest to prioritize the substitution of other water meters having a larger probability to generate apparent losses due to meter under-registration.

• In the central part of the rank (between 2,000 and 8,000), the two criteria provide similar results and the central accordance between the two rankings is preserved; this means that water meter age is the most important parameter in this part of the ranking.

• Finally, the last water meters to be replaced show again large differences between the two criteria with the _R/-based method suggesting the early substitution of meters that should be kept in place according to the sole age criterion

Fig. 3 Uncertainty analysis on water meters substitution ranking: the water meters are represented in groups of 100 elements each; the boxes represent the interval between 25 and 75 percentiles and the whiskers represent the 5 and 95 percentile.

Fig. 4 Comparison of rankings provided by the Rl-based median criterion and the age criterion.

The presented analysis shows that the RI-based criterion diverges from the age criterion in the two parts of the ranking that are mostly relevant to the water managers: the first groups of meters to be substituted, for which the RI-based criterion preserves some old water meters that may still have an acceptable performance, and the last groups of water meters to be substituted, for which the RI-based criterion suggests the early substitution of almost new water meters that may produce high apparent losses due to local operating conditions (such pressure and user's water consumption).

A substitution budget limitation was taken into account for the selection of the number of water meters which can be replaced. Consulting the business plan edited by the Palermo water utility (AMAP S.p.a), for the first year the available budget can be fixed equal to € 85,000 (equivalent to the average spent resources in the last three years). Assuming an average substitution cost of € 100/meter, the number of meters that can be replaced was 850 every year (about 7% of the meters installed in the sub-network).

After the first replacement (i.e. the first year in which the RI is determined and adopted), the method based on meter age allows to retrieve the 6.8% less than the method based on the RI; after the second replacement (i.e. the second year in which the RI is adopted), the percentage decreased around the 4.9% until reaching the 2.3% after the fifth replacement (Fig. 5). After the fifth year, the two methods provided similar results (less than 2% of difference always in favour of the RI application) until the 11th year of substitution is reached (after the substitution of around 9,500 meters). After that year the gap between the two methods starts growing slightly again reaching an asymptote to 3.5% in favour of the RI-based method. After 15 years, all the meters today present in the network will be substituted and the application of RI will always recovery 3.5% more volume than the common age-based method.



Fig. 5 Cumulated income and cost related to two different replacement strategy based on meter age and RI method.

Over the initial five years of application, the RI method resulted in a recovery of approximately 1.2 million of cubic meter corresponding to € 1.7 million whereas the method based on meter age provided an income of € 1.6 million (Fig. 5). This demonstrates that the composite indicator analysis provides added value through the selection of water meters to be replaced.

5. Conclusions

This study proposed a water meter substitution plan based on a composite "Replacement indicator". The complexity of the physical phenomena associated with metering errors in aging water meters does not allow meter replacement to be guided by single parameters, such as the meter age or the total metered volume. The proposed indicator considers three of the most influential parameters that can affect metering performance: meter age, total volume passing through the meter and network pressure.

A comparison of the proposed indicator with a common ranking procedure based on meter age showed that the proposed indicator can better select the meters to be replaced, fixing the annual replacement rate and allowing for a faster recovery of replacement costs.

The analysis showed that the W-based criterion provides additional information in the selection of the water meters to be substituted. This criterion is able to reduce the apparent losses more efficiently of the simple age criterion and this advantage cumulates over years providing a consolidated increase in apparent losses recovery equal to 3.5%.

The W-based criterion is characterized by some uncertainty that reflects particularly in the central part of the water meter substitution ranking. Such area is not really important for the water network manager because meters to be substituted are in the first part of the ranking (meters that have to be substituted as soon as possible depending on available budged) and in the last part of the ranking (containing some new meters that should be early substituted because the local pressure and user's consumption conditions may cause apparent losses higher than what is expected according to the meter age).


The authors would like to acknowledge the Project POFESR 2007-2013 SESAMO 'Sistema informativo integrato per l'acquisizione, gestione e condivisione di dati ambientali per il supporto alle decisioni' for financial support to the presented research.


Andey, S. P. and Kelkar, P. S., 2009. Influence of intermittent and continuous modes of water supply on domestic water consumption, Water Resouce and Managment, 23, 2555-2566.

Arregui, F. J., Cabrera, E., Cobacho, R. and Garcia-Serra J., 2006. Reducing apparent losses caused by meters inaccuracies. Water Practice &

Technology, 1(4), doi: 10.2166/WPT.2006093. Arregui, F. J., Cobacho, R., Cabrera, E. Jr. and Espert V., 2011. Graphical Method to Calculate the Optimum Replacement Period for Water Meters.

Journal of Water Resources Planning and Management, 137(1), doi:10.1061/(ASCE)WR.1943-5452.0000100. Arregui, F. J., Pardo, M. A., Parra, J. C. and Soriano, J., 2007. Quantification of meter errors of domestic users: a case study. Proceedings of the

Water Loss 2007 Conference, Bucharest, Romania. Arregui, F., Cabrera, E., Cobacho, R. and Garcia-Serra J., 2005. Key factors affecting water meter accuracy. In: Proceedings of the leakage 2005

conference, 12-14 September 2005, Halifax, Canada (CD-ROM edition). Arregui, F., Cabrera, E.Jr., Cobacho R. and Palau V., 2003. Management strategies for optimum meter selection and replacement. Water Science and

Technology: Water Supply, 3(1-2), 143-152. Cobacho, R., Arregui, F., Cabrera, E. and Cabrera E. Jr., 2008. Private water storage tanks: evaluating their inefficiencies. Water Practice &

Technology, 3(1), doi: 10.2166/WPT.2008025. Criminisi, A., Fontanazza, C.M., Freni, G. and La Loggia, G., 2009. Evaluation of the apparent losses caused by water meter under-registration in

intermittent water supply. Water Science and Technology, 60(9), 2373-2382. Cubillo, F., 2004 Droughts, risk management and reliability, Water Science and Technology, 4, 1-11.

De Marchis, M., Fontanazza, C. M., Freni, G., La Loggia, G., Napoli, E., Notaro, V., 2010. A model of the filling process of an intermittent

distribution network, Urban Water, 7, 321-333. De Marchis, M., Fontanazza, C. M., Freni, G., La Loggia, G., Napoli, E., Notaro, V., 2011. Analysis of the impact of the intermittent distribution by

modelling the network filling process, Journal of. Hydroinformatics, 13, 358-373. Directive of 17/12/1974 n° 75/33/CEE. "For the approximation of the legislations of the member states with respect to cold water meters". R.M. December 1974 p.866.

Ferreol, E., 2005 How to measure and reduce water meter park efficiency?, in: Proceedings of Leakage 2005 Conference, Halifax, Canada, September 2005.

Fontanazza, C. M., Freni, G., La Loggia, G., Notaro, V., Puleo, V. 2012. A composite indicator for water meter replacement in an urban distribution network. Urban Water Journal, 9(6), 419-428. doi:10.1080/1573062X.2012.690434.

Fontanazza, C.M., Freni, G., La Loggia, G. and Notaro, V., 2010. Effect of network pressure on apparent losses due to meters under-registration. Proceedings of the Water Loss 2010 Conference, Sao Paulo, Brazil.

Fontanazza, C.M., Freni, G., La Loggia, G., Notaro, V., 2008. Definition of performance indicators for urban water distribution systems in drought conditions. In: E. Cabrera Jr. and M.A. Pardo, eds. Performance Assessment of Urban Infrastructure Service. Drinking water, wastewater and solid waste, London: IWA Publishing, 35-46.

Freudenberg, M., 2003. Composite indicators of country performance: a critical assessment. Paris: OECD Publishing.

ISO 4064: 2005. Measurement of water flow in fully charged closed conduits - Meters for cold potable water and hot water.

Jacobs, R., Smith, P. and Goddard, M., 2004. Measuring performance: an examination of composite performance indicators. CHE Technical Paper Series 29, Centre for Health Economics, University of York.

Karavanas, A., Chaloulakou, A. and Spyrellis, N., 2009. Evaluation of the implementation of best available techniques in IPPC context: an environmental performance indicators approach. Journal of Cleaner Production, 17, 480-486.

Little, R. J. A. and Rubin, D. B., 2002. Statistical Analysis with Missing Data. Hoboken, New Jersey: Wiley Interscience, John Wiley & Sons.

Male, J. W., Noss, R. R., Moore, I. C. 1985. Identifying and reducing losses in water distribution systems. Noyes Publications, Park Ridge, N.J.

OECD - Organisation for Economic Co-operation and Development, 2008. Handbook on Constructing Composite Indicators. Methodology and User Guide, Paris, France: OECD Publishing.

Rizzo, A. and Cilia, J., 2005. Quantifying meter under-registration caused by the ball valves of roof tanks (for indirect plumbing systems). Proceedings of the Leakage 2005 Conference, Halifax, Canada.

Thornton, J. and Rizzo, A., 2002. Apparent losses, how low can you go? In: Proceedings of the leakage management conference, 20-22 November 2002, Lemesos, Cyprus.