Water Journal : Water Journal December 2012
small water & wastewater systems technical features 70 DECEMBER 2012 water Social benefits such as: • Treating waste locally; • Reduced issues with network expansion in built-up areas; • Education; • Equitable cost distribution – shifting of the financing burden to direct beneficiaries (user pays). What is Multi-Criteria Analysis (MCA) and Why use It? Decisions in the water industry are often complex and require decision makers to consider a wide range of perspectives and alternatives. The range of options and the complexity of tradeoffs have increased as principles of integrated water management, water-sensitive urban design and liveable cities have evolved. There are many methods available to compare sustainability impacts of different urban water options, and different decision makers prefer (or require) different methods (Fane, Blackburn and Chong, 2010). One method that has gained popularity in the water industry and is supported by the Water Services Association of Australia (WSAA) is multi- criteria analysis (MCA) (Lundie et al., 2008). MCA is a decision-making framework designed to help decision makers balance multiple objectives and multiple viewpoints, particularly when impacts are difficult to value in dollar terms (Asafu Adjaye, 2005). MCA ranks options based on a set of criteria developed in conjunction with stakeholders and the relative importance of criteria is represented by weights. A way of measuring against each criterion is agreed, but this does not have to be a dollar value. In fact, it does not have to be a value that can be quantified directly; a subjective, or qualitative assessment can be made. This allows decision makers to consider externalities even when the impacts cannot (or are too difficult to) be measured (Asafu Adjaye, 2005). MCA also makes it possible to include impacts that are outside the strict economic definition of externalities. How Do the elements of Multi-Criteria Analysis Bias Against Small Systems? MCA is a tool to aid complex decisions. However, like all tools it is only as good as the data used and the way it is applied. Limited data availability, limited knowledge on how to include the measures, or common decision-making pitfalls can all affect the fair assessment of options in an MCA process. However, these issues can have a greater effect on less well-understood alternatives, such as using small systems to complement larger centralised systems. These general issues include: • Common decision-making pitfalls; • Including risk and uncertainty; • Incorporating and valuing flexibility; • Ensuring consistent assessment boundaries in terms of e.g ., level of service provided, population served, timescales considered, components of infrastructure lifecycle considered, etc; • Identifying, selecting and valuing benefits and externalities; • Choosing the most appropriate metrics. Common decision-making pitfalls Several common decision-making flaws can particularly affect the fair assessment of small systems when using MCA. It is recognised that, in general, decision making has a strong bias towards preserving the status quo, seeking out evidence that confirms the current norms and making choices in ways that justify past choices (Hammond, Keeney and Raiffa, 1998). In the Australian water industry the current planning framework often preferences large centralised solutions (LECG Limited Asia Pacific, 2011). Most of the urban water planning is undertaken by the centralised utilities. The planners and engineers in these organisations have a large ‘intellectual capital’ in centralised systems management, and in the engineering community at large there is a lack of technical knowledge on the implementation and performance of smaller systems and limited education or training available (Etnier et al., 2007). For small systems being evaluated against large centralised options through an MCA process, this means that the criteria is more likely to be developed thinking about how it applies to large options. The risk and uncertainty of the smaller options will tend to be over-emphasised, while there will be over-confidence in the performance and value of larger options. It has been shown that in infrastructure decisions the benefits are often overestimated and the costs are often underestimated (Commonwealth of Australia, 2006a). One study of transport alternatives found that costs were 20–45% higher than originally estimated and benefits were 20–51% overestimated. Similar results were found in other areas of major infrastructure investment (Office of Financial Management, 2007). This bias can make large options look better value than they are. This also negates the flexibility and modular benefits of decentralised options. Including risk and uncertainty MCA can explicitly deal with risk and uncertainty through sensitivity analysis where key weights or scores are changed to see how the final decision may be affected (Mukheibir and Mitchell, 2011). However, perceptions of risk and uncertainty may also be implicitly included in MCA analysis through the early exclusion of options in the screening process, the way options are valued, and the inclusion of criteria such as customer acceptance, and this can lead to biases against small systems. Both federal and state treasury agencies (Commonwealth of Australia, 2006; Office of Financial Management, 2007) suggest pessimistic values should be used in options evaluation. Using pessimistic scenarios is a particular issue for small systems and newer technologies as there is more uncertainty surrounding their performance, full lifecycle costs and acceptability. Small systems reduce the consequences of failure and, when used in conjunction with centralised systems, could help to reduce vulnerability to natural shocks (Gikas and Tchobanoglous, 2009; Pinkham et al., 2004). However, small systems are also more vulnerable to misuse and shock loads (Etnier et al., 2007; Pinkham et al., 2004). Due to the public health aspects of water and wastewater services, decisions tend to avoid risk (Nelson, 2008; Productivity Commission, 2011; Water Corporation, 2011). This risk adversity affects smaller, less well-understood options and is compounded as decision makers commonly recall and place more emphasis on dramatic or bad outcomes (Hammond, Keeney and Raiffa, 1998). This can lead to the positive risk benefits of small systems being negated by the negative risks and results in the early exclusion of potential small options, or poor acceptability or protection of public health scores. Selecting lesser-known (and potentially more risky) technology could also lead to decisions requiring review by Treasury departments under government procurement guidelines (see, for example (NSW Government, 2010). As identified by ACTEW, the more layers of uncertainty added to the process (in this case an extra approval authority) the less likely it is to be favoured (Productivity Commission, 2011). This again can lead to the early exclusion of options or poor acceptability scores.
Water Journal February 2013
Water Journal November 2012-1