Scholarly article on topic 'Cost-benefit Analysis (CBA), or Multi-criteria Decision-making (MCDM) or Both: Politicians’ Perspective in Transport Policy Appraisal'

Cost-benefit Analysis (CBA), or Multi-criteria Decision-making (MCDM) or Both: Politicians’ Perspective in Transport Policy Appraisal Academic research paper on "Social and economic geography"

CC BY-NC-ND
0
0
Share paper
Academic journal
Transportation Research Procedia
OECD Field of science
Keywords
{decision-making / "cost-benefit analysis" / "multi-criteria decioson-making"}

Abstract of research paper on Social and economic geography, author of scientific article — Jan Anne Annema, Niek Mouter, Jafar Razaei

Abstract The scientific literature frequently discusses questions if cost-benefit analysis (CBA) or multi-criteria decision-making (MCDM) is the appropriate appraisal tool in transport policy-making, or a combination of both. Hardly any literature exists on the opinions of real transport policy decision-makers about appraisal tools such as CBA or MCDM (or both) which are actually supposed to help them. The aim of this paper is to discuss from a politicians’ perspective how a useful transport policy appraisal tool might look like. Twenty-one Dutch transport politicians were interviewed on their views on CBA. The results show that they use CBA but in a non-decisive manner and they find the aggregate outcome (the composite result) of CBAs pretentious. They seem especially interested in appraisal tools which show clearly to them the political important trade-offs of a transport policy. This paper proposes a possible approach for such a trade-off information sheet using both CBA and MCDM.

Academic research paper on topic "Cost-benefit Analysis (CBA), or Multi-criteria Decision-making (MCDM) or Both: Politicians’ Perspective in Transport Policy Appraisal"

Available online at www.sciencedirect.com

ScienceDirect

Transportation Research Procedia 10 (2015) 788 - 797

Transportation

Procedía

www.elsevier.com/locate/procedia

18th Euro Working Group on Transportation, EWGT 2015, 14-16 July 2015,

Delft, The Netherlands

Cost-benefit analysis (CBA), or multi-criteria decision-making (MCDM) or both: politicians' perspective in transport policy

appraisal

Jan Anne Annema*, Niek Mouter, Jafar Razaei

Delf University of Technology, TPM, Transport and Logistics, Jaffalaan 5, 2628 BXDelftt

Abstract

The scientific literature frequently discusses questions if cost-benefit analysis (CBA) or multi-criteria decisionmaking (MCDM) is the appropriate appraisal tool in transport policy-making, or a combination of both. Hardly any literature exists on the opinions of real transport policy decision-makers about appraisal tools such as CBA or MCDM (or both) which are actually supposed to help them. The aim of this paper is to discuss from a politicians' perspective how a useful transport policy appraisal tool might look like. Twenty-one Dutch transport politicians were interviewed on their views on CBA. The results show that they use CBA but in a non-decisive manner and they find the aggregate outcome (the composite result) of CBAs pretentious. They seem especially interested in appraisal tools which show clearly to them the political important trade-offs of a transport policy. This paper proposes a possible approach for such a trade-off information sheet using both CBA and MCDM.

© 2015 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license

(http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-review under responsibility of Delft University of Technology

Keywords: decision-making; cost-benefit analysis, multi-criteria decioson-making

1. Introduction

In the year 2000 the Dutch government decided to make cost-benefit analysis (CBA) mandatory for supporting large transportation infrastructure project decisions. A Multi-criteria decision-making^ (MCDM) was also considered as becoming the mandatory supporting tool but was declined for two main reasons: First, the basis for the assigned

Corresponding author. Tel.: +31 (0)15 2788912. E-mail address: J.A.Annema@tudelft.nl

^ MCDM is also referred to as MCDA (multi-criteria decision-making analysis) or MCA (Multi-criteria analysis). In this paper we use only the term MCDM for the sake of clarity.

2352-1465 © 2015 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license

(http://creativecommons.Org/licenses/by-nc-nd/4.0/).

Peer-review under responsibility of Delft University of Technology

doi: 10.1016/j.trpro.2015.09.032

weights was considered to be not always clear in MCDM. Second, double-counting of project effects would be more difficult to avoid in MCDM compared to CBA because strict criteria for the inclusion of effects are lacking. However, after 15 years of CBA practice in transport policy-making, this tool is not undisputed. Mouter et al. (2013) interviewed 86 key players in the Dutch CBA practice (consultants, civil servants, lobbyists, academics). One of their outcomes was that 'in the Dutch CBA practice there is agreement that CBA must have 'a' role in the appraisal process of spatial-infrastructure projects. However, despite this wide support for CBA, there is a lot of controversy among economists and spatial planners concerning the value that is and should be assigned to CBA in the decision-making process (p1). Dimitriou et al. (2013, p.23) came to a harsher judgment related to CBA: 'Of the 44 international infrastructure specialists interviewed in this study, 84% considered CBA to be inadequate as a tool to appraise Mega Transport Projects'.

This brings us to some questions. Is CBA actually the appropriate tool in the appraisal process of transport policies? Would MCDM be better? Or a combination of both? The scientific literature discusses these questions frequently (see further Section 2). This paper builds on this literature but attempts to bring the debate further by adding the views of politicians. We deem it important to take the politician's view into account in this scientific debate because, surely, these tools are aimed to support their decision-making.

Therefore, in this study twenty-one Dutch politicians actively involved in transport policy decision making were interviewed in person and asked what their view was on CBA. As far as we know this is the second time that the real decision-makers in transport projects are asked for their view on an instrument that actually is supposed to support their decision-making. The first study is a relatively old study by Nyborg (1996; 1998) who interviewed in person sixteen Norwegian politicians on how they use CBA in their decision-making (we discuss the results of her study briefly in Section 2). The previously mentioned studies by Mouter et al.(2013) and Dimitriou et al. (2013) interviewed only to a very limited extent 'real' politicians (Mouter et al. only 2 out of 86 and Dimitriou et al. 2 out of 44).

The aim of this paper is to discuss from a politicians perspective how a useful transport policy appraisal tool might look like. Twenty-one politicians is a small sample, and only Dutch politicians are interviewed. Thus, only some preliminary ideas can be discussed in this paper (albeit the politician's views on some issues are surprisingly consistent). Therefore, this paper should be considered a discussion paper.

We first discuss the literature which give arguments pro or against CBA and MCDM for transport policy-making (Section 2). In Section 3 we present the results of the interviews of 21 Dutch politicians how useful they consider CBA. Section 4 defines some criteria for useful supportive tools for politicians based on Section 3. In Section 5 we answer our questions by confronting the criteria as defined in Section 4 with the characteristics of CBA, MCDM or combined methods. We also propose a new possibility for a supportive tool in Section 5. Finally, in Section 6 we provide conclusions.

2. Literature on CBA and MCDM in transport policy assessment

CBA is a technique which is used by decision-makers (mostly governmental bodies) to appraise the efficiency of a policy. Multi-criteria decision-making refers to a class of decision-making methods based on which a number of alternatives are evaluated with respect to a number of criteria. The criteria can be quantitative or qualitative. Here we name just a few: analytic hierarchy process (AHP) (Saaty, 1977); TOPSIS (Technique for Order of Preference by Similarity to Ideal Solution) (Hwang and Yoon, 1981); ELECTRE (ELimination Et Choix Traduisant la REalite) (ELimination and Choice Expressing REality); multi-attribute evaluation using imprecise weight estimates (IMP) (Jessop, 2014); Best Worst Method (BWM) (Rezaei, 2015).

CBA is used in many countries for ex ante transport appraisal (e.g. Eliasson and Lundberg, 2012; Grant Muller et al., 2001, Hayashi and Morisugo, 2000; Odgaard et al., 2005). To which extent MCDM methods are used in real-world transport policy decision-making is less clear. Macharis and Bernardini (2015) mention that MCDM methods have gained importance as evaluation method for transport projects. They reviewed mostly academic publications (N = 276) concerning the application or the applicability of MCDM methods for transport projects, and it is not always entirely clear if these publications describe real-world applications or academic proposals on how MCDM methods might be used in transport policy-making.

Striking is that in the scientific literature hardly opinions can be found from the real decision-makers (the politicians) about the tools which are supposed to help them. The only exception, as mentioned already, is the study by Nyborg (1996; 1998). She interviewed 16 Norwegian member of parliament about their use of CBA in transport policy-making. She concluded that most of them found CBA useful, but not decisive in ranking projects. She found also that attitudes towards CBA varied along the left-right political axis, with politicians to the left being the most skeptical. For MCDM methods used in transport appraisal we could not find any study which evaluates politician's

opinions on these methods.

Next to CBA or MCDM results also other supportive information is provided to the transport policy decisionmakers such as Environmental Impact Assessment Reports (or Strategic Environmental Impact Assessment Reports), and all kinds of specialized impact report (e.g., on the wider economic impacts of new infrastructure). This paper focusses on CBA and MCDM methods because both tools aim for something special within the supportive decisionmaking information: both aim to encompass all effects that accrue from a transport project and aggregate all the information analyzed in the study in one final indicator. For example, CBA aggregate numbers are the net present value, benefit-to-cost ratio and internal rate of return. MCDM methods find an aggregated score for each alternative (e.g. projects)considering all the decision criteria.

In the scientific literature many arguments can be found pro and against CBA and MCDM as useful tools for supporting transport policy-making. We now give a brief overview.

The strong point of CBA according to academics is that it is theoretically unambiguous (e.g., Mackie et al., 2013; Macharis and Bernardini, 2015; Beria et al., 2012). This unambiguity makes CBA 'a common language, known and used worldwide' (Beria et al, 2012, p. 148), and 'widely used, firmly embedded in project appraisal (Browne and Ryan, 2011,p. 230). Also, the main outcome of CBA (the efficiency criterion) is seen as valuable input for decisionmaking on governmental spending (e.g., Beria et al., 2012). According to Mackie et al. (2014, p. 3) 'an important advantage with using CBA is that it is a way to overcome cognitive, structural and process-related limitations and biases in decision making' . The main argument against CBA is (often mentioned in all kinds of different wordings) that 'CBA is found to be inadequate to incorporate and assess multiple, often conflicting objectives, criteria and attributes like environmental and social issues which are usually intrinsically difficult to quantify' (Barfod and Salling, 2015, p.1). For some impacts of transport policies (such as equity impacts) it is as yet hardly possible to put monetary values on them, for other effects (e.g., intangible impacts such as agglomeration effects or environmental impacts such as nature and landscape deterioration due to new infrastructure) the monetization is highly questionable (e.g., Barfod et al., 2011; Macharis and Bernardini, 2015; Beria et al., 2012; Browne and Ryan, 2011; Guhnemann et al., 2012; Shang et al., 2004).

One of the main advantages of MCDM methods - often mentioned - is that these methods are able to incorporate criteria/factors which cannot be easily expressed as a monetary value, or the criteria which are inherently qualitative and cannot be quantified like environmental impacts (Guhnemann et al., 2012, p. 16). The advantage of having the freedom of potentially including every possible impact in MCDM is also mentioned by, for example, Barfod et al. (2011), Macharis and Bernardini (2015), Beria et al. (2012) and Barfod and Salling (2015). Relatively many authors stress the advantage that MCDM methods also facilitate the incorporation of the opinion of different stakeholders into the decision-making process (Macharis and Bernardini, 2015, Beria et al., 2012, Thomopoulos et al., 2009). The main disadvantages of MCDM methods all relate to worries about their subjectivity, arbitrariness and to worries about double-counting effects (e.g. Beria et al., 2012; Thomopoulos et al., 2009; Browne and Ryan, 2011, Macharis and Bernardini, 2015).

The two methods are also quite similar in several respects. Both need as input the impacts estimates of the transport policy studied compared to a 'do-nothing' or 'do-minimal' scenario. Thus, both appraisal methods are strongly depended on transportation model results (including the often large uncertainties in these results) and both have to deal with the inherent future uncertainty (related to the assumed reference scenario(s)). Another similarity is that both methods 'process' the transport project impacts in such a way that they can be aggregated into one single number. CBA uses the concepts of willingness to pay (WTP) for positive effects (or willingness to accept a monetary compensation, WTA, for negative impacts) and discounting in order to correct for time preferences to aggregate. MCDM uses the perceptions of experts/decision-makers/stakeholders to weight and, subsequently, to aggregate project impacts into one single number. Both methods use all kinds of methods (some relatively complicated, some highly debated) to estimate these WTPs, WTAs or perceptions. In short, it is not surprising that 'worries about transparency' is the corresponding disadvantage for the two methods according to the scientific literature (e.g., Beria et al., 2012; Browne and Ryan, 2011).

3. Politicians view on CBA

What are the views of Dutch politicians on CBA after 15 years of mandatory CBA to help them? Twenty-one Dutch politicians were interviewed to find their view on CBA (Table 1). The 21 politicians were selected because all were (or still are) actively involved in decision-making on transport policies. Nine were or are active on the national

political level (members of parliament acting as the spokespersons for transport policy-making) and thirteen were or are active on the regional political level (provincial or municipal transport executives) (one acted on both levels). Ten politicians could be regarded left-wing (socialists, social-democrats, green party), the other eleven more as central or right-wing politicians. The interviews were semi-structured and were held in person for about one hour. Interview reports were send afterwards to the interviewee. Only the reports agreed by the interviewees are used in this study. Table 1 summarizes their main points of view. Although the picture from Table 1 is somewhat scattered we think that four points mentioned are relatively common:

• They indicate that they use CBA information but the information is not decisive.

• The usefulness of the composite number (NPV) is doubted. NPV is incomplete in their view. Politicians seem more interested in a clear picture of the trade-offs.

• They find CBAs non-transparent.

• They suggest process improvements.

In contrast with the results of Nyborg (1996; 1998) we cannot clearly observe a difference in opinions between left-wing and right-wing politicians in our data.

4. Discussing criteria for 'useful' supporting tools

We will now discuss the four common points mentioned in Section 3 to find criteria for 'useful' supporting tools.

Not decisive

In a way this outcome is stating the obvious: surely, politicians base political decisions mainly on political considerations. This result also confirms one of the main outcomes of the study by Nyborg (1998). A CBA result is clearly for most of the politicians interviewed just one of the inputs in their decision-making process. They use CBA results but they are skeptical ('always doubt the results'), which related to the scientific criticism on CBA (Section 2) seems wise to do. Also, they seem to use CBA results strategically as one politician states (if CBA results suit me politically, I use the results; otherwise I do not). Has this non-decisiveness role of CBA in politics any implications for criteria for 'useful' supportive tools? We do not think so. CBA seems to play a supportive role politically anyhow, and, thus, should be of the highest quality contents-wise. Naturally, academic and applied research have always contributed to improving CBA methods, underlying transportation models, other estimate methods and so forth, and this non-decisive CBA role gives no reason to change this research and effort.

Supportive role of the composite number can be doubted

Politicians seem not particularly interested in the composite number of CBA (Table 1: 'analysis' is too pretentious; 'NPV is too pretentious'; 'NPV hides important trade-offs', 'Showing trade-offs is key role of CBA'). Additionally, the NPVs in transport CBAs are almost always viewed as incomplete by the politicians (in accordance with the scientists views, Section 2). Annema and Koopmans (2014) showed that in Dutch transport CBAs (N = 49) published in the period 2000 to 2011 environmental impacts are often treated qualitatively, and, thus, not included in the NPV presented to the politicians.

Table 1 The view of 21 Dutch politicians on the use of CBA for transport policy-making

About CBA

Advise

Member Parliament

Member Parliament

Member Parliament

Member Parliament

Social- CBA is a useful tool. However, the CBA result is not decisive. CBAs

democrat have difficulty to take into account subjective value like liveability

Liberal-left Political considerations can be an argument to deviate from CBA

wing outcomes. Non-monetized items are treated unbalanced (such as environmental impacts)

Liberal-right Useful tool but not decisive. A critical attitude towards CBA outcome is

wing key

Liberal-left Especially useful to structure 'thinking' about a project. It shows the

wing trade-offs. Do not apply the results mechanically

More freedom for researchers/consultants to present their own observations/insights

Try to monetize everything. Still, I will always have reasons not to use the CBA outcome

Better explanation

Showing trade-offs is key role of CBA

Member Parliament

Member Parliament

Member council

Provincial executive

ex Provincial executive and ex Member

Christian- An instrument to make ex ante impacts clear. The notion of 'analysis' is

democrat too pretentious

Socialist An instrument to show impacts. CBA can block decision-making

Socialist A non-transparent muddle of information. CBA just shows what the people in power want

Liberal-right A tool to rank alternatives and a tool to improve a plan/proposal. CBA is

wing not suitable to scrutinize innovative projects

Social- CBA gives an opportunity to compare. CBA signifies not the truth.

democrat CBAs cannot include every impact. The NPV is too pretentious.

In the end politics is more important than CBA

More transparency

More transparency

Politicians should be involved in the choices/models made/used in CBA

Parliament

Municipal Socialist

executive

Provincial Liberal-right

executive wing

Member Social-

Parliament democrat

Municipal Christian-

executive democrat

Provincial Liberal-right

executive wing

Municipal Social-

executive democrats

Municipal Green party

executive

ex Municipal Liberal-right

executive wing

Provincial Christian-

executive democrats

Provincial Christian-

executive democrats

Ex Member Green party

Parliament

Useful tool to show impacts. CBA is unreliable because the commissioning party (e.g., Minister) manipulates the input.

CBA has structured decision-making. CBA is an indication. Some benefits cannot be expressed in money.

CBA focusses the decision-making process; good reality check. Utility maximization is not the only truth

CBA supports decision-making in order to be able to choose between different project alternatives

Next to CBA other information also important

CBA does not replace political decision-making. Beware of technocracy

One can learn from a CBA but it cannot give answers. Making CBAs is very time consuming

CBA can be used to reconsider projects; to improve a project; and to prioritize projects. In the end, the decision is political

If CBA results suit me politically, I use the results; otherwise I do not

Systematic, objective information. Other criteria than CBA result play also important role

Supportive tool but weak in impacts such as nature, environment. In NPV very important political decisions can be hidden

More room/freedom for analyst

Also other information than CBA (such as EIA) is useful

Always doubt the results Include public participation

Organize learning about CBA within the government; organize the 'outside view'

Build own expertise within municipality

Use CBA-light for small projects

Independent review of CBA needed; more transparency (CBAs are a mess)

CBA's NPVs are used too absolute by politicians

Be explicit, be explicit, be explicit...

Here, an important gap might be noted between politicians and academics attempting to support decision-making. For, especially the analysis part seems interesting for academics to work on while our research shows that in practice this analysis part might be of less importance in the decision-making. In CBA much research is devoted on finding and applying the best scientific methods to estimate the 'willingness to pay' for all kinds of impacts, much debate is going on about the appropriateness of using certain discount rates, on how to include uncertainty and risks in the appraisal, and so forth. However, politicians seem more interested in a clear picture of the important trade-offs of a transport policy than the result of using these 'sophisticated' methods which shows in one number that the benefit-to-cost ratio of the trade-offs is finally positive (or negative). Related to this, an interesting quote is from a green party politician (bottom row, Table 1): 'Suppose Schiphol airport wants to extent. This might lead, as an example, to 3,500 extra jobs in the region and 85,000 extra people with sleep disturbance due to higher noise levels. This trade-off should be debated explicitly by politicians. In my view it might be interesting to express these changed impacts due to the extension in monetary terms (and I do not say not to do this) but this approach does not end the political debate. If the Minister has a higher preference for realizing the jobs compared to avoiding extra noise nuisance I want to hear this from her in a public debate and I want to be able to express my political views with respect to this trade-off (which would be another one). This signifies transparency for the citizens' In other words, this politician seems to imply that the composite number can lead to technocracy (instead of democracy) because it hides the real political choices.

What does this result mean for criteria for 'useful' supportive tools? We think that for really useful supportive tools more emphasis should be given in showing clearly the important trade-offs of a policy (and less attention to aggregating the different impacts into one number).

Transparency important

Many politicians mention their need for more transparency (Table 1). In our view transparency might be related to the composite number 'issue', as just discussed. On one hand, it seems attractive to have one number showing that a transport project is efficient or not. On the other hand, estimating this composite number requires a high number of assumptions and methods (impact models, methods to estimate willingness to pay or stakeholder preferences, and so forth) which makes the final number apparently opaque. Thus, a criterion for a transparent supportive tool could be to choose a less ambitious approach as already suggested in the previous paragraph (show important trade-offs of a policy, not (only) an aggregate result).

Process issues important

Interestingly also is that the 21 politicians often advise about process related CBA issues ('more freedom for researchers; include public participation; organize outside view; independent reviews, and so forth'). It is beyond the scope of this paper to evaluate if exactly these process proposals are useful (our sample of 21 Dutch politicians is very limited, naturally). However, important is that these suggestions of the politicians show that criteria for useful supportive tools should not only be limited to criteria which aim at getting the numbers right. The interviews with the 21 politicians also stress their need of processes that can provide trustworthy information. This outcome is in accordance with Beukers (2015) and Mouter (2014) who - based on interviews with many people involved in the Dutch CBA practice - show that also these people consider process-related improvements very important in order to enhance the quality of the Dutch CBA practice. Thus, a criterion for a useful supportive tool is the need of process instruments to prevent that CBAs become politically ineffective because the final decision-makers just do not trust their results. Most strongly voiced by one of the politicians in Table 1: 'CBA just shows what people in power want'.

5. Discussing CBA, MCDM or both based on the criteria

This brings us to the questions if CBA is actually the appropriate tool in the appraisal process of transport policies. Would MCDM still be better? Or a combination? The criteria as found by interviewing politicians induced

us to three considerations related to these questions (due to the low number of politicians interviewed, not to definitive answers).

First, CBA seems to be useful in itself. Since, most of the 21 politicians state that they use the tool. They use CBA results critically, non-decisive, and partially because they seem to be especially interested in the trade-offs of a policy, not in a composite number. Second, our results do not show clearly if MCDM would be better. From the viewpoint of the criteria 'showing clearly the important trade-offs', 'transparency' and 'process-related issues (trust in the results)', there seems to be no reason to judge MCDM better nor worse than CBA (Section 2). MCDM seems just to be different. Thirdly, a combination of CBA and MCDM might not be the best way forward because a tool combining two already quite difficult methods may even more hamper transparency and trust. Leaving aside that a combined tool also leads to a composite number which does not seem to get the highest attention from politicians. Examples of CBA - MCDM combination methods for transport policy appraisal can be found in Guhnemann et al. (2012), Barfod et al. (2011) and Barfod and Salling (2015). As a sideline, we would like to stress that in this paper we do not intend to criticize these specific methods proposed contents-wise.

Is a new or other appraisal tool or approach thinkable that fulfills the criteria? One of the most simple supportive practices which seem in accordance with our criteria is cited in Guhnemann et al. (2012, p. 16): "In transport appraisal practice in many countries, such as UK or Switzerland, no weightings are applied and decision makers are deliberately left to make their own trade-offs between criteria. Reasons for this include that the aggregation of impacts into one measure is seen as inadequate because it implies a potential compensation of impacts and reduces transparency of the results and, secondly, that it is difficult to establish accepted weights (Walter et al., 2006, cited in Guhnemann et al.). Also in Sweden decision-makers are provided with information on several criteria (efficiency, effectiveness, equity and sustainability) and it is up to them to make a trade-off (ASEK, 2013).

Another possible approach is sketched in Figure 1. The core of this idea is that politicians receive a 'trade-off sheet' showing the politically most important trade-offs of a transport policy proposal. CBA can provide the efficiency criterion and MCDM methods other aggregated criteria, for example, for the equity impact of the proposal and/or it's environmental impact. Politicians can use these aggregated criteria on a sublevel (e.g., economic efficiency in NPV; total environmental impact; total equity impact) in a non-weighted form in the political debate and, finally, in their voting. Naturally, also trade-offs on a more disaggregated level ('regional jobs versus more noise due to an airport extension'; 'lower travel times against more traffic casualties due to policy to increase maximum speeds on highways') can be given. The political need for this could be elicited by using some form of stakeholder participation (Figure 1).

Why still aggregating on a sublevel when this might conflict with the criterion 'transparency'? The reasons are twofold. First, the efficiency criterion seems in itself relevant information for politicians related to public spending (see Section 2). Second, environmental impacts and equity impacts of transport policies can be that divers that a perception based method (which MCDM is) might be useful to aggregate them into one number to reduce complexity in the decision-making process. An example of a method for aggregating equity impacts in transport infrastructure is the so-called SUMINI approach (Thomopoulus and Grant-Muller, 2013). SUMINI uses a MCDM approach (the heart is the Analytical Hierarchy Process) 'to quantify the implications of transport infrastructure improvements over a range of equity types' (p. 318).

Naturally, this 'trade-off sheet' approach is just a very rough idea which requires research to assess if it is a useful tool in supporting transport policy-making.

Figure 1 Possible supportive tool for transport policy-making: the trade-off sheet 6. Conclusions

There is hardly any literature on real transport policy decision-makers opinions about appraisal tools such as CBA and MCDM which are actually supposed to help them. Interviews with 21 Dutch transport politicians show that they use CBA but in a non-decisive manner. They find the aggregate outcome (the composite result) of CBAs pretentious. They seem especially interested in appraisal tools which show clearly to them the political important trade-offs of a transport policy. In this paper we formulated some criteria for a transport appraisal tool which aims to shows the trade-offs clearly, and compared CBA with MCDM in a qualitative manner. We think that the application of the proposed framework should be tested in some case studies to reveal its usefulness.

References

Annema, J.A., C, Koopmans (2014), The practice of valuing the environment in cost-benefit analyses in transport and spatial projects, Journal of

Environmental Planning and Management, DOI: 10.1080/09640568.2014.941975 ASEK., 2013, Arbetsgruppen for samhallsekonomiska kalkyl- och analysmetoder inom transportomrádet. Trafikverket, Stockholm Barfod, M.B., Salling, K.B. (2015), A new composite decision support framework for strategic and sustainable transport appraisals, Transport Research Part A, 72, pp. 1-15

Barfod, M.B., Salling, K.B., Leleur, S. (2011), Composite decision support by combining cost-benefit and multi-criteria decision analysis,

Decision Support Systems, 51, pp. 167 - 175 Beria, P., I. Maltese, I. Mariotti (2012), Multicriteria versus Cost Benefit Analysis: a comparative perspective in the assessment of sustainable mobility, Eur. Transp. Res. Rev., 4, pp. 137 - 152

Beukers, E., (2015), Shaking up the Cost benefit Analysis. Issues and directions for improvement when assessing integrated spatial transport

plans through a cost benefit analysis, Thesis, Universiteit van Amsterdam Browne, D., & Ryan, L. (2011). Comparative analysis of evaluation techniques for transport policies, Environmental Impact Assessment Review,

31 (3), pp. 226-233

Dimitriou, H. T., Ward, E. J., Wright, P.G. (2013), Mega transport projects—Beyond the 'iron triangle': Findings from the OMEGA research

programme, Progress in Planning, 86, pp. 1-43 Eliasson, J., Lundberg, M. (2012), Do Cost-Benefit Analyses Influence Transport Investment Decisions? Experiences from the Swedish

Transport Investment Plan 2010-21, Transport Reviews, 32 (1), pp. 29-48 Grant-Muller, S., Mackie, P., Nellthorp, J., Pearman, A. (2001), Economic appraisal of European transport projects - The state of the art

revisited, Transport Reviews, 21 (2), pp. 237-261 Guhnemann, A., Laird, J.L, Pearman, A.D. (2012), Combining cost-benefit and multi-criteria analysis to prioritise a national road infrastructure

programme, Transport Policy, 23, pp. 15-24 Hayashi, Y., and Morisugi, H. (2000), International comparison of background concept and methodology of transportation project appraisal, Transport Policy 7 (1), pp. 73-88

Hwang CL, Yoon K. Multiple attribute decision making: methods and applications: a state-of-the-art survey: Springer-Verlag New York; 1981 Jessop A. IMP: A decision aid for multiattribute evaluation using imprecise weight estimates. Omega. 2014;49:18-29

Macharis, C., Bernardini, A. (2015), Reviewing the use of Multi-Criteria Decision Analysis for the evaluation of transport projects: Time for a

multi-actor approach, Transport Policy, 37, p. 177 -186 Mackie, P., Worsley, T., Eliasson, J. (2014), Transport appraisal revisited, Research in Transportation Economics, 47 (1), pp. 3-18 Mouter, N. (2014), Cost-Benefit Analysis in Practice. A study of the way Cost-Benefit Analysis is perceived by key individuals in the Dutch

CBA practice for spatial-infrastructure projects, Thesis, Delft University of Technology Mouter, N., Annema, J.A., van Wee, G.P. (2013), Attitudes towards the role of Cost-Benefit Analysis in the decision-making process for spatial-

infrastructure projects: A Dutch case study, Transportation Research Part A, 58, pp. 1-14 Nyborg, K. (1996), Some Norwegian Politicians' Use of Cost-Benefit Analysis, Discussion Paper 169, Statistics Norway Nyborg, K. (1998), Some Norwegian Politicians' Use of Cost-Benefit Analysis, Public Choice, 95, pp. 381-401

Odgaard, T., Kelly, C., Laird, J. (2005), Current practice in project appraisal in Europe, in: Proceedings of the European Transport Conference. 35 October, Strasbourg, Association for European Transport Rezaei, J. (2015). Best-worst multi-criteria decision-making method. Omega, 53, 49-57.

Roy B. Classement et choix en présence de points de vue multiples (la méthode ELECTRE). Riro. 1968;2:57-75. Saaty TL. A scaling method for priorities in hierarchical structures. Journal of Mathematical Psychology. 1977;15:234-81.

Shang, J.S, Tjader, Y., Ding, Y. (2004), A Unified Framework for Multicriteria Evaluation of Transportation Projects, IEEE TRANSACTIONS

ON ENGINEERING MANAGEMENT, 51, (3), pp. 300 - 313 Thomopoulos, N., Grant-Muller, S. (2013), Incorporating equity as part of the wider impacts in transport infrastructure assessment: an application

of the SUMINI approach, Transportation, 40 (2), pp. 315 - 345 Thomopoulos, N., Grant-Muller, S., Tight, M.R. (2009), Incorporating equity considerations in transport infrastructure evaluation: Current

practice and a proposed methodology, Evaluation and Program Planning, 32, pp. 351-359 Walter, F., Lieb, S., Simmen, H. (2006). Nachhaltigkeitsbeurteilung des Bundes bei Strassenprojekten: NISTRA. Umweltrecht in der Praxis (URP), Vereinigung für Umweltrecht (VUR), 20 (5), pp. 571-586