Scholarly article on topic 'Priority-Setting Institutions in Health'

Priority-Setting Institutions in Health Academic research paper on "Economics and business"

Share paper
Academic journal
Global Heart
OECD Field of science

Abstract of research paper on Economics and business, author of scientific article — Amanda Glassman, Kalipso Chalkidou, Ursula Giedion, Yot Teerawattananon, Sean Tunis, et al.

Abstract The rationing problem is common to all health systems—the challenge of managing finite resources to address unlimited demand for services. In most low- and middle-income countries, rationing occurs as an ad hoc, haphazard series of nontransparent choices that reflect the competing interests of governments, donors, and other stakeholders. Yet in a growing number of countries, more explicit processes, with strengths and limitations, are under development that merit better support. Against this background, the purpose of the Center for Global Development Working Group, which is to examine how priorities are set currently, and to propose institutional arrangements that promote country ownership and improve health outcomes by more systematically managing this complex process of politics and economics, is discussed. Current global and national priority-setting practices in low- and middle-income countries, the potential for strengthened national institutions, and increased global support are reviewed. Recommendations for action are provided.

Academic research paper on topic "Priority-Setting Institutions in Health"

© 2012 World Heart Federation (Geneva). Published by Elsevier Ltd. All rights reserved.

VOL. 7, NO. 1, 2012 ISSN 221 1 - 8160/$36.00 DOI: 1 0.101 6/j.gheart.201 2.01 .007


Priority-Setting Institutions in Health

Recommendations from a Center for Global Development Working Group

Amanda Glassman ^ Kalipso Chalkidou Ursula Giedion ", Yot Teerawattananon Sean Tunis §, Jesse B. Bump Andres Pichon-Riviere "

Washington, DC, USA; Baltimore, MD, USA; London, United Kingdom; Bogota, Colombia; Nonthaburi, Thailand; and Buenos Aires, Argentina


The rationing problem is common to all health systems—the challenge of managing finite resources to address unlimited demand for services. In most low- and middle-income countries, rationing occurs as an ad hoc, haphazard series of nontransparent choices that reflect the competing interests of governments, donors, and other stakeholders. Yet in a growing number of countries, more explicit processes, with strengths and limitations, are under development that merit better support. Against this background, the purpose of the Center for Global Development Working Group, which is to examine how priorities are set currently, and to propose institutional arrangements that promote country ownership and improve health outcomes by more systematically managing this complex process of politics and economics, is discussed. Current global and national priority-setting practices in low- and middle-income countries, the potential for strengthened national institutions, and increased global support are reviewed. Recommendations for action are provided.

A fundamental challenge for all health systems is to allocate finite resources across the unlimited demand for health services. This is a rationing problem, regardless of whether it is explicitly addressed as such, because it requires active or passive choices about what services are provided to whom, at what time, and at whose expense. Inevitably, some demand goes unmet, which is a source of the intense pressure to provide more services within any given resource envelope. Efforts to reduce waste, increase quality, and improve efficiency are all responses to this pressure. Expanding healthcare costs are another reflection of the same forces. A recent Organisation for Economic Co-operation and Development (OECD) report [1] found that health spending growth exceeded economic growth

in almost all OECD countries over the past 15 years. In the context of worsening fiscal positions in the global recession and greater demand for services because of aging populations, as well as more complex and expensive health technologies, the pressure on OECD health systems to deliver more care with greater efficiency is unprecedented.

Policymakers and analysts working in low- and middle-income countries (LMIC) are concerned with the same issues. In many middle-income countries, economic growth is also accompanied by a greater proportion of gross domestic product devoted to health. This has led to a significant increase in health spending, often because of costly technologies of marginal benefit while most of the population remains without access to basic and

The authors have reported that they have no relationships relevant to the contents of this article to disclose.

From the t Center for Global Development, Washington, DC, USA; ^Department of International Health, Georgetown University, Washington, DC, USA; §Center for Medical Technology Policy, Baltimore, MD, USA; —National Institute for Health and Clinical Excellence, London, United Kingdom; ||Independent, Bogota, Colombia; 'Health Intervention and Technology Assessment Program, Department of Health Ministry of Public Health, Nonthaburi, Thailand; "Health Technology Assessment and Economic Evaluations, Institute for Clinical Effectiveness and Health Policy, University of Buenos Aires, Buenos Aires, Argentina. Correspondence: A. Glassman (

highly cost-effective care. Many LMIC have employed technical "priority-setting" approaches to argue for certain interventions over others. The burden of disease approach, applied by the World Health Organization (WHO) using the disability-adjusted life year (a standardized measure of disease burden), has been used to facilitate comparisons of one health problem versus another. Cost-effectiveness analysis is another commonly invoked technique, for instance as used by the United Kingdom's Department for International Development in its "value for money'' initiative. In general, these approaches are intended to maximize the impact (however defined) of health spending in LMIC, but because donors and others usually produce their own analyses using their own methods, there is no consensus on what should be prioritized.

The result of a myriad of actors championing a kaleidoscope of "priorities" is confusion. Advocates, researchers, and policy makers have labeled almost every disease, condition, medication, or intervention a "health priority." From rotavirus to cardiovascular disease, from leishmaniasis to cervical cancer, from vaccines to palliative care, the gamut of health needs and possible responses are prominent on the agendas of various groups. Competing advocacy efforts are not new and are likely to remain a key feature of the landscape. For instance, the recent UN High-Level Meeting on Non-Communicable Diseases [2] promoted at least 4 new conditions as global priorities, without recognizing the difficult reality that prioritizing some issues also means deprioritizing others. The problem is particularly acute for policy makers in many LMIC because they do not have institutional mechanisms that could assess various proffered ''priorities,'' evaluate political and economic constraints, and gather input from a broad range of stakeholders, including citizens.

In addition, many LMIC depend on external resources to finance their health systems, and many also rely on donors or other partners for technical assistance or implementation support. This adds an international dimension to the already complicated matter of setting priorities because reliance on external financing and advice leaves countries' domestic policy processes open to outside influence. A common outcome is a negotiated set of priorities that reflect some domestic needs and some technical, political, and economic considerations defined largely by the interests of donors. Few would argue that this process is optimal.

Coping with tradeoffs in the prevention and treatment of human immunodeficiency virus/ac-

quired immunodeficiency syndrome (HIV/AIDS) is among the most visible global rationing dilemmas facing policy makers and donors. Only 36% of those needing antiretroviral therapy were receiving treatment in Africa as of December 2009 [3]. Countries must decide who will receive lifesaving antiretroviral therapy, when, and under what circumstances. Yet in most cases, countries have not defined explicit rationing criteria, leading to a nontransparent process rife with opportunities for abuse by those with influence and partially or completely void of ethical considerations [4]. An Institute of Medicine panel on the future of HIV/AIDS in Africa [5] acknowledged this problem, calling for a ''decision-making process for resource allocation [that] incorporates robust safeguards not only against discrimination but also against arbitrary or self-serving exercises of power.''

Potential conflicts over priorities are already very complicated within the domestic sphere. For instance, important drivers of costs in middle-income countries have been rights-based legal arguments by which citizens have used the court system to compel the provision of often-expensive therapies. This strategy has been used in some South American countries where the right to health is enshrined in constitutional law. In these instances, citizens have forced the expansion of state-provided services, which leads to increased health system expenditures and/or cuts in other services. This practice will likely spread; civil society organizations in Uganda recently sued the government over 2 maternal deaths [6]. Because this strategy only compels the government to provide specific additional services, it does not address wider implications of how much should be spent, how the resources should be used, or what tradeoffs might result [7,8].

In many cases, health systems plan to provide far more services than is possible under their existing budgets, which creates the difficult problem of rationing the provision of promised services. Many plans have no budgets linked to the activities described in the plan. Ssengooba [9] illustrates this quandary in Uganda, where a package of services costing $28 per capita was expected to be delivered with an $8 per capita actual expenditure. More generally, governments in many LMIC tend to promise universal coverage to all those in need. For example, the constitution of Haiti states: ''the state of Haiti has the absolute obligation to guarantee the right to life, health and protection without distinction'' [10]. However, financial resources are

so low that this commitment cannot be fulfilled in practice. The lack of coherence between a limitless promise and limited resources leads to implicit rationing through waiting lines, low quality, inequities, and other mechanisms.

The multiplicity of priorities and the lack of institutional mechanisms at global and national levels to rationalize services and spending often result in poor overall system performance, as manifested in low coverage levels for highly effective health technologies, relatively excessive supply of high-cost services with limited clinical advantages, inefficient and underused health facilities, inequitable access to services, and ultimately poor health outcomes. Gains from reallocating toward more cost-effective health interventions can be substantial, illustrating the human and financial costs associated with weak and implicit rationing institutions. For example, a 2011 study by the Center for Global Health Research in India [11] estimates that a basic package of cost-effective health care would cost about one-half of current per capita public spending on health and reduce total deaths by 28%. For cardiovascular disease (CVD), Gazi-ano et al. [12] find that pharmacotherapy can be cost-effective in low resource settings and that CVD-related deaths could be reduced by up to 50%. Yet in most countries, the rationing compromises of the current implicit priority setting mechanisms are rarely made explicit to policy makers or the citizens they serve.

In this paper, we begin with the argument that for any given budget, systematic approaches to priority setting produce more health services and ultimately better health outcomes than do approaches that are haphazard or largely ad hoc. The large literature on priority setting shows widespread support for this position. However, most investigators have concentrated only on the technical aspects of priority setting, such as quantifications of the burden of disease or the cost-effectiveness of interventions. Although these technical contributions constitute key potential inputs into the process, priority setting is fundamentally a political exercise, because it involves the distribution of benefits and responsibilities. As Reich [13] observed in his critique of the 1993 World Development Report (WDR), technical advancements do not automatically translate into political action. Our review of the field finds that the process by which countries move toward more systematic approaches remains largely unanalyzed. Accordingly, we focus our attention on institutional mechanisms

that can help manage the politics of decision making by providing independent, technically sound assessments of health priorities and interventions. We begin by briefly reviewing global efforts to support more systematic priority setting. We then review national-level institutions that have tried to increase the influence of technical evidence in priority-setting decisions. We discuss the evidence and then identify important decisions and challenges that arise when establishing and managing priority-setting institutions. We conclude with recommendations for how countries can move toward more systematic priority setting to increase the effectiveness of their health spending and promote better lives for their citizens.


Global-level attempts to support priority setting have been undertaken for at least 3 decades, but these efforts have tended to focus on only a few aspects of priority setting, such as gathering evidence or creating methods, and have yet to result in broad improvements via more systematic decision making in LMIC. Perhaps the best-known of these projects has been the World Bank's 1993 WDR [14] and related literature, which was inspired by the Oregon Health Services Commission's effort to prioritize within the U.S. Medicaid program in the late 1980s [15]. This work introduced the global health community to the use of cost-effectiveness criteria to determine which health interventions would be publicly financed—and by extension donor-financed. These investigators argued that burden of disease estimates could be combined with a cost-effectiveness rank list of interventions to derive packages of services that would facilitate the largest improvement in health as measured by disability-adjusted life years, for any given budget. Murray et al. [16] distinguished between calculating these packages from a ''ground zero''—where the current budget is applied to the whole of disease burden in the absence of currently financed health interventions—versus a ''marginal expansion''—wherein a package would be constructed with an additional or marginal increase in budget applied to the current disease burden.

Criticisms of this approach have focused on the weak data on which estimates of burden, cost, and effectiveness relied; the value judgments implicit in disability-adjusted life year age weighting and discounting decisions; and treatment of equity issues

[17], as well as the political difficulties associated with translating a ground zero package into a public budget based on historical inputs [18]. Further, early cost-effectiveness analysis only considered health maximization as the priority-setting objective; later work has highlighted the need for multidimensional priority-setting objectives, including equity and fairness, financial protection, and existing health system capacity. Beyond these considerations, policy implementation and adoption issues have not been explored adequately.

Since the 1993 WDR [14], much work has focused on regularly updating estimates of the global burden of disease, injuries, and risk factors (1990, 2000-2002, 2004 thus far; 2005 and 2010 projections underway) via the Disease Control Priorities Project and a connected WHO/Institute for Health Metrics and Evaluation collaboration [19]. The availability of this information has likely played a role in the increasing number of LMIC-focused cost-effectiveness studies published in the peer-reviewed literature, which now number in the thousands. But their application in practice to actual priority-setting processes is likely to be very small.

Another growing area of activity relates to models and tools that policy makers could use to design their own packages or identify priority interventions. For instance, the 1994 Health Resource Allocation Model, by Murray et al. [16], is optimized for burden of disease, cost-effectiveness of available health interventions, and available health system infrastructure. Other examples include WHO's Choosing Interventions that are Cost-Effective (CHOICE) Project, which incorporates interactions between concurrent interventions and models the effect of scale on costs and effectiveness for every intervention and every combination at different levels of coverage [20,21]. The Lives Saved Tool [22], developed by Johns Hopkins University, allows users to compare the health impact of alternative coverage strategies over a period of time, but it does not model costs. The comprehensiveness of these models is growing. Evidence and Value: Impact on Decision Making (EVIDEM) combines a multicriteria decision analysis value matrix made up of 15 quantifiable components of decision (quality of evidence, disease, intervention, and economics) with a qualitative tool including 6 ethical and health system-related components of decision, and pilots the tool for a single disease in South Africa and Canada [23]. Baltussen et al. [24] use multicriteria decision analysis to prioritize interventions in 7 LMIC, concluding that methodolo-

gies like this ''can have far-reaching and constructive influences on policy formulation.'' Yet in all cases, the models remain in the literature or in pilots, and have rarely been deployed to inform ongoing policy-making processes.

The international community has also focused on improving the quality and availability of epide-miological, demographic, use, cost, and effectiveness data that feed into burden of disease and cost-effectiveness estimations, most notably via the Health Metrics Network [25], the Demographic and Health Surveys [26], and the Multi-Indicator Cluster Surveys [27]. Nevertheless, basic statistics on births, diseases, and deaths remain problematic [28]. (According to the UN Statistics Division, only around 20% of African countries provided complete information on births and deaths for at least 1 year in the 2003-2007 period [28].) Furthermore, little funding has gone to incentivize or support the regular, quality collection of these data and their use in actual decision making. For example, in spite of the importance of these data for assessing cost-effectiveness, value for money, and the impact of their investments, neither the GAVI Alliance nor the Global Fund for AIDS, Tuberculosis, and Malaria require standardized reporting on total public expenditure on health, relevant disease epidemiology, and the local costs ofprovision as pre-conditions to the receipt of their funding [29].

Among the few donor-sponsored initiatives to directly strengthen evidence-based resource allocation at the country level is the Supporting Independent Immunization and Vaccine Advisory Committees Initiative (SIVAC). SIVAC builds the capacity of National Immunization Technical Advisory Groups and, in some cases, provides support to carry out cost-effectiveness studies in support of new vaccine introduction or new vaccine technology [30]. However, the exclusive focus on vaccination creates its own challenges, as new vaccines are not necessarily compared with the appropriate counterfactual. For example, vaccines are frequently compared with a ground-zero situation-- a vaccine versus no intervention--even when screening and/or treatment can be viable alternatives. Although these efforts could help prioritizing among vaccines, they do not address prioritization between vaccines and other options, which severely limits their usefulness for national priority setting in health.

Another prioritization effort is the essential medicines list (EML). Defined as ''those [medi-

cines] that satisfy the priority health care needs of the population'' and launched in 1977, WHO— with the help of an international expert committee made up mainly of clinical pharmacists——creates lists of medicines ''with due regard to disease prevalence, evidence on efficacy and safety, and comparative cost-effectiveness'' to inform purchasing decisions in the context of national health agendas. A model list—updated every 2 years based on applications—is published online and frequently adopted (though not necessarily funded or provided) by governments in developing countries. As of 2011, 156 countries have adopted versions of the EML [31]. Whereas indicators to track access to essential medicines were only established in 2009 and information on funding is not available, a meta-analysis of 679 published studies (mostly from Sub-Saharan Africa) of variable design and quality reviewed medicines used in primary care in developing countries and found that the share of all prescribed medicines that are included on an EML has increased over time [32]. However, this does not imply availability of all or indeed most of the medicines listed on the EML. A study in India looked at 5 essential medicines for children included on the EML, finding that 4 of 5 were actually available in public facilities [33]. Conceptually, the weaknesses of EML relate to its exclusively primary care focus when many countries are seeking international advice on how to handle higher cost medications provided in hospitals, as well as the limited linkages between the medicines included on EML and the benefits, costs, and affordability implications of actually providing these medicines. Furthermore, EML have no local processes of updating or monitoring/evaluation attached. Finally, the EML approach has not been extended to devices, diagnostics, procedures, and other health interventions of increasing importance in LMIC.

In summary, global efforts to support national policy making have been limited to specific diseases, conditions, or types of technologies (such as drugs or vaccines) and have focused on the generation of global evidence, rather than generating country-specific data, supporting country decision making, building national institutions, or facilitating exchanges and collaboration. Interestingly, although efforts are global in scope, none of the actions mentioned directly help global funding agencies set priorities, in spite of a growing number of new technologies in the context of declining global health resources and evidence of widespread use of ineffective or unnecessary services.


We reviewed national-level attempts to make priority setting more systematic to understand what initiatives and processes have been taken on countries' own initiatives and to see what links there may be between these actions and some of the international research and data collection efforts. We find that national efforts to support explicit priority setting in health have taken 2 main paths: health benefits plans or lists (HBP) and health technology assessment (HTA) agencies. We review each separately.

Health benefits plans. As discussed, the World Bank's 1993 WDR and the WHO's Commission on Macroeconomics and Health both recommended a basic package of cost-effective care in LMIC as a priority-setting mechanism for LMIC. Table 1 [34—96] shows that this recommendation was influential; we identify at least 63 LMIC that have established an explicit positive and/or negative package, plan, or list of health interventions to be funded by public resources, insurance, or social security contributions (Table 1). Although HBP are sometimes thought to pertain exclusively to health insurance schemes, HBP are being used in all kinds of health systems, even those without purchaser—provider splits, as a means to structure resource allocation and potentially improve accountability, equity, enforcement of rights, and control of spending. In some countries, these benefits packages have become some of the main drivers of public health expenditure and thereby a key tool for strategic purchasing of health services. In a review of the 98 World Bank health projects approved between 2009 and 2011, 31—about one-third——refer to plans to develop or support a health benefits package. In addition, HBP have been used directly by donors in fragile states such as Afghanistan, Haiti, and South Sudan to contract with nongovernmental organizations for direct service provision. It is likely that the proliferation of HBP reflects the influence of recommendations from the 1993 WDR and WHO's Commission on Macroeconomics and Health, which both recommended a basic package of cost-effective care in LMIC as a priority-setting mechanism for LMIC. Table 1 shows that this recommendation was influential.

Drawing on the existing literature, we find that countries face challenges with HBP, and we summarize the evidence. We note that a systematic

Table 1. LMIC with health benefits plans

World Bank country grouping

Number Positive or negative list of health benefits

Eastern Europe and Central Asia 18

Latin America and Caribbean

Middle East and North Africa

Sub-Saharan Africa

Health insurance schemes: Azerbaijan (SAMHI decree 2008) [34]; Bulgaria (NHIF 1998) [35]; Croatia (HZZO) [36]; Estonia (EHIF 2001) [37]; Georgia (SMIC 1996) [38]; Hungary (OEP

1992) [39]; Kyrgyz Republic (MHIF 1996) [40]; Lithuania (NHIF) [41]; Macedonia (HIFM) [42]; Moldova (MHI 2002) [43]; Poland (NFZ 2003) [44]; Romania (NHIF 2002) [45]; Russia (MHIF

1993) [46]; Slovenia (HIIS 1992) [47]

Tax-funded systems: Armenia (SHA 1998) [48]; Kazakhstan (MOH 2005) [49]; Slovak Republic (MOH) [50]; Tajikistan (MOH 2007) [51]

Health insurance schemes: Argentina (PMO 1997) [52]; Chile (AUGE) [53]; Colombia (POS 1993)[54]; Dominican Republic (SENASA 2002) [55]; Nicaragua (Listado de Prestaciones Medicas Quirurgicas de Salud: Ursula Giedion, Personal Communication, February 27, 2012) [56]; Peru (PEAS 2009) [57]; Uruguaya (PIAS 2007) [58]

Tax-funded systems: Argentina (Plan Nacer 2005) [59]; Bolivia (SUMI 2003) [60]; Brazilb [61]; Honduras (IHSS 1995) [62]; Mexico (CAUSES 2001) [63]; Nicaragua (MINSA) [56] Health insurance schemes: Lao People's Democratic Republic (HEF 2008) [64]; Philippines (PhilHealth 2006) [65]; Vietnam (HCFP 2003) [66]

Tax-funded systems: Cambodia (HEF 2000) [67]; China (NRCMS 2003) [68] ; India (NRHM 2005) [69]; Malaysia (MOH) [70]; Thailand (UCS 2001) [71]

Health insurance schemes: Egypt (HIO 1964) [72] Israel (NIII 1995) [73]; Lebanon (CNSS) [74]; Malta (NHS 1956) [75]; Syria (NHIS 2010) [76]; Tunisia (CNAM 2007) [77]; United Arab Emirates (Daman 2006) [78]; West Bank and Gaza (GHI) [79]; Yemenc [80] Tax-funded systems: Bahrain (MOH 1979) [81]; Djibouti (MOH) [82]; Jordan (MOH 1965) [83]; Morocco (AMO 2005) [84]; Oman (MOH) [85]; Qatar (SCH) [86]; Saudi Arabia (NHS) [87] Health insurance schemes: Ghana (NHIS 2004) [88]; Kenya (NHIF 1998) [89]; Namibia (NMBFd) [90]; Nigeria (NHIS 1999) [91]; Senegal (CBHI) [92]; South Africa (NHIe) [93]; Tanzania (NHIF 1999) [94]; Uganda (NHIf) [95] Tax-funded systems: Zambia (MOH 1993) [96]

AMO = Compulsory Health Insurance; AUGE = Regime of Explicit Health Guarantees ; CAUSES = Catalogo Universal de Servicios de Salud; CBHI = Community-based Health Insurance; CNAM = La Caisse Nationale d'Assurance Maladie de Tunisie; CNSS = National Social Security Fund ; EHIF = Estonian Health Insurance Fund; GHI = Government Health Insurance ; HCFP = Health Care Pool Fund; HEF = Health Equity Funds; HIFM = Health Insurance Fund of Macedonia; HIIS = Health Insurance Institute of Slovenia; HIO = Health Insurance Organization; HZZO = Croatian Institute of Health Insurance; IHSS = Instituto Hondureno de Seguridad Social; LPMQSS =; MHI = Mandatory Health Insurance ; MHIF = Mandatory Health Insurance Fund (Kyrgyz Republic, Romania); MHIF = Mandatory Health Insurance Funds (Russia); MINSA = Ministerio de Salud; MOH = Ministry of Health ; NFZ = Narodowy Fundusz Zdrowia; NHI = National Health Insurance; NHIF = National Health Insurance Fund; NHIS = National Health Insurance Scheme; NHS = National Health Service; NIII = National Insurance Institute of Israel; NMBF = National Medical Benefit Fund; NRCMS = New Rural Cooperative Medical System; NRHM = National Rural Health Mission; OEP = National Health Care Fund/ Országos Egészségbiz-tosítási Pénztár; PEAS = Plan Esencial de Aseguramiento de Salud; PIAS = Plan Integral de Atención en Salud ; PMO = Compulsory Medical Plan; POS = Plan Obligatorio de Salud; SCH = Supreme Council of Health; SENASA = Proceso de traspaso al Seguro Nacional de Salud; SHA = State Health Agency; SAMHI = State Agency on Mandatory Health Insurance; SMIC = State Medical Insurance Company; SUMI = Seguro Universal Materno Infantil; UCS = Universal Coverage Scheme. a Benefits package was designed with a TA from the World Bank and AIF (U. Giedion and M. Dinarte, January 2012). b The lists are conditional on Department of Science and Technology (DECIT) evaluation. c In planning as of 2009. d In progress.

e Being established in 2012. f Proposed.

plan-by-plan review would be required to document all the characteristics of these efforts.

• HBP sometimes rely on poor data, no local data, and no cost information. Ghana's HBP and its contents, for example, were legislated before being costed

[97]. HBP in Latin America do not take future demographic and epidemiological trends into account

• HBP are not always linked to available resources or are unaffordable, leading to erosion of health and financial protection impact, as well as legal (and political) challenges. Chile's Regime of Explicit Health Guarantees (AUGE) only calculated the total cost

but never the additional cost required to provide guaranteed benefits [99]. Mexico's Catalogo Universal de Servicios de Salud (CAUSES) grew from 91 interventions in 2004 to 266 interventions in 2008, whereas the capitation payment associated with the package was only adjusted for inflation [100]. Where HBP have worked to structure expenditures, they are accompanied by changes in budgeting, from inputs to the packages, where input costs are wrapped up in the packaged interventions as in Colombia and Mexico. Many high-income countries also have packages—particularly the insurance systems of Europe, Australia, and Canada—but these are based on general major disease category classifications usually accompa-

nied by more detailed benefits schedules with associated fees (for example, a price attached to a diagnostic-related group, such as Medicare's Ambulatory Payment Classification).

• Lacking data on the costs and effectiveness of existing health system activities, HBP costs are generally calculated based on the ground-zero assumption, on a per capita basis rather than incrementally. This is the case in Chile, Honduras, and Peru [101].

• HBP can be vague, grouped in general categories with no specificity (Kenya), or grouped by diagnosis rather than interventions such that anything can be provided (Peru) [102].

• In some cases, there are no explicit criteria for defining the content of health plans, or—in some decentralized countries—plans vary by subnational entity and their financing depends on the wealth of that particular locality. This is the case in China for various health insurance programs [Z. Kun, October 2011], with consequences for care variation and equity. In countries that split social health insurance from public sector provision, there may also be multiple packages of different breadth, height, and depth, also with implications for equity. Similarly, criteria are often applied in an erratic, inconsistent way.

• HBP are often conceived as one-off exercises and no arrangements are in place to update analyses based on inflation, new information, or technological developments. Where updates occur and disinvestment decisions are taken, there are few methodological standards.

• Cost-effectiveness research is generally carried out and funded separately from the policy-making process, and there can be limited relevance to and connection with coverage and reimbursement decisions. In China, for example, primary and secondary research in health is funded separately by different government agencies, which makes it difficult for decision makers to get well-coordinated evidence that addresses their own questions [Z. Kun, October 2011].

• HBP are frequently designed by governments or consultants in isolation, often as a part of development bank programs. The government of Jamaica recently issued a request for proposals with Inter-American Development Bank funding that called for the design of a HBP in 6 months' time [103]. Colombia is an example of a HBP designed by technocrats based on cost-effectiveness, later challenged by a regulatory body [104]. Chile's AUGE package of guaranteed services was also delegated to technical staff at the Ministry of Health, with little opportunity for stakeholder or public input or review [105]. The legal contestation of health benefits is spreading in Latin America [Z. Kun, October 2011] and in South Africa [106,107]. Parliaments are also involved; in Colombia, Congress recently legislated public subsidy for treatment of rare, orphan diseases without regard

for trade-offs or costs [108]. (Another work estimated that the cost of treating Fabry disease alone, which would benefit between 31 and 37 persons was equivalent to affiliating 664,000 additional people in the subsidized regime of insurance [109].) Conflicts of interest of those participating in the evaluation of technologies and the design of HBP are generally undeclared.

• There is usually insufficient documentation of methods, processes, and actual decisions. In 2011, the Inter-American Development Bank [103] reviewed coverage decisions (decisions per se, criteria for decisions) for 20 high-cost drugs in 4 OECD and 4 Latin American countries and encountered an almost total lack of systematic information supporting coverage decisions in Latin America.

• Finally, the lack of a consultative, transparent, multis-takeholder process in HBP evidence evaluation, design, and adjustment is a prominent feature ofefforts in LMIC. This is a troubling pattern given the ethical, political, and legal issues involved in the priority-setting process. In South Africa in 1999, for example, university researchers using evidence-based criteria designed an essential package of hospital benefits for medical schemes that was established in regulation without any alteration and no public debate [110].

Clearly HBP have had a mixed track record, but their frequency is an expression of a trend toward more explicit rationing, motivated by growing population demands and related legal, political, fiscal, and health concerns, as well as increasing emphasis on results-based financing.

Health technology assessment entities. Another approach to priority setting has been the establishment of HTA entities to carry out clinical and economic evaluation and assessment of new medical technologies and to guide reimbursement decisions, mainly in middle-income countries such as Brazil, Czech Republic, Colombia, Croatia, Estonia, Poland, Thailand, and Uruguay [110]. (Whereas LMIC are the focus of this paper, the working group also examined HTA processes in European countries. All European countries have dedicated national HTA agencies that exploit economic evidence as part of applications to include new medicines for public reimbursement. This is particularly so in the case of innovative products or in situations in which the manufacturer is seeking a premium price [111]. Evidence used in HTA includes the safety, efficacy, effectiveness, cost, social, institutional, legal, and ethical implications of new technology adoption, although the precise use and methodology differs from agency to agency [112,113]. EUnetHTA, a European Commis-

sion—funded initiative for promoting some standardization of the methods for HTA across Europe, has been fairly successful in raising awareness and building a series of methodological tools to support those doing HTA in countries across Europe. European experience also illustrates that many HTA bodies involve a wide range of stakeholders including decision makers, health professionals, academics, patient representatives, and laypersons in the process [114,115]. Although agencies in middle-income countries did not initially connect HTA to the design of publicly funded benefits plans or coverage decisions, the transparency and rigor of the methodologies used to analyze new technologies have increased the influence of HTA agencies on the uses of public funding and have expanded the assessment mandate beyond medicines to devices, diagnostics, procedures, and other health interventions. For example, although recommendations are not binding, the National Health Fund in Poland is obliged to request and consider recommendations made by the HTA agency [110] and one of its first tasks was preparation of a ''basket of guaranteed services.'' Thailand's Health Intervention and Technology Assessment Program (HITAP) is now formally part of the team that designs the National Health Security Office's ''universal coverage'' package. In 2007, Korea made the use of HTA mandatory in decision making on drug and healthcare reimbursement under the Health Insurance Review Agency [116]. In Brazil, since mid-2011, new health technologies can only be listed for funding within the public health system provided their security, efficacy, and cost-effectiveness have been demonstrated and assessed by the Comissao Nacional de Incorpo-racao de Tecnologias. Similarly, the country's regulatory agency National Health Surveillance Agency (ANVISA) decides prices of new entrant medications based on comparative effectiveness. In Colombia, starting in 2012, a HTA institute will provide recommendations on whether evaluated technologies should or should not be included in the benefits package or whether, more generally, they should be financed with public resources. In Mexico, HTA was made mandatory ahead of a coverage decision in 2011 [117].

Although a more systematic assessment of the impact of each HTA agency is needed, the working group found that the savings associated with just 2 HITAP HTA-based recommendations—later adopted by the Ministry of Health leadership—

have, in addition to saving lives, far exceeded their cumulative operating costs to date [118,119].

In spite of their promise, there are a number of obstacles to the implementation of HTA in LMIC, similar to those observed in relation to HBP. First, evidence from Asia, Latin America, and Africa reveals a severe shortage of local technical and analytical research capacities for HTA compared with the capacities of high-income countries where HTA has long been used in policy decision making [120—122]. Second, the reviews of HTA reports in LMIC found that the majority of the studies were vulnerable to bias due to the poor quality of scientific evidence available and deficient reporting features [123]. This may be explained by an absence of standard methodological and process guidelines and limited research infrastructure in most LMIC [124—127]. Third, where HTA is not built into coverage decision-making, HTA research is not usually directed toward major health problems because many HTA studies in LMIC are funded by international organizations and pharmaceutical companies [128]. Given resource constraints, HTA studies have to be prioritized and focused on interventions that would assist decisions targeting major health problems that could subsequently have a large impact on population health and spending. Fourth, political timing is often at odds with time requirements necessary to conduct high-quality HTA [124] and supportive institutional infrastructure to take HTA decisions is missing. Fifth, many studies find that HTA is a new discipline among health professionals and decision makers in LMIC and the lack of a clear understanding of HTA among these potential users was identified as a major barrier regarding the use of HTA in policy and practice [129]. Similarly, in an article on HTA agencies in Central and Eastern Europe, Sorenson et al. [130] identify obstacles to HTA institutionalization as lack of capacity, lack of data, scoping difficulties, timeliness, and limited organizational infrastructure linking HTA findings to decision making and implementation. Finally, there is an enduring challenge in connecting the technical assessment work to national policy-making and coverage decision processes [131].

As is immediately apparent from the 2 priority-setting pathways commonly in use in LMIC, HBP and HTA have much in common, using similar methods of economic evaluation and criteria for decision making, though sometimes applied to technologies and services with different levels of

specificity. The growth of both of these policy instruments in LMIC over the last decade is motivated by a range of factors, among them growing population demands, increasing financial pressures (e.g., the IMF has recently established conditional-ity related to coverage decisions in publicly funded health care in Romania under the second review of the Stand-By Arrangement [132]), greater availability of new and generally higher cost technologies, persistent inequities in health and coverage, and——perhaps——maturation in democratic processes. Further, HBP and HTA are converging in middle-income countries, with HTA agencies or units being used to inform coverage decisions within HBP. HTA and HBP also have common limi-tations——data, local capacity, lack of legal frameworks, limited formal institutional structures, minimal stakeholder involvement, and sometimes limited connections to coverage decision making.


The Working Group on Priority-Setting Institutions for Health sees opportunities to support the creation, strengthening, and maintenance of systems in LMIC that inject objective assessment into decision making on publicly funded health services and technologies. These systems can and do help optimize health outcomes by increasing the technical content of difficult domestic political discussions that are often further complicated by donor preferences, industry pressures, and other forces. In general, we found that country efforts to improve priority setting fell into 2 groups. Some high-capacity countries—all were middle-income—have established robust institutions to support the priority-setting process. These countries acted largely on their own initiative. Progress in other countries tended to mirror the state of global-level research, meaning that technical plans had been developed, but very little progress had been made on managing the political process that actually sets priorities.

After examining trends and characteristics in priority-setting processes and institutions worldwide and building on the recommendation of Gie-dion et al. [133] to analyze priority setting as a multistep process from a systemic perspective, the working group adopted a ''7 + 7 framework'' that describes 7 principles (distilled from the original 15 [134]) and 7 core processes of priority setting that——if implemented, ideally under an explicit legal and institutional framework——could have the

potential to improve health for any desired level of health spending, while channeling and managing political, commercial, advocacy, and donor interests via a fair process. Without a legal framework for setting up and using HTA, it is difficult to ensure enforceability or budget. Equally, if the courts wish to enforce right-to-health legislation, but have no access to information and process related to evidence appropriateness and affordability, decisions will not have intended effects.

Building on the collective experience of entities currently using HTA to inform decisions, the working group recommends that HTA system governance follow 7 principles. It should be ethically sound, scientifically rigorous, transparent, consistent, independent from vested interests, contestable, and timely and enforceable [133,134].

The set of 7 processes that emerges from the 7 principles is an ''HTA system'' whose level of data and methodological complexity could be scaled according to country or funding agency circumstances. In this usage, HTA system does not refer exclusively to the technical and analytic function of assessing an individual technology or intervention, but instead to the entire decision-making process and context, including the legislative, regulatory, policy, payment, and reimbursement framework within which evidence is developed and used. We therefore use a broad definition of HTA developed by International Network of Agencies for Health Technology Assessment [135]: ''the systematic evaluation ofproperties, effects, and/or impacts of health care technology. It may address the direct, intended consequences of technologies as well as their indirect, unintended consequences. Its main purpose is to inform technology-related policymaking in health care. HTA is conducted by interdisciplinary groups using explicit analytical frameworks drawing from a variety of methods.'' Additionally, the term HTA system is chosen to reflect that priority setting involves multiple actors and processes and is based on inputs provided by the health systems, the legal framework, and the social values prevailing in each society, whereas the process can lead to different types of outputs such as coverage decisions, guidelines, protocols, or other evidence-based recommendations. The specific HTA system emerges from a country's priority-setting starting point: designing or adjusting a health benefits plan; establishing a positive or negative list; trying to speed cost-effective medical innovations to patients; deciding whether to finance a specific new technology or service.

The 7 processes are:

1. Registration. Registration or marketing authorization is a first step in the priority-setting process. Just obtaining regulatory approval in the country is a first step toward availability and the possibility of becoming a priority for public or donor spending. Some countries have used the registration process to proactively speed access to good value technologies, whereas others have been more passive, letting the process be initiated by industry or donors. Increasingly, pharmacoeconomic evidence and clinical data are required for drug applications [131]. Once a technology is registered in a country, products diffuse quickly to those with the ability to pay, creating immediate pressure for a public-sector coverage decision. Off-license use is also an issue. Off-license use can be good/necessary (as in pediatric medicines) or inappropriate (as in Colombia and the recent decision to use a certain brand of antiretroviral (ARV) as prophylaxis). Further, the potential for synergies between regulators and payers at the registration stage is large, given that security and efficacy of analyzed products are important evaluation dimensions.

2. Scoping. Depending on a country's starting point, it will be necessary to identify and select broadly defined technologies for evaluation. A poor country may start from a major, underaddressed burden of disease, such as CVD, and examine prevention and treatment alternatives. In other cases, a country may start from a costly device for which the incremental value for money is suspected to be low that, if funded, would potentially divert public monies from higher value uses, or a country might start with new technologies with great potential to generate health improvements or cost savings. Given more limited resources, the scoping exercise is especially challenging in LMIC where the number of technologies outside lists or packages will be large.

3. Cost-effectiveness. An analysis of the cost-effectiveness (CEA) or value for money of a technology should be undertaken using widely accepted methods, tools, and systematic evidence reviews. This is the area where most global efforts to support LMIC to date have centered, but they have not gone far enough to generate and use local data or to develop local cost-effectiveness thresholds. CEA should establish a comparative clinical benefit and an incremental cost-effectiveness ratio, ideally combining local data on costs and use with international data on efficacy, thus adapting and translating international evidence to local circumstances.

The identification of the appropriate counterfactual information should be given particular attention to avoid the zero-based scenario and to appropriately reflect the installed capacity of the health system. In addition, multiple quantitative criteria can be built into the cost-effectiveness analyses to reflect both health and nonhealth priorities in a country. For example, CEA can reflect equity considerations via differential weighting of different health states at different ages, a concern for poverty reduction via incorporation of financial protection criteria, and different discount rates given the time preferences of a given society, among others. A preliminary recommendation to adopt or not adopt is developed based on CEA using a country-specific threshold.

4. Budget impact. The budget impact of a preliminary recommendation emerging from CEA should be assessed. Budget impact analysis is a tool to predict the potential financial impact of the adoption and diffusion of a technology into a healthcare system with finite resources. Whereas CEA addresses the additional health benefits gained from investments in a technology, such as the cost per additional disability-adjusted life year gained, budget impact analysis addresses the affordability of the technology, for example, the net annual financial cost of adopting the technology for a finite number of years. This is essential for LMIC that have highly constrained fiscal resources in the health sector, yet is almost never conducted as part of the introduction of new technologies. A recent example is the adoption of the human papillomavirus vaccine by Rwanda [136]. Tools such as the Marginal Budgeting for Bottlenecks can be adapted for this use [137].

5. Deliberative process. A deliberative process should be run to examine the results of the CEA/ budget impact analysis, hear from stakeholders, and consider more subjective decision criteria such as the severity of conditions, the magnitude of individual health gain, the relative rarity of the disease condition, and trade-offs with ethical implications [138]. In processes worldwide, an appointed, a multidisciplinary committee that publicly discloses conflicts of interest is tasked with making final technology funding recommendations or decisions. Payers, providers, experts, industry/manufacturer representatives, and, sometimes, patient or public representatives are included in the deliberation, though voting members may be limited to a subset of the group. Culyer [139] describes the circumstances, common in LMIC, that make the use of

deliberative processes in healthcare priority-setting necessary and relevant:

.. .decisions have been delegated by a body with a democratic mandate to one without it; evidence from more than one expert discipline is involved; evidence from more than one profession is involved; stakeholders have conflicting interests; there are technical disputes to resolve and the evidence may be scientifically controversial; evidence gathered in one context is to be applied in another; there are issues of outcome, benefits, and costs that go beyond the conventional boundaries of medicine; there is substantial uncertainty about key values and risks that needs to be assessed and weighed; there are other social and personal values not taken into account in the scientific evidence; there are issues of equity and fairness; there are issues of implementability and operational feasibility involving knowledge beyond that of the decision makers; and wide public and professional ''ownership'' is desired.

Setting up a fair, deliberative process around a technical recommendation for coverage allows for debate on the ethical and equitable implications of decisions, such as who will receive ARV given limited resources, while providing a space for different interest groups to air concerns or bring new evidence to the table for consideration.

6. Decisions. Decisions should be guided by the results of the evidence and recommendations produced by the cost-effective, budget impact, and deliberative processes. Recommendations should be considered by the appropriate authority and when a coverage decision is made, it should be linked continuously to public budgets. Though seemingly self-evident, the HBP, as well as the national health plan, experience suggests that many policies and plans remain aspirational and unconnected to budget, or there is a one-time connection to budgets that can lead to erosion of effectiveness over time. Even though some advocate specific technology or program budget line items as the solution to this issue [140], this approach is ultimately problematic as it is unconnected with individuals entitled to receive a given intervention or technology. Depending on the setting, the decision to fund/cover may be devolved by the payer (insurance fund or government) to the multistakeholder committee carrying out the interpretation of the evidence through the deliberative process or it may remain, at least at the earlier stages of the process, the responsibility of the payer at either the central or local levels. In the latter case, the reasons for rejecting or deferring a positive or negative recommendation ought to be made public and be contestable.

7. Appeals, tracking, and evaluation. Finally, a coverage decision and/or the underlying recommendation and associated analyses should be contestable via an appeals process. This can range from the ''lighter'' option of a public review that can be triggered by a select group of stakeholders to a full-scale appeal, which may include formal resubmission of evidence and a public hearing. It is important that the right balance is struck between offering the opportunity to challenge a decision publicly, potentially leading to its reversal, and ensuring the overall process remains timely and insulated from vested——commercial or profes-sional——interests that may, as they have access to relevant resources, be more likely to challenge or delay unfavorable decisions. Finally, a more elaborate appeals process may be more relevant in settings where legal challenges and perceptions of procedural weakness have been common to the extent it may reduce the chances of judicial reviews and enhance stakeholders' faith in the decision-making process.

Decisions are tracked and evaluated, in the public domain. The implementation (degree of uptake) and longer-term impact (in terms of health outcomes and/or budgetary impact) of decisions need to be monitored and reported. Uptake evaluation can drive data collection systems that will in turn feed into further, better targeted and better informed, updates of the guidance. They can also form the basis for performance assessment of purchasers and providers at the local level, through helping to identify the most appropriate and least burdensome uptake metrics to be assessed. Finally, impact assessments can strengthen the case for using evidence and independent processes to inform coverage decisions, including in situations where additional investment is needed making the financial case for further investment, which may become increasingly important for both global donors and LMIC Ministries of Finance.

A regular assessment of whether the process needs repeating is also necessary. For example, in the event of new safety data, the process would restart at Step 1, or if new clinical or cost data becomes available, the process could restart at Step 3. Regular reviews and updates are critical in ensuring decisions are based on credible processes and up-to-date information.

There are a number of critical elements of a robust, sustainable HTA system that do not fit directly within the processes described. Priority-setting decisions must be made in the context of

Figure 1. Application of the 7 x 7 framework using a real case study from the British NICE and NHS. Reprinted, with permission, from [141].

legal, statutory, and regulatory policies that may support or undermine the ability to implement these decisions. For that reason, there is often a need to design HTA systems that are mindful of these contextual factors. In some cases, successful implementations of HTA-driven priority-setting policies require reconsideration of statutes and regulations. Furthermore, because of the limited availability of information about the comparative effectiveness of many health technologies and services, it will also be critical for the evaluation processes to be linked to systems for evidence development to address critical uncertainties in high-priority domains. Local or regional mechanisms for funding and implementing primary research will need to be organized to address critical uncertainties that arise in priority setting, so that the objective of policy making driven by effectiveness and value can be more consistently achieved.

The HTA system, whether global or national, should help to increase the rigor and relevance of evidence considered, provide a fair and transparent mechanism to manage the politics around resource allocation, connect evidence-based decisions to budgets, and create permanent institutional chan-

nels to consider resource allocation choices over time. The Box [141] describes how the HTA system works in the British National Health Service using the example of secondary prevention of CVD (Fig. 1).

In Figure 1 and the following explanatory text, we apply the 7 X 7 framework using a real case study from the NICE and the British National Health Service (NHS). Alongside countries such as Canada, Sweden, New Zealand, and Australia, Britain has one of the best developed systems of translating evidence into policy. This need not mean that the NICE institutional standard is (or ought to be) the model for other countries. Every country has to develop its own mechanism for identifying priorities and acting on them based on evidence and values. Equally, our choice of example need not imply that implementing the 7 X 7 framework is unattainable in a LMIC setting. Indeed, countries such as Thailand, China, and Brazil are building similar mechanisms. We chose this example because it is (perhaps because of the accessibility of NICE data and decisions and the fact they are in English) one of the better-documented cases of applying the 7 X 7 framework to real policy decisions.

• Secondary prevention of CVD: an example from the British National Health Service.

In low-income countries, over 75% of people with coronary heart disease or who had a stroke, did not receive any of the 4 effective drug types (aspirin, beta-blockers, angiotensin-converting enzyme [ACE] inhibitors/angiotensin receptor blockers [ARBs], and statins for myocardial infarctions; and aspirin, beta-blockers/diuretics/calcium channel blockers, ACE inhibitors/ARBs, and statins for stroke). For middle-income countries, the rates varied from 50% to over 70%. The use of statins between high- and low-income countries varied 20-fold, with the sickest/highest risk group patients, receiving the least care. The investigators conclude: ''although some patients receive appropriate treatments when they access health-care providers or hospitals, most do not receive basic effective therapies long term, with many individuals receiving no preventive treatment Prevalence of drug use was substantially lower in less economically developed countries than it was in developed countries, suggesting an urgent need for systematic approaches to understand and rectify the causes of the large treatment gap in secondary prevention globally.''

• Secondary prevention of myocardial infarction using pharmacological treatment.

1. Registration. The 4 drug classes recommended for secondary prevention of myocardial infarction are: ACE inhibitor (or ARB, if intolerant) + aspirin (or an alternative antiplatelet) + beta-blocker + statin (unless there is a contraindication or side effects reported). All 4 drug classes have been registered in Europe with the European Medicines Agency (EMA) (and/or in the United Kingdom by Medicines and Healthcare products Regulatory Agency [MHRA]), and all 4 are currently available in generic form at a price lower than the branded or branded generics, with ARBs being the last class to lose patent (losartan was the first drug in its class to become generic in March 2010, valsartan followed in 2011, and candesartan and irbesartan in 2012).

2. Scoping.

a. According to Royal College of General Practitioners and National Collaborating Centre for Primary Care [149], ''the annual incidence of myocardial infarction for men aged between 30 and 69 is about 600 per 100,000 and for women about 200 per 100,000. The British Heart Foundation (2004) has estimated that there are about 147,000 MIs per year in men of all ages in the UK and 121,000 in women, giving a total of 268,000 cases. In the UK, about 838,000 men and 394,000 women have had a myocardial infarction at some point in their lives.''

b. Reducing mortality from cardiovascular disease and secondary prevention of myocardial infarction (MI) are listed as key national priorities in the NHS Operating Framework of 2007/2008 [150] and the National Service Framework for Coronary Heart Disease: Winning the war on heart disease [151]. These nationally set priorities are important inputs in NICE's work program.

3. Cost-effectiveness analysis. Based on published research [152], the incremental cost of offering CVD medicines as secondary MI prevention, including the cost of the drugs and the visit to the primary care center, was estimated at £514. The incremental benefit was estimated at 0.049 quality-adjusted life years. Therefore, the cost per quality-adjusted life year was £10,816. Based on sensitivity analysis, pharmacological secondary prevention is deemed to be cost-effective even if the overall cost of the intervention is twice as high as the estimated cost. Furthermore, the current levels (baseline) of offering secondary prevention at a primary care setting across the United Kingdom are at 11.3%. With a primary care center level prevalence of MI of 0.75%, offering monetary incentives to encourage secondary prevention also becomes cost-effective.

4. Budget impact analysis. Using prescription and prevalence data [153] from IMS and the national Myocardial Infarction Audit Project; tariffs/Diagnosis-Related Group (DRG) costs; and unit costs from the British National Formulary and assumptions on the likely proportion people on 2, 3, and 4 drug combinations as practice changes over 1 year, the additional annual net budget impact of rolling out secondary prevention across the country for weighted drug combinations ranged from £0.3m to £2.1m (starting from a baseline current cost estimate of £9.2m). The cost of all four combinations is £195.6 per year per patient. Although the potential savings from implanting secondary prevention were not quantified, an uncomplicated acute MI costs the NHS approximately £3,500 and 24 h in the cardiac intensive care unit costs approximately £1,000 per day.

5. Deliberative process. A multidisciplinary committee [154] of experts and lay people, all of whom abide by a conflict-of-interest policy, was convened by NICE as per the published NICE process of operation [155] to discuss the evidence and make a recommendation as to whether the NHS ought to incentivize the pharmacological secondary prevention of MI. In addition to the clinical and economic data, the committee was presented with results of field-testing and stakeholder consultation. The minutes of the discussion and all the evidence was placed on NICE's Website. The committee recommended that a new indicator is included in the NHS's pay-for-per-

formance in primary care: ''The percentage of patients with a history of myocardial infarction (from 1 April 2011) currently treated with an ACE inhibitor (or ARB if ACE intolerant), aspirin or an alternative anti-platelet therapy, beta-blocker and statin (unless a contraindication or side effects are recorded).''

6. Decisions. The NHS Employers' organization and the professional association of British general practitioners will make the final decision as to whether pharmacological secondary prevention of MI will become part of the NHS's P4P scheme for primary care doctors across the country. During their annual negotiations, the 2 parties can choose from a menu of indicators/activities evaluated and recommended by NICE's committees. For 2011/2012, the 2 parties agreed to include the secondary prevention indicator in the Pay for Performance (P4P) scheme [156].

7. Appeals, tracking, and evaluations. Compliance with the P4P guidance (introduced in 2011) on secondary prevention of MI will be assessed through annual assessment ofdata collected through the online General Practitioners Results Database and primary care centers rewarded accordingly. In the meantime, there is evidence of uptake of NICE's advice on secondary prevention, with 95.5% of post-MI patients receiving at least one ofthe recommended medications [157]. In addition, data on CVD mortality is regularly collected through the national audit (Myocardial Ischaemia National Audit Project [MINAP]) and overall NHS performance assessed against the 2011/ 2012 Outcomes Framework. Finally, cost of prescribing and percentage of generic prescribing (also incen-tivized by NICE) for select drug classes such as ACE inhibitors and statins are monitored by region and benchmarking data made publicly available through NHS Prescription Services.

The recommendation and final decision are regularly reviewed as new clinical evidence emerges, including effectiveness and safety information, and also as unit costs and utilization volumes may differ from year to year or new pharmaceutical products are introduced to the UK market. The clinical guideline underpinning the recommendation on secondary prevention is currently (2012) being reviewed. The grounds for the review include: (1) the fact that the antiplatelet agent clopidogrel has gone off patent in the United Kingdom and new trial data showing that ARBs are of similar clinical effectiveness to ACE inhibitors and (2) combination of the 2 is not clinically better than monotherapy [158].


The working group has identified 6 major obstacles to the further development of HTA systems in

LMIC and has developed a series of risk-mitigating strategies:

Scope is large and unmanageable; demands are urgent. As noted earlier, many benefits plans are designed under unreasonable time frames associated with a window of opportunity for broader health reform. That initial effort will result in a list and some notional cost but, to mitigate the risks associated with this usual practice, efforts could be made, perhaps as part of the reform legislation, to ensure that a long-term institution be built, charged with updating and revising plans, coverage decisions, or negative lists. As agencies that frequently provide technical assistance and funding for the development of benefits plans, the multilateral development banks can ensure that institutions are part of the agenda for policy dialogue.

Further, starting small with a vision for scaling up is the most practical mitigating strategy. The Thai, Polish, and Colombian experiences suggest that initially a small-scale effort will be most feasible to build confidence on the approach and methodologies, to educate policy makers to create demand, to train professionals and stakeholders involved in deliberations to prepare/oversee and understand the evidence, and to address 1 or 2 key health burdens or technologies in an integral way. The SIVAC and Malaria Vaccine Initiative (MVI) programs are the seeds of such efforts that need to be connected with a permanent institutional entity, budget affordability assessments, and budget decision making. Where the courts or congresses/parliaments have been active in legislating health priorities, it will be vital to involve these groups in the designs of the processes and the small-scale pilots.

Coping with capacity shortfalls. The capacity to actually carry out clinical and economic evaluations is a major feasibility concern. However, whereas capacity to do these evaluations is many times limited in-country, the entity could call for nationalinternational partnerships to prepare core evidence packages or simply contract out to a foreign university with expertise (as has been done in Poland in some cases). In parallel, as has been done in Thailand, scholarships and sabbaticals can be arranged to build more capacity in the medium term. Finally, capacity building driven by demand and within the context of a budding HTA system may be the most efficient way of developing fit-for-purpose human resources. This is what the Thai model has shown and is an experience shared by relatively resource-rich countries such as England where health eco-

nomic capacity burgeoned in response to the British National Institute for Health and Clinical Excellence's (NICE) requests for evaluations. Understanding roles and responsibilities in

fragmented settings. Many countries struggle with existing priority-setting arrangements that are fragmented among multiple agencies and individuals. A baseline mapping of who does what is needed in order to build a coordinated priority-setting policy and to understand whether a ''new'' institution is required and where it should be positioned or whether the institutional functions should be assumed by an existing entity. Such an analysis has been carried out in Colombia during the preparatory phase of the design of a new health technology evaluation institute [142]. Can fair processes work in the context of weak governance and corruption? Given the emphasis of existing institutions on the role of expert and stakeholder committees to consider the evidence and reach a recommendation, there is a worry that such processes may be illegitimate if subject to political manipulation or if top leadership does not support the concept. Some ways to manage these threats are through: clearly defined procedural rules for engagement, conflict-of-interest disclosures, and audits; elections within professional bodies; and limiting voting rights to government and committee chairs. The roles of physicians' organizations have been mixed. Clinical engagement and leadership throughout priority-setting processes are essential but there is also a need to ensure that professional and commercial interests do not capture the process of evidence assessment and decision making. In many LMIC, for example, there is a need to distinguish between physician-led scientific associations and unions focused on wage negotiations.

However, it is worth noting that priority-setting entities supported by external donors already exist in many low-income countries. At least 1 priority-setting committee—the Country Coordinating Committee (CCM)—exists in every Global Fund beneficiary country, including the most impoverished fragile states. The CCM is charged with setting priorities in disease control and prevention, consulting with stakeholders, and preparing the country proposal, including the budget to the Global Fund. Yet, in spite of adequate funding and a mandate, the CCMs infrequently use locally adapted cost-effectiveness and budget impact studies to decide on the optimal mix of technologies, do not assess whether their budgets

reflect true costs or whether efficiencies can be made, do not always assess the availability of new technologies that might be cost-saving or more effective, sometimes fails to address the rationing problem directly, and so on. The CCMs also lack defined processes to carry out their functions. In at least in this context, it seems feasible to build out the capacity of CCMs to set priorities more rigorously and in consultation.

In the end, for any HTA/priority-setting activity to gain traction within a country, there ought to be a clearly articulated political demand and longer-term political backing. Without such backing, any attempt to rationalize priority setting, build capacity, and generate data will be unlikely to succeed.

Intransigent data problems. In low-income settings particularly, there will be very limited data with which to carry out clinical and economic evaluations. Beyond the inevitable suggestion that more support be given to the production of these data (and asking for what is needed is a start), in the interim, countries may need to rely on data from ''similar'' countries, efficacy and effectiveness estimates from studies conducted elsewhere, and assumptions about cost and use based on small samples and outdated demographic projections. Although this will be frustrating, problematic studies may galvanize greater demand for measurement, and perhaps motivate donors to support routine, consolidated data collection and disease surveillance at scale. Alternatively, entities can commission original data collection. A donor-funded project in India, for example, is collecting representative data on costs in primary healthcare clinics as an input into a more accurate cost-effectiveness estimate and affordability analysis [143]. Weak links to decision making. A final problem encountered in many HTA systems in high-income countries is the inefficiency with which the assessment programs are linked to decision making. More progress has been made recently, though for many years, the technical production of HTA reports received only limited attention in coverage, benefit, or budgetary decisions. It is possible that a similar phenomenon could occur in LMIC; in Malaysia, for example, whereas HTA conducted within the Ministry of Health on drugs has an impact on listing in the ministry's formulary, HTA on devices and practices has no impact on reimbursement decisions [144]. In a study in Latin America, policy makers identified the lack of a transparent and clearly defined link between HTA and the

decision-making processes as a major barrier for the effective application of HTA to resource allocation decisions [145]. This risk is mitigated by the more pronounced fiscal pressures under which LMIC operate, lessons learned from the high-income countries, as well as potentially proactive global support to an integral HTA system.


Based on this framing of an HTA system and its challenges in different settings, the working group recommends direct, substantive support to the creation and development ofboth global and domestic HTA systems. Within domestic actions, the working group made a distinction between strategies relevant to middle-income countries and those relevant to low-income countries.

Global HTA facility. A global HTA facility should be created to provide sustained technical and consultative support to global funding agencies and LMIC governments. Global funders have an ongoing need for rigorous economic evaluation of technologies to be funded, from antiretroviral medication to bed nets to determining the best strategy for funding CVD prevention and multi-drug-resistant tuberculosis control. For example, a comparison of least-cost equivalent antiretroviral therapy with the standard first-line regimens in 43 WHO ''3 by 5'' focus countries finds that 59% of countries use drug combination regimens that are more expensive than available alternatives, demonstrating the capacity for funders to achieve similar results at a reduced cost.

A Global HTA facility might also undertake the accreditation of national HTA systems and evaluations. Building off the working group findings and recommendations as well as instruments already developed to assess clinical guidelines [146], standards for a system could be developed that would establish the full range of analytic and decision-making components of a working system. Functional and quality standards might include, among other things, basic analytic skills sets, defined relationship to decision-making authorities, transparency, public engagement, appeals, HTA/ regulatory alignment, appeals mechanisms, basic analytical methods standards, or a link to research priorities. Donors such as the Global Fund could require that accredited HTA systems be in place and in use as conditions of grants and loans. The global HTA facility could also track the health and budget impact of coverage decisions in a selec-

tion of countries over time to inform future reforms.

A global HTA entity could also derive economies of scale in the generation and adaptation of evidence dossiers, following toolkits and glossaries already developed [124]. As new technologies come online, there is little need for each country to repeat core analyses, but a great need to appropriately adapt HTA conducted in other countries to the epidemiological, cost and utilization profiles domestically. Recent work on a geographic ''trans-ferability checklist'' for HTA will also be useful [147]. There are also great synergies in the sharing and benchmarking of coverage decisions among countries at similar levels of gross domestic product per capita, a simple strategy that could provide local decision makers with the ammunition needed to adopt a new cost-effective technology, to tailor high-cost interventions to high-need subgroups, or to say no to inappropriate and expensive interventions. This approach has been used by the Inter-American Development Bank in the context of a regional project and has been used to inform coverage decision-making during a 2010 update of Colombia's benefits plan [143,151]

Also, international funders, especially international organizations such as the World Health Organization, the World Bank, or multinational pharmaceutical companies should be well aware of the difficulties in using HTA conducted in 1 setting for another setting, especially when the studies involve economic parameters and social preferences. Because when they commission economic evaluation studies, they are likely to use economic evaluations to inform policy decisions in multiple settings. As a result, they should request the use of multinational trials for estimating clinical efficacy of the intervention, not only to introduce the intervention more widely, but also to provide for the collection of local data used in local HTA.

The support of this facility would be guided by a deep, current knowledge of the history and best practices of HTA systems developed in other countries, applying lessons learned in those situations to ensure that best practices are replicated and common mistakes are avoided. Above all, a tailored approach to technical assistance, with practitioners in each country relating directly to each other will be useful and will help protect and enhance the independence and status of those charged with carrying out this very difficult task.

Direct support to LMIC. Direct support to LMIC that are creating or growing their own

HTA systems could take a number of forms. Existing capacity-building efforts via evidence creation, tools, and methods could be more directly targeted to government counterparts charged with carrying out priority-setting work. Further, hands-on technical pilots and demonstration projects—from the relevant starting point—would be a way to engage with policy makers on real-time concerns. Advice and financial support to generate data on effectiveness, efficacy, medical practice, and patient use patterns in-country have been highlighted as a priority in Asia [148].

Coaching via procedural advice and knowledge exchange among countries, facilitated by a global facility or a regional network, will also be essential. Exchanging examples of legislation, process guidelines including conflict-of-interest management, handling of confidential data, stakeholder involvement, and overall governance and oversight can prevent "reinventing the wheel.'' Providing support to communications and public awareness-raising can also be useful to build stakeholder support and demonstrate the commonality of problems and solutions.

The working group made an important distinction between low-income and high-income countries, and the need to develop HTA system options for low-income settings that would build on bodies such as National Immunization Techni-

cal Advisory Groups and CCM that are in some cases already setting priorities for public and donor budgets. Even in countries that only do HBP in the context of results-based financing or performance-based contracting and/or where donors take a more direct role in priority setting, a version of an HTA system could be put in place, which would relieve donors of the difficult technical and ethical decisions that are taking place implicitly, without adequate technical justification, and in the absence of local participation.

Given the global economic outlook and anticipated drops in aid, how LMIC spend their own money will be a main determinant of the size and pace of health improvement in the future. Helping countries develop HTA systems that will increase value for money is a way forward.


The authors acknowledge the members of the Priority-Setting Institutions for Health Working Group, whose membership is described at Working group members have informed the arguments and policy messages included in this article, but have not yet reviewed the recommendations presented here, and are therefore not responsible for any errors and opinions.


1. OECD. OECD Health Data. OECD Health Statistics (database), updated February 14, 2011. Available at: healthdata. Accessed January 19, 2012.

2. UN General Assembly. Political Declaration of the High-Level Meeting of the General Assembly on the Prevention and Control of Non-Communicable Diseases. September 16, 2011. Available at: http:// symbol=A/66/L.1. Last accessed March 2, 2012.

3. Institute of Medicine. Preparing for the Future of HIV/AIDS in Africa: A Shared Responsibility. November 2010. Available at: http://www.iom. edu/~/media/Files/Report%20Files/ 2010/Preparing-for-the-Future-of-HIVAIDS-in-Africa-A-Shared-Res ponsibility/Future%20of%20HIV% 20AIDS%202010%20Report%20

Brief.pdf. Last accessed March 2, 2012.

4. Rosen S, Sanne I, Collier A, Simon JL. Rationing antiretroviral therapy for HIV/AIDS in Africa: choices and consequences. PLoS Med 2005; 2:303. Available at: http://www.eq pdf. Last accessed March 2, 2012.

5. National Research Council. Preparing for the future of HIV/AIDS in Africa: a shared responsibility. Washington, DC: The National Academies Press, 2011.

6. Dugger CW. Maternal deaths focus harsh light on Uganda. New York Times. July 29, 2011. Available at: 30/world/africa/30uganda.html?_r=2 &pagewanted=all. Accessed January 19, 2012.

7. Perehudoff SK, Laing RO, Hogerzeil HV. Access to essential medicines in national constitutions. Bull World

Health Organ 2010;88:800. Available at: volumes/88/11/10-078733/en/index. html. Last accessed March 2, 2012.

8. Tsai T. Second chance for health reform in Colombia. Lancet 2010; 375:109-10. Available at: http://www. PIIS0140-6736(10)60033-5/fulltext. Last accessed March 2, 2012.

9. Tashobya CK, Ssengooba F, Cruz V. Health Systems Reforms in Uganda: processes and outputs. Health Systems Development Programme. London, UK: London School of Hygiene & Tropical Medicine; 2003.

10. Strengthening Monitoring and Evaluation of National Health Plans and Strategies: Operationalizing the CHeSS Framework. Report of Technical Meeting. July 14-15, 2010, Glion, Switzerland. Available at: http://www.internationalhealth nts/chess_technical_meeting_-_glion, _14-15july2010_-_final_report_EN. pdf. Accessed March 23, 2011.

11. Reddy KS, Patel V, Jha P, et al. Towards achievement of universal health care in India by 2020: a call to action. Lancet 2011;377:760-8.

12. Gaziano TA, Opie LH, Weinstein MC. Cardiovascular disease prevention with a multidrug regimen in the developing world: a cost-effectiveness analysis. The Lancet 2006; 368:679-86.

13. Reich MR. Review the World Bank World Development Report. Available at 10.1086/452312. Accessed February 23, 2012.

14. World Bank. World Development Report 1993: Investing in health. Available at: pdf/WorldDevelopmentReport1993. pdf. Accessed January 19, 2012.

15. Kitzhaber JA. Prioritising health services in an era of limits: the Oregon experience. BMJ 1993;307: 373-7.

16. Murray CJ, Kreuser J, Whang W. Cost-effectiveness analysis and policy choices: investing in health systems. Bull World Health Organ 1994;72:663-74.

17. Bobadilla JL, Cowley P, Musgrove P, Saxenian H. Design, content and financing of an essential national package of health services. Bull World Health Organ 1994;72: 653-62.

18. Anand S, Hanson K. Disability-adjusted life years: a critical review. J Health Econ 1997;16:685-702. Available at: http://economics.ouls. Last accessed March 2, 2012.

19. Glassman A, Reich MR, Laserson K, Rojas F. Political analysis of health reform in the Dominican Republic. Health Policy Plan 1999;14:115-26.

20. Institute for Health Metrics and Evaluation. Global Burden of Disease Study. Available at: http:// Accessed January 10, 2012.

21. Evans DB, Adam T, Edejer TT, et alfor the WHO CHOICE Millennium Development Goals Team. Time to reassess strategies for improving health in developing countries. BMJ 2005;331:1133-6. Available at: choice/publications/p_2005_MDG_ series_introduction.pdf. Last accessed March 2, 2012.

22. Innovative Approach to Global Health and Poverty: Capitalism for Good. Available at: http://www. Accessed January 10, 2012.

23. Goetghebeur MM, Wagner M, Khoury H, Rindress D, Gregoire JP, Deal C. Combining multicriteria decision analysis, ethics and health technology assessment: applying the EVIDEM decision-making framework to growth hormone for Turner syndrome patients. Cost Eff Resour Alloc 2010;8:4.

24. Hutubessy RCW, Baltussen RMPM, Tan Torres-Edejer T, Evans DB. Generalised cost -effectiveness analysis: an aid to decision making in health. Applied Health Economics and Health Policy 2002;1:89-95.

25. Health Metrics Network. What Is HMN? Available at: http://www. hmn/en/index.html. Accessed January 19, 2012.

26. Measure DHS--Demographic and Health Surveys. Available at: http: // Accessed January 19, 2012.

27. UNICEF. Statistics and Monitor-ing--Multiple Indicator Cluster Surveys. Available at: http://www.unicef. org/statistics/index_24302.html. Accessed January 10, 2012.

28. UN Statistics Division. Available at: aphic/CRVS/VS_availability.htm. Last accessed March 2, 2012.

29. The Global Fund Adopts New Strategy to Save 10 Million Lives [Press release]. November 23, 2011. Available at: http://www.theglobal 2011-11-23_The_Global_Fund_ado pts_new_strategy_to_save_10_million_ lives_by_2016/. Accessed January 10, 2012.

30. Bryson M, Duclos P, Jolly A, Bryson J. A systematic review of national immunization policy making processes. Vaccine 2010;28(Suppl. 1):A6-A12.

31. WHO. The WHO Essential Medicines List: 30th Anniversary. Available at: medicines/events/fs/en/index.html. Accessed January 10, 2012.

32. WHO, Harvard Medical School and Harvard Pilgrim Health. Medicines Use in Primary Care in Developing and Transitional Countries: Fact Book Summarizing Results from Studies Reported Between 1990 and 2006. Available at: http://www. ary_care_8April09.pdf. Accessed January 10, 2012.

33. Available at: http://web.ebscohost. com/ehost/detail?vid=5&hid=7&sid =d58d9836-3f3e-4d7e-b8fc-8d25ab c350d7%40sessionmgr15&bdata=Jn NpdGU9ZWhvc3QtbGl2ZQ%3d% 3d#db=mnh&AN=21772768. Accessed January 10, 2012.

34. USAID, PHCS, Abt Associates. Azerbaijan Primary Healthcare

Strengthening Project: Year 2 Annual Report--September 1, 2008-August 31, 2009. Available at: http://pdf. Accessed January 19, 2012.

35. H3OK. National Health Insurance Fund--Bulgaria. Available at: http:// Accessed January 19, 2012.

36. HZZO. What You Should Know About the Usage of Health Care Services During Temporary Stay in the Republic of Croatia. Available at: eng.php. Accessed January 19, 2012.

37. Estonian Health Insurance Fund. Available at: http://www.haigekassa. ee/eng/ehif. Accessed January 19, 2012.

38. European Observatory on Health Systems and Policies. Georgia-Health System Review. Health Syst Transition 2009;2. Available at: http: // pdf_file/0003/85530/E93714.pdf. Accessed January 19, 2012.

39. Orszagos Egeszsegbiztositasi Penztar. [Health Care in Hungary]. Available at: pageid=34,1&_dad=portal&_schema= PORTAL. Accessed January 19, 2012.

40. European Observatory on Health Systems and Policies. Kyrgyzstan--Health System Review. Health Syst Transition 2011;13. Available at: assets/pdf_file/0017/142613/ e95045.pdf. Accessed January 19, 2012.

41. VLK. National Health Insurance Fund under the Ministry of Health. Available at: info&id=175. Accessed January 19, 2012.

42. Health Insurance Fund of Macedonia. Health Insurance Fund of Macedonia. Available at: http://www. Accessed January 19, 2012.

43. Richardson E, Roberts B, Sava V, Menon R, McKee M. Health insurance coverage and health care access in Moldova. Health Policy Plan 2011 Mar 26 [E-pub ahead of print]. Available at: 26/heapol.czr024.abstract. Accessed January 19, 2012.

44. Narodowy Fundusz Zdrowia [Ministry of Health--Poland]. Available at: Accessed January 19, 2012.

45. European Observatory on Health Systems and Policies. Romania--Health System Review. Health Syst Transition 2008;10. Available at: http: // pdf_file/0008/95165/E91689.pdf. Accessed January 19, 2012.

46. Federal Fund—Russia. Available at: portal/top/index. Accessed January

19, 2012.

47. Zavod za zdravstveno zavarovanje Slovenije. Health Insurance Institute of Slovenia. Available at: http://www. Accessed January 19, 2012.

48. Armenian Health Insurance Fund. Available at: Accessed January 19, 2012.

49. Ministry of Health of Republic of Kazakhstan. Welcome! Available at: wakka=/Eng. Accessed January 19, 2012.

50. Ministerstvo zdravotnictva SR. Slovakia Ministry of Health. Available at: Accessed January 19, 2012.

51. [Ministry of Health—Tajikistan.] Available at: en/. Accessed January 19, 2012.

52. Pan American Health Organization. Profile of the Health Services System Argentina. February 25, 2002. Available at: hq/dmdocuments/2010/Health_Sys-tem_Profile-Argentina_2002.pdf. Accessed January 19, 2012.

53. Auge en Linea — Fonasa. Available at: Accessed January 19, 2012.

54. Ministry of Social Protection, Republic of Colombia. Ultima jornada de socialización del texto final de la GPC sobre el manejo del Asma infantil. Available at: http://www. Accessed January 19, 2012.

55. SeNaSa. Seguro Nacional de Salud. Available at: Accessed January 19, 2012.

56. Gobierno de Reconciliacion y Unidad Nacional [Ministry of Health—Nicaragua]. Available at: http://www. Accessed January 19, 2012.

57. MINSA inicio difusion del Plan Esencial de Aseguramiento de Salud (PEAS). Available at: http:// 22/minsa-inicio-difusion-del-plan-esencial-de-aseguramiento-de-salud-peas/. Accessed January 19, 2012.

58. Aran D. Sistema de salud de Uruguay. Available at: Last accessed March 2, 2012.

59. Plan Nacer. Noticias. Available at: Accessed January 19, 2012.

60. World Bank. Health Sector Reform in Bolivia: A Decentralization Case Study. Washington, DC: World Bank, 2004. Available at: http:// 9ziYC&pg=PA4&lpg=PA4&dq=

Bolivia+SUMI&source=bl&ots= l0K2rDKYnN&sig=CPvafEssR7ux MSsWIni48rak_tg&hl=en&ei=P1N 6Tuj0E0b40gGYztncAg&sa=X& oi=book_result&ct=result&resnum= 9&ved=0CGkQ6AEwCA#v=onepage &q=Bolivia%20SUMI&f=false; Bolivia_OP_213.pdf. Accessed January 19, 2012.

61. Lista de Medicamentos de Referencia. Available at: http://www.scribd.-com/doc/73742931/Lista-de-Medi-camentos-SUS-I. Last accessed March 2, 2012.

62. IHSS. Instituto Hondureno de Seguridad Social. Available at: http: // Accessed January 19, 2012.

63. Comision Nacional de Proteccion Social en Salud/Seguro Popular. Catalogo Universal de Servicios de Salud (CAUSES); 2010. Available at: images/contenidos/Causes/catalogo_ 2010.pdf. Accessed January 19, 2012.

64. United Nations. United Nations Economic and Social Commission for Asia and the Pacific. 2009. Promoting sustainable strategies to improve access to health care in the Asian and Pacific Region. Available at: hds/pubs/2529/pub2529.pdf. Last accessed March 2, 2012.

65. PhilHealth. Phillippine Health Insurance Corporation home page. Available at: Accessed January 19, 2012.

66. World Bank Group. Vietnam: Partnership in Health. Available at: http: // en&sl=vi&u=http://web.worldbank.







Accessed January 19, 2012.

67. Ir P, Bigdeli M, Van Damme W. Translating knowledge into policy and action to promote health equity: the Health Equity Fund policy process in Cambodia 2000-2008. Health Policy 2010;96:200-9.

68. World Bank. Reforming China's Rural Health System. Available at: Accessed January 19, 2012.

69. Ministry of Health and Family Welfare, Government of India. National Rural Health Mission. Available at: NRHM.htm. Accessed January 19, 2012.

70. Ministry ofHealth Malaysia. Official portal. Available at: http://www.moh. Accessed January 19, 2012.

71. Hughes D, Leethongdee S. Universal coverage in the Land of Smiles: lessons from Thailand's 30 Baht health reforms. Health Affairs 2007; 26:999-1008. Available at: http:// 26/4/999.full. Accessed January 19, 2012.

72. Health Insurance Organization. Home page. Available at: http://hioegypt. org/english.aspx. Accessed January 19, 2012.

73. National Insurance Institute of Israel. Home page. Available at: http: // page/Pages/default.aspx. Accessed January 19, 2012.

74. CNSS-Lebanon. Available at: http: // Accessed January 19, 2012.

75. European Observatory on Health Care Systems. Health Care Systems in Transition—Malta. Available at: assets/pdf_file/0008/95129/E67140. pdf. Accessed January 19,

76. Joint Learning Network. Syria: Healthcare Reforms. Available at: blog/2011/jun/29/syria-healthcare-reforms. Accessed January 19, 2012.

77. Republic of Tunisia, Ministry of Social Affairs. Home page. Available at: Accessed January 19, 2012.

78. Daman Health. Home page. Available at: eDamanApp/en/home/index.html? version=R110.8.9. Accessed January 19, 2012.

79. World Bank. West Bank and Gaza Health Policy Report: Reforming Prudently Under Pressure—Health Financing Reform and the Rationalization of Public Sector Health Expenditures. Available at: BANKGAZA/Resources/WBGHealth PolicyReport.pdf. Accessed January 19, 2012.

80. WHO. Country Cooperation Strategy for WHO and the Republic of Yemen: 2008—2013. Available at: cooperation_strategy/ccs_yem_en. pdf. Accessed January 19, 2012.

81. Ministry of Health, Bahrain. Home page. Available at: http://www.moh. Accessed January 19, 2012.

82. Ministry of Health, Djibouti. Available at: djibalger.php. Accessed October 28, 2011.

83. WHO. Country Cooperation Strategy for WHO and Jordan: 2003— 2007. Available at: http://www.who.

int/countries/en/cooperation_strategy_ jor_en.pdf. Accessed January 19, 2012.

84. Agence Nationale de l'Assurance Maladie. Missions. Available at: http: // php?id_espace=1&id_srub=2. Accessed January 19, 2012.

85. Ministry of Health-Sultanate of Oman. Available at: http://www. Accessed January 19, 2012.

86. State of Qatar Supreme Council of Health. Home page. Available at: Accessed January 19, 2012.

87. Kingdom of Saudi Arabia, Ministry of Health. Home page. Available at: Default.aspx. Accessed January 19, 2012.

88. National Health Insurance Scheme. NHIS Benefits Package. Available at: Accessed January 19, 2012.

89. NHIF. Home page. Available at: Accessed January 19, 2012.

90. Smit N. SSC mulls national medical aid. The Namibian [online newspaper]. September 15,2011. Available at http: // national/full-story/archive/2011/sep tember/article/ssc-mulls-national-medical-aid/. Accessed January 19, 2012.

91. National Health Insurance Scheme. Welcome page. Available at: http:// Accessed January 19, 2012.

92. Jutting J. Health Insurance for the Rural Poor? Available at: http:// 2510517.pdf. Accessed January 19, 2012.

93. Khumalo G. SA gears for National Health Insurance. August 12, 2011. Available at: http://www.southafrica. info/about/health/nhi-120811.htm. Accessed January 19, 2012.

94. National Health Insurance Fund. Home page. Available at: http://www. Accessed January 19, 2012.

95. Uganda Health Insurance to Improve Lives. Available at: http: // html. Last accessed March 2, 2012.

96. USAID, HSSP. Strengthening Human Resources for Health. Occasional Paper Series 1. April 2006. Available at: Accessed January 19, 2012.

97. Ramachandra S, Hsiao W. Ghana: initiating social health insurance. Social health insurance for developing nations. Washington, DC: World Bank; 2007.

98. Glassman A, Gaziano TA, Bouillon Buendia CP, Guanais de Aguiar FC. Confronting the chronic disease burden in Latin America and the

Caribbean. Health Aff (Millwood) 2010;29:2142-8.

99. Bitran R, Escobar L. The politics of the AUGE health reform in Chile: a case study. Washington, DC: Results for Development Institute; 2008.

100. Giedion U, Panopoulou G, Gomez-Fraga S. Disefio y ajuste de los planes explícitos de beneficios: el caso de Colombia y México. CEPAL; 2009.

101. Prieto, Cid. IDB regional study on the design and adjustment of health benefits packages; 2011.

102. Hsiao WC, Shaw P. Social health insurance for developing nations. Washington, DC: World Bank; 2007.

103. Inter-American Development Bank. Terms of reference for study on Jamaica health sector. Sustainable financing and reform of the health sector to improve effectiveness, efficiency and quality of care in Jamaica, August 2011.

104. Yamin AE, Vera OP. The role of courts in defining health policy: the case of the Colombian Constitutional Court. Harvard Law School Working Paper; 2008. Available at: http:// documents/Yamin_Parra_working_ paper.pdf. Accessed January 19, 2012.

105. Landwehr C, Bohm K. Delegation and institutional design in healthcare rationing. Governance 2011;24: 665-88. Available at: http://online

1468-0491.2011.01542.x/full. Accessed January 19, 2012.

106. Budlender D. The South African Women's Budget Initiative. Paper presented at: Meeting on Women and Political Participation: 21st Century Challenges; March 24-26, 1999; New Delhi, India. Available at: %20CD%20subsection%204.2/4.2.1a %20South%20African%20Women% 20Budget%20Initiative.htm. Accessed January 19, 2012.

107. McLeod H. Defining the Benefit package: National Health Insurance Policy Brief 10. Innovative Medicines South Africa, 2010. Available at: Library/NHI/policy%20brief%2010/ IMSA%20NHI%20Policy%20Brief% 2010%20Defining%20the%20Benefit %20Package%20vF.pdf. Accessed February 22, 2012.

108. Congreso de la Republica. Ley 1392 de Julio 02 de 2010. Diario Oficial 47558 de 2010. Available at: http:// a1/a107146d-684d-4593-9ad3-5a88 fc855e35.pdf. Accessed January 19, 2012.

110. Bochenek T. Note on comparative analysis of reference countries con-

ducted as part of the working, group. 2011.

111. Sorenson C, Drummond M, Kris-tensen FB, et al. How can the impact of health technology assessment be enhanced? Copenhagen, WHO Regional Office for Europe (Policy Brief). 2008. Available at: http://www. euro.who.int_data/assets/pdf_file/ 0019/73225/E93420.pdf. Last accessed March 2, 2012.

112. Schreyogg J, Stargardt T, Velasco-Garrido M, Busse R. Defining the 'Health Benefit Basket' in nine European countries. Eur J Health Econ 2005. Available at: http:// PMC1388078/. Accessed February 23, 2012.

113. Nielsen CP, Funch TM, Kristensen FB. Health technology assessment: research trends and future priorities in Europe. Journal of Health Services Research & Policy. July 2011, vol. 16. Available at: http://jhsrp.rsmjournals. com/content/16/suppl_2/6. Accessed February 23, 2012.

114. European Commission. Public Health Policy. Available at: http:// assessment/policy/index_en.htm. Last accessed March 2, 2012.

115. Dalli J. Commissioner Dalli Addresses the EUnetHTA Conference 2011: HTA in National and Cross-Border Healthcare in Europe [Transcript]. December 8, 2011. Available at: commission_2010-2014/dalli/docs/ speech_08122011_en.pdf. Last accessed March 2, 2012.

116. AsiaLink. Collection of HTcAsia-Link newsletters. Available at: Accessed January 19, 2012.

117. Oortwijn W, Mathijssen J, Banta D. The role of health technology assessment on pharmaceutical reimbursement in selected middle-income countries. Health Policy 2010;95:174-84.

118. First Step Program Evaluation Report 2010. Research for development of an optimal policy strategy for prevention and control of cervical cancer in Thailand. Available at: 10481. Accessed February 22, 2012.

119. Health Intervention and Technology Assessment Program, HITAP. Evaluating HITAP: 2 years on HITAP's responses to key recommendations. December 2009. Available at: http:// first_step_eng.pdf. Last accessed March 2, 2012.

120. Gavaza P, Shepherd M, Shcherbak-ova N, Khoza S. The state of health economics and pharmaceoconomics research in Russia: a systematic

review. J Pharm. Health Serv Res 2010;1:113—21.

121. Iglesias C, Drummond M. Healthcare decision making processes in Latin America: problems and prospects for the use of economic evaluation. Int J Technol Assess Health Care 2005;21:1—14.

122. Tarn YH, Hu S, Kamae I, et al. Health-care systems and pharmaco-economic research in Asia-Pacific Region. Value Health 2008;11(Sup-pl. 1):S137-55. Available at: http:// 20-Tarn.pdf. Accessed January 19, 2012.

123. Reglamento Interior de la Comision Interinstitucional del Cuadro Basico y Catalogo de Insumos del Sector Salud. Gobierno de Mexico, Diario Oficial de la Federacion, Miercoles 22 de junio de 2011.

124. Teerawattananon Y, Russell S, Mugford M. A systematic review of economic evaluation literature in Thailand: are the data good enough to be used by policy-makers? Phar-macoeconomics 2007;25:146—79.

125. Walker J, Pan E, Johnston D, Adler-Milstein J, Bates DW, Middleton B. The value of health care information exchange and interoperability. Health Aff 2005;Suppl Web Exclu-sives:W5-10—18. Available at: http: // suppl/2005/02/07/hlthaff.w5.10.DC1. Last accessed March 2, 2012.

126. Mulligan JA, Walker D, Fox-Rush-by J. Economic evaluations of non-communicable disease interventions in developing countries: a critical review of the evidence base. Cost Eff Resour Alloc 2006;4:7.

127. ISPOR. ISPOR Global Health Systems Road Map. Available at: http:// Accessed February 22, 2012.

128. Walker D, Teerawattananon Y, Anderson R, Richardsong. General-isability, transferability, complexity and relevance. In: Shemlit I, Mug-ford M, Vale L, et al. eds. Evidence-based decisions and economics: health care, social welfare, education and criminal justice; 2010:56—66.

129. Chaikledkaew U, Lertpitakpong C, Teerawattananon Y, Thavorncha-roensap M, Tangcharoensathien V. The current capacity and future development of economic evaluation for policy decision-making: a survey among researchers and decision-makers in Thailand. Value Health 2009;12(Suppl. 3):S31—5.

130. Sorenson C, Kanavos P, Drummond M. Ensuring value for money in healthcare—the role of HTA in the European Union. In: Financing sustainable healthcare in Europe: new approaches for new outcomes (Cox Report), Luxembuorg: Ministry of

Health, Luxembourg and Sitra, the Finnish Innovation Fund; 2007

131. Yothasamut J, Tantivess S, Teera-wattananon Y. Using economic evaluation in policy decision-making in Asian countries: mission impossible or mission probable? Value Health 2009;12(Suppl. 3):S26—30.

132. International Monetary Fund. Romania: Second Review under the Stand-By Arrangement and Request for Modification of Performance Criteria——Staff Report [Press release]. October 2011. Available at: ft/scr/2011/cr11297.pdf. Accessed January 19, 2012.

133. Gutierrez G, Munoz A. The need for a systemic process in priority setting. IDB strengthening priority-setting processes technical brief series. Washington, DC: IDB, 2012.

134. International Working Group for HTA Advancement. Are key principles for improved health technology assessment supported and used by health technology assessment organizations? Int J Technol Assess Health Care. Available at: http:// Fulltext?type=1&fid=7029668&jid= THC&volumeId=26&issueId=01& aid=7029660. Last accessed March 2, 2012.

135. Goodman G.S. HTA 101: Introduction to Health Technology Assessment. 2004. Available at http://www. pdf. Accessed February 22, 2012.

136. Binagwaho A, Wagner C, Nutt C. HPV vaccine in Rwanda: different disease, same double standard. Lancet 2011;378:1916. Available at: http:// cle/PIIS0140-6736(11)61837-0/full text. Accessed January 19, 2012.

137. Partnership for Maternal, Newborn, and Child Health. Marginal Budgeting for Bottlenecks: UNICEF/ The World Bank [Online costing tool]. Available at: http://www. costing_tools/en/index12.html. Accessed January 19, 2012.

138. Miljeteig I, Sayeed SA, Jesani A, Johansson KA, Norheim OF. Impact of ethics and economics on end-of-life decisions in an Indian neonatal unit. Pediatrics 2009;124:e322—8.

139. Culyer AJ, Sculpher M. Lessons from health technology assessment. Available at: Accessed February 27, 2012.

140. NHS. NHS Evidence. Accreditation process. Available at: https://www. tation-process. Accessed February

23, 2012.

141. Yusuf S, Islam S, Chow CK, et al. Use of secondary prevention drugs for cardiovascular disease in the community in high-income, middle-income, and low-income countries (the PURE Study): a prospective epidemiological survey. The Lancet 2011;378:1231—43.

142. Inter-American Development Bank. Proyecto de apoyo al fortalecimiento del proceso de priorizacion en salud en Colombia. Unpublished technical brief, forthcoming 2012.

143. Doherty J, Govender R. The Cost-Effectiveness of Primary Care Services in Developing Countries: A Review of International Literature. Disease Control Priorities Project Working Paper 37. December 2004. Available at: http:// Accessed January 19, 2012.

144. Shafie AA. Setting national healthcare priorities in Asia: experiences from Malaysia. Paper presented at: BMJ-NICE Conference; September 30, 2011; London, UK.

145. Pichon-Riviere A, Augustovski F, Rubinstein A, Marti SG, Sullivan SDDrummond MF. Health technology assessment for resource allocation decisions: are key principles relevant for Latin America? Int J Technol Assess Health Care 2010;26:421—7.

146. Goeree R, He J, O'Reilly D, et al. Transferability of health technology assessments and economic evaluations: a systematic review of approaches for assessment and application. Clinicoecon Outcomes Res 2011;3:89—104.

147. Development of a toolkit and glossary to aid in the adaptation of health technology assessment (HTA) reports for use in different contexts. Available at: http://www. Accessed January 19, 2012.

148. Yang BM, Lee KC. Growing application of pharmacoeconomics and outcomes research in health-care decision-making in the Asia-Pacific region. Value Health 2009; 12(Suppl. 3):S1—2.

149. Royal College of General Practitioners and National Collaborating Centre for Primary Care.

150. NHS. The NHS in England: operating framework for 2007-08. Available at: en/Publicationsandstatistics/Publica tions/PublicationsPolicyAndGuidance/ DH_063267. Accessed February 22, 2012.

151. National Service Framework for Coronary Heart Disease: Winning the war on, heart disease 2004. Available at: en/Publicationsandstatistics/Publica-tions/PublicationsPolicyAndGuidance/

DH_4094275. Accessed February 22, 2012.

152. National Primary Care Research and Development Centre, University of York Health Economics Consortium. Health Economic Report: Net Benefit Analysis. July 2010. Available at: nicemedia/live/13071/50048/50048. pdf. Accessed January 19, 2012.

153. National Institute for Health and Clinical Excellence. Quality Outcomes Framework Programme: NICE Cost Statement, July 2010. Available at: nicemedia/live/13071/50101/50101. pdf. Accessed January 19, 2012.

154. National Institute for Health and Clinical Excellence. Primary Care Quality and Outcomes Framework

Indicator Advisory Committee Recommendations. Available at: live/13071/50050/50050.pdf. Accessed January 19, 2012.

155. National Institute for Health and Clinical Excellence. Developing Clinical and Health Improvement Indicators for the Quality and Outcomes Framework: Interim Process Guide, May 2009. Available at: 32/QOFProcessGuide.pdf. Accessed January 19, 2012.

156. Summary of 2011/12 QOF indicator changes, points, and thresholds [Table]. Available at: http://www.bma. 2011_v3_tcm41-204734.pdf. Accessed January 19, 2012.

157. National Institute for Health and Clinical Excellence. Available at: Accessed January 19, 2012.

158. National Institute for Health and Clinical Excellence, Centre for Clinical Practice. Review of Clinical Guideline (CG48)--Secondary Prevention in Primary and Secondary Care for Patients Following a Myo-cardial Infarction. February 2011. Available at: nicemedia/live/11008/53347/53347. pdf. Accessed January 19, 2012.