Scholarly article on topic 'European collaboration on relative effectiveness assessments: What is needed to be successful?'

European collaboration on relative effectiveness assessments: What is needed to be successful? Academic research paper on "Economics and business"

CC BY-NC-ND
0
0
Share paper
Academic journal
Health Policy
OECD Field of science
Keywords
{"Relative effectiveness" / "Relative effectiveness assessment" / Pharmaceuticals / "Health technology assessment" / "European collaboration" / Reimbursement}

Abstract of research paper on Economics and business, author of scientific article — Sarah Kleijnen, Wil Toenders, Folkert de Groot, Mirjana Huic, Elisabeth George, et al.

Abstract Objective The objective of this study is to identify the possible barriers and critical success factors for the implementation of European collaboration in the field of relative effectiveness assessment (REA) of drugs. Methods Data were gathered through semi-structured interviews with representatives from eight European health technology assessment (HTA) organisations involved in assessment of drugs for coverage decision-making (AAZ, AIFA, AHTAPol, HAS, HVB, IQWIG, NICE and ZiN). Results Potential barriers identified mainly relate to methodology, resources and challenges with implementation in the respective national processes (e.g. legal restrictions). The most critical success factors for production of cross-border assessments were the continuous cooperation of competent partners, and the quality and timely availability of the assessment. Conclusion Further adaptation of the process and methods is required for optimal collaboration. In the near future it can be expected that cross-border assessments will meet in particular the needs of smaller/middle-sized European countries and also European countries with less developed HTA systems as the potential efficiency/quality gains are the highest for these countries. Therefore, national implementation of cross-border assessments is especially likely in these countries in the coming years. Once more experience is gained with cross-border assessments, and successes become more evident, efficiency/quality gains may also be likely for some larger countries with well established processes.

Academic research paper on topic "European collaboration on relative effectiveness assessments: What is needed to be successful?"

HEALTH

POLICY

Contents lists available at ScienceDirect

Health Policy

journal homepage www.elsevier.com/locate/healthpol

European collaboration on relative effectiveness assessments: What is needed to be successful?

Sarah Kleijnena b'*, Wil Toendersc, Folkert de Grootc, Mirjana Huicd, Elisabeth Georgef, Beate Wieselerg, Mira Pavlovich, Anna Bucsicsij, Paolo D. Sivieroe, Martin van der Graaffa, Rafai Rdzanyk, Finn B0rlum Kristensenl m, Wim Goettschab

a National Health Care Institute (Zorginstituut Nederland), PO Box 320, 110 XH Diemen, The Netherlands b University of Utrecht, PO Box 80082,3508 TB Utrecht, The Netherlands c ToendersdeGroot Consultants, Boomstede 281,3608 AN Maarssen, The Netherlands

d Agency for Quality and Accreditation in Health Care and Social Welfare (Agencija za kvalitetu i akreditaciju u zdravstvu i socijalnoj skrbi), Planinska 13, HR-10000 Zagreb, Croatia

e Italian Medicines Agency (Agenzia Italiana del Farmaco), Via del Tritone, 181, 00187 Rome, Italy f National Institute for Health and Care Excellence, 10 Spring Gardens, London SW1A2BU, England, United Kingdom g The Institute for Quality and Efficiency in Healthcare (Institut für Qualität und Wirtschaftlichkeit im Gesundheidswesen), Im Mediapark 8, D-50670 Köln, Germany

h French National Authority for Health (Haute Autorité de Santé), 2, avenue du Stade de France, 93218 Saint-Denis La Plaine Cedex, France

i Association of Austrian Social Insurance Institutions (Hauptverband der Österreichischen Sozialversicherungsträger), Postfach 600, 1031 Vienna, Austria

j University ofVienna, Brünner Straße 72, 1210 Vienna, Austria

k Agency for Health Technology Assessment and Tariff System in Poland (Agencja Oceny Technologii Medycznych i Taryfikacji), ul. I. Krasickiego 26, 02-611 Warsaw, Poland

l Danish Health and Medicines Authority (Sundhedsstyrelsen), Axel Heides Gade 1, 2300 Copenhagen S, Denmark m University of Southern Denmark, Campusvej 55, DK5230 Odense M, Denmark

ARTICLE INFO

ABSTRACT

Article history:

Received 30 October 2014

Received in revised form 28 January 2015

Accepted 31 January 2015

Keywords:

Relative effectiveness

Relative effectiveness assessment

Pharmaceuticals

Health technology assessment

European collaboration

Reimbursement

Objective: The objective of this study is to identify the possible barriers and critical success factors for the implementation of European collaboration in the field of relative effectiveness assessment (REA) of drugs.

Methods: Data were gathered through semi-structured interviews with representatives from eight European health technology assessment (HTA) organisations involved in assessment of drugs for coverage decision-making (AAZ, AIFA, AHTAPol, HAS, HVB, IQWIG, NICE and ZiN).

Results: Potential barriers identified mainly relate to methodology, resources and challenges with implementation in the respective national processes (e.g. legal restrictions). The most critical success factors for production of cross-border assessments were the continuous cooperation of competent partners, and the quality and timely availability of the assessment.

Conclusion: Further adaptation of the process and methods is required for optimal collaboration. In the near future it can be expected that cross-border assessments will meet in particular the needs of smaller/middle-sized European countries and also European countries with less developed HTA systems as the potential efficiency/quality gains

* Corresponding author at: Postbus320,1110 AH Diemen, The Netherlands. Tel.: +31 020 797 8008; mobile: +31 06 21853421. E-mail address: skleijnen@zinl.nl (S. Kleijnen).

http://dx.doi.org/10.1016lj.healthpol.2015.01.018

0168-8510/© 2015 The Authors. Published by Elsevier Ireland Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4XI/).

are the highest for these countries. Therefore, national implementation of cross-border assessments is especially likely in these countries in the coming years. Once more experience is gained with cross-border assessments, and successes become more evident, efficiency/quality gains may also be likely for some larger countries with well established processes.

©2015 The Authors. Published by Elsevier Ireland Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. Introduction

Insurance entities in many nations use health technology assessments (HTA) to prioritise drugs for reimbursement [1]. European countries employ different analytical frameworks to guide their assessments of drugs as part of the coverage decision making process [2]. Most assessments include a variety of criteria, of which clinical effectiveness and safety relative to the comparator (also referred to as relative or comparative effectiveness) are the most widely used. Examples of other relevant criteria are cost-effectiveness, budget impact, drug/innovative characteristics, availability of therapeutic alternatives, equity considerations, public health impact and research and development [2].

Although coverage decisions for drugs in Europe are mostly made at a national or regional level and may differ between countries, this does not preclude member states from sharing the scientific assessments on which their decisions are based [3]. Increased European collaboration and harmonisation in the field of relative effectiveness assessment (REA)/health technology assessment (HTA) may save resources and prevent duplication of work for both manufacturers and coverage decision agencies [2-4]. However, it could also carry the risks of losing local con-textualisation, of the application of standards that are not universally accepted and slowing the rate of development and innovation in the analytical disciplines supporting the assessments [3].

In 2005, the European Commission established that a REA, a specific element of HTA, is a relevant tool to identify the most valuable drugs, and allow containment of drug costs as well as a fair reward for innovation. The Commission set up a Working Group on Relative Effectiveness as part of the High Level Pharmaceutical Forum (2006-2008, see Fig. 1) with the aim to support Member States in applying REAs. The High Level Pharmaceutical Forum consisted of Ministries of the Member States as well as relevant stakeholders. In 2008, the High Level Pharmaceutical Forum endorsed the definition of relative effectiveness as: the extent to which an intervention does more good than harm compared to one or more intervention alternatives for achieving the desired results when provided under the usual circumstances of health care practice [5]. As this definition includes harms, the concept is similar to what in many countries is referred to as the net therapeutic benefit or relative therapeutic value. This is confirmed by the statement of the High Level Pharmaceutical Forum that the aim of REA is to compare healthcare interventions in daily practice and classifying them according to their added therapeutic value.

The work of the High Level Pharmaceutical Forum on REA was continued by the European network for Health Technology Assessment (EUnetHTA) (see Fig. 1). Between 2010 and 2012, work package 5 of EUnetHTA Joint Action 1 (WP5/JA1), developed several products and tools that aim to facilitate collaboration in this field [6]. Most important were a structure for jointly producing and reporting scientific relative effectiveness information for trans-national use (HTA Core Model® for Rapid Relative Effectiveness Assessment of Pharmaceuticals) and methodological guidelines on issues relevant to REAs. The development of the HTA Core Model® for Rapid REA of Pharmaceuticals was based on the work of EUnetHTA on the original HTA Core Model® [7] but adapted to suit the expectation and requirements of rapid REAs of drugs [6]. The nine methodological guidelines focus on issues that assessors are frequently challenged with: endpoints (clinical endpoints, composite endpoints, surrogate endpoints, safety and quality of life), comparisons (choice of appropriate comparator and direct and indirect comparisons) and level of evidence (internal validity of randomised controlled trials and applicability) [8]. Both Model and Guidelines were tested by a pilot assessment of the drug pazopanib for renal cell carcinoma [9]. Currently, the Model and Guidelines are being further piloted as part of WP5 Joint Action 2 (WP5/JA2, 2012-2015) [10,11].

As European countries have different healthcare systems with their own dynamics, they may have different challenges for collaboration in the field of REA. Literature data on challenges, barriers and factors facilitating international collaboration on cross-border HTAs is scarce [12-14]. Recently Huic et al. [14] concluded that timely and efficient collaborative HTA processes on relative efficacy/effectiveness and safety on different types of health technologies are possible in Europe but there are still barriers to overcome.

In order to maximise the likelihood of successful collaboration in the field of REA, we wanted to research the challenges and success factors on collaboration in more detail. Therefore, the objective of this study is to identify the possible barriers and critical success factors for the implementation of European collaboration in the field of REA of drugs.

2. Methods

2.1. Study design and period

A qualitative cross-sectional study was conducted for which eight interviews were performed between 8th and 16th January 2013.

High Level Pharmaceutical

Forum: Concepts and definitions

WP5 EUnetHTA JA1: Development of methodology

for joint assessments

WP5 EUnetHTA JA2: Testing methodology and procedure

of joint assessments in 10 pilots

EUnetHTA JA3: Integration of tools and method in national setting

Fig. 1. European collaboration on relative effectiveness assessment of drugs. The historical context of collaboration in the field of relative effectiveness assessment of drugs in Europe.

A cross-border assessment of relative effectiveness of pharmaceuticals in Europe was defined as an assessment of pharmaceuticals reported and co-produced by at least two countries. Preferably, more countries (five) should be involved in reviewing the report. The results of the assessment (or parts of them) can then be available to be used by (other) European countries in their national assessment.

2.2. Participants

Data were gathered through semi-structured interviews with representatives from eight European HTA organisations involved in the assessment of drugs for coverage decision making, which participated in WP5/JA1. The representatives were nominated for the interviews by the HTA organisations participating in WP5/JA1. For this study a limited number of HTA organisations were chosen (eight) as it was not possible to interview all organisations within the available timeframe. In addition, it was assumed that based on the selection of organisations sufficient diversity in opinions was guaranteed for this study. The selection of the organisations was based on including the four most populous European jurisdictions (France, Germany, Italy and England), two middle-sized countries (Austria and the Netherlands) and two central/eastern European countries (Poland and Croatia). The representatives of HTA organisations were contacted by email. All respondents approached participated in the study. The organisations are presented in Table 1.

2.3. Survey instrument and data collection

A standardised data extraction form (questionnaire) was developed with input from WP5/JA1 members. The form included a total of 23 questions (see Supplement 1) divided in three parts. The 1st part was focused on expectations from European collaboration in the field of REA (Q1-Q3); the 2nd part on potential challenges for European collaboration in the field of REA (Q4-Q18) and the 3rd part on critical success factors for cross-border assessments (Q19-Q23).

The questionnaire was sent to the interviewees in advance. Data were captured via a 1-h telephone interview. The interviews were not recorded. All interviews were conducted by W.T. (PharmD) experienced in coverage

assessments of drugs. The average age of the interviewees at the time of the interviews was 50 and 3/8 (38%) were male.

2.4. Data analysis

The results were processed as qualitative information. The results of the interviews were handled confidentially. The answers were checked by a second researcher for inconsistencies and clarity. In case of inconsistencies/lack of clarity a follow-up contact with the respondent was established asking for further explanation. The results were aggregated into a summary. In order to guarantee anonymous presentation of the data, the exact number of countries giving a specific response is not presented, unless all countries agreed. Anonymous participant quotations are presented to illustrate the themes. The summary was sent to the respondents for validation. Input received as a result of the validation was processed accordingly.

The COREQ check-list, a 32-item checklist for explicit and comprehensive reporting of qualitative studies, was used for reporting the methods section [15].

3. Results

3.1. Expectations from European collaboration in the field of REA

3.1.1. Extent of collaboration

Collaboration (in the broadest sense, from exchange of information to cross-border production of assessments) between European Union (EU) member states on REAs was seen as a positive development.

The respondents agreed that member states should first focus on collaboration in the field of relative effectiveness, before addressing health economic issues. Several respondents indicated that they think there are less differences in REAs between EU countries and regions within countries (= to be referred to as countries) compared with economic assessments. Opinions differed regarding the desired extent of the cooperation. Some respondents were in favour of cross-border production of assessments that can be used directly in the national decision-making process. Others were more reserved and would like to limit the collaboration to exchange of information and/or

Table 1

Jurisdictions and organisations included in the study.

Jurisdiction Assessment Agency

Austria Hauptverband der Österreichischen Sozialversicherungsträger (HVB)

Croatia Agencija za kvalitetu i akreditaciju u zdravstvu i socijalnoj skrbi (AAZ)

France Haute Autorité de Santé (HAS)

Germany Institut für Qualität und Wirtschaftlichkeit im Gesundheidswesen (IQWIG)

Italy Agenzia Italiana del Farmaco (AIFA)

Netherlands Zorginstituut Nederland (former CVZ)

Poland Agencja Oceny Technologii Medycznych (AHTAPol)

England National Institute for Health and Clinical Excellence (NICE)

methodology development. A mandatory cross-border production of assessments was not preferred by any of the respondents.

3.1.2. Influence of cross-border assessments on national autonomy and expertise

Most respondents considered coverage in healthcare to be a politically sensitive issue as it is expected that no country would be willing to give up its autonomy in decision-making. However, it was also indicated that the sensitive aspects are less likely to be in the technical domains of a REA but more likely to be caused by economic, societal and nationally specific elements. Therefore, most respondents agreed that the production of cross-border REAs does not conflict with the principle of subsidiarity. The subsidiarity principle is defined in Article 5 of the Treaty on EU and in this case refers to the principle that decision making regarding healthcare and social security is a national competence in EU Member States.

It was indicated by some respondents that if (part of) a cross-border assessment would be included in the national procedure, this would influence the autonomy of the HTA organisation to some extent as it would be difficult to explain an interpretation of relative effectiveness which is different to the one presented in the cross-border assessment. This would at least require a transparent explanation.

All respondents agreed that cross-border collaboration would lead to an increase in national expertise by sharing insights and information. It was indicated that frequent personal contacts between staff from different HTA organisations (e.g. international meetings, internships) are relevant to build trust. None of the respondents expected a reduction in the number of staff needed for the national process.

3.1.3. Influence of cross-border assessments on the time to market

Most respondents considered it unlikely that a cross-border assessment will lead to a decrease in the time to market of new drugs since a national process will still be needed and the REA is only a part of the coverage decision making process. It was mentioned by some that cross-border REAs initially may slow down the process in countries with an already well established procedure, thereby negatively influencing the average duration of time to market. On the other hand, in countries that have just started with such assessments it is to be expected that

cross-border assessments could lead to a faster assessment and thereby speed up the national decision making.

Two respondents pointed out that cross-border assessments of new drugs may result in freeing up of more resources on a national level for activities such as spending time on exploring social/ethical values or (re)assessing drugs that are already reimbursed.

3.1.4. Influence of cross-border assessments on the quality of decision making

Most respondents felt the general quality level of decision making among European countries would be raised by cross-border assessments, although maybe not compared with the present level in some individual countries. A small majority of the respondents expected that the transparency in decision making may increase to some degree. It was indicated by one respondent that making both the submission dossier of the marketing authorisation holder as well as the final assessment publicly available would probably lead to the highest transparency.

3.2. Potential challenges for European collaboration in the field of REA

3.2.1. Potential challenges for the uptake of the HTA Core Model and/or Guidelines for Rapid REA of Pharmaceuticals in the national setting

There were diverging opinions regarding potential problems for implementing the HTA Core Model and/or Guidelines for Rapid REA of Pharmaceuticals in the national setting. Whereas some respondents did not foresee any problems, others identified several challenges. The potential challenges that were mentioned by respondents are listed in Table 2.

We identified nine key potential challenges. A number of the challenges identified refer to potential differences between the national way of working and EUnetHTA Model and Guidelines regarding content and the use of more or less advanced methodology, and how strictly or pragmatically methodologies are applied. One example of differences in content was that in some countries unpublished data from the registration-dossier can be included in the dossier, while in other countries unpublished data are not accepted. Another respondent indicated that there is too much emphasis on safety which may duplicate information from EMA. A further example was that effectiveness is not evaluated separately in every country, as in some countries it is integrated into the cost-effectiveness

Table 2

Potential challenges for uptake of the HTACore Model and/or Guidelines for Rapid Relative Effectiveness Assessment of Pharmaceuticals in national setting.

• National assessments may not have the same content as assessments produced with the Model/Guidelines

• In national assessments methodologies may not be applied as strictly or pragmatically as in assessments produced with the Model/Guidelines (e.g. use of surrogate endpoints)

• For national assessments less/more advanced methodology may be used (e.g. use of indirect comparisons)

• The resources associated with having to adapt the Model's outcomes to the national procedure

• Implementation may lead to longer timelines which is a problem as the assessment of new drugs has to be done in a timely manner

• Implementation may lead to loss of autonomy in deciding on the outcome of the relative effectiveness assessment.

• Legal restrictions that hinder the use of the Model/Guidelines (e.g. use of unpublished data or inclusions off-label treatments)

• There may be variance in the interpretation of the methods in the Model between assessors and/or countries

• It is questioned whether the Model is suitable for submission based assessments (e.g. more suitable for 'de novo' assessments)

evaluation. An example of different attitudes to the strictness with which to apply methodologies that was mentioned was the divergence in acceptance of surrogate outcomes.

Further, it was indicated that the time and resources associated with having to adapt the Model's outcomes to the national procedure may be an issue. This may be due to differences in content or because for some countries a translation of an assessment produced with the Model is required for national implementation. It was also pointed out by one respondent that the current format of assessments produced with the Model is not very suitable for consultation or decision making due to its repetitive and lengthy nature.

3.2.2. Potential challenges for the production of cross-border assessments

3.2.2.1. Methodological challenges. The most frequently mentioned methodological problem for the production of a cross-border assessment was the choice of the comparator. It was indicated that the principles for choosing the com-parator(s) can vary between countries. For instance, not all countries include drugs used off label or best supportive care as a comparator. It was also indicated that the process for choosing the right comparator may differ (e.g. based on the proposal of the manufacturer or on input from a national physician organisation).

Another methodological challenge mentioned was the difference between countries in accepting indirect comparisons, if direct evidence is lacking. Further, it was noticed that countries may differ in the acceptance of intermediate or surrogate endpoints (e.g. glycosylated haemoglobin values versus outcomes on microvascular and macrovascular complications in diabetes patients). Finally, it was indicated that some countries prefer evidence from randomised clinical trials whereas other countries also consider observational studies.

3.2.2.2. Process, time constraints and resources. Almost all respondents indicated that the work on a cross-border assessment has to start very early in order for the results to be available in time for national decision making. Some respondents even indicated that they may prefer to start the national procedure along with the cross-border assessments due to limited timelines.

It was also mentioned that resources and time are needed to achieve consensus between countries, for example on which comparators/studies/outcomes should be included. In addition, it was estimated that broadening the

scope to meet multiple demands could lead to additional pressure on the already limited time and resources available.

Finally, it was mentioned that it may cost extra resources to have staff that can write reports in English.

3.3. Critical success factors for cross-border assessments

A total of 14 critical success factors for the cross-border assessments were identified by the respondents. They are listed in Table 3.

The success factors in general refer to either political aspects, building trust between countries, the quality of the assessments or the process and its management. It was clear from the interviews that a high quality assessment is expected which should be adaptable to the national situation. The report should be transparent, have a clear structure and be both comprehensive and succinct. In addition it was mentioned that there should be agreement on methodology, but also enough room to meet demands of individual countries.

Respondents emphasised that differences in methodological approaches may be solved by identifying them at an early stage. For instance, it was mentioned that the potential challenge of different comparators can be solved by including multiple (indirect) comparisons in the assessment. Further, it was emphasised that a significant part of the scoping-process should be done long before the marketing authorisation date, e.g. identifying stakeholders to be consulted, choice of comparators, outcomes and the population. It was also emphasised that in this case, the risk that an application for regulatory registration could be rejected or withdrawn should be taken into account.

Several respondents indicated that a cross-border report should be produced by at least two countries, acting as author and co-author. Further, to enhance the quality, it was suggested that a number of additional reviewers would be desirable, ranging from a couple to all countries that agreed to use the assessment. It was indicated that in order to make the project a success, the steady cooperation of at least five to seven countries is probably needed.

3.3.1. Added value to national reports and other important factors

Most respondents saw some opportunities for adding useful elements to their national reports by using a cross-border assessment. The following were mentioned: a standardised literature search, extensive background

Table 3

Critical success factors for cross-border assessments.

• Willingness to cooperate and high degree of commitment by different countries

• Voluntary use of the assessment

• Investment in personal contacts for building trust between participating countries

• Competent, independent, qualified assessment teams in the participating countries

• Transparent process and output

• Clear and comprehensive structure of the report

• High quality of the assessment which should be transferable to the national situation

• Common understanding and agreement on methodology, but with enough room to meet demands of individual countries as well

• Efficient project management at the European level

• Timely communication of relevant information about the planned assessment to national organisations so they plan their national assessment accordingly

• Early agreement on scope and participating countries

• Need for a clear and effective topic selection process that is relevant for the majority of countries

• Report production by an authoring country, a co-authoring country and involvement of several countries in reviewing the assessment

• Cooperation with stakeholders with involvement of

• manufacturers for submitting a dossier

• manufacturers, doctors and patients for scoping and commenting on the draft report

information, multiple comparisons, information about coverage decisions/recommendations in other countries, a refinement of categories of added therapeutic value, organisational and ethical aspects and modelling (e.g. indirect comparisons). One respondent also indicated that it would be desirable to have information on the prices and (hopefully) risk-sharing schemes in other countries, although this was also mentioned to be unlikely due to confidentiality issues. Further it was mentioned that a re-evaluation after three years would be appreciated.

The respondents mentioned that the length of the report is of less importance, although preferably it should be comprehensive but succinct. Some recommended that the technical data could be included as an appendix.

4. Discussion

Whereas several studies have analysed and discussed differences and similarities between countries in criteria, methods or outcomes of coverage assessments of drugs [2,16-20], the potential for harmonisation/collaboration between countries has only been studied by a few [3,4,14,21]. Kleijnen et al. [4] studied the similarities and differences in the assessment of relative effectiveness in 29 jurisdictions. The authors concluded that some important methodological aspects of REA are approached in a similar way in many jurisdictions, indicating that collaboration on assessments may be feasible. This study adds to the evidence base by identifying actual barriers and success factors for collaboration between countries in the field of REA of drugs.

Our study found that there is willingness to cooperate in the development and standardisation of methods for relative effectiveness in Europe and that it is expected to lead to an increase of quality and expertise throughout Europe. However, the desired extent of the collaboration varies between countries. Potential barriers identified mainly relate to methodology, resources and challenges with implementation in the national process (e.g. legal restrictions). The most critical success factors for production of cross-border assessments were the continuous cooperation of competent partners, and the quality and timely availability of the assessment.

The results of this study confirm some of the findings of Hutton et al. [3].The results confirm that harmonisation of HTA across jurisdictions should not, and cannot, aim to produce a single decision on coverage and utilisation of a technology. However, harmonisation and/or collaboration are more likely on the evaluation of clinical evidence.

Moreover, the results confirm the conclusion from Huic et al. [14] that the smaller/middle-sized countries and also countries that have less developed HTA systems show the highest willingness to conduct cross-border assessments. This may relate to the fact that these countries will profit most from efficiency gains from cross-border assessments. In countries with a well embedded and legalised structure it may be more difficult to adopt methods used in other counties or change the process compared with countries with a less well-developed system. Further, the willingness of an individual country to collaborate will be highly dependent on whether or not it can use the resulting report in the national setting (whether the report fulfils the specific needs and demands). For example, different comparators were included by Germany and England in their assessment of ticagrelor for the treatment of acute coronary syndrome as legal requirements stipulate that in Germany the comparator has to be licensed for the indication, whereas in England also 'off-label' comparators are accepted [22].

At this time point, there is some doubt whether a cross-border assessment may lead to a decrease in the time to market of new drugs in Europe as a cross-border assessment would not replace the subsequent national process. The assessment has to be integrated in the national process along with other evidence (such as on cost-effectiveness), legal considerations and social values. Further streamlining of national processes and cross-border assessments could reduce the total time needed. Naturally, a shorter time to the market would be of benefit to manufacturers and patients. However, it is not clear yet whether this can be established in the short run. A potential benefit for manufacturers that is more realistic in the next years may be a reduction of resources because of less duplication of information. Manufacturers spend considerable resources in preparing jurisdiction-specific submission files. Therefore, the use of the (same) REA part of the submission file in several jurisdictions may lead to efficiency gains.

Our study shows that some issues need to be worked out in more detail in order to increase European collaboration in the field of REAs of new drugs. Therefore it is essential that the following recommendations should be taken into account for future cross-border assessment:

o The cross-border assessments should not limit the time available for the national assessments and decision making. In order for the cross-border assessment to be available in a timely fashion, the scoping should start in a very early stage, preferably before the marketing authorisation, including identification of relevant stakeholders that should be included in the process. The risk that an application for marketing authorisation could be rejected or withdrawn should be accounted for. o The selection of the topics (drugs to be assessed) and the countries participating in the cross-border assessment should be transparent. o The scoping process should identify the needs of the different countries, especially regarding multiple comparators, direct and indirect comparisons, the use of both intermediate and final endpoints (patient-relevant and clinically significant endpoints), and the analysis of relevant subgroups. The goal should be to offer an optimum number of analyses to meet the needs of as many countries as possible, but with careful consideration of relevance, workload and time constraints. o Each country should be able to select that part of the assessment which is relevant to its national context and it should be easy to tailor those parts towards the national needs. Therefore a common understanding and agreement on methods needs to be developed with sufficient room for variation to meet as many individual demands as possible. The structure ofthe report should be clear and comprehensive and the use of the assessment should be voluntary.

o For producing a cross-border report, by definition at least two countries must be involved, acting as author and co-author. But to raise the quality, a number of additional reviewers is desirable, ranging from a couple to all countries that agreed to use the assessment. In order to make cross-border assessments a success in the long run, the steady cooperation of at least five to seven countries is probably desirable. Efficient project management at the European level is needed. o Optimal transparency of decision criteria is needed to explain potential different outcomes of the coverage process in the individual countries. It should be actively communicated that the legal frameworks and the purchasing power of EU countries vary considerably and that therefore different decisions in the member states are possible.

Our study has some limitations. This study included a limited set of HTA organisations as it was not possible to interview all European organisations within the available timeframe. Therefore the identified potential barriers and success factors may not be exhaustive. However, five out of eight organisations were from large countries. As argued before, large countries may be more reluctant towards collaboration resulting in a critical panel for the study.

Therefore, the results of this study may present a relatively critical attitude compared with the European average, but it is also likely that the majority of the potential barriers are identified. In addition, it was assumed that based on the selection of organisations sufficient diversity in opinions was guaranteed for this study. Finally, the differences in the terminology used in different countries may have affected the accuracy of some data. Further research that would include a wider set of HTA organisations from small, middle-sized and Central/Eastern European countries would be helpful for confirming the current findings.

In conclusion, there is willingness to cooperate in the development and standardisation of methods for REAs in Europe. However the desired extent of the collaboration varies between countries. In the near future it can be expected that cross-border assessments will meet in particular the needs of smaller/middle-sized European countries and also European countries with less developed HTA systems as the potential efficiency/quality gains are the highest for these countries. Therefore, national implementation of cross-border assessments is especially likely in these countries in the coming years. Further adaptation of the process and methods is required for optimal collaboration. Once more experience is gained with cross-border assessments, and successes become more evident, efficiency/quality gains may also be likely for some larger countries with well established processes.

Conflict of interest

None of the authors have declared a (potential) conflict of interest.

Acknowledgements

This work was produced by EUnetHTA Joint Action 1 which was co-funded by a grant (no. 20092302) from the European Union and the participating HTA organisations.

Appendix A. Supplementary data

Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/ j.healthpol.2015.01.018.

References

[1] Robinson JC, Howell S. Specialty pharmaceuticals: policy initiatives to improve assessment, pricing, prescription, and use. Health Affairs 2014;33(0ctober(10)):1745-50.

[2] Sorenson C, Drummond M, Kanavos P. Ensuring value for money in healthcare: the role of health technology assessment in the European Union Observatory studies series, no. 11. Copenhagen, Denmark: World Health Organisation Regional Office for Europe; 2008, ISBN 9789289071833.

[3] Hutton J, Trueman P, Facey K. Harmonization of evidence requirements for health technology assessment in reimbursement decision making. International Journal of Technology Assessment in Health Care 2008;24(4):511-7.

[4] Kleijnen S, George E, Goulden S, d'Andon A, Vitre P, Osiftska B, et al. Relative effectiveness assessment of pharmaceuticals: similarities and differences in 29 jurisdictions. Value in Health 2012;15(September-0ctober (6)):954-60.

[5] European Commission Enterprise and Industry. High level pharmaceutical forum 2005-2008. Conclusions and recommendations.

European Commission; 2008, ISBN 978-92-79-09863-5. Available from http://ec.europa.eu/enterprise/sectors/healthcare/files/docs/ pharmaforum-finaLconclusions-brochure.en.pdf [accessed

05.05.14].

[6] Kleijnen S, Pasternack I, Van de Casteele M, Rossi B, Cangini A, Di Bidino R, et al. Standardised reporting for rapid relative effectiveness assessments of pharmaceuticals. International Journal of Technology Assessment in Health Care 2015;30, http://dx.doi.org/10.1017/S0266462314000609.

[7] Lampe K, Mäkelä M, Garrido MV, Anttila H, Autti-Rämö I, Hicks NJ, et al. The HTA core model: a novel method for producing and reporting health technology assessments. International Journal of Technology Assessment in Health Care 2009;25(Suppl. 2):9-20.

[8] EUnetHTA WP5 Joint Action 1. Methodological guidelines for rapid REA of pharmaceuticals; March 2013. Available from: http://www. eunethta.eu/eunethta-guidelines [accessed 28.03.14].

[9] EUnetHTA Work Package 5 Joint Action 1. Pazopanib for the treatment of advanced renal cell carcinoma. Pilot assessment using the draft HTA Core Model for Rapid Relative Effectiveness Assessment model; December 2012. Available from: http://www.eunethta. eu/sites/5026.fedimbo.belgium.be/files/WP5%20JA1%20Pilot%20 Pazopanib%20Report+Appendix.pdf [accessed 28.03.14].

[10] EUnetHTA WP5 Joint Action 2 Strand A. Zostavax for the prevention ofherpes zoster and postherpetic neuralgia; September 2013. Available from: http://www.eunethta.eu/sites/5026.fedimbo.belgium. be/files/Zostavax_main%20report%20including%20appendices_ 20130922.pdf [accessed 28.03.14].

[11] EUnetHTA WP5 Joint Action 2 Strand A. Canagliflozin for the treatment of type 2 diabetes mellitus; February 2014. Available from: http://www.eunethta.eu/sites/5026.fedimbo.belgium.be/files/WP5_ SA-2_canagliflozin_for_the_treatment_of_diabetes_mellitus.pdf [accessed 28.03.14].

[12] Hailey D, Marshall D, Sampietro-Colom L, Rico R, Granados A, Asua J, et al. International collaboration in health technology assessment: a study of technologies used in management of osteoporosis. Health Policy 1998;43:233-41.

[13] Estrada MD, Asplund K, Barrington R, Faisst K, Kahan JP, Kjonniksen I, et al. European joint assessments and coordination of findings and

resources International Journal of Technology Assessment in Health Care 2002;8:321-60.

[14] Huic M, Nachtnebel A, Zechmeister I, Pasternak I, Wild C. collaboration in health technology assessment (EUnetHTA joint action, 2010-2012): four case studies. International Journal of Technology Assessment in Health Care 2013;29(July (3)):323-30.

[15] Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care 2007;19(December (6)):349-57.

[16] Clement FM, Harris A, Li JJ, Yong K, Lee KM. Using effectiveness and cost-effectiveness to make drug coverage decisions: a comparison of Britain, Australia, and Canada. The Journal of the American Medical Association 2009;302(0ctober(13)):1437-43.

[17] Levy AR, Mitton C, Johnston KM, Harrigan B, Briggs AH. International comparison of comparative effectiveness research in five jurisdictions: insights forthe US. Pharmacoeconomics 2010;28(10):813-30.

[18] Franken M, Nilsson F, Sandmann F, de Boer A, Koopmanschap M. Unravelling drug reimbursement outcomes: a comparative study of the role of pharmacoeconomic evidence in Dutch and Swedish reimbursement decision making. Pharmacoeconomics 2013;31(September (9)):781-97.

[19] Spinner DS, Birt J, Walter JW, Bowman L, MauskopfJ, Drummond MF, et al. Do different clinical evidence bases lead to discordant health-technology assessment decisions? An in-depth case series across three jurisdictions. ClinicoEconomics and Outcomes Research 2013;5:69-85.

[20] Paris V, Belloni A. Value in pharmaceutical pricing, OECD health working papers, no. 63. OECD; 2013. Available from: http://dx.doi.org/10.1787/5k43jc9v6knx-en [accessed 15.01.15].

[21] Kristensen FB1, Gerhardus A. Health technology assessments: what do differing conclusions tell us? British Medica Journal 2010;341(0ctober):c5236.

[22] George E, Sidhu R, Kaiser T, Schüler YB, Wieseler B. A comparison of the NICE and IQWIG/G-BA assessments of ticagrelor for ACS. Poster presented at ISPOR, Berlin; November 2012. Available at: http://www.ispor.org/congresses/Berlin1112/casestudyabstracts. aspx [accessed 05.01.15].