Scholarly article on topic 'Interrupted time series analysis in drug utilization research is increasing: systematic review and recommendations'

Interrupted time series analysis in drug utilization research is increasing: systematic review and recommendations Academic research paper on "Clinical medicine"

Share paper
Academic journal
Journal of Clinical Epidemiology
OECD Field of science
{ARIMA / "Drug utilization" / Pharmacoepidemiology / Review / "Segmented regression" / "Time series"}

Abstract of research paper on Clinical medicine, author of scientific article — Racquel Jandoc, Andrea M. Burden, Muhammad Mamdani, Linda E. Lévesque, Suzanne M. Cadarette

Abstract Objectives To describe the use and reporting of interrupted time series methods in drug utilization research. Study Design and Setting We completed a systematic search of MEDLINE, Web of Science, and reference lists to identify English language articles through to December 2013 that used interrupted time series methods in drug utilization research. We tabulated the number of studies by publication year and summarized methodological detail. Results We identified 220 eligible empirical applications since 1984. Only 17 (8%) were published before 2000, and 90 (41%) were published since 2010. Segmented regression was the most commonly applied interrupted time series method (67%). Most studies assessed drug policy changes (51%, n = 112); 22% (n = 48) examined the impact of new evidence, 18% (n = 39) examined safety advisories, and 16% (n = 35) examined quality improvement interventions. Autocorrelation was considered in 66% of studies, 31% reported adjusting for seasonality, and 15% accounted for nonstationarity. Conclusion Use of interrupted time series methods in drug utilization research has increased, particularly in recent years. Despite methodological recommendations, there is large variation in reporting of analytic methods. Developing methodological and reporting standards for interrupted time series analysis is important to improve its application in drug utilization research, and we provide recommendations for consideration.

Academic research paper on topic "Interrupted time series analysis in drug utilization research is increasing: systematic review and recommendations"

Accepted Manuscript

Interrupted Time Series Analysis in Drug Utilization Research is Increasing: Systematic Review and Recommendations

Racquel Jandoc, Andrea M. Burden, Muhammad Mamdani, Linda E. Levesque, Suzanne M. Cadarette

PII: S0895-4356(15)00123-7

DOI: 10.1016/j.jclinepi.2014.12.018

Reference: JCE 8839

To appear in: Journal of Clinical Epidemiology

Received Date: 16 April 2014 Revised Date: 28 November 2014 Accepted Date: 24 December 2014

Please cite this article as: Jandoc R, Burden AM, Mamdani M, Levesque LE, Cadarette SM, Interrupted Time Series Analysis in Drug Utilization Research is Increasing: Systematic Review and Recommendations, Journal of Clinical Epidemiology (2015), doi: 10.1016/j.jclinepi.2014.12.018.

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

1 Interrupted Time Series Analysis in Drug Utilization Research is Increasing: Systematic

2 Review and Recommendations

3 Racquel Jandoc1, Andrea M. Burden1, Muhammad Mamdani1'2'3, Linda E. Lévesque4, Suzanne

4 M. Cadarette1*

5 1 Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto ON, Canada

6 2 Applied Health Research Center, La Ka Shing Knowledge Institute, St. Michael's Hospital,

7 Toronto ON, Canada

8 3 Institute of Health Policy, Management, and Evaluation, University of Toronto, Toronto ON,

9 Canada

10 4 Department of Public Health Sciences, Queen's University, Kingston ON, Canada

14 Correspondence to: Suzanne M. Cadarette, Leslie L. Dan Pharmacy Building, University of

15 Toronto, 144 College Street, Toronto, Ontario, M5S 3M2 Canada. Tel: 416-978-2993, Fax: 41616 978-8511, E-mail:

1 Support and Financial Disclosure Declaration

3 Conflicts of interest:

4 Racquel Jandoc - none

5 Andrea M. Burden - none

6 Muhammad Mamdani - none related to this work

7 Linda E. Levesque - none

8 Suzanne M. Cadarette - none

10 Financial support:

11 Project support - Ontario Ministry of Research and Innovation (ER09-06-043)

12 Racquel Jandoc - CIHR Training Program in Bridging Scientific Domains for Drug Safety and

13 Effectiveness

14 Suzanne M. Cadarette - CIHR New Investigator Award (MSH-95364)

1 Abstract

2 Objective: To describe the use and reporting of interrupted time series methods in drug

3 utilization research.

4 Study Design and Setting: We completed a systematic search of MEDLINE®, Web of

5 Science®, and reference lists to identify English-language articles through to December 2013 that

6 employed interrupted time series methods in drug utilization research. We tabulated the number

7 of studies by publication year, and summarized methodological detail.

8 Results: We identified 220 eligible empirical applications since 1984. Only 17 (8%) were

9 published before 2000, and 90 (41%) were published since 2010. Segmented regression was the

10 most commonly applied interrupted time series method (67%). Most studies assessed drug policy

11 changes (51%, n=112); 22% (n=48) examined the impact of new evidence, 18% (n=39)

12 examined safety advisories, and 16% (n=35) examined quality improvement interventions.

13 Autocorrelation was considered in 66% of studies, 31% reported adjusting for seasonality, and

14 15% accounted for non-stationarity.

15 Conclusion: Use of interrupted time series methods in drug utilization research has increased,

16 particularly in recent years. Despite methodological recommendations, there is large variation in

17 reporting of analytic methods. Developing methodological and reporting standards for

18 interrupted time series analysis is important to improve its application in drug utilization

19 research, and we provide recommendations for consideration.

20 Key Words: ARIMA, drug utilization, pharmacoepidemiology, review, segmented regression,

21 time series

22 Running title: Systematic review of interrupted time series methods

23 Word counts: Abstract - 199, Main text - 2887 (including acknowledgements)

What is new?

• There has been an increase in the application of interrupted time series analysis in drug utilization research, particularly in recent years.

• We identified large variation in methodological considerations reported in empirical applications.

• Developing methodological and reporting standards for interrupted time series analysis is important to improve its application in drug utilization research, and we provide recommendations for consideration, Table 2.

2 1. Introduction

3 Interrupted time series analysis is the strongest and most commonly used quasi-experimental

4 design to assess the impact of an intervention when a randomized controlled trial is not feasible

5 [1-6]. This method has been applied in a variety of disciplines, and was first introduced to the

6 field of health services research in 1981 to evaluate the impact of regionalized perinatal care [7].

7 Interrupted time series methods use aggregate data collected over equally spaced time intervals

8 before and after an intervention, with the key assumption that data trends prior to the intervention

9 can be extrapolated to predict trends had the intervention not occurred [3]. Routinely maintained

10 pharmacy and medical databases provide rich data sources to apply interrupted time series

11 methods [3].

12 Several methodological issues need to be considered when completing an interrupted

13 time series analysis. First, given the serial nature of the design; autocorrelation, non-stationarity,

14 and seasonality need to be considered [3, 8]. Autocorrelation refers to the serial dependence of

15 outcome measure error terms. For example, prescription patterns closer to each other may be

16 more similar than those further apart [3, 5, 6]. The presence of autocorrelation can be assessed

1 using the Ljung-Box chi-square statistic [9] or Durbin-Watson statistic [3, 5, 6, 10], and

2 corrected for if necessary. Non-stationary data exhibit an underlying trend that is unrelated to the

3 intervention. For example, the use of a drug commonly increases once it enters the market [8].

4 Non-stationary can be tested using the augmented Dickey-Fuller test [11]. Seasonality represents

5 regular seasonal fluctuations in the outcome, for example, use of medications to treat influenza.

6 When present, terms for seasonality (e.g. months) should be included in the model [3, 6]. Failing

7 to account for autocorrelation, non-stationarity, and seasonality may lead to biased results.

8 The number of data points available for analysis, and the number of observations within

9 each data point, are important when using interrupted time series analysis. Although there is no

10 gold standard, it is generally agreed that more data points and observations are better. Depending

11 on the minimum effect size and the amount of variation, a minimum of nine data points pre-,

12 post-, and when applicable, between interventions [4, 12]; and at least 100 observations per data

13 point is encouraged [3]. A larger number of observations in each data point provides more stable

14 estimates and thus reduces the variability and outliers within a time series analysis. Data point

15 outliers that are explainable, such as a sudden peak in drug dispensing in anticipation of a drug

16 restriction policy, can be controlled for using an indicator term [3]. Outliers that result from

17 random variation can be treated as regular data points [3]. A larger number of data points also

18 permits more stable estimates for forecasting pre-intervention trends had the intervention not

19 occurred. In general however, caution should be used when forecasting beyond the data points

20 observed in interrupted time series analyses. Another caveat when conducting interrupted time

21 series analysis relates to possible outcome measure ceiling or floor effects. For example, when

22 studying the impact of an intervention in improving the proportion of patients treated with a

23 drug, the outcome has a natural ceiling of 100%, and thus depending on the initial level of

1 measurement, minimal change in the outcome may be observed [13]. Authors must consider

2 ceiling and floor effects when designing their study and interpreting results.

3 A clear intervention time point helps to identify pre- and post-intervention data points,

4 yet if intervention effects are gradual or delayed, then a lag period may be considered [3].

5 Lagged intervention effects can be accounted for by excluding the lag period from the analysis,

6 modeling the lag period as a separate segment in the time series [3], or using a ramp function in

7 autoregressive integrated moving average (ARIMA) models [14]. Here, graphical figures

8 displaying the results of interrupted time series analysis are particularly useful. Even without

9 statistical output, figures allow readers to visually examine baseline trends, the time point at

10 which the intervention occurred, and the impact of the intervention [3, 5]. All interrupted time

11 series studies should therefore include graphical display to facilitate interpretation of study

12 results.

13 The main threat to validity in interrupted time series analysis relates to time-varying

14 confounding, such as changes in outcome coding, co-interventions, or changes in the population

15 under study [3-5, 15]. These threats need to be considered at the individual study level and

16 require intimate knowledge of the data and healthcare utilization trends. The use of a comparison

17 outcome in the same population, or a comparison group using the same outcome in a group not

18 exposed to the intervention, helps to alleviate concerns related to time-varying confounding [3,

19 5]. Indeed, an advantage of interrupted time series analysis is the ease in stratifying results by

20 different groups [5].

21 Interrupted time series analysis has been applied in a variety of disciplines, however its use

22 to study the impact of healthcare interventions on drug utilization has not been well described.

1 The purpose of our study was to describe the use and reporting of interrupted time series

2 methods in drug utilization research.

3 2. Methods

4 We completed a systematic MEDLINE® keyword and Web of Science® citation search to

5 identify all English-language articles that employed interrupted time series methods to study

6 drug utilization in humans. Empirical applications that examined the impact of interventions at

7 the population level, including: drug policy changes, new evidence in the form of guideline

8 changes or major publications, quality improvement interventions, and government or media

9 safety advisories; on prescription drug utilization were eligible. We defined drug utilization as

10 the number or proportion of: drug(s) dispensed, patients dispensed a drug, or patients meeting an

11 adherence target. Systematic reviews, methodological contributions, letters to the editor, and

12 conference abstracts were excluded since the focus was on use and reporting of empirical

13 applications. We also excluded single institution studies so we could focus on population-based

14 interventions that may be more generalizable.

15 We first searched MEDLINE from inception (1946) to December 2013 with keyword

16 terms related to time series analysis and drug utilization (Appendix A). We then used Web of

17 Science® to perform a citation search of methodological papers identified in the keyword search

18 [3, 5, 16], and a commonly cited paper [7]. Finally, we manually searched reference lists from all

19 methodological contributions [B.1-10], review papers [B.11-14], and eligible empirical

20 applications [B.15-204] identified in the keyword and citation searches to identify additional

21 empirical applications. Two authors (RJ and AMB) independently completed each search and

22 reviewed articles for eligibility. Discrepancies were resolved through discussion with a third

23 author (SMC). A proportional Venn diagram was created to illustrate the number of empirical

1 applications identified by each search strategy. The number of empirical applications was then

2 plotted by publication year.

3 We abstracted the following characteristics for each application: intervention(s) of interest,

4 primary data source, and methodological detail (time intervals, outcome measure, interrupted

5 time series methods used, and methodological considerations reported). As described above,

6 there are several methodological considerations in interrupted time series analysis, and we have

7 taken care to abstract whether authors reported these; however, we were unable to evaluate

8 aspects that would require access to each study's raw data. Thus methodological considerations

9 abstracted included reporting of: autocorrelation, non-stationarity, and seasonality; use of a

10 comparison group; clearly defined time points; number of pre- and post-intervention points;

11 outliers; forecasting; and absolute and/or relative changes with confidence intervals or standard

12 errors. Additional considerations abstracted included the use of lag periods, sensitivity analysis,

13 and graphical figures to display results. One author (RJ) abstracted all data and a second author

14 (AMB) verified all abstracted data. All methodological considerations were summarized using

15 descriptive statistics.

17 3. Results

18 Of 1917 unique articles identified, 10 were methodological contributions [B.1-10], 4 were

19 review papers [B.11-14], and 220 were eligible empirical applications [B.15-234] (Fig. 1,

20 Appendix B).

21 *Insert Fig. 1*

22 Each search strategy proved important, with 52 (24%) empirical applications identified

23 solely by the keyword search, 33 (15%) identified solely by the citation search, 30 (14%)

24 identified solely by the reference list search; and only 35 (16%) identified by all three search

1 strategies (Fig. 2). Most segmented regression papers (92 of 133, 69%) were identified by the

2 citation search, whereas most ARIMA papers (26 of 30, 84%) were identified by the keyword

3 search. One eligible paper that did not appear in our original search was identified by a reviewer

4 during the peer-review process.

5 *InsertFig. 2*

6 The first empirical application was published in 1984, yet relatively few (n=17, 8%) were

7 published before the year 2000 (Fig. 3). Since 2000, use has increased with an average of 15

8 (SD=8.2) applications published per year. Forty-one percent (n=90) were published in the last

9 four years, with a high of 31 articles published in 2013.

10 *Insert Fig. 3*

11 Table 1 summarizes the characteristics of the 220 empirical applications identified, of

12 which 92% utilized administrative pharmacy databases. Policy changes were the most common

13 interventions evaluated (51%), followed by new evidence (22%), safety advisories (18%), and

14 quality improvement interventions (16%). Seventy-one percent examined prescriptions

15 dispensed (22% number, 35%, proportion and 14% standard dose) as the primary outcome

16 measure, and 29% used the number or proportion of patients dispensed the drug of interest or

17 meeting an adherence target. Most applications examined drug utilization over monthly (76%) or

18 quarterly (14%) intervals. Of the 200 papers (91%) reporting detailed methods, segmented

19 regression (67%), ARIMA models (16%), and linear regression (11%) were the most commonly

20 applied analyses. Other analytical methods included generalized estimating equations, logistic,

21 nonlinear, and Poisson models. Fifty percent (n=67) of papers using segmented regression

22 applied a linear model.

1 Of all empirical studies, 146 (66%) reported testing for autocorrelation (77% of ARIMA

2 papers, and 73% of segmented regression papers), 68 (31%) reported adjusting for seasonality

3 (52% of ARIMA, and 29% of segmented regression), and 32 (15%) reported testing for non-

4 stationarity (65% of ARIMA, 9% of segmented regression). One-third (35%) of all empirical

5 studies reported the use of a comparison group, 70% reported absolute and/or relative impacts

6 with confidence intervals or standard errors, and 28% reported including lag periods in their

7 models. Most articles (85%) clearly reported the intervention time point(s) of interest, and 84%

8 of studies included a graph, yet only 39% reported the number of pre- and post-intervention data

9 points included in their analysis (range: 3 to 72 data points). One-fifth (21%) of applications

10 conducted a sensitivity analysis.

11 *Insert Table 1*

13 4. Discussion

14 We examined the application and reporting of interrupted time series analysis methods in drug

15 utilization research. Use of interrupted time series analysis has increased since the year 2000, a

16 finding noted in other recent reviews of innovative methods in pharmacoepidemiology [17, 18].

17 The most common interrupted time series methods were segmented regression (67%), ARIMA

18 models (16%), and linear regression (11%). When executing time series models, several

19 methodological aspects are important and may impact the validity of the model. We identified

20 that the majority of eligible articles employing ARIMA models addressed autocorrelation (77%),

21 non-stationarity (65%), seasonality (52%). Since ARIMA models inherently account for

22 autocorrelation, non-stationarity, and seasonality [19]; it is possible that authors may have

23 chosen not to report these considerations. In contrast, segmented regression models do not

24 intrinsically account for autocorrelation, non-stationarity, and seasonality; and thus it is

1 imperative to consider each [3]. However, only 73% of segmented regression studies reported

2 testing for autocorrelation, 29% reported adjustment for seasonality, and only 9% reported

3 consideration of non-stationarity.

4 Over 80% of interrupted time series analyses cited a clearly defined intervention time

5 point, and used figures to graphically display results; however, other methodological issues were

6 poorly reported. Explicit reporting of all methodological considerations may improve awareness

7 of their importance as well as the interpretation of interrupted time series studies. Therefore,

8 based on prior suggestions [3-6, 12, 20], we recommend the following be reported in all

9 interrupted time series applications: 1) autocorrelation, non-stationarity, and seasonality

10 considerations; 2) intervention time point(s) and lag periods; 3) the number of data points pre-,

11 post- and between-intervention(s); 4) specific statistical regression methods and the

12 appropriateness of a linear model when applied; and 5) absolute and/or relative changes from

13 baseline (intervention impact) with statistical significance. We also recommend that all

14 interrupted time series studies: 1) use graphical display with clearly defined time point(s) to

15 present results; 2) comment on: the minimum number of observations per data point, data

16 variability, ceiling or floor effects; and 3) consider the use of a comparison group. Authors are

17 also encouraged to discuss possible data or co-intervention confounding issues and provide a

18 rationale if no comparison group was considered. These recommendations are summarized in

19 Table 2, and build from the Strengthening the Reporting of Observational Studies in

20 Epidemiology (STROBE) statement [21].

21 * Insert Table 2 *

22 Our systematic review is subject to some limitations related to our literature search

23 strategy and inclusion criteria. First, we recognize that the lack of Medical Subject Headings

1 (MeSH terms) and standardized terminology to describe the interrupted time series design may

2 have resulted in some missed applications. Although segmented regression was first introduced

3 to healthcare research in 1981 [7], and a seminal method paper was published in 2002 [3], we

4 found that many papers did not use the term "segmented regression" to describe their analysis.

5 Therefore particular attention to each study's statistical analysis was required during data

6 abstraction to determine the type of interrupted time series method used. Second, our search was

7 limited in ability to identify applications that are not indexed in either of the databases used

8 (MEDLINE®, Web of Science®). Indeed, during the peer-review of our manuscript, a blind

9 reviewer identified one eligible article [B.109] that is not indexed in the databases used and

10 therefore was not identified in our original search.

11 Third, by restricting inclusion to studies that examined prescription drug utilization

12 defined by the number or proportion of prescription drugs dispensed or patients dispensed a

13 drug; we will have missed interrupted time series analyses with different drug outcomes, such as

14 illicit drug use, drug sales, or drug market share [22, 23]. Fourth, we acknowledge that studies

15 examining single institution interventions (n=59, Appendix C) were excluded so we could focus

16 on population-based interventions that may be more generalizable. Despite potentially missing

17 some applications, we feel that our results that identify an increase in the number of applications

18 in recent years, and conclusions of the general trends of methods and underreporting of statistical

19 considerations, would still hold. Indeed, 66% of the 59 single institution studies used segmented

20 regression analysis, similar to our finding that 67% of studies included in our review employed

21 segmented regression analysis.

22 Finally, we acknowledge that our review is limited by what authors have reported or

23 presented in their studies, which may not reflect the true methodological rigour of each study.

1 Therefore, the large variation in reporting that we identified may not indicate inappropriate use

2 of interrupted time series methods, but rather a need for reporting standards to facilitate quality

3 reporting, application, and interpretation of interrupted time series results.

4 A major strength of our systematic review is the use of multiple search strategies to

5 identify articles. Our keyword and citation search yielded 190 eligible articles (86% overall),

6 with only 31% identified in both. We attribute this small overlap to a lack of MeSH terms for

7 time series analysis. The additional reference list search of eligible articles identified another 30

8 (14% overall) eligible applications not captured in our prior searches. This observation

9 corroborates the importance of using multiple search strategies as identified in prior reviews of

10 new statistical methods [17, 18]. We encourage future systematic reviews to use a similar

11 proportional Venn diagram to clarify search strategy yield.

12 In summary, we identified an increase in the number of applications of interrupted time

13 series analysis to examine interventions in drug utilization, particularly in recent years. When

14 properly executed, interrupted time series analysis is a valuable method to evaluate the success,

15 failure, or unintended consequences of healthcare interventions on drug utilization [24].

16 However, there is large variation in the reporting of interrupted time series methods. Developing

17 methodological and reporting standards for interrupted time series analysis is important to

18 improve its application in drug utilization research. We provide a summary table of

19 methodological and reporting recommendations for researchers to consider when completing

20 interrupted time series analyses.

1 Acknowledgements

2 The authors declare no conflict of interest.

3 This research was supported by an Ontario Ministry of Research and Innovation Early

4 Research Award held by Dr. Suzanne Cadarette. Dr. Cadarette was supported by a Canadian

5 Institutes of Health Research (CIHR) New Investigator Award (MSH-95364). Racquel Jandoc

6 received support from the CIHR Training Program in Bridging Scientific Domains for Drug

7 Safety and Effectiveness. Authors thank Mina Tadrous, PharmD, MS, University of Toronto,

8 Gina Matesic, MA, MLIS, MEd, University of Toronto, Giulia Consiglio, BSc, University of

9 Toronto, Joanna Bielecki, Research Librarian, University of Toronto, and our anonymous

10 reviewers for insightful discussions or comments.

11 This research was presented at the International Conference on Pharmacoepidemiology

12 and Therapeutic Risk Management (ICPE) in Montreal QC, Canada, August 2013, the Canadian

13 Association for Population Therapeutics (CAPT) Annual Conference in Toronto ON, Canada,

14 November 2013, and the Canadian Association for Health Services and Policy Research

15 (CAHSPR) Annual Conference in Toronto ON, Canada, May 2014. Participation at ICPE was

16 supported by an ICPE Travel Scholarship, participation at CAPT was supported by a CAPT

17 Student Bursary and the Leslie Dan Faculty of Pharmacy Student Experience Fund, and

18 participation at CAHSPR was supported by a University of Toronto School of Graduate Studies

19 Conference Grant.

1 References

2 [1] Grimshaw J, Campbell M, Eccles M, Steen N. Experimental and quasi-experimental designs

3 for evaluating guideline implementation strategies. Fam Pract 2000;17 Suppl 1:S11-6.

4 [2] Harris AD, McGregor JC, Perencevich EN, Furuno JP, Zhu JK, Peterson DE, et al. The use

5 and interpretation of quasi-experimental studies in medical informatics. J Am Med Inform Assoc

6 2006;13:16-23.

7 [3] Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of

8 interrupted time series studies in medication use research. J Clin Pharm Ther 2002;27:299-309.

9 [4] Briesacher BA, Soumerai SB, Zhang F, Toh S, Andrade SE, Wagner JL, et al. A critical

10 review of methods to evaluate the impact of FDA regulatory actions. Pharmacoepidemiol Drug

11 Saf 2013;22:986-94.

12 [5] Penfold RB, Zhang F. Use of Interrupted Time Series Analysis in Evaluating Health Care

13 Quality Improvements. Acad Pediatr 2013;13:S3 8-S44.

14 [6] Cochrane Effective Practice and Organisation of Care Review Group. Interrupted time series

15 analyses. 2013

16 <http://epoc. cochrane. org/sites/epoc. cochrane. org/files/uploads/21 %20Interrupted%20time%20s

17 eries%20analyses%202013%2008%2012.pdf> Accessed 1 September 2014.

18 [7] Gillings D, Makuc D, Siegel E. Analysis of interrupted time series mortality trends: An

19 example to evaluate regionalized perinatal care. Am J Public Health 1981;71:38-46.

20 [8] Lagarde M. How to do (or not to do) ... Assessing the impact of a policy change with routine

21 longitudinal data. Health Policy Plan 2012;27:76-83.

22 [9] Ljung GM, Box GEP. On a measure of lack of fit in time series models. Biometrika

23 1987;67:297-303.

24 [10] Durbin J, Watson G. Testing for serial correlation in least squares regression I. Biometrika

25 1950;37:409-28.

26 [11] Dickey DA, Fuller WA. Distribution of the estimators for autoregressive time series with a

27 unit root. J Am Stat Assoc 1979:427-31.

28 [12] Zhang F, Wagner AK, Ross-Degnan D. Simulation-based power calculation for designing

29 interrupted time series analyses of health policy interventions. J Clin Epidemiol 2011;64:1252-

30 61.

31 [13] Gottman J, Rushe R. The analysis of change: Issues, fallacies, and new ideas. Journal of

32 consulting and clinical psychology 1993;61:907-10.

33 [14] Jirovec MM. Time-series analysis in nursing research: ARIMA modeling. Nursing research

34 1986;35:315-9.

1 [15] Cochrane Effective Practice and Organisation of Care Review Group. Suggested risk of bias

2 criteria for EPOC reviews: Risk of bias for interrupted time series (ITS) studies. 2013

3 <

4 s%20criteria%20for%20EP0C%20reviews.pdf> Accessed 1 September 2014.

5 [16] Zhang F, Wagner AK, Soumerai SB, Ross-Degnan D. Methods for estimating confidence

6 intervals in interrupted time series analyses of health interventions. J Clin Epidemiol

7 2009;62:143-8.

8 [17] Consiglio GP, Burden AM, Maclure M, McCarthy L, Cadarette SM. Case-crossover study

9 design in pharmacoepidemiology: systematic review and recommendations. Pharmacoepidemiol

10 Drug Saf 2013;22:1146-53.

11 [18] Tadrous M, Gagne JJ, Sturmer T, Cadarette SM. Disease risk score as a confounder

12 summary method: systematic review and recommendations. Pharmacoepidemiol Drug Saf

13 2013;22:122-9.

14 [19] Box GEP, Jenkins GM. Time series analysis: forecasting and control. San Francisco:

15 Holden-Day; 1976.

16 [20] Mandell MB. Obtaining interval estimates of policy impacts from interrupted time-series.

17 Eval Rev 1987;11:631-59.

18 [21] von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The

19 Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement:

20 guidelines for reporting observational studies. J Clin Epidemiol 2008;61:344-9.

21 [22] Garabedian LF, Ross-Degnan D, Ratanawijitrasin S, Stephens P, Wagner AK. Impact of

22 universal health insurance coverage in Thailand on sales and market share of medicines for non-

23 communicable diseases: an interrupted time series study. BMJ open 2012;2.

24 [23] Gorman DM, Charles Huber J, Jr. Do medical cannabis laws encourage cannabis use? Int J

25 Drug Policy 2007;18:160-7.

26 [24] Huesch MD, Ostbye T, Ong MK. Measuring the effect of policy interventions at the

27 population level: Some methodological concerns. Health Econ 2012;21:1234-49.

28 [25] Green CJ, Maclure M, Fortin PM, Ramsay CR, Aaserud M, Bardal S. Pharmaceutical

29 policies: Effects of restrictions on reimbursement. Cochrane Database Syst Rev

30 2010;4:CD008654.

1 Table 1. Characteristics of interrupted time series applications in drug utilization, n=220a



Drug policy changes 112 50.9

Co-payments or cost-sharing 33 15.0

New drug or drug withdrawal 15 6.8

Prior authorization 21 9.5

Reimbursement changes 43 19.5

New evidence 48 21.8

Guideline changes 15 6.8

Major publications 33 15.0

Quality improvement interventions 35 15.9

Educational 18 8.2

Other quality improvement 17 7.7

Safety advisories 39 17.7 Data source

Administrative data 203 92.3

Medical charts 10 4.5

Survey data 7 3.2 Time point intervals

Monthly 166 75.5

Quarterly 31 14.1

Other (e.g. annually, bi-annually, bi-weekly, weekly) 23 10.5 Primary outcome measure

Patient 64 29.1

Number 15 6.8

Proportion 49 22.3

Prescriptions 157 71.4

Number 49 22.3

Proportion 77 35.0

Standardized dose (e.g. daily defined dose) 31 14.1

Interrupted time series method reportedb 200 90.9

ARIMA models 31 15.5

Reported invervention function (e.g. point, ramp, step)c 18 58.1

Linear regression 22 11.0

Segmented regression 134 67.0

Linear modelsd 67 50.0

Other modelsd (e.g. GEE, logistic, nonlinear) 9 6.7

Not specifiedd 58 43.3

Other regression (e.g. GEE, logistic, nonlinear, Poisson) 14 7.0 Statistical considerations reported

Autocorrelation 146 66.4

Comparison group 77 35.0

Confidence intervals or standard errors reported with estimates 153 69.5

Forecasting using pre-intervention trends 64 29.1

Graphical figures to display results 184 83.6

Lag periods 61 27.7

Non-stationarity 32 14.5

Number of pre- and post-intervention data points 86 39.1

Outliers 17 7.7

Seasonality 68 30.9

Sensitivity analysis 45 20.5

Time point clearly defined 186 84.5

1 ARIMA: Autoregressive integrated moving average; GEE: Generalized estimating equation

2 aSome characteristics are not mutually exclusive, thus proportions add to greater than 100%

3 bFor papers reporting detailed time series methods only, n=200

4 cFor papers reporting ARIMA only, n=31

5 dForpapers conducting segmented regression only, n=134

Table 2. Methodological and reporting recommendations for interrupted time series studiesa Item No


Title and abstract


Background/ rationale


1 Indicate the study design (interrupted time series) in the title or abstract

Provide background regarding the intervention and setting under investigation to support the study rationale and methods

(a) State specific objectives and any pre-specified hypotheses

(b) Distinguish between primary and secondary objectives




Data sources and measurement


4 Define the intervention time point(s) used in the analysis

5 (a) List eligibility criteria and methods of selection

(b) Define any subgroups

(c) Consider including a comparison group not exposed to the intervention as a secondary group of participants

6 (a) List data source(s)

(b) Comment on data completeness, validity, and changes in data coverage over time

Statistical methods



(b) (c)

(f) (g)

(b) (c)

Define all variables Outcome variable(s) Descriptive and stratifying variable(s) Comment on change in variable coding over time Consider including details of variable coding in supplemental material, e.g., Appendix or research website

Report all statistical methods

• Study time intervals, e.g., monthly, quarterly

• Regression model, e.g., ARIMA, linear, segmented

o For ARIMA models, indicate the intervention function, e.g.,

point, ramp, or step o Indicate the appropriateness of linear model(s) when applied

• Number of pre-, post- and between intervention data points Define the study period and number of pre-intervention data points used in forecasting

Indicate how autocorrelation, non-stationarity, and seasonality were tested and handled

Consider a lag period if intervention effects are gradual or delayed Define and distinguish between primary and secondary or sensitivity analyses

Consider use of comparison outcome(s) and/or population(s) not exposed to the intervention(s) as secondary analyses Report statistical software used for analysis

Report the number of individuals and/or observations in each group analyzed

Consider use of a flow diagram

Describe characteristics and indicate any missing data

Item No


Outcome data

Main results

Other analyses


Key results



10 (a) Report the number of outcomes examined over the study period

(b) Report the average, minimum, and maximum number of outcomes across time intervals

(c) Report on data variability

(d) Comment on outliers, and floor or ceiling effects where relevant

11 (a) Present results using a graphical display with intervention time

point(s) clearly defined

(b) Consider including forecasted results graphically

(c) Report absolute and/or relative change(s) and their significance, e.g. clinical or policy and statistical

12 Report additional results (secondary and sensitivity analyses) in the manuscript, Appendix, or research website

13 Summarize key results with reference to study objectives

14 (a) Provide context related to possible confounding

• Discuss relevant co-interventions that occurred during the study period

• Comment on the stability of participant characteristics over time

• Comment on the stability of outcome coding over time

(b) Discuss results of comparison analyses, or provide a rationale if no comparison group was considered

15 (a) Discuss limitations of the study

(b) Comment on data variability and appropriateness of the number of data points

(c) Comment on ceiling or floor effects, and outliers where relevant

(d) Discuss direction and magnitude of any potential bias

16 Provide overall interpretation of results considering objectives, limitations, results from similar studies, and other relevant evidence


Other information

Funding 17 List funding source(s) and role of funders

References 18 Reference methodological papers that support statistical methods _employed_

1 ARRIMA: Autoregressive integrated moving average; GEE: Generalized estimating equation

2 aItems adapted from the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement[2l]

10 11 12

20 21 22 23

Fig. 1. Flow diagram of systematic search results. MEDLINE was used for the keyword search

using search terms (Appendix A), and Web of Science was used for the citation search [3, 5, 7, 16]

Reviewer, n=1

9 Fig. 2. Proportional Venn diagram of search result yield of empirical applications by search

10 strategy, n=220. The size of each circle is proportional to the relative number of articles

11 identified. MEDLINE® keyword search terms are listed in Appendix A, and four papers were

12 used in the Web of Science citation search [3, 5, 7, 16]. The reference search included all

13 eligible empirical applications (n=190), methods (n=10), and reviews (n=4) identified by the

14 keyword and citation searches. One article not indexed in MEDLINE® or Web of Science®

15 databases was identified by a reviewer during the peer-review process.

-«tmtDis~oocnoT-(NcO'«tmtDis~oocn 0000000000000)010)010)010)010)01




Fig. 3. Number of interrupted time series empirical applications in drug utilization research, by publication year, n=220.

1 Appendices

2 Appendix A: List of keyword terms used for MEDLINE® search

3 1. Time

4 2. Time trend$.tw.

5 3. Trend analys$.tw.

6 4. Time series analys$.tw.

7 5. Forecast model$.tw.

8 6. Intervention analys$.tw.

9 7.

10 8. Medicat$.mp.

11 9. Pharmaceutic?. mp.

12 10. Prescri$.mp.

13 11. Pharmacoepidemiolog$.mp.

14 12. Dispens$.mp.

15 13. Drug

16 14. Or/1-6

17 15. Or/7-13

18 16. 14 and 15

19 17. Limit 16 to English language

20 Search terms 8 and 9 were adopted from Green et al.

1 Appendix B: List of identified references (methodological papers, reviews, and empirical

2 applications)

4 [B.1] Fretheim A, Soumerai SB, Zhang F, Oxman AD, Ross-Degnan D. Interrupted time-series

5 analysis yielded an effect estimate concordant with the cluster-randomized controlled trial result.

6 J Clin Epidemiol 2013;66:883-7.

7 [B.2] Kong MY, Cambon A, Smith MJ. Extended Logistic Regression Model for Studies with

8 Interrupted Events, Seasonal Trend, and Serial Correlation. Comm Stat Theory Methods

9 2012;41:3528-43.

10 [B.3] Linden A, Adams JL. Applying a propensity score-based weighting model to interrupted

11 time series data: improving causal inference in programme evaluation. J Eval Clin Pract

12 2011;17:1231-8.

13 [B.4] Mandell MB. Obtaining interval estimates of policy impacts from interrupted time-series.

14 Eval Rev 1987;11:631-59.

15 [B.5] Pape UJ, Millett C, Lee JT, Car J, Majeed A. Disentangling secular trends and policy

16 impacts in health studies: use of interrupted time series analysis. J R Soc Med 2013;106:124-9.

17 [B.6] Penfold RB, Zhang F. Use of Interrupted Time Series Analysis in Evaluating Health Care

18 Quality Improvements. Acad Pediatr 2013;13:S3 8-S44.

19 [B.7] Schneeweiss S, Maclure M, Soumerai SB, Walker AM, Glynn RJ. Quasi-experimental

20 longitudinal designs to evaluate drug benefit policy changes with low policy compliance. J Clin

21 Epidemiol 2002;55:833-41.

22 [B.8] Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of

23 interrupted time series studies in medication use research. J Clin Pharm Ther 2002;27:299-309.

24 [B.9] Zhang F, Wagner AK, Ross-Degnan D. Simulation-based power calculation for designing

25 interrupted time series analyses of health policy interventions. J Clin Epidemiol 2011;64:1252-

26 61.

27 [B.10] Zhang F, Wagner AK, Soumerai SB, Ross-Degnan D. Methods for estimating confidence

28 intervals in interrupted time series analyses of health interventions. J Clin Epidemiol

29 2009;62:143-8.

30 [B.11] Lagarde M. How to do (or not to do) ... Assessing the impact of a policy change with

31 routine longitudinal data. Health Policy Plan 2012;27:76-83.

1 [B.12] Mandryk JA, Mackson JM, Horn FE, Wutzke SE, Badcock CA, Hyndman RJ, et al.

2 Measuring change in prescription drug utilization in Australia. Pharmacoepidemiol Drug Saf

3 2006;15:477-84.

4 [B.13] Schneeweiss S, Maclure M, Walker AM, Grootendorst P, Soumerai SB. On the

5 evaluation of drug benefits policy changes with longitudinal claims data: the policy maker's

6 versus the clinician's perspective. Health Policy 2001;55:97-109.

7 [B.14] Zuckerman IH, Lee E, Wutoh AK, Xue ZY, Stuart B. Application of regression-

8 discontinuity analysis in pharmaceutical health services research. Health Serv Res 2006;41:550-

10 [B.15] Acheampong P, Cooper G, Khazaeli B, Lupton DJ, White S, May MT, et al. Effects of

11 MHRA drug safety advice on time trends in prescribing volume and indices of clinical toxicity

12 for quinine. Br J Clin Pharmacol 2013;76:973-9.

13 [B.16] Adams AS, Zhang F, LeCates RF, Graves AJ, Ross-Degnan D, Gilden D, et al. Prior

14 authorization for antidepressants in Medicaid: effects among disabled dual enrollees. Arch Intern

15 Med 2009;169:750-6.

16 [B.17] Adnan W, Zaharan NL, Bennett K, Wall CA. Trends in co-prescribing of angiotensin

17 converting enzyme inhibitors and angiotensin receptor blockers in Ireland. Br J Clin Pharmacol

18 2011;71:458-66.

19 [B.18] Alahdab OG, Crealey G, Scott MG, Mairs J, McElnay JC. Product standardisation as a

20 tool to control prescribing costs - a case study of alginate liquid preparations. Int J Pharm Pract

21 2013;21:73-81.

22 [B.19] Andersson K, Petzold MG, Sonesson C, Lonnroth K, Carlsten A. Policy changes in the

23 pharmaceutical reimbursement schedule affect drug expenditures? Interrupted time series

24 analysis of cost, volume and cost per volume trends in Sweden 1986-2002. Health Policy

25 2006;79:231-43.

26 [B.20] Atchessi N, Ridde V, Haddad S. Combining user fees exemption with training and

27 supervision helps to maintain the quality of drug prescriptions in Burkina Faso. Health Policy

28 Plan 2013;28:606-15.

29 [B.21] Austin PC, Mamdani MM. Impact of the pravastatin or atorvastatin evaluation and

30 infection therapy-thrombolysis in myocardial infarction 22/Reversal of Atherosclerosis with

31 Aggressive Lipid Lowering trials on trends in intensive versus moderate statin therapy in

32 Ontario, Canada. Circulation 2005;112:1296-300.

1 [B.22] Austin PC, Mamdani MM, Tu K. The impact of the Women's Health Initiative study on

2 incident clonidine use in Ontario, Canada. J Clin Pharmacol 2004;11 :e191-4.

3 [B.23] Bambauer KZ, Adams AS, Zhang F, Minkoff N, Grande A, Weisblatt R, et al. Physician

4 alerts to increase antidepressant adherence - Fax or fiction? Arch Intern Med 2006;166:498-504.

5 [B.24] Barron TI, Bennett K, Feely J. Impact of high dose statin trials on hospital prescribers.

6 Eur J Clin Pharmacol 2007;63:65-72.

7 [B.25] Bennie M, Bishop I, Godman B, Barbui C, Raschi E, Campbell S, et al. Are specific

8 initiatives required to enhance prescribing of generic atypical antipsychotics in Scotland?:

9 International implications. Int J Clin Pract 2013;67:170-80.

10 [B.26] Bennie M, Bishop I, Godman B, Campbell S, Miranda J, Finlayson AE, et al. Are

11 prescribing initiatives readily transferable across classes: the case of generic losartan in

12 Scotland? Qual Prim Care 2013;21:7-15.

13 [B.27] Bergen H, Hawton K, Murphy E, Cooper J, Kapur N, Stalker C, et al. Trends in

14 prescribing and self-poisoning in relation to UK regulatory authority warnings against use of

15 SSRI antidepressants in under-18-year-olds. Br J Clin Pharmacol 2009;68:618-29.

17 [B.28] Bijlsma MJ, Hak E, Bos J, De Jong-van den Berg LTW, Janssen F. Assessing the effect

18 of a guideline change on drug use prevalence by including the birth cohort dimension: the case of

19 benzodiazepines. Pharmacoepidemiol Drug Saf 2013;22:933-41.

20 [B.29] Blais L, Boucher JM, Couture J, Rahme E, LeLorier J. Impact of a cost-sharing drug

21 insurance plan on drug utilization among older people. J Am Geriatr Soc 2001;49:410-4.

22 [B.30] Blais L, Couture J, Rahme E, LeLorier J. Impact of a cost sharing drug insurance plan on

23 drug utilization among individuals receiving social assistance. Health Policy 2003;64:163-72.

24 [B.31] Breen CL, Degenhardt LJ, Bruno RB, Roxburgh AD, Jenkinson R. The effects of

25 restricting publicly subsidised temazepam capsules on benzodiazepine use among injecting drug

26 users in Australia. Med J Aust 2004;181:300-4.

27 [B.32] Briesacher BA, Soumerai SB, Field TS, Fouayzi H, Gurwitz JH. Nursing home residents

28 and enrollment in Medicare Part D. J Am Geriatr Soc 2009;57:1902-7.

29 [B.33] Briesacher BA, Soumerai SB, Field TS, Fouayzi H, Gurwitz JH. Medicare part D's

30 exclusion of benzodiazepines and fracture risk in nursing homes. Arch Intern Med

31 2010;170:693-8.

1 [B.34] Brufsky JW, Ross-Degnan D, Calabrese D, Gao X, Soumerai SB. Shifting physician

2 prescribing to a preferred histamine-2-receptor antagonist. Effects of a multifactorial intervention

3 in a mixed-model health maintenance organization. Med Care 1998;36:321-32.

4 [B.35] Brunt ME, Murray MD, Hui SL, Kesterson J, Perkins AJ, Tierney WM. Mass media

5 release of medical research results: an analysis of antihypertensive drug prescribing in the

6 aftermath of the calcium channel blocker scare of March 1995. J Gen Intern Med 2003;18:84-94.

7 [B.36] Bucsics A, Godman B, Burkhardt T, Schmitzer M, Malmstrom RE. Influence of lifting

8 prescribing restrictions for losartan on subsequent sartan utilization patterns in Austria:

9 implications for other countries. Expert Rev Pharmacoecon Outcomes Res 2012;12:809-19.

10 [B.37] Campbell NRC, McAlister FA, Brant R, Levine M, Drouin D, Feldman R, et al. Temporal

11 trends in antihypertensive drug prescriptions in Canada before and after introduction of the

12 Canadian Hypertension Education Program. J Hypertens 2003;21:1591-7.

13 [B.38] Campbell NRC, Tu K, Brant R, Duong-Hua M, McAlister FA, Canadian Hypertension

14 Education Program Outcomes Research Task F. The impact of the Canadian Hypertension

15 Education Program on antihypertensive prescribing trends. Hypertension 2006;47:22-8.

16 [B.39] Carracedo-Martinez E, Pia-Morandeira A, Figueiras A. Impact of a health safety warning

17 and prior authorisation on the use of piroxicam: a time-series study. Pharmacoepidemiol Drug

18 Saf 2012;21:281-4.

19 [B.40] Chen H, Nwangwu A, Aparasu R, Essien E, Sun S, Lee K. The impact of Medicare Part

20 D on psychotropic utilization and financial burden for community-based seniors. Psychatr Serv

21 2008;59:1191-7.

22 [B.41] Choudhry NK, Fischer MA, Avorn J, Schneeweiss S, Solomon DH, Berman C, et al. At

23 Pitney Bowes, Value-Based Insurance Design Cut Copayments And Increased Drug Adherence.

24 Health Aff 2010;29:1995-2001.

25 [B.42] Choudhry NK, Fischer MA, Avorn JL, Lee JL, Schneeweiss S, Solomon DH, et al. The

26 impact of reducing cardiovascular medication copayments on health spending and resource

27 utilization. J Am Coll Cardiol 2012;60:1817-24.

28 [B.43] Choudhry NK, Zagorski B, Avorn J, Levin R, Sykora K, Laupacis A, et al. Comparison of

29 the impact of the Atrial Fibrillation Follow-Up Investigation of Rhythm Management trial on

30 prescribing patterns: a time-series analysis. Ann Pharmacother 2008;42:1563-72.

31 [B.44] Clarke G, Dickerson J, Gullion CM, DeBar LL. Trends in youth antidepressant dispensing

32 and refill limits, 2000 through 2009. J Child Adolesc Psychopharmacol 2012;22:11-20.

1 [B.45] Cohen A, Rabbani A, Shah N, Alexander GC. Changes in glitazone use among office-

2 based physicians in the U.S., 2003-2009. Diabetes Care 2010;33:823-5.

3 [B.46] Damiani G, Federico B, Silvestrini G, Bianchi CBNA, Anselmi A, Iodice L, et al. Impact

4 of regional copayment policy on selective serotonin reuptake inhibitor (SSRI) consumption and

5 expenditure in Italy. Eur J Clin Pharmacol 2013;69:957-63.

6 [B.47] Delaney JAC, McClelland RL, Furberg CD, Cooper R, Shea S, Burke G, et al. Time

7 trends in the use of anti-hypertensive medications: results from the Multi-Ethnic Study of

8 Atherosclerosis. Pharmacoepidemiol Drug Saf 2009;18:826-32.

9 [B.48] Delate T, Mager DE, Sheth J, Motheral BR. Clinical and financial outcomes associated

10 with a proton pump inhibitor prior-authorization program in a Medicaid population. Am J Manag

11 Care 2005;11:29-36.

12 [B.49] Dormuth CR, Morrow RL, Carney G. Trends in health care utilization in British

13 Columbia following public coverage for tiotropium. Value Health 2011;14:600-6.

14 [B.50] Dorsey ER, Rabbani A, Gallagher SA, Conti RM, Alexander GC. Impact of FDA black

15 box advisory on antipsychotic medication use. Arch Intern Med 2010;170:96-103.

16 [B.51] Dowell D, Tian LH, Stover JA, Donnelly JA, Martins S, Erbelding EJ, et al. Changes in

17 fluoroquinolone use for gonorrhea following publication of revised treatment guidelines. Am J

18 Public Health 2012;102:148-55.

19 [B.52] Du DT, Zhou EH, Goldsmith J, Nardinelli C, Hammad TA. Atomoxetine use during a

20 period of FDA actions. Med Care 2012;50:987-92.

21 [B.53] Ehrenpreis ED, Deepak P, Sifuentes H, Devi R, Du H, Leikin JB. The metoclopramide

22 black box warning for tardive dyskinesia: Effect on clinical practice, adverse event reporting,

23 and prescription drug lawsuits. Am J Gastroenterol 2013;108:866-72.

24 [B.54] Farley JF, Dusetzina SB. Medicaid prescription drug utilization and expenditures

25 following Part D. J Health Care Poor Underserved 2010;21:715-28.

26 [B.55] Feldstein AC, Vollmer WM, Smith DH, Petrik A, Schneider J, Glauber H, et al. An

27 outreach program improved osteoporosis management after a fracture. J Am Geriatr Soc

28 2007;55:1464-9.

29 [B.56] Fischer MA, Schneeweiss S, Avorn J, Solomon DH. Medicaid prior-authorization

30 programs and the use of cyclooxygenase-2 inhibitors. N Engl J Med 2004;351:2187-94.

1 [B.57] Fisher JE, Zhang Y, Sketris I, Johnston G, Burge F. The effect of an educational

2 intervention on meperidine use in Nova Scotia, Canada: a time series analysis.

3 Pharmacoepidemiol Drug Saf 2012;21:177-83.

4 [B.58] Foulon V, Svala A, Koskinen H, Chen TF, Saastamoinen LK, Bell JS. Impact of

5 regulatory safety warnings on the use of antidepressants among children and adolescents in

6 Finland. J Child Adolesc Psychopharmacol 2010;20:145-50.

7 [B.59] Fretheim A, Havelsrud K, MacLennan G, Kristoffersen DT, Oxman AD. The effects of

8 mandatory prescribing of thiazides for newly treated, uncomplicated hypertension: interrupted

9 time-series analysis. PLoS Med 2007;4:e232.

10 [B.60] Gadzhanova S, Roughead EE, Loukas K, Vajda J. Improvement in metformin and insulin

11 utilisation in the Australian veteran population associated with quality use of medicines

12 intervention programs. Pharmacoepidemiol Drug Saf 2011;20:359-65.

13 [B.61] Gadzhanova SV, Roughead EE, Bartlett MJ. Improving cardiovascular disease

14 management in Australia: NPS MedicineWise. Med J Aust 2013;199:192-5.

15 [B.62] Gamble J-M, Johnson JA, Majumdar SR, McAlister FA, Simpson SH, Eurich DT.

16 Evaluating the introduction of a computerized prior-authorization system on the completeness of

17 drug exposure data. Pharmacoepidemiol Drug Saf 2013;22:551-5.

18 [B.63] Garg RK, Fulton-Kehoe D, Turner JA, Bauer AM, Wickizer T, Sullivan MD, et al.

19 Changes in Opioid Prescribing for Washington Workers' Compensation Claimants After

20 Implementation of an Opioid Dosing Guideline for Chronic Noncancer Pain: 2004 to 2010. J

21 Pain 2013;14:1620-8.

22 [B.64] Gilson AM, Fishman SM, Wilsey BL, Casamalhuapa C, Baxi H. Time series analysis of

23 California's prescription monitoring program: impact on prescribing and multiple provider

24 episodes. J Pain 2012;13:103-11.

25 [B.65] Godman B, De Bruyn K, Miranda J, Raschi E, Bennie M, Barbui C, et al. Generic

26 atypical antipsychotic drugs in Belgium: their influence and implications. J Comp Eff Res

27 2013;2:551-61.

28 [B.66] Godman B, Persson M, Miranda J, Skiold P, Wettermark B, Barbui C, et al. Changes in

29 the utilization of venlafaxine after the introduction of generics in Sweden. Appl Health Econ

30 Health Policy 2013;11:383-93.

31 [B.67] Godman B, Wettermark B, Miranda J, Bennie M, Martin A, Malmstrom RE. Influence of

32 multiple initiatives in Sweden to enhance ARB prescribing efficiency following generic losartan;

33 findings and implications for other countries. Int J Clin Pract 2013;67:853-62.

1 [B.68] Gomes T, Juurlink DN, Moore I, Maguire JL, Mamdani MM. The impact of federal

2 warnings on publically funded desmopressin utilization among children in Ontario. J Pediatr

3 Urol 2012;8:249-53.

4 [B.69] Graham JJ, Timmis A, Cooper J, Ramdany S, Deaner A, Ranjadayalan K, et al. Impact of

5 the National Service Framework for coronary heart disease on treatment and outcome of patients

6 with acute coronary syndromes. Heart 2006;92:301-6.

7 [B.70] Guo JJ, Curkendall S, Jones JK, Fife D, Goehring E, She D. Impact of cisapride label

8 changes on codispensing of contraindicated medications. Pharmacoepidemiol Drug Saf

9 2003;12:295-301.

10 [B.71] Guthrie B, Clark SA, Reynish EL, McCowan C, Morales DR. Differential impact of two

11 risk communications on antipsychotic prescribing to people with dementia in Scotland:

12 Segmented regression time series analysis 2001-2011. PLoS One 2013;8:e68976.

13 [B.72] Hagen BF, Armstrong-Esther C, Quail P, Williams RJ, Norton P, Le Navenec C-L, et al.

14 Neuroleptic and benzodiazepine use in long-term care in urban and rural Alberta: characteristics

15 and results of an education intervention to ensure appropriate use. Int Psychogeriatr

16 2005;17:631-52.

17 [B.73] Hartung DM, Carlson MJ, Kraemer DF, Haxby DG, Ketchum KL, Greenlick MR. Impact

18 of a Medicaid copayment policy on prescription drug and health services utilization in a fee-for-

19 service medicaid population. Med Care 2008;46:565-72.

20 [B.74] Hartung DM, Touchette DR, Ketchum KL, Haxby DG, Goldberg BW. Effects of a prior-

21 authorization policy for celecoxib on medical service and prescription drug use in a managed

22 care Medicaid population. Clin Ther 2004;26:1518-32.

23 [B.75] Hashim S, Gomes T, Juurlink D, Hellings C, Mamdani M. The rise and fall of the

24 thiazolidinediones: impact of clinical evidence publication and formulary change on the

25 prescription incidence of thiazolidinediones. J Popul Ther Clin Pharmacol 2013;20:e238-42.

26 [B.76] Hawton K, Bergen H, Simkin S, Brock A, Griffiths C, Romeri E, et al. Effect of

27 withdrawal of co-proxamol on prescribing and deaths from drug poisoning in England and

28 Wales: time series analysis. BMJ 2009;338:b2270.

29 [B.77] Hawton K, Bergen H, Simkin S, Wells C, Kapur N, Gunnell D. Six-year follow-up of

30 impact of co-proxamol withdrawal in England and Wales on prescribing and deaths: time-series

31 study. PLoS Med 2012;9:e1001213.

32 [B.78] Hazlet TK, Blough DK. Health services utilization with reference drug pricing of

33 histamine(2) receptor antagonists in British Columbia elderly. Med Care 2002;40:640-9.

1 [B.79] Hemmelgarn BR, Zhang JG, Manns BJ, James MT, Quinn RR, Ravani P, et al.

2 Nephrology Visits and Health Care Resource Use Before and After Reporting Estimated

3 Glomerular Filtration Rate. JAMA 2010;303:1151-8.

4 [B.80] Hesse U, Godman B, Petzold M, Martin A, Malmstrom RE. Impact of delisting ARBs,

5 apart from losartan, on ARB utilisation patterns in Denmark: implications for other countries.

6 Appl Health Econ Health Policy 2013;11:677-85.

7 [B.81] Hillman JJ, Zuckerman IH, Lee E. The impact of the Women's Health Initiative on

8 hormone replacement therapy in a Medicaid program. J Womens Health 2004;13:986-92.

9 [B.82] Horn FE, Mandryk JA, Mackson JM, Wutzke SE, Weekes LM, Hyndman RJ.

10 Measurement of changes in antihypertensive drug utilisation following primary care educational

11 interventions. Pharmacoepidemiol Drug Saf 2007;16:297-308.

12 [B.83] Huang SH, Hsu CN, Yu SH, Cham TM. Impact of drug price adjustments on utilization

13 of and expenditures on angiotensin-converting enzyme inhibitors and angiotensin receptor

14 blockers in Taiwan. BMC Public Health 2012;12:288.

15 [B.84] Huybrechts KF, Palmsten K, Mogun H, Kowal M, Avorn J, Setoguchi-Iwata S, et al.

16 National trends in antidepressant medication treatment among publicly insured pregnant women.

17 Gen Hosp Psychiatry 2013;35:265-71.

18 [B.85] Hynd A, Roughead EE, Preen DB, Glover J, Bulsara M, Semmens J. The impact of co-

19 payment increases on dispensings of government-subsidised medicines in Australia.

20 Pharmacoepidemiol Drug Saf 2008;17:1091-9.

21 [B.86] Hynd A, Roughead EE, Preen DB, Glover J, Bulsara M, Semmens J. Increased patient co-

22 payments and changes in PBS-subsidised prescription medicines dispensed in Western Australia.

23 Aust N Z J Public Health 2009;33:246-52.

24 [B.87] Jackevicius CA, Tu JV, Demers V, Melo M, Cox J, Rinfret S, et al. Cardiovascular

25 outcomes after a change in prescription policy for clopidogrel. N Engl J Med 2008;359:1802-10.

26 [B.88] Juurlink DN, Mamdani MM, Lee DS, Kopp A, Austin PC, Laupacis A, et al. Rates of

27 hyperkalemia after publication of the Randomized Aldactone Evaluation Study. N Engl J Med

28 2004;351:543-51.

29 [B.89] Kabir Z, Feely J, Bennett K. Primary care prescribing patterns in Ireland after the

30 publication of large hypertension trials. Br J Clin Pharmacol 2007;64:381-5.

31 [B.90] Kales HC, Zivin K, Kim HM, Valenstein M, Chiang C, Ignacio RV, et al. Trends in

32 antipsychotic use in dementia 1999-2007.[Erratum appears in Arch Gen Psychiatry. 2011

1 May;68(5):466 Note: Ignacio, Rosalindo [corrected to Ignacio, Rosalinda V]]. Arch Gen

2 Psychiatry 2011;68:190-7.

3 [B.91] Kephart G, Skedgel C, Sketris I, Grootendorst P, Hoar J. Effect of copayments on drug

4 use in the presence of annual payment limits. Am J Manag Care 2007;13:328-34.

5 [B.92] Kephart G, Sketris IS, Bowles SK, Richard ME, Cooke CA. Impact of a criteria-based

6 reimbursement policy on the use of respiratory drugs delivered by nebulizer and health care

7 services utilization in Nova Scotia, Canada. Pharmacotherapy 2005;25:1248-57.

8 [B.93] Koranek AM, Smith TL, Mican LM, Rascati KL. Impact of the CATIE trial on

9 antipsychotic prescribing patterns at a state psychiatric facility. Schizophr Res 2012;137:137-40.

10 [B.94] Kornfield R, Watson S, Higashi AS, Conti RM, Dusetzina SB, Garfield CF, et al. Effects

11 of FDA advisories on the pharmacologic treatment of ADHD, 2004-2008. Psychatr Serv

12 2013;64:339-46.

13 [B.95] Kurdyak PA, Juurlink DN, Mamdani MM. The effect of antidepressant warnings on

14 prescribing trends in Ontario, Canada. Am J Public Health 2007;97:750-4.

15 [B.96] Kurian BT, Ray WA, Arbogast PG, Fuchs DC, Dudley JA, Cooper WO. Effect of

16 regulatory warnings on antidepressant prescribing for children and adolescents. Arch Pediatr

17 Adolesc Med 2007;161:690-6.

18 [B.97] Lam NN, Jain AK, Hackam DG, Cuerden MS, Suri RS, Huo CY, et al. Results of a

19 randomized controlled trial on statin use in dialysis patients had no influence on statin

20 prescription. Kidney Int 2009;76:1172-9.

21 [B.98] Langley TE, Huang Y, Lewis S, McNeill A, Coleman T, Szatkowski L. Prescribing of

22 nicotine replacement therapy to adolescents in England. Addiction 2011;106:1513-9.

23 [B.99] Langley TE, Huang Y, McNeill A, Coleman T, Szatkowski L, Lewis S. Prescribing of

24 smoking cessation medication in England since the introduction of varenicline. Addiction

25 2011;106:1319-24.

26 [B.100] Langley TE, Szatkowski L, McNeill A, Coleman T, Lewis S. Prescribing of nicotine

27 replacement therapy to cardiovascular disease patients in England. Addiction 2012;107:1341-8.

28 [B.101] Law MR, Lu CY, Soumerai SB, Graves AJ, LeCates RF, Zhang F, et al. Impact of two

29 Medicaid prior-authorization policies on antihypertensive use and costs among Michigan and

30 Indiana residents dually enrolled in Medicaid and Medicare: results of a longitudinal, population-

31 based study. Clin Ther 2010;32:729-41.

1 [B.102] Lee E, Maneno MK, Wutoh AK, Zuckerman IH. Long-term effect of the Women's

2 Health Initiative study on antiosteoporosis medication prescribing. J Womens Health

3 2010;19:847-54.

4 [B.103] Lee E, Wutoh AK, Xue Z, Hillman JJ, Zuckerman IH. Osteoporosis management in a

5 Medicaid population after the Women's Health Initiative study. J Womens Health 2006;15:155-

7 [B.104] Lee I-H, Bloor K, Hewitt C, Maynard A. The effects of new pricing and copayment

8 schemes for pharmaceuticals in South Korea. Health Policy 2012;104:40-9.

9 [B.105] Libby AM, Brent DA, Morrato EH, Orton HD, Allen R, Valuck RJ. Decline in treatment

10 of pediatric depression after FDA advisory on risk of suicidality with SSRIs. Am J Psychiatry

11 2007;164:884-91.

12 [B.106] Libby AM, Orton HD, Valuck RJ. Persisting decline in depression treatment after FDA

13 warnings. Arch Gen Psychiatry 2009;66:633-9.

14 [B.107] Linton A, Bacon T, Peterson M. Proton-pump inhibitor utilization associated with the

15 change to nonpreferred formulary status for esomeprazole in the TRICARE formulary. J Manag

16 Care Pharm 2009;15:42-54.

17 [B.108] Lo-Ciganic W-H, Boudreau RM, Gray SL, Zgibor JC, Donohue JM, Perera S, et al.

18 Changes in cholesterol-lowering medications use over a decade in community-dwelling older

19 adults. Ann Pharmacother 2013;47:984-92.

20 [B.109] Lu CY, Ross-Degnan D, Stephens P, Liu B, Wagner AK. Changes in use of antidiabetic

21 medications following price regulations in China (1999-2009). J Pharm Health Serv Res

22 2013;4:3-11.

23 [B. 110] Lu CY, Law MR, Soumerai SB, Graves AJ, LeCates RF, Zhang F, et al. Impact of prior

24 authorization on the use and costs of lipid-lowering medications among Michigan and Indiana

25 dual enrollees in Medicaid and Medicare: results of a longitudinal, population-based study. Clin

26 Ther 2011;33:135-44.

27 [B.111] Lu CY, Soumerai SB, Ross-Degnan D, Zhang F, Adams AS. Unintended Impacts of a

28 Medicaid Prior Authorization Policy on Access to Medications for Bipolar Illness. Med Care

29 2010;48:4-9.

30 [B.112] MacBride-Stewart SP, Elton R, Walley T. Do quality incentives change prescribing

31 patterns in primary care? An observational study in Scotland. Fam Pract 2008;25:27-32.

1 [B.113] MacCara ME, Sketris IS, Comeau DG, Weerasinghe SD. Impact of a limited

2 fluoroquinolone reimbursement policy on antimicrobial prescription claims. Ann Pharmacother

3 2001;35:852-8.

4 [B.114] Maio V, Gagne JJ. Impact of ALLHAT publication on antihypertensive prescribing

5 patterns in Regione Emilia-Romagna, Italy. J Clin Pharm Ther 2010;35:55-61.

6 [B.115] Majumdar SR, McAlister FA, Soumerai SB. Synergy between publication and

7 promotion: comparing adoption of new evidence in Canada and the United States. Am J Med

8 2003;115:467-72.

9 [B.116] Mamdani M, McNeely D, Evans G, Hux J, Oh P, Forde N, et al. Impact of a

10 fluoroquinolone restriction policy in an elderly population. Am J Med 2007;120:893-900.

11 [B. 117] Mamdani M, Warren L, Kopp A, Paterson JM, Laupacis A, Bassett K, et al. Changes in

12 rates of upper gastrointestinal hemorrhage after the introduction of cyclooxygenase-2 inhibitors

13 in British Columbia and Ontario. CMAJ 2006;175:1535-8.

14 [B.118] Manns B, Laupland K, Tonelli M, Gao S, Hemmelgarn B. Evaluating the impact of a

15 novel restricted reimbursement policy for quinolone antibiotics: A time series analysis. BMC

16 Health Serv Res 2012;12.

17 [B.119] Marshall D, Gough J, Grootendorst P, Buitendyk M, Jaszewski B, Simonyi S, et al.

18 Impact of administrative restrictions on antibiotic use and expenditure in Ontario: time series

19 analysis. J Health Serv Res Policy 2006;11:13-20.

20 [B. 120] Marshall DA, Willison DJ, Grootendorst P, LeLorier J, Maclure M, Kulin NA, et al. The

21 effects of coxib formulary restrictions on analgesic use and cost: regional evidence from Canada.

22 Health Policy 2007;84:1-13.

23 [B.121] Martikainen JE, Hakkinen U, Enlund H. Adoption of new antiglaucoma drugs in

24 Finland: impact of changes in copayment. Clin Ther 2007;29:2468-76.

25 [B.122] Martin BC, McMillan JA. The impact of implementing a more restrictive prescription

26 limit on Medicaid recipients. Effects on cost, therapy, and out-of-pocket expenditures. Med Care

27 1996;34:686-701.

28 [B.123] Miller ME, Gengler DJ. Medicaid case management: Kentucky's Patient Access and

29 Care Program. Health Care Financ Rev 1993;15:55-69.

30 [B.124] Morales DR, Donnan PT, Daly F, Van Staa T, Sullivan FM. Impact of clinical trial

31 findings on Bell's palsy management in general practice in the UK 2001-2012: interrupted time

32 series regression analysis. BMJ Open 2013;3.

1 [B.125] Muller A, Baker JA. Evaluation of the Arkansas Medicaid primary care physician

2 management program. Health Care Financ Rev 1996;17:117-33.

3 [B.126] Musleh S, Kraus S, Bennett K, Zaharan NL. Irish Medicines Board safety warnings: do

4 they affect prescribing rates in primary care? Pharmacoepidemiol Drug Saf 2011;20:979-86.

5 [B.127] Nyasulu JCY, Muchiri E, Mazwi S, Ratshefola M. NIMART rollout to primary

6 healthcare facilities increases access to antiretrovirals in Johannesburg: An interrupted time

7 series analysis. S Afr Med J 2013;103:232-6.

8 [B.128] Okano GJ, Rascati KL, Wilson JP, Remund DD. A comparison of antihypertensive

9 medication utilization before and after guideline changes using the Department of Defense

10 Prescription Database. Ann Pharmacother 1999;33:548-53.

11 [B. 129] Olfson M, Marcus SC, Druss BG. Effects of Food and Drug Administration warnings on

12 antidepressant use in a national sample. Arch Gen Psychiatry 2008;65:94-101.

13 [B.130] Ong M, Catalano R, Hartig T. A time-series analysis of the effect of increased

14 copayments on the prescription of antidepressants, anxiolytics, and sedatives in Sweden from

15 1990 to 1999. Clin Ther 2003;25:1262-75.

16 [B.131] Ooba N, Yamaguchi T, Kubota K. The Impact in Japan of Regulatory Action on

17 Prescribing of Dopamine Receptor Agonists Analysis of a Claims Database between 2005 and

18 2008. Drug Saf 2011;34:329-38.

19 [B.132] Pamer CA, Hammad TA, Wu Y-T, Kaplan S, Rochester G, Governale L, et al. Changes

20 in US antidepressant and antipsychotic prescription patterns during a period of FDA actions.

21 Pharmacoepidemiol Drug Saf 2010;19:158-74.

22 [B.133] Pearson S-A, Soumerai S, Mah C, Zhang F, Simoni-Wastila L, Salzman C, et al. Racial

23 disparities in access after regulatory surveillance of benzodiazepines. Arch Intern Med

24 2006;166:572-9.

25 [B.134] Pettersson B, Hoffmann M, Wandell P, Levin LA. Utilization and costs of lipid

26 modifying therapies following health technology assessment for the new reimbursement scheme

27 in Sweden. Health Policy 2012;104:84-91.

28 [B.135] Piccinni C, Raschi E, Poluzzi E, Puccini A, Cars T, Wettermark B, et al. Trends in

29 antiarrhythmic drug use after marketing authorization of dronedarone: comparison between

30 Emilia Romagna (Italy) and Sweden. Eur J Clin Pharmacol 2013;69:715-20.

1 [B.136] Pichetti S, Sorasith C, Sermet C. Analysis of the impact of removing mucolytics and

2 expectorants from the list of reimbursable drugs on prescription rates: a time-series analysis for

3 France 1998-2010. Health Policy 2011;102:159-69.

4 [B.137] Piening S, Reber KC, Wieringa JE, Straus SMJM, de Graeff PA, Haaijer-Ruskamp FM,

5 et al. Impact of safety-related regulatory action on drug use in ambulatory care in the

6 Netherlands. Clin Pharmacol Ther 2012;91:838-45.

7 [B.138] Pinheiro SP, Kang EM, Kim CY, Governale LA, Zhou EH, Hammad TA. Concomitant

8 use of isotretinoin and contraceptives before and after iPledge in the United States.

9 Pharmacoepidemiol Drug Saf 2013;22:1251-7.

10 [B.139] Pluss-Suard C, Pannatier A, Ruffieux C, Kronenberg A, Muhlemann K, Zanetti G.

11 Changes in the Use of Broad-Spectrum Antibiotics after Cefepime Shortage: a Time Series

12 Analysis. Antimicrob Agents Chemother 2012;56:989-94.

13 [B.140] Polinski JM, Brookhart MA, Glynn RJ, Schneeweiss S. Medicare part D's impact on

14 antipsychotic drug use and costs among elderly patients without prior drug insurance. J Clin

15 Psychopharmacol 2012;32:3-10.

16 [B.141] Reiss SK, Ross-Degnan D, Zhang F, Soumerai SB, Zaslavsky AM, Wharam JF. Effect

17 of switching to a high-deductible health plan on use of chronic medications. Health Serv Res

18 2011;46:1382-401.

19 [B.142] Roblin DW, Platt R, Goodman MJ, Hsu J, Nelson WW, Smith DH, et al. Effect of

20 increased cost-sharing on oral hypoglycemic use in five managed care organizations - How much

21 is too much? Med Care 2005;43:951-9.

22 [B. 143] Ross JS, Jackevicius C, Krumholz HM, Ridgeway J, Montori VM, Alexander GC, et al.

23 State Medicaid programs did not make use of prior authorization to promote safer prescribing

24 after rosiglitazone warning. Health Aff 2012;31:188-98.

25 [B.144] Ross-Degnan D, Simoni-Wastila L, Brown JS, Gao XM, Mah C, Cosler LE, et al. A

26 controlled study of the effects of state surveillance on indicators of problematic and non-

27 problematic benzodiazepine use in a Medicaid population. International Journal of Psychiatry in

28 Medicine 2004;34:103-23.

29 [B.145] Ross-Degnan D, Soumerai SB, Fortess EE, Gurwitz JH. Examining product risk in

30 context. Market withdrawal of zomepirac as a case study. JAMA 1993;270:1937-42.

31 [B. 146] Roughead EE, Kalisch Ellett LM, Ramsay EN, Pratt NL, Barratt JD, LeBlanc VT, et al.

32 Bridging evidence-practice gaps: improving use of medicines in elderly Australian veterans.

33 BMC Health Serv Res 2013;13:514.

1 [B.147] Roughead EE, Ramsay E, Pratt N, Gilbert AL. NSAID Use in Individuals at Risk of

2 Renal Adverse Events An Observational Study to Investigate Trends in Australian Veterans.

3 Drug Saf 2008;31:997-1003.

4 [B.148] Roughead EE, Zhang F, Ross-Degnan D, Soumerai S. Differential effect of early or late

5 implementation of prior authorization policies on the use of Cox II inhibitors. Med Care

6 2006;44:378-82.

7 [B.149] Sanfelix-Gimeno G, Cervera-Casino P, Peiro S, Lopez-Valcarcel BG, Blazquez A,

8 Barbera T. Effectiveness of safety warnings in atypical antipsychotic drugs: an interrupted time-

9 series analysis in Spain. Drug Saf 2009;32:1075-87.

10 [B.150] Santa-Ana-Tellez Y, Mantel-Teeuwisse AK, Dreser A, Leufkens HGM, Wirtz VJ.

11 Impact of over-the-counter restrictions on antibiotic consumption in Brazil and Mexico. PLoS

12 ONE [Electronic Resource] 2013;8:e75550.

13 [B.151] Schneeweiss S, Maclure M, Dormuth CR, Glynn RJ, Canning C, Avorn J. A therapeutic

14 substitution policy for proton pump inhibitors: clinical and economic consequences. Clin

15 Pharmacol Ther 2006;79:379-88.

16 [B.152] Schneeweiss S, Maclure M, Soumerai SB. Prescription duration after drug copay

17 changes in older people: methodological aspects. J Am Geriatr Soc 2002;50:521-5.

18 [B.153] Schneeweiss S, Patrick AR, Maclure M, Dormuth CR, Glynn RJ. Adherence to beta-

19 blocker therapy under drug cost-sharing in patients with and without acute myocardial infarction.

20 American Journal of Managed Care 2007;13:445-52.

21 [B.154] Schneeweiss S, Patrick AR, Maclure M, Dormuth CR, Glynn RJ. Adherence to statin

22 therapy under drug cost sharing in patients with and without acute myocardial infarction - A

23 population-based natural experiment. Circulation 2007;115:2128-35.

24 [B.155] Schneeweiss S, Patrick AR, Pedan A, Varasteh L, Levin R, Liu N, et al. The effect of

25 Medicare Part D coverage on drug use and cost sharing among seniors without prior drug

26 benefits. Health Aff 2009;28:w305-16.

27 [B.156] Schneeweiss S, Soumerai SB, Glynn RJ, Maclure M, Dormuth C, Walker AM. Impact

28 of reference-based pricing for angiotensin-converting enzyme inhibitors on drug utilization.

29 CMAJ 2002;166:737-45.

30 [B.157] Schneeweiss S, Soumerai SB, Maclure M, Dormuth C, Walker AM, Glynn RJ. Clinical

31 and economic consequences of reference pricing for dihydropyridine calcium channel blockers.

32 Clin Pharmacol Ther 2003;74:388-400.

1 [B.158] Sen B, Blackburn J, Morrisey MA, Kilgore ML, Becker DJ, Caldwell C, et al. Did

2 copayment changes reduce dealth Service utilization among CHIP enrollees? Evidence from

3 Alabama. Health Serv Res 2012;47:1603-20.

4 [B.159] Serumaga B, Ross-Degnan D, Avery AJ, Elliott RA, Majumdar SR, Zhang F, et al.

5 Effect of pay for performance on the management and outcomes of hypertension in the United

6 Kingdom: interrupted time series study. BMJ 2011;342.

7 [B.160] Sheldon TA, Cullum N, Dawson D, Lankshear A, Lowson K, Watt I, et al. What's the

8 evidence that NICE guidance has been implemented? Results from a national evaluation using

9 time series analysis, audit of patients' notes, and interviews. BMJ 2004;329:999.

10 [B.161] Shrank WH, Choudhry NK, Tong A, Myers J, Fischer MA, Swanton K, et al. Warnings

11 without guidance patient responses to an FDA warning about ezetimibe. Med Care 2012;50:479-

12 84.

13 [B.162] Shrank WH, Patrick AR, Pedan A, Polinski JM, Varasteh L, Levin R, et al. The effect of

14 transitioning to Medicare Part D drug coverage in seniors dually eligible for Medicare and

15 Medicaid. J Am Geriatr Soc 2008;56:2304-10.

16 [B.163] Simoni-Wastila L, Ross-Degnan D, Mah C, Gao XM, Brown J, Cosler LE, et al. A

17 retrospective data analysis of the impact of the New York triplicate prescription program on

18 benzodiazepine use in Medicaid patients with chronic psychiatric and neurologic disorders. Clin

19 Ther 2004;26:322-36.

20 [B.164] Siriwardena AN, Fairchild P, Gibson S, Sach T, Dewey M. Investigation of the effect of

21 a countywide protected learning time scheme on prescribing rates of ramipril: interrupted time

22 series study. Fam Pract 2007;24:26-33.

23 [B.165] Sketris IS, Kephart GC, Frail DM, Skedgel C, Allen MJ. The effect of deinsuring

24 chlorpropamide on the prescribing of oral antihyperglycemics for Nova Scotia Seniors'

25 Pharmacare beneficiaries. Pharmacotherapy 2004;24:784-91.

26 [B.166] Slekovec C, Leroy J, Vernaz-Hegi N, Faller J-P, Sekri D, Hoen B, et al. Impact of a

27 region wide antimicrobial stewardship guideline on urinary tract infection prescription patterns.

28 Int J Clin Pharm 2012;34:325-9.

29 [B.167] Smith DH, Perrin N, Feldstein A, Yang XH, Kuang D, Simon SR, et al. The impact of

30 prescribing safety alerts for elderly persons in an electronic medical record - An interrupted time

31 series evaluation. Arch Intern Med 2006;166:1098-104.

1 [B.168] Soumerai SB, Avorn J, Gortmaker S, Hawley S. Effect of government and commercial

2 warnings on reducing prescription misuse: the case of propoxyphene. Am J Public Health

3 1987;77:1518-23.

4 [B.169] Soumerai SB, Avorn J, Ross-Degnan D, Gortmaker S. Payment restrictions for

5 prescription drugs under Medicaid. Effects on therapy, cost, and equity. N Engl J Med

6 1987;317:550-6.

7 [B.170] Soumerai SB, McLaughlin TJ, Rossdegnan D, Casteris CS, Bollini P. Effects of limiting

8 Medicaid drug-reimbursement benefits on the use of psychotropic agents and acute mental health

9 services by patients with schizophrenia. N Engl J Med 1994;331:650-5.

10 [B.171] Soumerai SB, Ross-Degnan D, Gortmaker S, Avorn J. Withdrawing payment for

11 nonscientific drug therapy. Intended and unexpected effects of a large-scale natural experiment.

12 JAMA 1990;263:831-9.

13 [B.172] Sreeharan V, Madden H, Lee JT, Millett C, Majeed A. Improving Access to

14 Psychological Therapies and antidepressant prescribing rates in England: a longitudinal time-

15 series analysis. Br J Gen Pract 2013;63:e649-53.

16 [B.173] Starmans B, Janssen R, Schepers M, Verkooijen M. The effect of a patient charge and a

17 prescription regulation on the use of antihypertension drugs in Limburg, The Netherlands. Health

18 Policy 1994;26:191-206.

19 [B.174] Starner CI, Fenrick B, Coleman J, Wickersham P, Gleason PP. Rosiglitazone prior

20 authorization safety policy: a cohort study. J Manag Care Pharm 2012;18:225-33.

21 [B.175] Stewart KA, Natzke BM, Williams T, Granger E, Casscells SW, Croghan TW. Temporal

22 trends in anti-diabetes drug use in tricare following safety warnings in 2007 about rosiglitazone.

23 Pharmacoepidemiol Drug Saf 2009;18:1048-52.

24 [B.176] Sun SX, Lee KY, Bertram CT, Goldstein JL. Withdrawal of COX-2 selective inhibitors

25 rofecoxib and valdecoxib: impact on NSAID and gastroprotective drug prescribing and

26 utilization. Curr Med Res Opin 2007;23:1859-66.

27 [B.177] Szatkowski L, Coleman T, McNeill A, Lewis S. The impact of the introduction of

28 smoke-free legislation on prescribing of stop-smoking medications in England. Addiction

29 2011;106:1827-34.

30 [B.178] Tamblyn R, Laprise R, Hanley JA, Abrahamowicz M, Scott S, Mayo N, et al. Adverse

31 events associated with prescription drug cost-sharing among poor and elderly persons. JAMA

32 2001;285:421-9.

1 [B.179] Thamer M, Ray NF, Henderson SC, Rinehart CS, Sherman CR, Ferguson JH. Influence

2 of the NIH Consensus Conference on Helicobacter pylori on physician prescribing among a

3 Medicaid population. Med Care 1998;36:646-60.

4 [B.180] Thamer M, Zhang Y, Lai D, Kshirsagar O, Cotter D. Influence of safety warnings on

5 ESA prescribing among dialysis patients using an interrupted time series. BMC Nephrol

6 2013;14:172.

7 [B.181] Tu K, Mamdani MM, Jacka RM, Forde NJ, Rothwell DM, Tu JV. The striking effect of

8 the Heart Outcomes Prevention Evaluation (HOPE) on ramipril prescribing in Ontario. CMAJ

9 2003;168:553-7.

10 [B.182] Tu K, Mamdani MM, Tu JV. Hypertension guidelines in elderly patients: is anybody

11 listening? Am J Med 2002;113:52-8.

12 [B.183] Udell JA, Fischer MA, Brookhart MA, Solomon DH, Choudhry NK. Effect of the

13 Women's Health Initiative on osteoporosis therapy and expenditure in Medicaid. J Bone Miner

14 Res 2006;21:765-71.

15 [B.184] Usher C, Teeling M, Bennett K, Feely J. Effect of clinical trial publicity on HRT

16 prescribing in Ireland. Eur J Clin Pharmacol 2006;62:307-10.

17 [B.185] Valiyeva E, Herrmann N, Rochon PA, Gill SS, Anderson GM. Effect of regulatory

18 warnings on antipsychotic prescription rates among elderly patients with dementia: a population-

19 based time-series analysis. CMAJ 2008;179:438-46.

20 [B.186] Valuck RJ, Libby AM, Orton HD, Morrato EH, Allen R, Baldessarini RJ. Spillover

21 effects on treatment of adult depression in primary care after FDA advisory on risk of pediatric

22 suicidality with SSRIs. Am J Psychiatry 2007;164:1198-205.

23 [B.187] van Driel ML, Vander Stichele R, Elseviers M, De Sutter A, De Maeseneer J,

24 Christiaens T. Effects of an evidence report and policies lifting reimbursement restrictions for

25 acid suppressants: analysis of the Belgian national database. Pharmacoepidemiol Drug Saf

26 2008;17:1113-22.

27 [B. 188] van Kasteren MEE, Mannien J, Kullberg B-J, de Boer AS, Nagelkerke NJ, Ridderhof M,

28 et al. Quality improvement of surgical prophylaxis in Dutch hospitals: evaluation of a multi-site

29 intervention by time series analysis. J Antimicrob Chemother 2005;56:1094-102.

30 [B.189] Vegter S, Kolling P, Toben M, Visser ST, de Jong-van den Berg LTW. Replacing

31 hormone therapy-is the decline in prescribing sustained, and are nonhormonal drugs substituted?

32 Menopause 2009;16:329-35.

1 [B.190] Venekamp RP, Rovers MM, Verheij TJM, Bonten MJM, Sachs APE. Treatment of acute

2 rhinosinusitis: discrepancy between guideline recommendations and clinical practice. Fam Pract

3 2012;29:706-12.

4 [B.191] Verbiest MEA, Chavannes NH, Crone MR, Nielen MMJ, Segaar D, Korevaar JC, et al.

5 An increase in primary care prescriptions of stop-smoking medication as a result of health

6 insurance coverage in the Netherlands: population based study. Addiction 2013;108:2183-92.

7 [B.192] Wagner AK, Ross-Degnan D, Gurwitz JH, Zhang F, Gilden DB, Cosler L, et al. Effect

8 of New York state regulatory action on benzodiazepine prescribing and hip fracture rates. Ann

9 Intern Med 2007;146:96-103.

10 [B.193] Wagner AK, Soumerai SB, Zhang F, Mah C, Simoni-Wastila L, Cosler L, et al. Effects

11 of state surveillance on new post-hospitalization benzodiazepine use. Int J Qual Health Care

12 2003;15:423-31.

13 [B.194] Wang PS, Patrick AR, Dormuth CR, Avorn J, Maclure M, Canning CF, et al. The impact

14 of cost sharing on antidepressant use among older adults in British Columbia. Psychatr Serv

15 2008;59:377-83.

16 [B.195] Wang TJ, Stafford RS, Ausiello JC, Chaisson CE. Randomized clinical trials and recent

17 patterns in the use of statins. American heart journal 2001;141:957-63.

18 [B.196] Weatherby LB, Walker AM, Fife D, Vervaet P, Klausner MA. Contraindicated

19 medications dispensed with cisapride: temporal trends in relation to the sending of 'Dear Doctor'

20 letters. Pharmacoepidemiol Drug Saf 2001;10:211-8.

21 [B.197] Weiss K, Blais R, Fortin A, Lantin S, Gaudet M. Impact of a Multipronged Education

22 Strategy on Antibiotic Prescribing in Quebec, Canada. Clinical infectious diseases: an official

23 publication of the Infectious Diseases Society of America 2011;53:433-9.

24 [B.198] Wheeler BW, Metcalfe C, Gunnell D, Stephens P, Martin RM. Population impact of

25 regulatory activity restricting prescribing of COX-2 inhibitors: ecological study. Br J Clin

26 Pharmacol 2009;68:752-64.

27 [B.199] Wijeysundera DN, Mamdani M, Laupacis A, Fleisher LA, Beattie WS, Johnson SR, et

28 al. Clinical evidence, practice guidelines, and -blocker utilization before major noncardiac

29 surgery. Circulation 2012;5:558-65.

30 [B.200] Wijlaars L, Nazareth I, Petersen I. Trends in Depression and Antidepressant Prescribing

31 in Children and Adolescents: A Cohort Study in The Health Improvement Network (THIN).

32 PLoS One 2012;7:e33181.

1 [B.201] Winkelmayer WC, Asslaber M, Bucsics A, Burkhardt T, Schautzer A, Wieninger P, et

2 al. Impact of reimbursement changes on statin use among patients with diabetes in Austria.

3 Wiener Klin Wochenschr 2010;122:89-94.

4 [B.202] Wright NMJ, Roberts AJ, Allgar VL, Tompkins CNE, Greenwood DC, Laurence G.

5 Impact of the CSM advice on thioridazine on general practitioner prescribing behaviour in

6 Leeds: time series analysis. Br J Gen Pract 2004;54:370-3.

7 [B.203] Zechnich AD, Greenlick M, Haxby D, Mullooly J. Elimination of over-the-counter

8 medication coverage in the Oregon Medicaid population: the impact on program costs and drug

9 use. Med Care 1998;36:1283-94.

10 [B.204] Zhang Y, Adams AS, Ross-Degnan D, Zhang F, Soumerai SB. Effects of prior

11 authorization on medication discontinuation among Medicaid beneficiaries with bipolar disorder.

12 Psychatr Serv 2009;60:520-7.

13 [B.205] Briesacher BA, Zhao Y, Madden JM, Zhang F, Adams AS, Tjia J, et al. Medicare part D

14 and changes in prescription drug use and cost burden: national estimates for the Medicare

15 population, 2000 to 2007. Med Care 2011;49:834-41.

16 [B.206] Donnelly N, McManus P, Dudley J, Hall W. Impact of increasing the re-supply interval

17 on the seasonality of subsidised prescription use in Australia. Aust N Z J Public Health

18 2000;24:603-6.

19 [B.207] Dormuth CR, Glynn RJ, Neumann P, Maclure M, Brookhart AM, Schneeweiss S.

20 Impact of two sequential drug cost-sharing policies on the use of inhaled medications in older

21 patients with chronic obstructive pulmonary disease or asthma. Clin Ther 2006;28:964-78.

22 [B.208] Fischer MA, Choudhry NK, Winkelmayer WC. Impact of Medicaid prior authorization

23 on angiotensin-receptor blockers: can policy promote rational prescribing? Health Aff

24 2007;26:800-7.

25 [B.209] Gibson TB, Wang S, Kelly E, Brown C, Turner C, Frech-Tamas F, et al. A value-based

26 insurance design program at a large company boosted medication adherence for employees with

27 chronic illnesses. Health affairs (Project Hope) 2011;30:109-17.

28 [B.210] Haas JS, Kaplan CP, Gerstenberger EP, Kerlikowske K. Changes in the use of

29 postmenopausal hormone therapy after the publication of clinical trial results. Ann Intern Med

30 2004;140:184-8.

31 [B.211] Jackevicius CA, Anderson GM, Leiter L, Tu JV. Use of the statins in patients after acute

32 myocardial infarction: does evidence change practice? Arch Intern Med 2001;161:183-8.

1 [B.212] Katz LY, Kozyrskyj AL, Prior HJ, Enns MW, Cox BJ, Sareen J. Effect of regulatory

2 warnings on antidepressant prescription rates, use of health services and outcomes among

3 children, adolescents and young adults. CMAJ 2008;178:1005-11.

4 [B.213] Lurk JT, DeJong DJ, Woods TM, Knell ME, Carroll CA. Effects of changes in patient

5 cost sharing and drug sample policies on prescription drug costs and utilization in a safety-net-

6 provider setting. Am J Health Syst Pharm 2004;61:267-72.

7 [B.214] Majumdar SR, Almasi EA, Stafford RS. Promotion and prescribing of hormone therapy

8 after report of harm by the Women's Health Initiative. JAMA 2004;292:1983-8.

9 [B.215] Mamdani M, Juurlink DN, Kopp A, Naglie G, Austin PC, Laupacis A. Gastrointestinal

10 bleeding after the introduction of COX 2 inhibitors: ecological study. BMJ 2004;328:1415-6.

11 [B.216] Mandryk JA, Wai A, Mackson JM, Patterson C, Bhasale A, Weekes LM. Evaluating the

12 impact of educational interventions on use of antithrombotics in Australia. Pharmacoepidemiol

13 Drug Saf 2008;17:160-71.

14 [B.217] Marra F, Patrick DM, White R, Ng H, Bowie WR, Hutchinson JM. Effect of formulary

15 policy decisions on antimicrobial drug utilization in British Columbia. J Antimicrob Chemother

16 2005;55:95-101.

17 [B.218] Marshall JK, Grootendorst PV, O'Brien BJ, Dolovich LR, Holbrook AM, Levy AR.

18 Impact of reference-based pricing for histamine-2 receptor antagonists and restricted access for

19 proton pump inhibitors in British Columbia. CMAJ 2002;166:1655-62.

20 [B.219] McManus P, Donnelly N, Henry D, Hall W, Primrose J, Lindner J. Prescription drug

21 utilization following patient co-payment changes in Australia. Pharmacoepidemiol Drug Saf

22 1996;5:385-92.

23 [B.220] Mishra G, Kok H, Ecob R, Cooper R, Hardy R, Kuh D. Cessation of hormone

24 replacement therapy after reports of adverse findings from randomized controlled trials: evidence

25 from a British Birth Cohort. Am J Public Health 2006;96:1219-25.

26 [B.221] Morden NE, Zerzan JT, Rue TC, Heagerty PJ, Roughead EE, Soumerai SB, et al.

27 Medicaid prior authorization and controlled-release oxycodone. Med Care 2008;46:573-80.

28 [B.222] Morrow RL, Carney G, Wright JM, Bassett K, Sutherland J, Dormuth CR. Impact of

29 rosiglitazone meta-analysis on use of glucose-lowering medications. Open Med 2010;4:e50-9.

30 [B.223] Motheral B, Fairman KA. Effect of a three-tier prescription copay on pharmaceutical

31 and other medical utilization. Med Care 2001;39:1293-304.

1 [B.224] Nelson AA, Jr., Reeder CE, Dickson WM. The effect of a Medicaid drug copayment

2 program on the utilization and cost of prescription services. Med Care 1984;22:724-36.

3 [B.225] Nemeroff CB, Kalali A, Keller MB, Charney DS, Lenderts SE, Cascade EF, et al.

4 Impact of publicity concerning pediatric suicidality data on physician practice patterns in the

5 United States. Arch Gen Psychiatry 2007;64:466-72.

6 [B.226] Ruiter R, Visser LE, van Herk-Sukel MP, Geelhoed-Duijvestijn PH, de Bie S, Straus

7 SM, et al. Prescribing of rosiglitazone and pioglitazone following safety signals: analysis of

8 trends in dispensing patterns in the Netherlands from 1998 to 2008. Drug Saf 2012;35:471-80.

9 [B.227] Shah BR, Juurlink DN, Austin PC, Mamdani MM. New use of rosiglitazone decreased

10 following publication of a meta-analysis suggesting harm. Diabet Med 2008;25:871-4.

11 [B.228] Shorr RI, Fought RL, Ray WA. Changes in antipsychotic drug use in nursing homes

12 during implementation of the OBRA-87 regulations. JAMA 1994;271:358-62.

13 [B.229] Simoens S, De Bruyn K, Miranda J, Bennie M, Malmstrom R, Godman B. Measures to

14 enhance angiotensin-receptor blocker prescribing efficiency in Belgium following generic

15 losartan: impcat and implications for the future. J Pharm Health Serv Res 2013;4:173-91.

16 [B.230] Smalley WE, Griffin MR, Fought RL, Sullivan L, Ray WA. Effect of a prior-

17 authorization requirement on the use of nonsteroidal antiinflammatory drugs by Medicaid

18 patients. N Engl J Med 1995;332:1612-7.

19 [B.231] Valluri S, Zito JM, Safer DJ, Zuckerman IH, Mullins CD, Korelitz JJ. Impact of the

20 2004 Food and Drug Administration pediatric suicidality warning on antidepressant and

21 psychotherapy treatment for new-onset depression. Med Care 2010;48:947-54.

22 [B.232] Wilkinson JJ, Force RW, Cady PS. Impact of safety warnings on drug utilization:

23 marketplace life span of cisapride and troglitazone. Pharmacotherapy 2004;24:978-86.

24 [B.233] Xie F, Petitti DB, Chen W. Prescribing patterns for antihypertensive drugs after the

25 Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial: report of

26 experience in a health maintenance organization. Am J Hypertens 2005;18:464-9.

27 [B.234] Yin W, Basu A, Zhang JX, Rabbani A, Meltzer DO, Alexander GC. The effect of the

28 Medicare Part D prescription benefit on drug utilization and expenditures. Ann Intern Med

29 2008;148:169-77.

1 Appendix C: List of references examining single institutional interventions

2 [C.1] Aiken AM, Wanyoro AK, Mwangi J, Juma F, Mugoya IK, Scott JAG. Changing Use of

3 Surgical Antibiotic Prophylaxis in Thika Hospital, Kenya: A Quality Improvement Intervention

4 with an Interrupted Time Series Design. PLoS One 2013;8.

5 [C.2] Aldeyab MA, Kearney MP, Scott MG, Aldiab MA, Alahmadi YM, Elhajji FWD, et al. An

6 evaluation of the impact of antibiotic stewardship on reducing the use of high-risk antibiotics and

7 its effect on the incidence of Clostridium difficile infection in hospital settings. J Antimicrob

8 Chemother 2012;67:2988-96.

9 [C.3] Andersen SE, Knudsen JD, Bispebjerg Intervention G. A managed multidisciplinary

10 programme on multi-resistant Klebsiella pneumoniae in a Danish university hospital. BMJ Qual

11 Saf 2013;22:907-15.

12 [C.4] Anokbonggo WW, Ogwal-Okeng JW, Obua C, Aupont O, Ross-Degnan D. Impact of

13 decentralization on health services in Uganda: a look at facility utilization, prescribing and

14 availability of essential drugs. East Afr Med J 2004;Suppl:S2-7.

15 [C.5] Ansari F, Gray K, Nathwani D, Phillips G, Ogston S, Ramsay C, et al. Outcomes of an

16 intervention to improve hospital antibiotic prescribing: interrupted time series with segmented

17 regression analysis. J Antimicrob Chemother 2003;52:842-8.

18 [C.6] Berbatis CG, Maher MJ, Plumridge RJ, Stoelwinder JU, Zubrick SR. Impact of a drug

19 bulletin on prescribing oral analgesics in a teaching hospital. Am J Hosp Pharm1982;39:98-100.

20 [C.7] Bornard L, Dellamonica J, Hyvernat H, Girard-Pipau F, Molinari N, Sotto A, et al. Impact

21 of an assisted reassessment of antibiotic therapies on the quality of prescriptions in an intensive

22 care unit. Med Mal Infect 2011 ;41:480-5.

23 [C.8] Boruett P, Kagai D, Njogo S, Nguhiu P, Awuor C, Gitau L, et al. Facility-level intervention

24 to improve attendance and adherence among patients on anti-retroviral treatment in Kenya-a

25 quasi-experimental study using time series analysis. BMC Health Serv Res 2013;13.

26 [C.9] Buising KL, Thursky KA, Black JF, MacGregor L, Street AC, Kennedy MP, et al.

27 Improving antibiotic prescribing for adults with community acquired pneumonia: Does a

28 computerised decision support system achieve more than academic detailing alone?--A time

29 series analysis. BMC Med Inform Decis Mak 2008;8:35.

30 [C.10] Buising KL, Thursky KA, Robertson MB, Black JF, Street AC, Richards MJ, et al.

31 Electronic antibiotic stewardship--reduced consumption of broad-spectrum antibiotics using a

32 computerized antimicrobial approval system in a hospital setting. J Antimicrob Chemother

33 2008;62:608-16.

34 [C.11] Buyle F, Vogelaers D, Peleman R, Van Maele G, Robays H. Implementation of

35 guidelines for sequential therapy with fluoroquinolones in a Belgian hospital. Pharm World Sci

36 2010;32:404-10.

1 [C.12] Cairns KA, Jenney AWJ, Abbott IJ, Skinner MJ, Doyle JS, Dooley M, et al. Prescribing

2 trends before and after implementation of an antimicrobial stewardship program. Med J Aust

3 2013;198:262-6.

4 [C.13] Church EC, Mauldin PD, Bosso JA. Antibiotic resistance in Pseudomonas aeruginosa

5 related to quinolone formulary changes: an interrupted time series analysis. Infect Control Hosp

6 Epidemiol 2011;32:400-2.

7 [C.14] Colombet I, Sabatier B, Gillaizeau F, Prognon P, Begue D, Durieux P. Long-term effects

8 of a multifaceted intervention to encourage the choice of the oral route for proton pump

9 inhibitors: an interrupted time-series analysis. Qual Saf Health Care 2009;18:232-5.

10 [C.15] Cook PP, Rizzo S, Gooch M, Jordan M, Fang X, Hudson S. Sustained reduction in

11 antimicrobial use and decrease in methicillin-resistant Staphylococcus aureus and Clostridium

12 difficile infections following implementation of an electronic medical record at a tertiary-care

13 teaching hospital. J Antimicrob Chemother 2011 ;66:205-9.

14 [C.16] Cortina S, Somers M, Rohan JM, Drotar D. Clinical effectiveness of comprehensive

15 psychological intervention for nonadherence to medical treatment: a case series. J Pediatr

16 Psychol 2013;38:649-63.

17 [C.17] Cortoos PJ, Gilissen C, Mol PGM, Van den Bossche F, Simoens S, Willems L, et al.

18 Empirical management of community-acquired pneumonia: impact of concurrent A/H1N1

19 influenza pandemic on guideline implementation. J Antimicrob Chemother 2011;66:2864-71.

21 [C.18] Durieux P, Nizard R, Ravaud P, Mounier N, Lepage E. A clinical decision support system

22 for prevention of venous thromboembolism: effect on physician behavior. JAMA

23 2000;283:2816-21.

25 [C.19] Eisenstein EL, Wojdyla D, Anstrom KJ, Brennan JM, Califf RM, Peterson ED, et al.

26 Evaluating the Impact of Public Health Notification Duke Clopidogrel Experience. Circ-

27 Cardiovasc Qual Outcomes 2012;5:767-74.

28 [C.20] Elligsen M, Walker SAN, Pinto R, Simor A, Mubareka S, Rachlis A, et al. Audit and

29 feedback to reduce broad-spectrum antibiotic use among intensive care unit patients: a controlled

30 interrupted time series analysis. Infect Control Hosp Epidemiol 2012;33:354-61.

31 [C.21] Everitt DE, Soumerai SB, Avorn J, Klapholz H, Wessels M. Changing surgical

32 antimicrobial prophylaxis practices through education targeted at senior department leaders.

33 Infect Control Hosp Epidemiol 1990;11:578-83.

34 [C.22] Fowler S, Webber A, Cooper BS, Phimister A, Price K, Carter Y, et al. Successful use of

35 feedback to improve antibiotic prescribing and reduce Clostridium difficile infection: a

36 controlled interrupted time series. J Antimicrob Chemother 2007;59:990-5.

1 [C.23] Graber CJ, Hutchings C, Dong F, Lee W, Chung JK, Tran T. Changes in antibiotic usage

2 and susceptibility in nosocomial Enterobacteriaceae and Pseudomonas isolates following the

3 introduction of ertapenem to hospital formulary. Epidemiol Infect 2012;140:115-25.

4 [C.24] Gregoire J-P, Moisan J, Potvin L, Chabot I, Verreault R, Milot A. Effect of drug

5 utilization reviews on the quality of in-hospital prescribing: a quasi-experimental study. BMC

6 Health Serv Res 2006;6:33.

7 [C.25] Gurwitz JH, McLaughlin TJ, Fish LS. The effect of an Rx-to-OTC switch on medication

8 prescribing patterns and utilization of physician services: the case of vaginal antifungal products.

9 Health Serv Res 1995;30:672-85.

10 [C.26] Gurwitz JH, Noonan JP, Soumerai SB. Reducing the use of H2-receptor antagonists in the

11 long-term-care setting. J Am Geriatr Soc 1992;40:359-64.

12 [C.27] Guterman JJ, Chernof BA, Mares B, Gross-Schulman SG, Gan PG, Thomas D.

13 Modifying provider behavior: a low-tech approach to pharmaceutical ordering. J Gen Intern Med

14 2002;17:792-6.

15 [C.28] Hadi U, Keuter M, van Asten H, van den Broek P, Study Grp Antimicrobial R.

16 Optimizing antibiotic usage in adults admitted with fever by a multifaceted intervention in an

17 Indonesian governmental hospital. Trop Med Int Health 2008; 13:888-99.

18 [C.29] Hartung DM, Evans D, Haxby DG, Kraemer DF, Andeen G, Fagnan LJ. Effect of Drug

19 Sample Removal on Prescribing in a Family Practice Clinic. Ann Fam Med 2010;8:402-9.

20 [C.30] Hulgan T, Rosenbloom ST, Hargrove F, Talbert DA, Arbogast PG, Bansal P, et al. Oral

21 quinolones in hospitalized patients: an evaluation of a computerized decision support

22 intervention. J Intern Med 2004;256:349-57.

23 [C.31] Jump RLP, Olds DM, Seifi N, Kypriotakis G, Jury LA, Peron EP, et al. Effective

24 Antimicrobial Stewardship in a Long-Term Care Facility through an Infectious Disease

25 Consultation Service: Keeping a LID on Antibiotic Use. Infect Control Hosp Epidemiol

26 2012;33:1185-92.

27 [C.32] Kaplan RC, Psaty BM, Kriesel D, Heckbert SR, Smith NL, Gillett C, et al. Replacing

28 short-acting nifedipine with alternative medications at a large health maintenance organization.

29 Am J Hypertens 1998;11:471-7.

30 [C.33] Lewis GJ, Fang X, Gooch M, Cook PP. Decreased resistance of Pseudomonas aeruginosa

31 with restriction of ciprofloxacin in a large teaching hospital's intensive care and intermediate care

32 units. Infect Control Hosp Epidemiol 2012;33:368-73.

33 [C.34] Lowe DO, Lummis H, Zhang Y, Sketris IS. Effect of educational and policy interventions

34 on institutional utilization of wet nebulization respiratory drugs and portable inhalers. J Clin

35 Pharmacol 2008;15:e334-43.

1 [C.35] Magedanz L, Silliprandi EM, dos Santos RP. Impact of the pharmacist on a

2 multidisciplinary team in an antimicrobial stewardship program: a quasi-experimental study. Int

3 J Clin Pharm 2012;34:290-4.

4 [C.36] McCoy AB, Waitman LR, Gadd CS, Danciu I, Smith JP, Lewis JB, et al. A computerized

5 provider order entry intervention for medication safety during acute kidney injury: a quality

6 improvement report. Am J Kidney Dis 2010;56:832-41.

7 [C.37] McPhail GL, Weiland J, Acton JD, Ednick M, Chima A, VanDyke R, et al. Improving

8 evidence-based care in cystic fibrosis through quality improvement. Arch Pediatr Adolesc Med

9 2010;164:957-60.

10 [C.38] Meyer E, Buttler J, Schneider C, Strehl E, Schroeren-Boersch B, Gastmeier P, et al.

11 Modified guidelines impact on antibiotic use and costs: duration of treatment for pneumonia in a

12 neurosurgical ICU is reduced. J Antimicrob Chemother 2007;59:1148-54.

14 [C.39] Meyer E, Schwab F, Pollitt A, Bettolo W, Schroeren-Boersch B, Trautmann M. Impact of

15 a Change in Antibiotic Prophylaxis on Total Antibiotic Use in a Surgical Intensive Care Unit.

16 Infection 2010;38:19-24.

17 [C.40] Mol PGM, Wieringa JE, NannanPanday PV, Gans ROB, Degener JE, Laseur M, et al.

18 Improving compliance with hospital antibiotic guidelines: a time-series intervention analysis. J

19 Antimicrob Chemother 2005;55:550-7.

20 [C.41] Mukamal KJ, Markson LJ, Flier SR, Calabrese D. Restocking the sample closet: results of

21 a trial to alter medication prescribing. J Am Board Fam Pract 2002;15:285-9.

22 [C.42] Naughton C, Feely J, Bennett K. A RCT evaluating the effectiveness and cost-

23 effectiveness of academic detailing versus postal prescribing feedback in changing GP antibiotic

24 prescribing. J Eval Clin Pract 2009;15:807-12.

25 [C.43] Persell SD, Kaiser D, Dolan NC, Andrews B, Levi S, Khandekar J, et al. Changes in

26 performance after implementation of a multifaceted electronic-health-record-based quality

27 improvement system. Med Care 2011 ;49:117-25.

28 [C.44] Peto Z, Benko R, Matuz M, Csullog E, Molnar A, Hajdu E. Results of a Local Antibiotic

29 Management Program on Antibiotic Use in a Tertiary Intensive Care Unit in Hungary. Infection

30 2008;36:560-4.

31 [C.45] Price J, Cheek E, Lippett S, Cubbon M, Gerding DN, Sambol SP, et al. Impact of an

32 intervention to control Clostridium difficile infection on hospital- and community-onset disease;

33 an interrupted time series analysis. Clin Microbiol Infect 2010;16:1297-302.

34 [C.46] Schwartz DN, Abiad H, DeMarais PL, Armeanu E, Trick WE, Wang Y, et al. An

35 educational intervention to improve antimicrobial use in a hospital-based long-term care facility.

36 J Am Geriatr Soc 2007;55:1236-42.

1 [C.47] Simon SR, Soumerai SB. Failure of Internet-based audit and feedback to improve quality

2 of care delivered by primary care residents. Int J Qual Health Care 2005;17:427-31.

3 [C.48] Smith J, Kong MY, Cambon A, Woods CR. Effectiveness of Antimicrobial Guidelines

4 for Community-Acquired Pneumonia in Children. Pediatrics 2012;129:E1326-E33.

5 [C.49] Soumerai SB, Avorn J, Taylor WC, Wessels M, Maher D, Hawley SL. Improving choice

6 of prescribed antibiotics thorugh concurrent reminders in an educational order form. Med Care

7 1993;31:552-8.

8 [C.50] Sousa D, Castelo-Corral L, Gutierrez-Urbon J-M, Molina F, Lopez-Calvino B, Bou G, et

9 al. Impact of ertapenem use on Pseudomonas aeruginosa and Acinetobacter baumannii imipenem

10 susceptibility rates: collateral damage or positive effect on hospital ecology? J Antimicrob

11 Chemother 2013;68:1917-25.

12 [C.51] Spaulding A, Fendrick AM, Herman WH, Stevenson JG, Smith DG, Chernew ME, et al.

13 A controlled trial of value-based insurance design - The MHealthy: Focus on Diabetes (FOD)

14 trial. Implement Sci 2009;4.

15 [C.52] Stenner SP, Chen Q, Johnson KB. Impact of generic substitution decision support on

16 electronic prescribing behavior. J Am Med Inform Assoc 2010;17:681-8.

17 [C.53] Talpaert MJ, Rao GG, Cooper BS, Wade P. Impact of guidelines and enhanced antibiotic

18 stewardship on reducing broad-spectrum antibiotic usage and its effect on incidence of

19 Clostridium difficile infection. J Antimicrob Chemother 2011;66:2168-74.

20 [C.54] Tangden T, Eriksson B-M, Melhus A, Svennblad B, Cars O. Radical reduction of

21 cephalosporin use at a tertiary hospital after educational antibiotic intervention during an

22 outbreak of extended-spectrum beta-lactamase-producing Klebsiella pneumoniae. J Antimicrob

23 Chemother 2011;66:1161-7.

24 [C.55] Thomas SK, Hodson J, McIlroy G, Dhami A, Coleman JJ. The Impact of Direct

25 Healthcare Professional Communication on Prescribing Practice in the UK Hospital Setting: An

26 Interrupted Time Series Analysis. Drug Saf 2013;36:557-64.

27 [C.56] Umscheid CA, Hanish A, Chittams J, Weiner MG, Hecht TEH. Effectiveness of a novel

28 and scalable clinical decision support intervention to improve venous thromboembolism

29 prophylaxis: a quasi-experimental study. BMC Med Inform Decis Mak 2012;12.

30 [C.57] van Hees BC, de Ruiter E, Wiltink EH, de Jongh BM, Tersmette M. Optimizing use of

31 ciprofloxacin: a prospective intervention study. J Antimicrob Chemother 2008;61:210-3.

32 [C.58] Willemsen I, Cooper B, van Buitenen C, Winters M, Andriesse G, Kluytmans J.

33 Improving quinolone use in hospitals by using a bundle of interventions in an interrupted time

34 series analysis. Antimicrob Agents Chemother 2010;54:3763-9.

1 [C.59] Yeo CL, Chan DSG, Earnest A, Wu TS, Yeoh SF, Lim R, et al. Prospective audit and

2 feedback on antibiotic prescription in an adult hematology-oncology unit in Singapore. Eur J

3 Clin Microbiol Infect Dis 2012;31:583-90.