0Drugs (2014) 74:737-755 DOI 10.1007/s40265-014-0214-8
REVIEW ARTICLE
Management of Asthma and Chronic Obstructive Pulmonary Disease with Combination Inhaled Corticosteroids and Long-Acting ß-Agonists: A Review of Comparative Effectiveness Research
Douglas W. Mapel • Melissa H. Roberts
Published online: 6 May 2014
© The Author(s) 2014. This article is published with open access at Springerlink.com
Abstract The value of combination therapy with inhaled corticosteroids and long-acting p-agonists (ICS/LABA) is well recognized in the management of asthma and chronic obstructive pulmonary disease (COPD). Despite differences in the pharmacological properties between two well-established ICS/LABA products (budesonide/formoterol and fluticasone/salmeterol), data from randomized clinical trials (RCTs) and meta-analyses suggest that these two products perform similarly under RCT conditions. In contrast, a few recently reported real-world comparative effectiveness studies have suggested that there are substantial differences between ICS/LABA combination treatments in terms of clinical and healthcare outcomes in patients with asthma or COPD. The purpose of this article is to provide a brief review of the benefits, as well as the limitations, of comparative effectiveness research (CER) in the therapeutic area of asthma and COPD. We conducted a structured literature review of the current CER studies on ICS/LABA combinations in asthma and COPD. These articles were then used to illustrate the unique challenges of CER studies, providing a summary of study results and limitations. We focus particularly on difficult biases and confounding factors that may be introduced before, during, and after the initiation of therapy. Beyond being a review of these two ICS/LABA combination treatments, this article is intended to help those who wish to assess the quality of CER published projects in asthma and COPD, or guide investigators who wish to design new CER studies for chronic respiratory disease treatments.
D. W. Mapel (&) • M. H. Roberts Lovelace Clinic Foundation, 2309 Renard Place SE, Albuquerque, NM 87106, USA e-mail: dmapel@comcast.net
Key Points
A number of new treatments for asthma and chronic obstructive pulmonary disease (COPD) were recently released, with several more on the horizon. This has sparked interest in the comparative effectiveness among the available inhaled therapies.
The natural histories of asthma and COPD presentation and progression present many unique challenges for comparative effectiveness research. Comorbidities, disease heterogeneity, and poor treatment adherence are just a few of the problems that can introduce bias into the analysis if not effectively addressed in the study design.
1 Introduction
The goal of comparative effectiveness research (CER) is to measure the real-life benefits and risks of treatments. The Institute of Medicine defines CER as ''the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policymakers to make informed decisions that will improve healthcare at both the individual and population levels'' [1]. Proof of whether or not a treatment can work is known as efficacy, while the benefit of that treatment in routine clinical practice is known as effectiveness. Efficacy is usually established by randomized clinical trials (RCTs), which are considered to be the 'gold
standard' for proof of treatment benefit, but RCTs do have important limitations, including highly selected study populations and artificial clinical conditions. Effectiveness studies are needed to demonstrate that treatments still have the intended benefits when they are used in broader unse-lected patient populations and routine clinical practice. CER attempts to capture the differences in clinical benefits among similar treatments when used in the general population.
The availability of well-established inhaled combination corticosteroid and long-acting p-agonist (ICS/LABA) products [budesonide/formoterol (BFC) and fluticasone/ salmeterol (FSC)] in the management of asthma and chronic obstructive pulmonary disease (COPD) creates an opportunity to examine the benefits, as well as limitations, of CER in the therapeutic area of chronic respiratory disease. Both of these combination inhalers have proven efficacy in both asthma and COPD (a third combination, mometasone/formoterol, is approved only for asthma), and data from RCTs and meta-analyses suggest that these products perform similarly under controlled RCT conditions [2-7]. The role of ICS/LABA combination therapy for patients with persistent asthma is well established, while the role of combination therapy in COPD is not as clear (Fig. 1). The only head-to-head comparisons of ICS/ LABA combinations in asthma found that any differences in efficacy between them were slight, and in their primary endpoints, not statistically significant [6]. In contrast, some recently reported real-world, comparative effectiveness studies have suggested that there are differences between ICS/LABA combination treatments in a variety of clinical outcomes. These discrepancies are intriguing, but there are several unique features of asthma and COPD disease pathophysiology, progression, and management that need to be considered when interpreting CER studies, as well as very important limitations in study design. In this review, we examine the clinical trajectory of ICS/LABA use in asthma and COPD and how study design problems during different time periods may result in significant biases in CER research. We present a current review of published CER studies that have directly compared BFC with FSC, and examine how they may or may not have dealt with these study design issues. Finally, we summarize the practical, clinical implications of these CER studies as well as the knowledge gaps that remain, and look at what lessons can be learned for the development of new therapeutic options.
2 Literature Review
We conducted a literature review using both PubMed and Thomson Reuters Web of Science databases for studies
comparing ICS/LABA treatments, spanning the time period 1 January 1997 through 2 October 2013. Multiple searches were conducted. Search terms included fluticasone, salmeterol, budesonide, and formoterol. Initially the search was broad and also included terms for mometasone furoate and beclomethasone formoterol. A total of 330 unique citations were identified from the broad search process. The abstracts for these were reviewed, and based on that information the following were excluded from further review: 124 review articles, 118 clinical trial studies, 35 non-relevant studies (e.g. generic studies, inhaler mechanism effectiveness, only ICS treatment, or simulation study), and 14 non-study documents (book chapters, viewpoints, or editorials). We then reviewed the remaining 39 articles in detail. Of these, 25 articles did not present comparative effectiveness findings for the BFC and FSC or were duplicative of ones that did, and 14 articles described observational comparative effectiveness studies of BFC and FSC; 9 for asthma and 5 for COPD. Highlights of these studies are summarized in Tables 1, 2, 3 and 4 [9-22].
3 The Trajectory of Asthma and Chronic Obstructive Pulmonary Disease (COPD) Treatment
A major difference between RCTs and CER is that the index date, or 'day one' of treatment, in an RCT for an asthma or COPD medication typically starts when the patient is at a stable baseline, at least several weeks after their last exacerbation. In real-life, asthma or COPD patients usually start taking medications when they are sick. In studies that have examined the natural history of COPD in the time before and after the first diagnosis of COPD, the diagnosis typically occurs during an exacerbation, respiratory infection, or non-respiratory acute medical event, and respiratory medications are often dispensed along with antibiotics and other drugs during that time [23, 24]. Asthma diagnoses are also more likely to be made at times when patients are symptomatic [25, 26]. Therefore, ICS/LABA treatment in the real-world is usually prescribed when asthma and COPD patients are unstable and often have other complicating problems such as infections or cardiovascular complications. When contemplating a CER study in asthma and COPD, it is useful to partition the time periods into the time of the initial treatment with a new medication (the index date), the time before the new medication was dispensed (the baseline period), and the time after the medication was dispensed (the follow-up period). Each of these periods is associated with unique factors that result in selection biases, measurement errors, or potential confounding that are likely to affect clinical outcomes. In the following sections, we will examine these features by each time period in detail, and examine how
Intermittent Asthma
Step 1
Preferred: SABA PRN
Persistent Asthma: Daily Medication Consult with asthma specialist if step 4 care or higher is required. Consider consultation at step 3.
Step 2
Preferred: Low-dose ICS
Alternative:
Cromolyn, LTRA, Nedocromil, or Theophylline
Step 3
Preferred:
Low-dose ICS ♦ LABA OR
Medium-dose ICS Alternative:
Low-dose ICS + either LTRA, Theophylline, or Zileuton
Step 4
Preferred:
Medium-dose ICS
♦ LABA
Alternative:
Medium-dose ICS
♦ either LTRA, Theophylline, or Zileuton
Step 5
Preferred:
High-dose ICS + LABA
Consider Omalizumab for patients who have allergies
Step 6
Preferred:
High-dose
ICS ♦ LABA ♦ oral
corticosteroid
Consider Omalizumab for patients who have allergies
Each step: Patient education, environmental control, and management of comorbidities.
Steps 2-4: Consider subcutaneous allergen immunotherapy for patients who have allergic asthma (see notes).
Quick-Relief Medication for All Patients
• SABA as needed for symptoms. Intensity of treatment depends on severity of symptoms: up to 3 treatments at 20-minute intervals as needed. Short course of oral systemic corticosteroids may be needed.
• Use of SABA >2 days a week for symptom relief (not prevention of EIB) generally indicates inadequate control and the need to step up treatment.
Step up if needed
(first, check adherence, environmental control, and comorbid conditions)
I Assess I
control I
Step down if possible
(and asthma is well controlled at least 3 months)
"Recommendation 4: ACP, ACCP, ATS, and ERS recommend that clinicians prescribe monotherapy using either long-acting inhaled anti-cholinergics or long-acting inhaled ß-agonists for symptomatic patients with COPD and FEN^ <60% predicted. (Grade: strong recommendation, moderate-quality evidence). Clinicians should base the choice of specific monotherapy on patient preference, cost, and adverse effect profile."
"Recommendation 5: ACP, ACCP, ATS, and ERS suggest that clinicians may administer combination inhaled therapies (long-acting inhaled anticholinergics, long-acting inhaled ß-agonists, or inhaled corticosteroids) for symptomatic patients with stable COPD and FEVj <60% predicted (Grade: weak recommendation, moderate-quality evidence)."
Fig. 1 Role of ICS/LABA combination therapy in (a) asthma and (b) COPD [8]. Asthma guidelines reproduced with permission from the National Asthma Education and Prevention Program [8]. Refer to the original document for full guideline notes (http://www.nhlbi.nih. gov/guidelines/asthma/09_sec4_lt_12.pdf, accessed 17 March 2014). ACP American College of Physicians, ACCP American College of
Chest Physicians, ATS American Thoracic Society, COPD chronic obstructive pulmonary disease, EIB exercise-induced bronchospasm, ERS European Respiratory Society, FEVj forced expiratory volume in 1 second, ICS inhaled corticosteroid, LABA long-acting inhaled b2-agonist, LTRA leukotriene receptor antagonist, SABA short-acting inhaled b2-agonist
these potential problems have been handled in the CER studies for ICS/LABA treatment to date.
4 The Index Date
In CER of ICS/LABA combinations, the index date is commonly the date of the first prescription fill for the new
treatment after the asthma or COPD diagnosis is established. In Table 1, the majority are longitudinal cohort analyses with index dates that are based on this definition. The index date is a logical choice as the start of the analysis period because it is safe to assume that the patient first started using the ICS/LABA inhaler on or near that date, and whatever benefits and side effects attributable to the treatment will begin to accrue at that time. However, that
Table 1 The index date—study design characteristics
Reference Study population Inclusion criteria Exclusion criteria Outcome measures
[9] Belgian pulmonary clinics Diagnosis of asthma for Asthma treatments other than Asthma control as measured by the
BFC 157 >6 months rescue inhalers Juniper ACQ score
FSC 156 Persistent asthma FEV1 >60 % predicted Age >15 years Enrolled April-November 2003
[10] German insurance database First Rx for ICS/LABA between COPD Probability of 'full treatment
BFC 1,456 June 2001 and June 2005 Any ICS/LABA use before success' based on SABA use.
FSC 982 Age >12 years Diagnosis of asthma in baseline [ICD-10 J45^16] Enrolled 12 months pre-and post-baseline June 2001 addition of other asthma
medications between weeks 10 and 52, ICS/LABA change, OCS fills, hospitalizations, or specialist referrals
[H] US First Rx for ICS/LABA between 1 COPD (ICD-9 491.xx, 492.xx, Appropriate indication for ICS/
insurance database July 2007 and 30 June 2008 496) LABA prior to index date
BFC 1,417 Age 12-64 years
FSC 14,788 Diagnosis of asthma (ICD-9
493.xx) during baseline Continuously enrolled 12 months before index
[12] US First Rx for ICS/LABA in 2007 Cystic fibrosis, lung cancer. Appropriate indication for ICS/
insurance database Continuously enrolled 12 months COPD, tuberculosis LABA prior to index date
BFC 993 before index (ICS or LTRA use; asthma-related
FSC 23,238 Age >12 years ED visit or hospitalization; OCS
use; or >6 SABA canister fills)
Diagnosis of asthma in baseline
[ICD-9 493.x]
[13] UK GPRD First Rx for ICS/LABA between COPD Treatment success (time to first
database May 2001 and January 2006 Any ICS/LABA use before asthma hospitalization, OCS fill.
BFC 6,918 FSC 16,157 Age 16-65 years May 2001 ICS/LABA switch, anti-
Diagnosis of asthma in baseline Enrolled in GPRD <12 months inflammatory fill)
[ICD-9 493.0, 493.1, 493.9] pre-index date Complete treatment success (time
to first SABA fill)
Enrolled 1 year before and after
index date Refills for either ICS/LABA Switches between ICS/LABA Hospitalizations for asthma Office visits for asthma Asthma specialist referrals OCS, SABA, LABA, xanthene, LTRA, ICS, antibiotic, or other asthma prescriptions
Reference Study population Inclusion criteria Exclusion criteria Outcome measures
Quebec healthcare database BFC 1,449 FSC 9,381 1,264 matched
Hungarian pulmonary clinic BFC-EF N= 53 BFC or FSC N= 58
Italian
respiratory clinics BFC-EF 473 BFC-DPI 453 FSC-DPI 454
Quebec healthcare database BFC 1,286 FSC 3,969 1,131 matched 1:1 cohorts
Italian
respiratory clinics3 BFC-EF 301 BFC-DPI 145 FSC-DPI 123
>1 fill for ICS/LABA in 2002 or 2003
No fill for ICS/LABA in the previous 12 months
Age between 16 and 65 years Diagnosis of asthma in baseline
[ICD-9 493.0, 493.1, 493.9] Enrolled 1 year before and after
index date Moderate to severe asthma diagnosed at least 6 months pre-index Age >18 years
Treated with ICS/LABA at least
4 weeks May-August 2008 Last exacerbation more than
6 weeks pre-index Diagnosis of asthma for at least
6-months Adults enrolled in 1 of 56
pulmonary clinics Enrolled January-October 2009
(PRISMA cross-sectional study) >1 fill for BFC or FSC from 1 February 2003 to 31 January 2007
No fill for ICS/LABA in the
previous 12 months Age >40 years
Diagnosis of COPD in baseline
[ICD-9 491, 492, 496] Enrolled > 1 year before and after
index date Diagnosis of asthma for at least
6 months Adults enrolled in 1 of 56
pulmonary clinics Enrolled in the PRISMA cross-sectional study
Other respiratory diseases
Serious chronic disease Obesity
Any acute illnesses within 6 weeks
Participants in other clinical trials
Other serious disabling
diseases Pregnancy
Diagnosis of asthma in baseline
(ICD-9 493.0, 493.1, 493.9) >182 days supply of OCS
during baseline or follow-up Switching between ICS/LABA products in year after index date
Participants in other clinical trials
Other serious disabling
diseases Pregnancy
Hospitalizations or ED visits for
asthma OCS fills
SABA doses per week Respiratory specialist visits Ambulatory clinic visits Treatment adherence
Asthma control assessment
questionnaire Doses of SABA per week Spirometry
EQ-5D™
Healthcare utilization
Number of COPD exacerbations
ED visits and hospitalizations for
COPD Claims for OCS Claims for tiotropium Doses of SABA per day Doses of ipratropium per day Treatment adherence
EQ-5D™
Healthcare utilization Exacerbations
Use of other asthma medications
Reference Study population Inclusion criteria Exclusion criteria Outcome measures
[19] US insurance database BFC 3,390 FSC 90,070 3,385 matched 1:1 cohorts Diagnosis of COPD in baseline [ICD-9 491, 492, 496] Age >40 years No fill for ICS/LABA in the previous 6 months Enrolled 6 months pre- and 3-6 months post-index date OCS possession >50 % of baseline Respiratory cancer Healthcare utilization Healthcare costs Exacerbation events COPD and pneumonia-related utilization and costs Adherence to treatment
[20] US insurance database BFC 3,852 FSC 3,852 [11,380 total for SABA analysis] Index dates from 1 January 2007 to 31 January 2009 Diagnosis of COPD in baseline (ICD-9 491, 492, 496) Age >40 years No fill for ICS/LABA in the previous 6 months Enrolled 6 months pre- and 3-6 months post-index date Index dates from 1 January 2006 to 30 June 2010 OCS possession >50 % of baseline Respiratory cancer COPD exacerbations before and after index date Number of SABA fills before and after index date Association between SABA use at baseline and risk of exacerbations during baseline and follow-up
[21] Swedish healthcare database BFC 7,155 FSC 2,738 2,734 matched 1:1 cohorts Diagnosis of COPD (ICD 10-CM code J44) between 1 January 1999 and 31 December 2009 None COPD exacerbations Inhaled medication fills Healthcare utilization
[22] Swedish healthcare database BFC 7,155 FSC 2,738 2,734 matched 1:1 cohorts Diagnosis of COPD (ICD 10-CM code J44) between 1 January 1999 and 31 December 2009 None Yearly pneumonia rate (ICD 10J- 10 to J-18) Time from index date to first pneumonia event. Mortality related to pneumonia
The index date was defined as the first day of ICS/LAB A treatment in each study, except references 8 (first day of montelukast treatment) and 14 (day of study enrollment)
ACQ Asthma Control Questionnaire, ACT Asthma Control Test, BFC budesonide/formoterol, COPD chronic obstructive pulmonary disease, DPI dry powder inhaler, ED emergency department, EF extrafine, EQ-5D™ standardized instrument for generating a single index value for health-related quality of life, FSC fluticasone/salmeterol, FEV1 forced expiratory volume in 1 second, GPRD UK General Practice Research Database, ICD-9 International Classification of Diseases, Ninth Revision, ICD-10 ICD, Tenth Revision, ICD-10-CM ICD-10, clinical modification, ICS inhaled corticosteroid, LABA long-acting (5-agonist, LTRA leukotriene receptor antagonist, OCS oral corticosteroid, SABA short-acting (5-agonist a Patients who completed the 12-month follow-up visit of the PRISMA study
Table 2 The baseline period—patient characteristics prior to the index date
Reference Baseline duration Comorbidity/confounding variables Severity measures Baseline treatments
[9] Not defined None Juniper ACQ score Physician's and Patient's global well-being rating at the index date Not assessed
[10] 12 months Rhinitis, GERD Hospitalization or asthma Specialist referral ICS, LABA, or SABA use OCS use ICS + LABA separately
[11] 12 months ICD-9 diagnoses of sinusitis. Hospitalization or ED visit for ICS or LTRA use
allergic rhinitis, respiratory asthma OCS use <21 days
infections, GERD
SABA use >6 canisters
[12] 12 months Age, sex, region, Charlson Hospitalization or ED visit for ICS use
comorbidity score, insomnia. asthma OCS claims
allergic rhinitis, sinusitis. OCS use >2 courses LTRA use
GERD, psychiatric disorders
SABA use >6 canisters SABA doses per week
[13] 12 months None Number of asthma prescription fills Hospitalization for asthma Other medication and health care use ICS or OCS claims LTRA, xanthine, or ipratropium use SABA, LABA, or SABA/ ipratropium Nasal corticosteroids
[14] 12 months Area of residence, social Hospitalization or ED visit for ICS dose
assistance, respiratory asthma OCS claims
physician referral or as initial LABA, LTRA, and
prescriber theophylline use SABA doses per week
[15] Not defined Allergy history, smoking status Lung function (spirometry, PEF) Asthma control Asthma maintenance and reliever therapies
[16] 3 months Smoking, occupation, educational level, concomitant diseases. ACT Hospitalization or ED visit for asthma or asthma Past asthma therapies
medical asthma consults Specialist referral
[17] 12 months Matched on age, sex, treatment COPD exacerbations (ED visits. Tiotropium, OCS, LABA,
year, # COPD hospitals, OCS fills) theophylline, and ipratropium claims
[18] Not defined Smoking, body mass index, concomitant diseases, EQ-5D™ ACT Not assessed
Reference Baseline duration Comorbidity/confounding variables Severity measures Baseline treatments
[19] 6 months Age, sex, region, treatment year, months of follow-up, Elixhauser comorbidities, asthma, obstructive sleep apnea, heart disease, serious lung conditions COPD hospitalizations Pneumonia hospitalizations COPD ED visits OCS use Antibiotic use SABA use Ipratropium use
[20] 6 months Same as Terzano et al. [18] COPD hospitalizations, ED, or clinic visits OCS, antibiotic, SABA, or ipratropium claims
[21] 2 years Age, sex, diabetes, asthma, cancer, rheumatoid arthritis, heart failure, hypertension and stroke Lung function measurements Outpatient visits for acute COPD Number of previous hospitalizations Number of prescriptions for antibiotics, oral steroids, tiotropium, ipratropium, ICSs, SABAs, LABAs, angiotensin receptor blockers, [¡-blockers, statins, calcium antagonists and thiazides
[22] 2 years Age, sex, diabetes, asthma, cancer, rheumatoid arthritis, heart failure, hypertension and stroke Lung function measurements Outpatient visits for acute COPD Number of previous hospitalizations Number of prescriptions for antibiotics, oral steroids, tiotropium, ipratropium, ICSs, SABAs, LABAs, angiotensin receptor blockers, fS-blockers, statins, calcium antagonists and thiazides
ACT Asthma Control Test, ACQ Asthma Control Questionnaire, COPD chronic obstructive pulmonary disease, ED emergency department, EQ-5D™ standardized instrument for generating a single index value for health-related quality of life, GERD gastroesophageal reflux disease, ICD-9 International Classification of Diseases, Ninth Revision, ICS inhaled corticosteroid, LABA long-acting ß-agonist, LTRA leukotriene receptor antagonist, OCS oral corticosteroid, PEF peak expiratory flow, SABA short-acting ß-agonist
Table 3 The follow-up period—treatment comparisons, adjustments for adherence, and tests of assumptions
Reference
Duration
BFC-FSC comparison method
Treatment adherence
Sensitivity analyses2
2 months 12 months
(cross-section)
12 months or person-years on treatment
12 months
(cross-section)
1 year
Mean change in ACQ score after
initiation of montelukast Logistic regression models for treatment success with adjustment for prior treatment (plus age and physician specialty for secondary definition of success)
Proportions of BFC and FSC cohorts who had indications for combined treatment at the index date Logistic regression models adjusting for demographics, physician specialty, and comorbidities Proportions of BFC and FSC cohorts who had indications for combined treatment before the index date Logistic regression models adjusting for demographics and comorbidities Survival and Poisson regression models
1. Intent-to-treat
2. On-treatment only
Matching on age and baseline asthma severity measures, with Poisson regression models for unmatched confounders Linear regression models for adjusted doses of SABA
At baseline:
proportions of extra-fine BFC and DPI patients with well-controlled asthma Mean symptoms and asthma control
scores At baseline:
proportions of extra-fine BFC, BFC, and FSC users with well-controlled asthma
Not assessed
49.4 % of BFC and 45 % of FSC did
not refill the initial prescription Treatment differences were not significant for the 586 patients with one refill within 4 months Not assessed
Not assessed
20 % of BFC and 15 % of FSC did
not refill the initial prescription Mean duration of persistent use 5 months for each
Mean number of claims in 1 year was used to describe adherence
Not assessed
Only patients who were on therapy for at least 5 days within the last 4 weeks were compared
Global assessment of well-being
Stratified sub-analyses of persons with >1 refill, and for >1 refills within 4 months
Repeated analysis after excluding persons with allergic rhinitis
Comparison of intent-to-treat and
on-treatment analysis methods Age >50 years
First ICS/LABA canister must
have 120 puffs None
Current smokers were compared with non-smokers
Multivariate analyses for EQ-5D™ and healthcare utilization
Reference Duration BFC-FSC comparison method Treatment adherence Sensitivity analyses2
[17] 12 months At 12-month follow-up Only patients who were on therapy for Multivariate analyses for EQ-
Proportions of extra-fine BFC, BFC, the last 4 weeks at the end of the 5D™ and healthcare utilization
and FSC users with well-controlled year were compared (n = 569, from
asthma 1,017 at the beginning of the year and from 739 who were contacted at
Proportions having exacerbations at 1 year)
any time or in last 3 months of
follow-up
[18] 12 months Crude proportions, numbers, and rates of outcomes in the follow-up year Crude and adjusted multivariate relative risks and adjusted mean differences Percent of days with drug supply was 52.3 % for BFC and 51.5 % for FSC, and mean number of fills were 6.9 and 6.8, respectively None
[19] 3-6 months Baseline costs after propensity Medication possession ratio for BFC Log-transformed costs, cost
matching and FSC analysis without outlier
Utilization during follow-up and
respiratory drug fills
[20] 3-6 months COPD utilization before and after index date Exacerbations before and after index date stratified by SABA use Adjusted hazard ratios for exacerbations before and after index date stratified by SABA use Medication possession ratio for BFC and FSC SABA utilization stratified
[21] Unlimited Yearly exacerbation rates and COPD- related events during follow-up Inhaled medication use during follow-up Not assessed Propensity model was varied Poisson regression modified to use only events occurring up to 1 month after first ICS/LABA switch
[22] Unlimited Pneumonia event rates Adjusted rate ratios Proportional hazard models Not assessed Pneumonia rates associated with OCS or LABA use, asthma, age, sex, or pneumonia pre-index
ACQ Asthma Control Questionnaire, BFC budesonide/formoterol, COPD chronic obstructive pulmonary disease, DPI dry powder inhaler, EQ-5D standardized instrument for generating a single index value for health-related quality of life, FSC fluticasone/salmeterol, ICS inhaled corticosteroid, LABA long-acting fi-agonist, NA not applicable, OCS oral corticosteroid, SABA short-acting fi-agonist
a Tests of assumptions or potential biases
Table 4 Study results, conclusions, and analysis limitations
Reference Results Conclusions Limitations2
[9] The effect of add-on montelukast on improving the ACQ score did not differ significantly for FSC (14.3-8.2) and BFC (13.5-6.6) Addition of montelukast for patients with persistent symptoms while on FSC or BFC may result in significantly improved asthma control Open-label study without placebo control
[10] Full treatment success was achieved by 68.3 % BFC users had a higher likelihood of treatment success Poor treatment compliance
of BFC patients as compared to 62.5 % of FSC and lower healthcare costs Retrospective database limitations
patients (p = 0.006)
The odds for full success was greater for BFC in
adjusted models
Asthma-related and total healthcare costs were
lower among the BFC users
[11] More patients in the BFC group had >1 Just under 40 % of ICS/LABA users met the criteria for Incomplete capture of severity information and non-
indication for combined therapy initiation (55.6 appropriate initiation of combined therapy compliance in claims databases
vs. 37.7 % of FSC) [p < 0.001]. The adjusted Patients treated with BFC were more likely to have
odds for appropriate therapy favored BFC indications for controlled therapy
(1.79: 1.60-2.00)
[12] More patients in the BFC group had >1 A significantly higher proportion of BFC users met the Claims database limitations, such as incomplete data
indication for combined therapy initiation (58.4 criteria for appropriate use of ICS/LABA therapy about asthma severity
vs. 37.5 % of FSC) [p < 0.001]. The adjusted
odds for appropriate therapy favored BFC
(2.01: 1.76-2.30)
[13] Asthma hospitalizations and GP visits were BFC and FSC have similar effectiveness Observational data limitations, including slightly
similar BFC users had lower relative doses of inhaled higher severity among FSC users
Time to exacerbation or need to change corticosteroid but were more likely to be prescribed an
controller therapy were similar additional corticosteroid or LABA medication
[14] BFC users had fewer ED visits, hospitalizations Subjects initiating BFC have better outcomes than those Prescription fills may not accurately reflect actual
for asthma, and fills for OCS, and SABAs than initiating FSC treatment use
FSC Diagnosis of asthma is not confirmed
[15] BFC-EF users were more likely to have well-controlled asthma (56.6 vs. 36.2 %) and better daytime symptoms, rescue medication use, and asthma control scores BFC-EF demonstrated better asthma control compared with DPIs None discussed
[16] More BFC-EF users had controlled asthma (ACT Asthma control was achieved in a majority of this Observational study design with potential for
score >20) than BFC DPI or FSC users (76 vs population, with the results favoring the EF BFC unmeasured or unrecognized confounding factors
69 % and 71 %, respectively) formulation
[17] Among patients who completed the 12-month Improved asthma control was observed over the Observational study design with potential for
follow-up visit: 12-month follow-up in each treatment group unmeasured or unrecognized confounding factors
More BFC-EF users had controlled asthma (ACT BFC-EF had significantly better asthma control and
score >20) than BFC-DPI or FSC users (83 vs quality-of-life measures as compared to the large
79 % and 77 %, respectively) particle BFC-DPI and FSC users
EQ-5D™ scores were higher for BFC-EF and
FSC users
Reference
Results
Conclusions
Limitations2
COPD exacerbations, OCS claims, use of SABAs, and treatment adherence were not significantly different BFC users were less likely to have a COPD-related ED visit or hospitalization, or claims for tiotropium or ipratropium bromide
There were no significant differences in COPD-related utilization events, including hospitalizations, ED visits, and clinic visits, and no difference in exacerbations or total costs. BFC users had fewer fills for SABA and ipratropium in the follow-up period, but there were no differences in other drugs
Exacerbations resulting in hospitalizations or ED visits are reduced by BFC and FSC by equivalent amounts, approximately 20 % overall
Persons who have the greatest SABA use at
baseline have the most benefit Increased SABA use is associated with increased COPD exacerbations at baseline and follow-up BFC users had lower rates of exacerbations and
exacerbation-related utilization BFC users had fewer fills for other inhaled
medications, except formoterol FSC users had higher rates of pneumonia and mortality related to pneumonia, although all-cause mortality was the same
Patients treated with BFC were less likely to have ED visits and hospitalizations for COPD, and used less anticholinergic medication. However, due to the observational study design, "we cannot conclude with certainty that the medication was the only factor responsible for the observed differences"
For most outcomes of interest, BFC and FSC showed comparable real-world effectiveness
BFC and FSC had equivalent effectiveness in reduction in ED visits and hospitalizations for COPD exacerbations, but BFC users had lower subsequent need for SABA fills. Increased SABA use is a predictor of exacerbations among COPD patients treated in the general population
Long-term treatment with BFC was associated with fewer exacerbations than persons with FSC in patients with moderate to severe COPD
There is a difference between BFC and FSC in the risk of pneumonia and pneumonia-related events among COPD patients
Observational study design with potential for biases Level of COPD severity not directly measured Possible residual confounding COPD diagnoses not all confirmed by spirometry Excluded individuals who switched ICS/LABA
medications Observational study design limitations
The accuracy of COPD diagnosis could not be verified
Limited information on disease severity Results and conclusions are limited to the population observed
Possible residual confounding COPD diagnoses not all confirmed by spirometry 6-month follow-up time prevents study of longer-term effects
Retrospective data limitations Unmeasured confounding
COPD diagnoses not all confirmed by spirometry
Unknown/unmeasured confounding factors Accuracy of COPD or pneumonia diagnoses could not be verified
ACQ Asthma Control Questionnaire, ACT Asthma Control Test, BFC budesonide/formoterol, COPD chronic obstructive pulmonary disease, DPI dry powder inhaler, ED emergency department, EF extrafine, EQ-5D standardized instrument for generating a single index value for health-related quality of life, FSC fluticasone/salmeterol, GP general practice, ICS inhaled corticosteroid, LABA long-acting fi-agonist, OCS oral corticosteroid, SABA short-acting fi-agonist a Major study limitations as noted by the authors
assumption may not be true in clinics where free drug samples are dispensed, wherein there may be a gap of weeks to months between the office visit when treatment was started and the first ICS/LABA prescription drug fill, resulting in a bias in follow-up time. Therefore, one needs to be familiar with the policies about sampling in the health systems one is studying and the likelihood that patients are dispensed medications that do not appear in the database.
Unfortunately, studies rarely describe what was happening to the patient on the index date that caused them to be treated. These circumstances are important because they are likely to affect the patient's outcome during follow-up. For example, if the patient was experiencing an asthma or COPD exacerbation at the time it was prescribed, their risk for another exacerbation during the next few months will be substantially higher than for someone who had not had a recent exacerbation [27, 28]. If the ICS/LABA prescription was provided during an emergency department visit, it is likely that the patient will have different adherence with long-term treatment than someone who is prescribed one during a scheduled office visit [29]. If the prescription is written by a pulmonologist, then it is more likely that there will have been a pulmonary function test or other testing confirming the diagnosis than if written by a primary care provider. If the index date is at or near the time of a hos-pitalization, then associated comorbidities such as heart disease or pneumonia may be more likely to affect subsequent outcomes [30].
One way to deal with the problem ofheterogeneous clinical presentations is to match patients based on the clinical setting where they started ICS/LABA treatment, such as a clinic visit, ED visit, or hospitalization. Another way is to match patients based on the specialty of the prescribing physician, such as pulmonary or allergy specialists versus primary care provider. In studies where COPD or asthma exacerbations are an outcome of interest, it is very important to match on exacerbations that occurred on or near the index date because one of the best predictors of future exacerbations is the history of prior exacerbations [27, 31, 32].
One also needs to be aware that physicians may not be free to choose which ICS/LABA product the patient will get; the choice is often made by pharmacy benefit managers who negotiate a lower price for one ICS/LABA product or the other, which then becomes the more readily available product on the formulary [33]. In that circumstance, the fact that a patient is not using the less expensive product is an indication that there is something unusual about the patient that may affect their outcomes. When comparing COPD or asthma patients among several health systems, one must be aware of those with a formulary that heavily favors one product because differences among heavily biased cohorts may be more likely to reflect group clinical characteristics than treatment effects.
4.1 Index Dates in Cross-Sectional Studies
A few of the CER studies in asthma were based in respiratory clinics and are purely cross-sectional, meaning that all patients at the initiation of the research have already been taking their medications for weeks to years before the first day of the study (Table 1). The index date for these cross-sectional studies is usually the date of enrollment, and because of the variable length of medication use at the start, lead time bias is inevitable. Clinic-based studies are necessary when the endpoints of interest are asthma symptom control and health-related well-being, which are measured using standardized questionnaires. These projects also tended to be more similar to RCTs in that they excluded persons who had a recent exacerbation, they excluded those with COPD or other serious chronic diseases, and they included a relatively homogeneous selected population of persons who most likely had severe asthma at one time and were thus referred to a specialist's office. While reducing confounding influences, these criteria also limit the generalizability of the results. Unlike RCTs, these projects are obviously not blinded or randomized, making them highly susceptible to the biases that affect any observational study.
4.2 Misclassification Errors in Asthma and COPD Comparative Effectiveness Research (CER)
Misclassification errors (misdiagnoses) are common among persons labeled with either asthma or COPD. One primary care-based clinical study found that almost half of their patients with a physician diagnosis of COPD did not have airflow obstruction when tested by spirometry [34]. In addition, in a study of 496 randomly selected patients diagnosed with asthma from eight cities in Canada, 150 (30 %) did not have asthma after a complete evaluation including methacholine challenge testing. [35] For asthma, there can also be difficulties in accurately distinguishing between levels of disease severity [36-38]. To decrease the risk of disease misclassification, some database studies require multiple outpatient COPD or asthma diagnoses over a specified time period, or at least one diagnosis during a hospitalization. Hospitalization diagnoses are assumed to be more valid because patients are more likely to have had a more thorough assessment than during an office visit, although that is not always a safe assumption. [39]
4.3 Inclusion and Exclusion Criteria
In asthma or COPD CER studies there is a tendency to use the same inclusion and exclusion criteria used in asthma or COPD RCTs. It is often driven by a desire to demonstrate
effectiveness in CER that is similar to efficacy in RCT. However, when doctors in the real-world prescribe asthma or COPD medications, they are not limited to the strict inclusion and exclusion criteria that are used for RCTs. Like it or not, physicians will immediately begin testing the boundaries unexplored by RCTs as soon as a drug is on the market. Relative effectiveness of treatment in the general population should reflect treatment in the general population. If there is a desire to replicate RCT results, then secondary analyses that systematically examine the impact of various inclusion and exclusion criteria can serve that purpose and additionally may yield unique and useful information.
Exclusion criteria deserve special consideration in CER studies because that is where the unintended effects of new treatments are likely to be found. Drug studies attempting to prove efficacy in RCTs typically exclude unstable patients, those with severe comorbidities, or persons who are not likely to be compliant with therapy, because these patients increase safety monitoring concerns while also reducing the power to observe clinical benefits. However, excluding these patients also reduces the power of RCTs to capture negative side effects, particularly those that are more likely in the excluded populations. One of the unique advantages of CER is that the risk or benefits of treatment for these excluded, unstudied patients can now be captured and described.
It is common for COPD patients to have other respiratory diagnoses, especially asthma. In a cohort of 42,565 COPD patients treated in managed care systems in the US, 27 % also had an asthma diagnosis within the last year [40]. The impact on measured outcomes of having a combined asthma and COPD diagnosis is variable depending on the outcome, but most studies have found increased risk for exacerbations [41]. Furthermore, most observational database studies do find that a combined diagnosis of COPD and asthma does affect treatment, particularly the selection of ICS/LABA combinations versus inhaled anti-muscarinic treatments [42]. Unfortunately, most CER studies in COPD to date have simply excluded patients with any diagnosis of asthma. Lung cancer is common in older COPD cohorts (4-7 % in cross-sectional cohorts) [40], and because of its very high case fatality rate, it has a significant impact in longitudinal studies [43]. Depending on the objectives of the study, investigators may need to make a systematic effort to capture specific respiratory comorbidities.
Another common way to reduce selection bias and create balanced ICS/LABA cohorts similar to RCT populations is to use case-control methods. It is possible to match on just a few major demographic and clinical factors [17], but most studies have used propensity score matching (PSM) techniques which allow more comprehensive
matching using a broader array of clinical variables (Table 1). The obvious benefit of matching is that known confounding factors are more likely to be balanced among the comparison groups, but PSM still has a number of very important limitations [44]. First, matching limits the study cohorts to the size of the smaller treatment population. If one of the treatment groups is substantially smaller than the other, then most of the persons in the larger group are dropped from the analysis, which leads to sample selection bias and may seriously compromise the generalizability of the results. Second, PSM will help balance the selected variables, but any unselected or unmeasured variables that affect outcomes may continue to be unbalanced. This is very important in observational studies where attitudes about treatment and adherence with therapy are often not measurable. Finally, when one matches on a clinical parameter, then one's ability to measure how that factor affects the outcomes is compromised [45]. For example, if men are more likely than women to benefit from the reduction in COPD or asthma exacerbations attributable to ICS-LABA treatment, then matching by sex will bias the risk estimates for exacerbations among men towards the null, and eliminate the possibility of examining interactive effects between ICS-LABA treatment, sex, and exacerbation risk.
5 The Baseline Period
Utilization data collected during the baseline period is often used to establish the diagnosis of asthma or COPD, capture items that are associated with disease severity, and identify comorbid diseases and conditions (Table 2). The duration of the baseline period is typically 12 months, which helps to ensure that the treatment dispensed on the index date is a new treatment. Importantly, a 12-month baseline also allows for the seasonal variability in asthma and COPD exacerbations. Within the baseline period, the timing of events may be important, in particular those occurring just prior to the index date. For example, an emergency department visit with a discharge diagnosis of bronchitis that occurred 1 week prior to hospitalization for asthma is most likely a very different syndrome than that of a patient who was seen in a pulmonologist's office
6 months before the index date with a chronic cough.
The approach to dealing with confounders is one of the
most important differences between RCTs and CER. In randomized trials, treatment groups have randomly and equally distributed known (e.g. age, comorbidities, sex) and unknown (e.g. preferences and attitudes toward treatment) confounding influences, leaving treatment as the major difference between groups that may impact outcomes. The challenge of CER is to adequately adjust for
the many confounding factors that exist in these retrospective treatment populations. In asthma and COPD, comorbid illnesses such as cardiovascular disease or psychiatric illness are examples of known and very complex confounders. It is important to keep in mind that even if it were possible to fairly balance every known confounding factor retrospectively, unknown factors will remain unbalanced and could become sources of residual confounding.
To capture baseline comorbidities that are likely to affect subsequent treatment choices or outcomes, it is best to use one of the standardized classification systems that have been adapted for use in electronic databases containing International Classification of Diseases, Ninth Revision (ICD-9) or ICD, Tenth Revision (ICD-10) codes [46]. The Charlson Comorbidity Index, and database versions such as the Charlson-Deyo Index, are popular for identifying and weighting prognostically significant comorbidities, although the validity of the original weighting scheme is limited due to the improvement in some disease outcomes since the first version was published over 20 years ago [47]. The Elixhauser system captures a broader range of comorbidities, and software for database versions are publically available on the US Agency for Healthcare Research and Quality website [48]. However, there are comorbidities of special interest in asthma and COPD, such as obstructive sleep apnea, cor pulmonale, allergic rhinitis, vascular disease, gastroesophageal reflux, and cardiac arrhythmias, that are missed by some of these standardized systems. Therefore, it is usually necessary to supplement standardized comorbidity classification tools with the other respiratory diseases and conditions associated with COPD and asthma that are likely to affect outcomes.
The baseline period may also be used to estimate the severity of asthma or COPD. It is reasonable to expect that patients with unstable asthma or COPD will have more hospitalizations and unscheduled clinic visits. Emergency department visits may also be indicators for severity; however, many patients, even in managed care systems, may use emergency departments as their primary care clinic, and thus emergency department visits not leading to hospitalization tend to be more like clinic visits than hospital stays [27]. Patients with more severe disease may also have more documentation of respiratory symptoms in the form of diagnosis codes; for example, dyspnea (ICD-9 codes 786.05/786.09) or wheezing (ICD-9 codes 786.07) [49]. Oxygen use, or codes for hypoxemia or respiratory failure, are also evidence of severe lung disease.
Medication use during the baseline period is also a useful indicator of respiratory disease severity. Increased rescue medication use (either short-acting p-agonists and/ or ipratropium bromide) in the baseline period is associated
with increased COPD exacerbations before and after the index date [20]. Increased use of rescue medication is also a marker for poor asthma control [50]. Thorough description of baseline use of both the total number and types of medications used (e.g. oral steroids, xanthenes, leukotriene inhibitors) is useful for characterization of the study populations.
6 The Follow-up Period
The follow-up period has unique challenges (Table 3). The biggest challenge is dealing with the poor treatment adherence among asthma and COPD patients. As previously noted, the index date is commonly associated with an exacerbation of asthma or COPD. Fortunately, the natural history of most COPD exacerbations is that the symptoms will resolve and the patient will return to their usual health status prior to the exacerbation [51]. But as asthma and COPD symptoms decrease, the benefits of maintenance medications such as ICS/LABAs become less obvious to patients, and they quite often conclude that they do not need them anymore. Experience varies, but approximately 25 % of persons initially dispensed an ICS/LABA will not get it refilled, and by 6 months more than half will have discontinued it [29]. Although adherence with combined inhalers tends to be better than for use of the individual component medications, studies of ICS/LABA combinations show that continuity of treatment in either asthma or COPD is very poor [52-54]. Even in RCTs such as the TORCH study where patients were very closely monitored, 44 % of those on placebo and 34 % on full treatment stopped the medication before the end of the study, resulting in a bias that affected the results [55].
The conventional treatment comparison method for RCTs is the intent-to-treat analysis, wherein patients are kept in their original treatment groups even if they have stopped taking the medicine. Many respiratory CER studies also simply follow the intent-to-treat design, even though the adherence with treatment is far worse than that of RCTs and a large proportion of patients labeled as treated with BFC or FSC were dispensed only one inhaler for the entire follow-up period. This obviously creates misclassification errors, and importantly makes it far more likely that any observed differences between treatment groups are biased by factors associated with selection of treatment than by any effect of the treatment itself.
While patients who are non-compliant create mis-classification errors and other biases, the patients who are compliant with ICS/LABA treatment create another problem—bias by indication. Patients who have more severe asthma and COPD are more likely to experience chronic symptoms, have higher risk of recurrent exacerbations, and
thus use more respiratory medications. Without effectively addressing the problem of bias by indication, ICS/LABA combinations can appear to increase the risk of asthma and COPD exacerbations in the general populations, when in fact the treatment is simply acting as a marker for more severe disease.
Although treatment adherence is a fundamental problem for CER studies, most ICS/LABA studies to date have not been very thorough in capturing or describing how medications were actually used, or adjusting for adherence in the analysis (Table 3). There are notable exceptions; for example, the project by Suissa et al. [13] specifically addressed the adherence problem by using both intent-to-treat analyses and analyses while on-treatment only. Adjusting for treatment compliance during follow-up is a very difficult problem that is the subject of ongoing research. Because of the problem of bias by indication, simply making treatment a time-dependent variable is more likely to introduce bias than adjust for it [56]. Micro-simulation models and marginal structural models are new techniques that address the complex competing problems of disease severity and treatment adherence, but have only recently been applied to COPD [57].
Exacerbations are a popular endpoint for CER because they have significant impact on other disease outcomes such as quality of life, healthcare costs, hospitalizations, disability, and mortality [58]. However, defining exacerbations is controversial, especially in retrospective studies. The most common definitions for exacerbations of CER are based on utilization, which is a very practical approach for studies where utilization and cost are major endpoints [59]. Outpatient exacerbations are usually defined as outpatient visits associated with prescription fills for oral corticoste-roids (OCSs) and/or antibiotics, standard treatments for acute asthma and COPD exacerbations. Occasional use of oral steroids can be a reasonable indicator of an acute exacerbation, but use of only antibiotics is less clear, especially now that chronic use of azithromycin to prevent COPD exacerbations is growing in popularity. If exacerbations are the primary endpoint in CER, chart abstraction may be needed to validate the sensitivity and specificity of exacerbation definitions.
When utilization is used to define exacerbations, one also has to decide how closely the outpatient or hospital visit must be associated with the prescription fill to reasonably conclude that they are related. Most have required that fills occur within 3 days of a respiratory-related visit. Exacerbations of asthma and COPD are known to linger from a few days to several weeks, and in one prospective clinical study of exacerbations, the median duration was 12 days [60]. It is important to designate the expected duration of exacerbations so that follow-up visits for one exacerbation do not get counted as multiple exacerbations.
There are no methods that have near perfect sensitivity or specificity for capturing COPD or asthma exacerbations, but the utilization method has demonstrated useful reliability across many patient populations and study designs.
Misclassification errors and extreme outliers can be a problem in the follow-up period, particularly for persons labeled with asthma who have vocal cord dysfunction or conversion disorders, or COPD patients who have hospi-talizations that last for weeks. Some misdiagnosed asthma patients can have multiple hospitalizations, including events that result in intubation and intensive care unit stays. Pneumonia is often the cause of acute exacerbations of COPD that result in hospitalization, and hospitalizations for COPD and pneumonia are substantially longer and more costly than those without pneumonia [61]. Just a few outliers or misclassified patients can substantially skew clinical outcomes, especially costs, and bias CER of ICS/ LABA combinations [58]. Projects using administrative databases need to be aware of the possibility of extreme outliers and develop decision rules a priori for exclusion of extraordinary cases that bias results.
Confounding by events associated with comorbidities is also a problem in the follow-up period. In a nationwide, population-based study of comorbidities associated with COPD, Baty et al. [30] identified a few with the most prognostic significance among COPD patients who had hospitalizations related to COPD (N = 160,317 patients). Lung cancers, lymphatic neoplasms, obesity-hypoventila-tion syndrome, pseudomonas pneumonia, and secondary polycythemia were associated with reduced time to rehospitalization, while asthma was associated with longer time to rehospitalization. The large number of comorbidi-ties associated with COPD as compared to age- and sex-matched controls suggests that a CER project is very likely to find random differences in comorbidities between treated COPD groups by chance alone.
6.1 Sensitivity Analyses
Sensitivity analyses are secondary analyses that are intended to examine assumptions about important covariates, the impact of specific inclusion/exclusion criteria, and the differences among various analysis methods. Sensitivity analyses are especially important in studies that use multivariable analyses, where the effects of interactions among covariates can be obscured. Inherent in any study are a multitude of decisions about classifications and methods. To the extent possible, decisions that have a greater likelihood in impacting findings should be investigated through sensitivity analyses. For example, decisions about exacerbation event duration, assumptions about treatment adherence, or definitions of exacerbations that could vary by the number of days between a doctor's visit with a primary
respiratory diagnosis and an outpatient claim for an OCS, may need to be tested across a range of alternate values to see what impact these assumptions may have had on the final results.
7 Interpretation of Results and Conclusions in CER
An important aspect of CER studies in asthma and COPD is that they often have multiple outcome measures of interest (Table 1) which increases the probability that there will be at least one 'statistically significant' difference between treatments by chance alone (Table 4). Another aspect to consider is that the database studies often include thousands of patients, so their size alone gives them power to detect very small differences between the treatment groups. However, a small difference between treatment groups with ap-value well below 0.05, or even 0.001, does not prove that the difference is either clinically significant or due to a treatment effect. Absolute standardized differences can provide better information than p-values about effect size, and are starting to appear more in published CER studies. Statistical tests help determine the degree to which differences can be attributed to random errors, but study design factors are more important determinants of establishing causality [62]. One must always keep in mind that retrospective analyses, because of their high likelihood of bias, are not reliable proof of causal associations. However, that is not to say that the results are not valuable. An analysis that explores the sources of variability and imprecision among outcomes can reveal very useful insights into the clinical factors that help determine the success and failures of treatments in real life.
Determining a clinically important and significant difference between treatment groups in CER, as opposed to a statistically significant difference, is a subjective assessment. In most retrospective comparisons from observational databases, odds ratio differences of less than 2.0 should be regarded with caution [63]. A helpful way to understand the clinical relevance of a difference in outcome between two treatments is to present a comprehensive assessment of all demographic and clinical factors associated with an outcome in the study population, then compare the magnitude of the treatment-related differences with the other clinical factors affecting outcome [64]. For example, a 72 % difference in pneumonia incidence between two COPD treatment groups with p-value <0.001 may sound like an important finding [22]. However, if the incidence of pneumonia more than doubles with COPD severity, various comorbidities, and advanced age, then a 72 % difference could be within the range expected from selection biases, misclassification errors, and unmeasured confounders [65].
8 Conclusions
Naturally there is a lot of interest in which treatment is better when there are choices within the same class. However, because of the limitations in retrospective study designs and databases, retrospective analyses of population databases and observational cohorts can rarely be conclusive about the causal associations between treatments and outcomes. In our compulsion to see which treatment is better, we often lose sight of the fact that both treatments are substantially improving outcomes as compared to those not treated, and the relative difference between treatments is within the range of random variation, not only statistically but also in terms of the robustness of the study design. The intense focus on proving which treatment is 'better' can be a distraction away from understanding their role and relationships with the other determinates of overall health outcomes for asthma and COPD patients.
Given the many potential sources of bias that we have identified in CER studies of asthma and COPD, one might wonder if these studies are at all reliable or interpretable. However, if one embraces these sources of variation instead of hiding or ignoring them, and one directly addresses them in CER studies, then a truer depiction of how asthma and COPD patients are managed in real-life can be recognized. Both reproducibility, referring to method or measurement precision, and repeatability, referring to agreement across similar populations, are important concepts for both RCTs and CER studies. Indeed, much of the value of CER studies lies in the ability to conduct multiple CER studies using equivalent methods in an efficient manner using minimal resources. Additional CER is needed to understand how these treatments are being used in patients in the general population who do not fit the restrictive inclusion and exclusion criteria of RCTs. There is also a need for studies that compare various analysis approaches and statistical methods to deal with some of the study design limitations we have discussed, particularly in treatment adherence and bias by indication.
Source of Funding None.
Conflict of interest Douglas W. Mapel and Melissa H. Roberts are previous recipients of research grants from AstraZeneca, Glaxo-SmithKline, Boehringer Ingelheim, and Pfizer Inc.
Open Access This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.
References
1. Institute of Medicine. Initial national priorities for comparative effectiveness research. Washington, DC: The National Academies Press; 2009.
2. Edwards SJ, Gruffydd-Jones K, Ryan DP. Systematic review and meta-analysis of budesonide/formoterol in a single inhaler. Curr Med Res Opin. 2007;23(8):1809-20.
3. Bateman E, Nelson H, Bousquet J, Kral K, Sutton L, Ortega H, et al. Meta-analysis: effects of adding salmeterol to inhaled corticosteroids on serious asthma-related events. Ann Intern Med. 2008;149(1):33-42.
4. Mapel DW, Hurley JS, Dalal AA, Blanchette CM. The role of combination inhaled corticosteroid/long-acting beta-agonist therapy in COPD management. Prim Care Respir J. 2010;19(2): 93-103.
5. Tamm M, Richards DH, Beghe B, Fabbri L. Inhaled corticosteroid and long-acting beta2-agonist pharmacological profiles: effective asthma therapy in practice. Respir Med. 2012;106(Suppl 1):S9-19.
6. Lasserson TJ, Ferrara G, Casali L. Combination fluticasone and salmeterol versus fixed dose combination budesonide and for-moterol for chronic asthma in adults and children. Cochrane Database Syst Rev. 2011;(12):CD004106.
7. Nannini LJ, Poole P, Milan SJ, Holmes R, Normansell R. Combined corticosteroid and long-acting beta2-agonist in one inhaler versus placebo for chronic obstructive pulmonary disease. Cochrane Database Syst Rev. 2013;11:CD003794.
8. Qaseem A, Wilt TJ, Weinberger SE, Hanania NA, Criner G, van der Molen T, et al. Diagnosis and management of stable chronic obstructive pulmonary disease: a clinical practice guideline update from the American College of Physicians, American College of Chest Physicians, American Thoracic Society, and European Respiratory Society. Ann Intern Med. 2011;155(3):179-91.
9. Dupont L, Potvin E, Korn D, Lachman A, Dramaix M, Gusman J, et al. Improving asthma control in patients suboptimally controlled on inhaled steroids and long-acting beta2-agonists: addition of montelukast in an open-label pilot study. Curr Med Res Opin. 2005;21(6):863-9.
10. Aballea S, Cure S, Vogelmeier C, Wiren A. A retrospective database study comparing treatment outcomes and cost associated with choice of fixed-dose inhaled corticosteroid/long-acting beta-agonists for asthma maintenance treatment in Germany. Int J Clin Pract. 2008;62(12):1870-9.
11. Blanchette CM, Culler SD, Ershoff D, Gutierrez B. Association between previous health care use and initiation of inhaled corti-costeroid and long-acting beta2-adrenergic agonist combination therapy among US patients with asthma. Clin Ther. 2009;31(11): 2574-83.
12. Ye X, Gutierrez B, Zarotsky V, Nelson M, Blanchette CM. Appropriate use of inhaled corticosteroid and long-acting beta(2)-adrenergic agonist combination therapy among asthma patients in a US commercially insured population. Curr Med Res Opin. 2009;25(9):2251-8.
13. Suissa S, Dell'Aniello S, Ernst P. Effectiveness of combination therapies in asthma: an observational study. Pulm Pharmacol Ther. 2009;22(3):194-8.
14. Blais L, Beauchesne MF, Forget A. Acute care among asthma patients using budesonide/formoterol or fluticasone propionate/ salmeterol. Respir Med. 2009;103(2):237-43.
15. Muller V, Galffy G, Eszes N, Losonczy G, Bizzi A, Nicolini G, et al. Asthma control in patients receiving inhaled corticosteroid and long-acting beta2-agonist fixed combinations. A real-life study comparing dry powder inhalers and a pressurized metered dose inhaler extrafine formulation. BMC Pulm Med. 2011;11:40.
16. Allegra L, Cremonesi G, Girbino G, Ingrassia E, Marsico S, Nicolini G, et al. Real-life prospective study on asthma control in Italy: cross-sectional phase results. Respir Med. 2012;106(2): 205-14.
17. Blais L, Forget A, Ramachandran S. Relative effectiveness of budesonide/formoterol and fluticasone propionate/salmeterol in a
1-year, population-based, matched cohort study of patients with chronic obstructive pulmonary disease (COPD): effect on COPD-related exacerbations, emergency department visits and hospi-talizations, medication utilization, and treatment adherence. Clin Ther. 2010;32(7):1320-8.
18. Terzano C, Cremonesi G, Girbino G, Ingrassia E, Marsico S, Nicolini G, et al. 1-year prospective real life monitoring of asthma control and quality of life in Italy. Respir Res. 2012;13:112.
19. Roberts M, Mapel D, Petersen H, Blanchette C, Ramachandran S. Comparative effectiveness of budesonide/formoterol and flutica-sone/salmeterol for COPD management. J Med Econ. 2011;14(6):769-76.
20. Mapel D. Effectiveness of inhaled combined corticosteroid/long-acting bronchodilator treatment in reducing COPD exacerbations and short-acting bronchodilator use. J Clin Outcomes Manag. 2013;20(2):60-8.
21. Larsson K, Janson C, Lisspers K, Jorgensen L, Stratelis G, Telg
G, et al. Combination of budesonide/formoterol more effective than fluticasone/salmeterol in preventing exacerbations in chronic obstructive pulmonary disease: the PATHOS study. J Intern Med. 2013;273(6):584-94.
22. Janson C, Larsson K, Lisspers KH, Stallberg B, Stratelis G, Goike
H, et al. Pneumonia and pneumonia related mortality in patients with COPD treated with fixed combinations of inhaled cortico-steroid and long acting beta2 agonist: observational matched cohort study (PATHOS). BMJ. 2013;346:f3306.
23. Mapel DW, Robinson SB, Dastani HB, Shah H, Phillips AL, Lydick E. The direct medical costs of undiagnosed chronic obstructive pulmonary disease. Val Health. 2008;11(4):628-36.
24. Akazawa M, Halpern R, Riedel AA, Stanford RH, Dalal A, Blanchette CM. Economic burden prior to COPD diagnosis: a matched case-control study in the United States. Respir Med. 2008;102(12):1744-52.
25. Yeatts K, Davis KJ, Sotir M, Herget C, Shy C. Who gets diagnosed with asthma? Frequent wheeze among adolescents with and without a diagnosis of asthma. Pediatrics. 2003;111(5 Pt 1):1046-54.
26. Adams RJ, Wilson DH, Appleton S, Taylor A, Dal Grande E, Chittleborough CR, et al. Underdiagnosed asthma in South Australia. Thorax. 2003;58(10):846-50.
27. Mapel DW, Schum M, Lydick E, Marton JP. A new method for examining the cost savings of reducing COPD exacerbations. Pharmacoeconomics. 2010;28(9):733-49.
28. Lane SJ, Petersen H, Seltzer JM, Blanchette CM, Navaratnam P, Allen-Ramey F, et al. Moderate symptom-based exacerbations as predictors of severe claims-based exacerbations in asthma. J Asthma. 2013;50(6):642-8.
29. Charles MS, Blanchette CM, Silver H, Lavallee D, Dalal AA, Mapel D. Adherence to controller therapy for chronic obstructive pulmonary disease: a review. Curr Med Res Opin. 2010;26(10): 2421-9.
30. Baty F, Putora PM, Isenring B, Blum T, Brutsche M. Comor-bidities and burden of COPD: a population based case-control study. PloS One. 2013;8(5):e63285.
31. McCarren M, McDermott MF, Zalenski RJ, Jovanovic B, Marder D, Murphy DG, et al. Prediction of relapse within eight weeks after an acute asthma exacerbation in adults. J Clin Epidemiol. 1998;51(2):107-18.
32. Hurst JR, Vestbo J, Anzueto A, Locantore N, Mullerova H, TalSinger R, et al. Susceptibility to exacerbation in chronic obstructive pulmonary disease. N Engl J Med. 2010;363(12): 1128-38.
33. Ali OG, Mantrala M. Pharma rebates, pharmacy benefit managers and employer outcomes. Health Care Manag Sci. 2010;13(4): 281-93.
34. Joo MJ, Au DH, Fitzgibbon ML, McKell J, Lee TA. Determinants of spirometry use and accuracy of COPD diagnosis in primary care. J Gen Intern Med. 2011;26(11):1272-7.
35. Pakhale S, Sumner A, Coyle D, Vandemheen K, Aaron S. (Correcting) misdiagnoses of asthma: a cost effectiveness analysis. BMC Pulm Med. 2011;11:27.
36. Teeter JG, Bleecker ER. Relationship between airway obstruction and respiratory symptoms in adult asthmatics. Chest. 1998;113(2):272-7.
37. Bacharier LB, Strunk RC, Mauger D, White D, Lemanske RF Jr, Sorkness CA. Classifying asthma severity in children: mismatch between symptoms, medication use, and lung function. Am J Respir Crit Care Med. 2004;170(4):426-32.
38. Fuhlbrigge AL, Carey VJ, Finkelstein JA, Lozano P, Inui TS, Weiss ST, et al. Validity of the HEDIS criteria to identify children with persistent asthma and sustained high utilization. Am J Manag Care. 2005;11(5):325-30.
39. Thomsen RW, Lange P, Hellquist B, Frausing E, Bartels PD, Krog BR, et al. Validity and underrecording of diagnosis of COPD in the Danish National Patient Registry. Respir Med. 2011;105(7):1063-8.
40. Mapel DW, Dutro MP, Marton JP, Woodruff K, Make B. Identifying and characterizing COPD patients in US managed care. A retrospective, cross-sectional analysis of administrative claims data. BMC Health Serv Res. 2011;11:43.
41. Menezes AM, Montes de Oca M, Perez-Padilla R, Nadeau G, Wehrmeister FC, Lopez-Varela MV, et al. Increased risk of exacerbation and hospitalization in subjects with an overlap phenotype: COPD-asthma. Chest. 2014;145(2):297-304.
42. Dalal AA, Roberts MH, Petersen HV, Blanchette CM, Mapel DW. Comparative cost-effectiveness of a fluticasone-propionate/ salmeterol combination versus anticholinergics as initial maintenance therapy for chronic obstructive pulmonary disease. Int J Chron Obstruct Pulm Dis. 2011;6:13-22.
43. Anthonisen NR, Skeans MA, Wise RA, Manfreda J, Kanner RE, Connett JE, et al. The effects of a smoking cessation intervention on 14.5-year mortality: a randomized clinical trial. Ann Intern Med. 2005;142(4):233-9.
44. Streiner DL, Norman GR. The pros and cons of propensity scores. Chest. 2012;142(6):1380-2.
45. Schlesselman JJ. Case-control studies. New York: Oxford University Press; 1982.
46. Austin SR, Wong YN, Uzzo RG, Beck JR, Egleston BL. Why summary comorbidity measures such as the Charlson comorbid-ity index and Elixhauser score work. Med Care. Epub 23 May 2013.
47. Needham DM, Scales DC, Laupacis A, Pronovost PJ. A systematic review of the Charlson comorbidity index using Canadian administrative databases: a perspective on risk adjustment in critical care research. J Crit Care. 2005;20(1):12-9.
48. Southern DA, Quan H, Ghali WA. Comparison of the Elixhauser and Charlson/Deyo methods of comorbidity measurement in administrative data. Med Care. 2004;42(4):355-60.
49. Mapel DW, McMillan GP, Frost FJ, Hurley JS, Picchi MA, Ly-dick E, et al. Predicting the costs of managing patients with chronic obstructive pulmonary disease. Respir Med. 2005;99(10): 1325-33.
50. Silver HS, Blanchette CM, Kamble S, Petersen H, Letter MA, Meddis D, et al. Relationship between short-acting beta2-
adrenergic agonist use and healthcare costs. Am J Manag Care. 2011;17(1):19-27.
51. Seemungal TA, Donaldson GC, Bhowmik A, Jeffries DJ, Wed-zicha JA. Time course and recovery of exacerbations in patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med. 2000;161(5):1608-13.
52. Cramer JA, Bradley-Kennedy C, Scalera A. Treatment persistence and compliance with medications for chronic obstructive pulmonary disease. Can Respir J. 2007;14(1):25-9.
53. Foster JM, Smith L, Bosnic-Anticevich SZ, Usherwood T, Sawyer SM, Rand CS, et al. Identifying patient-specific beliefs and behaviours for conversations about adherence in asthma. Intern Med J. 2012;42(6):e136-44.
54. Patel M, Perrin K, Pritchard A, Williams M, Wijesinghe M, Weatherall M, et al. Accuracy of patient self-report as a measure of inhaled asthma medication use. Respirology. 2013;18(3): 546-52.
55. Vestbo J, Anderson JA, Calverley PM, Celli B, Ferguson GT, Jenkins C, et al. Bias due to withdrawal in long-term randomised trials in COPD: evidence from the TORCH study. Clin Respir J. 2011;5(1):44-9.
56. Fisher LD, Lin DY. Time-dependent covariates in the Cox proportional-hazards regression model. Annu Rev Public Health. 1999;20:145-57.
57. Suarez D, Borras R, Basagana X. Differences between marginal structural models and conventional models in their exposure effect estimates: a systematic review. Epidemiology. 2011; 22(4):586-8.
58. Mapel DW, Roberts MH. New clinical insights into chronic obstructive pulmonary disease and their implications for phar-macoeconomic analyses. Pharmacoeconomics. 2012;30(10): 869-85.
59. Rodriguez-Roisin R. Toward a consensus definition for COPD exacerbations. Chest. 2000;117(5 Suppl 2):398S-401S.
60. Aaron SD, Donaldson GC, Whitmore GA, Hurst JR, Ramsay T, Wedzicha JA. Time course and pattern of COPD exacerbation onset. Thorax. 2012;67(3):238-43.
61. Andreassen SL, Liaaen ED, Stenfors N, Henriksen AH. Impact of pneumonia on hospitalizations due to acute exacerbations of COPD. Clin Respir J. 2014;8(1):93-9.
62. Rothman KJ, Lash TL, Greenland S. Modern epidemiology. Philadelphia: Lippincott Williams & Wilkins; 2008.
63. Streiner DL, Norman GR. Mine is bigger than yours: measures of effect size in research. Chest. 2012;141(3):595-8.
64. Mapel D, Schum M, Yood M, Brown J, Miller D, Davis K. Pneumonia among COPD patients using inhaled corticosteroids and long-acting bronchodilators. Prim Care Respir J. 2010;19(2):109-17.
65. Mullerova H, Chigbo C, Hagan GW, Woodhead MA, Miravitlles M, Davis KJ, et al. The natural history of community-acquired pneumonia in COPD patients: a population database analysis. Respir Med. 2012;106(8):1124-33.
66. National Asthma Education and Prevention Program, Third Expert Panel on the Diagnosis and Management of Asthma. Expert panel report 3: guidelines for the diagnosis and management of asthma. Bethesda (MD): US Department of Health and Human Services/National Institutes of Health/National Heart, Lung, and Blood Institute; 28 Aug 2007.
