CC BY
0Health Services Research
RESEARCH ARTICLE Open Access
Performance measures of the specialty referral process: a systematic review of the literature
James P Guevara1'2'3'4*, Diane Hsu1,2 and Christopher B Forrest2,4
Abstract
Background: Performance of specialty referrals is coming under scrutiny, but a lack of identifiable measures impedes measurement efforts. The objective of this study was to systematically review the literature to identify published measures that assess specialty referrals.
Methods: We performed a systematic review of the literature for measures of specialty referral. Searches were made of MEDLINE and HealthSTAR databases, references of eligible papers, and citations provided by content experts. Measures were eligible if they were published from January 1973 to June 2009, reported on validity and/or reliability of the measure, and were applicable to Organization for Economic Cooperation and Development healthcare systems. We classified measures according to a conceptual framework, which underwent content validation with an expert panel.
Results: We identified 2,964 potentially eligible papers. After abstract and full-text review, we selected 214 papers containing 244 measures. Most measures were applied in adults (57%), assessed structural elements of the referral process (60%), and collected data via survey (62%). Measures were classified into non-mutually exclusive domains: need for specialty care (N = 14), referral initiation (N = 73), entry into specialty care (N = 53), coordination (N = 60), referral type (N = 3), clinical tasks (N = 19), resource use (N = 13), quality (N = 57), and outcomes (N = 9). Conclusions: Published measures are available to assess the specialty referral process, although some domains are limited. Because many of these measures have been not been extensively validated in general populations, assess limited aspects of the referral process, and require new data collection, their applicability and preference in assessment of the specialty referral process is needed.
Keywords: Primary Care, Specialists, Referral-Consultation, Coordination, Quality of Care
Background
Access to specialty care in developed countries occurs mainly through primary care referrals [1]. Specialists provide important services related to resolution of clinical uncertainty, provision of long-term medical therapy for patients with unusual or complex medical problems, and provision of specialized technologies [2,3]. Nonetheless, a clear consensus on the availability and role of specialists is lacking, which likely contributes to the marked variation in use of specialty care across regions and countries [4].
The optimal availability of specialty care has been a long-standing matter of contentious debate [5]. Workforce
* Correspondence: guevara@email.chop.edu
1PolicyLab: Center to Bridge Research, Practice, & Policy, The Children's
Hospitalof Philadelphia, Philadelphia, PA, USA
Fulllist of author information is available at the end of the article
models that extrapolate current levels of specialist utilization while accounting for demographic trends project an impending shortage, particularly in the U.S [6]. These models assume that current demand for specialists per population will remain constant or even grow over time [7]. Others have argued that more appropriate use of the skills and expertise of specialists would bring supply and demand into balance within the constraints of future workforce supply [8,9]. For example, some large integrated health systems use fewer specialists per 1,000 members than conventional open-access systems and achieve comparable outcomes [10-12].
While consensus on the role of primary care clinicians in specialty referrals has been established, similar consensus on the normative role of specialists has not been achieved [4]. For example, primary care clinicians initiate
O© 2011 Guevara et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons "WlVhMl Csntral Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
referrals and coordinate specialty care when diagnostic uncertainty arises or treatment becomes complex [13,14]. Responsibilities for primary care clinicians in this role include ensuring that the health needs of patients are met, services are integrated across providers and over time, and patients are linked with relevant community resources [15-17]. The benefits of these primary care tasks have been established empirically [13,18].
Empirical research on the role of specialists in the referral process is less well defined and mainly involves comparisons of primary care vis-à-vis specialist care without strong methodological rigor [19]. Overall, the literature suggests that care provided by specialists compared with that provided by generalists is more costly due to the addition of expensive tests and highly selective treatments [20-22]. In addition, care provided by specialists is more likely to be evidence based within the specialist's area of expertise but may be less evidence-based and associated with poorer outcomes outside their expertise. Several studies have examined care provided jointly by primary care clinicians and specialists for patients with chronic disease and have found that it is associated with better outcomes in comparison with either acting alone [23-26].
With health care costs rising, efforts to control costs and improve the efficiency of the specialty referral process have gained renewed traction in current healthcare debates [27-29]. Measures of the specialty referral process are needed to inform the ongoing debate on the availability and appropriate role of specialists in the healthcare delivery system. To address this need, we sought to identify published measures that assess the performance of the specialty care process including the interface between primary care and specialty care. Such measures will be of interest to healthcare organizations to allow them to benchmark current practices, assist providers in meeting standards of care, and determine the most rational ways to organize the primary-specialty care interface. Such measures will also be of interest to academic researchers who seek to better understand the complexities of the current referral process and to develop and test innovative improvements in specialty referrals. Therefore, the aims of the study were to 1) identify published measures of the specialty referral process through a systematic review of the literature, 2) categorize the measures according to a conceptual framework of the specialty referral process, and 3) assess the content validity of identified metrics, the conceptual framework developed, and the assignment of metrics to given domains of the framework among a group of specialty referral content experts.
Methods
Conceptual Framework
We developed a conceptual framework to guide the identification and categorization of measures of the
specialty referral process (Figure 1). This framework was developed from a review of the literature on specialty referrals and conceptual frameworks and underwent several iterations based on suggestions from a panel of 10 content experts [4,30-32]. According to the framework, the specialty referral process represents a series of health-related events or stages (i.e. domains) to diagnose and/or manage a health condition. In the first stage, a patient considers whether to seek care for a health problem from a specialist (Evaluation of Need for Specialty Care). Consultation regarding the referral decision may be made with a referral source such as a primary care clinician or emergency medicine clinician. Once a decision is made to seek a referral, the initiation of a referral involves the reason(s) for referral, urgency of referral, and selection of specific specialty (Referral Initiation). In the next stage, a patient makes an appointment with a specialist (Entry into Specialty Care), which can be delayed or hindered by a number of access-related barriers including geographic, organizational, and financial factors. Once a patient visit is completed, a specialist may direct a number of diagnostic or therapeutic procedures, consider duration of follow-up, and suggest additional referrals (Clinical Tasks). The extent of the specialist involvement may be short-term and involve consultation for a diagnosis or specified procedure or may be long-term and involve co-managed care or transferred care (Referral Type). Information regarding clinical tasks and their outcomes can be exchanged between the specialist and referring clinician, and care can be distributed between the specialist and the referring clinician (Coordination). The clinical tasks involved in the referral process will consume an amount and monetary value of health services (Resource Use) and reflect a degree of quality consistent with best available evidence based on the patient's and provider's expectations of the referral (Quality). The referral process from initiation to completion may impact on the patient's health problem and his or her overall health and quality of life (Outcomes).
Search Strategy
We conducted a systematic review of the published literature to identify relevant papers that contain measures of the specialty referral process. We searched the electronic databases MEDLINE and HealthSTAR from January 1973 to June 2009, to coincide with the emergence of managed care papers in the published literature to the present. We used a search strategy intended to be highly sensitive for identification of specialty referral articles from a previous study [4]:
(("Primary health care"[MeSH Terms] OR generalists
[Text Word]) AND ("specialism"[MeSH Terms] OR
Evaluation of Need for Specialty Care
Thind Party
чThird party-referral
Self-refierral
Referral Decision
■ Reason for referral
■ Specialist selection
■ Referral Rate
Referral Type
Entry into * Cognitive
Specialty consultation
Care • Procedural
consultation
- Attendance • Co-management
■ Accessibility with shared care
• Co-management
with principal care
Clinical Tasks
■ Problem
recognition
■ Diagnosis
■ Treatment
■ Reassessment
Coordination
■ Communication
■ Integration of care
Quality
* Referral satisfaction
* Approprlaten ess
* Effectiveness
* Efficiency
* Patient-centeredness -Safety
* Equity •Timeliness
Outcomes
■ Patient-ne ported outcomes
■ Clinical outcomes
■ Mortality
■ Unintended consequences
Figure 1 Specialty Referral Process. Referral process is a series of health-related events beginning when a patient has an assessment of need for specialty care, progresses to the decision to make a referraland enter into specialty care, and ends with the completion of referred care. The process encompasses communication and coordination between referring and consultant clinicians regarding tasks to be completed. The overall referralprocess results in a given resource utilization, quality, and outcomes for a patient.
special*[Text Word] OR subspecial*[Text Word])) AND ((coordination[Text Word] OR communica-tion[Text Word] OR shared decision making[Text Word] OR co-management[Text Word] OR shared care[Text Word] OR integrated care[Text Word] OR multidisciplinary care[Text Word]) OR "referral and consultation"[MeSH Terms]))
The reference lists of all eligible papers were reviewed for additional eligible papers. In addition, published papers nominated by content experts were reviewed for eligibility. This research was granted an exemption from review by the Institutional Review Board at the Chil-drens Hospital of Philadelphia.
Eligibility Criteria
Studies were eligible if they were (1) published in peer-reviewed journals or government reports in 1973 or later, (2) performed in one of the member Organization for Economic Cooperation and Development (OECD) countries or Israel, (3) contained an operational measure (s) of the referral process with prior or current data supporting the measure's validity (content, concurrent, or construct) or reliability (inter-rater, test-retest, or
internal consistency), and (4) addressed referrals to a recognized medical specialty. Operational measures included instruments, questionnaires, and other tools that purport to measure a component of the specialty referral framework. Measures did not have to be specific to referrals but could report on the outcomes or quality of referrals. We accepted articles that demonstrated validity or reliability of the metric within the presentation of results of the paper (e.g., a metric that changes in hypothesized directions with outcome variables) or made reference to a prior publication that reported on the measure's validity or reliability.
A variety of exclusion criteria were employed to increase the specificity of article selection. First, we excluded studies that employed interventions which defined the duration and type of specialist involvement; surveys in which general attitudes of the referral process were the focus; and, studies of hypothetical referral scenarios, because we wanted to capture measures involved in actual referral behavior. Second, we excluded review papers, although we utilized these to identify papers containing referral metrics. Third, we excluded studies involving only referrals to clinicians not part of the American Board of Medical Specialties (e.g. physical
therapists and dentists), since these may not involve clinician-to-clinician communication and coordination of referrals. We did capture referrals to psychologists, as they were often viewed along with psychiatrists as comprising mental health referrals in the literature. Fourth, we excluded referrals to hospital-based specialists including emergency departments, anesthesiologists, radiologists, and inpatient units of hospitals due to the urgent nature of many of these referrals. Fifth, we excluded curbside consultations, since data on these consultations are difficult to capture.
The title and abstract of all identified papers were reviewed for potential eligibility. The full-text of each potentially eligible paper was then reviewed for eligibility independently by three investigators (JG, DH, CF), two of which were assigned to each paper. Differences between investigators were settled by consensus of all investigators as to whether the pre-specified inclusion and exclusion criteria were met. After a training period of approximately 50 papers, we had a final overall agreement rate of 87% with k = 0.60 (p < 0.001).
Eligible papers were abstracted using standardized abstraction forms that had been piloted prior to study initiation. Since papers may contain more than one relevant referral measure, we abstracted information on all metrics contained in each eligible paper that met eligibility criteria. We abstracted information on each metric's definition as given in the published paper, its components and formula for calculation (numerator, denominator), data type (integer, ratio, proportion), data source (administrative data, surveys, chart abstraction), validity measure, reliability measure, and use in the paper. We categorized each metric according to the Donabedian framework of structure, process, and outcome [33]. We also abstracted information from each paper on the specific specialty involved, the patient population (children, adults), and disease state involved if any. Eligible metrics were then categorized into one or more domains and sub-domains of the conceptual framework of the specialty referral process (Figure 1).
To clarify measures and obtain missing information, we contacted corresponding authors of eligible papers to provide explanations on how metrics were constructed, how they were validated, what their purpose was, and to obtain a copy of the instruments if available. Metrics were linked to all source studies, defined as the first published study containing an identified metric. Subsequent studies that contained a given metric were also linked to source studies.
Expert Panel
We identified a pool of content experts on specialty referrals through our literature search and by nomination of other experts. We selected a pool of 10 who
agreed to assess the content validity of the metric set and the corresponding conceptual framework and to propose additional papers not previously identified in our literature search. The expert panel included health services researchers, clinical administrators, insurance executives, and practicing physicians from the U.S. and the U.K. Names of panel members can be found in the acknowledgement section.
Results
We identified 4,225 studies from our search of MED-LINE and HealthSTAR databases supplemented by additional papers from content experts and searches of the reference lists of eligible papers (Figure 2). After accounting for 1,261 duplicate articles, we identified 2,964 unique articles and government reports. Based on review of abstracts, we excluded 2,452 studies based on exclusion criteria and pulled the full text of 512 papers for a more comprehensive review. After reviewing the full-text of these potentially eligible papers, we excluded an additional 298 articles, which left 214 eligible papers containing 244 unique referral metrics (Additional File 1). We attempted to contact 126 corresponding authors, 24 of whom had no valid contact information. A majority of the remaining authors (90 of 102, 88%) responded to our inquiries and provided us with additional information or the full instruments if available. Based on this correspondence, we were able to obtain additional information for 129 metrics that was not readily apparent in the source studies' methodologies. Of these, 79 metrics (61%) were
Files Identified N = 4225 Electronic Databases = 2440 Content Expert Files =1314 Bibliographies = 471
Duplicates N- 1261
Unique Articles Identified N = 2964 Electronic Databases =1531 Content Expert Files = 1314 Bibliographies = 471
Abstracts Excluded N = 2452
Full Text Pulled N = 512 Electronic Databases = 158 Content Expert Files = 260 Bibliographies = 263
Ineligible Papers N = 298 Reasons for Exclusion 213 Not a validated measure 47 Hypothetical scenarios or attitudinal evaluations 17 Irrelevant to outpatient visits or telemedicine consultations 16 Interventions that stipulate elements of the referral process
Eligible Papers N = 214 Electronic Databases = 61 Content Expert Files = 108 Bibliographies = 140
Unique Metrics N = 244
Figure 2 Flow chart of Literature Search.
available in online resources, were provided by corresponding authors, or were embedded in the full text of papers.
A majority of the eligible metrics (57%) were assessed exclusively in adult patient populations, while a quarter (26%) of the metrics were assessed exclusively in children and the remainder (17%) were assessed in both child and adult populations. The majority of the measures (63%) examined referrals to medical specialties, as opposed to surgical and mental health specialties. The most common individual subspecialty reported in the studies was mental health (42%), followed by otolaryn-gology (30%), dermatology (28%), neurology (23%), general surgery (23%), cardiovascular medicine (22%), and
ophthalmology (21%) or orthopedic surgery (21%). A number of measures (27%) were evaluated in multiple subspecialties.
With consultation from the expert panel, measures were categorized into domains and sub-domains of the conceptual framework (Table 1). The majority of measures were from one of four domains: Referral Initiation (73 metrics, 30%), Entry into Specialty Care (53 metrics, 22%), Coordination (60 metrics, 26%), and Quality (57 metrics, 23%). Referral Initiation metrics included mainly measures of the reason for referral and selection of specialist (27 metrics) and rates of referral (42 metrics). Entry into Specialty Care metrics were comprised of measures of accessibility (46 metrics) with few assessing attendance at the
Table 1 Metric Domains, Sub-domains, and Exemplary Metrics
Domain
Sub-domains
Metrics
Metric Example
Evaluation of Need for Specialty Care (n ■ 14)
ReferralInitiation (n = 73)
Entry into Specialty Care (n = 53)
Coordination (n = 60) Referral Type (n = 3)
ClinicalTasks (n = 19)
Resource Use (n = 13) Quality (n = 57)
Outcomes (n = 1
Referral Source
Referral Decision Reason for referral Specialist selection Referral rate
Attendance
Accessibility Communication Integration of care Consultation Co-management
0 4 27 42
Problem recognition 10
Diagnosis 18
Treatment 17
Reassessment 12
Services 12
Costs 13
Appropriateness 13
Effectiveness 4
Efficiency 1
Equity 0
Patient-centeredness 8
Referral satisfaction 21
Safety 0
Timeliness 23
Health status 3
Mortality 2
Functional status 3
Unintended 0 Consequences
Proportion of self-referred visits [38]
Proportion of referrals for advice on diagnosis and treatment [3] Proportion of PCPs who referred at patient request [39] # Referrals to a specialist per 100 PCP visits [40]
7 Proportion of referred patients who attended first specialty visit [41]
46 Total# providers per 100,000 people per state [42]
48 Proportion of specialist reports by letter or e-mail [43]
17 Proportion of PCPs who received feedback from a specialist [43]
3 Proportion of referrals for procedural consultation [44]
3 Proportion of referrals for co-management with shared care [44]
Proportion of letters including a patient's condition [45] Proportion of letters including a patient's diagnosis [45] Proportion of letters including treatment recommendation [45] Proportion of letters including follow-up arrangements [45] Primary and specialty visits per patient per 30 days [46] Totalannualspecialty expenditures [47]
Proportion of patients who received an unnecessary referral[48] Proportion of PCPs who adhered to a specialists' recommendations [49]
Proportion of PCPs who believed a specialist was minimizing costs
Proportion of patients who thought that specialty care was helpful
Proportion of patients who were satisfied with specialty care [52]
Average length of time spent for a specialty visit [53] Change in depression score on Hopkins Symptom Checklist [54] Proportion of surviving patients per year [55] Change in score on Pain Interference Scale [56]
specialist visit. Coordination metrics consisted of measures of communication of the reason and expectations for referral to the specialist (48 metrics), with few assessing integration of care with referring clinicians (17 metrics). Quality metrics were mainly comprised of timeliness of initial specialty visits (23 metrics) and satisfaction (21 metrics). There were no identified quality measures assessing the safety or equity of referrals and few assessing the effectiveness, efficiency, or patient-centeredness of referrals. There were few measures that comprised Need for Specialty Care (14 metrics, 6%), Clinical Tasks (19 metrics, 8%), Referral Type (3 metrics), Resource Use (13 metrics, 5%), or Outcomes (9 metrics, 4%).
A majority of eligible metrics (60%) assessed structural features of referrals (Table 2), while a minority assessed processes (34%) or outcomes (19%). Most metrics (98%) included a measure of validity with the most common type being construct validity (88%), followed by content validity (42%). Few measures included criterion validity (7%) or an assessment of reliability (16%).
Eligible measures were derived from several different data sources, which were grouped into one of the following three categories: questionnaire, administrative data, or chart review. To improve the accuracy and precision of data, researchers utilized several data sources in
collecting information for an individual metric, so a single operational measure may not be mutually exclusive to a particular type of data. Questionnaires included self-administered and staff-aided instruments completed by the patient, primary care physician, or specialist, and a majority of the measures (62%) were derived from this source of information (Table 2). Approximately one fifth of the metrics (20%) utilized administrative data, which included information readily available in health plans, public databases, scheduling and billing databases, and physician claims files. Roughly one third (31%) of the measures were derived from chart reviews, which included referral letters, paper and electronic medical records, and referral logs maintained by office managers. Very few metrics (1%) in this review did not report on data sources.
Discussion
In this review, we found that available measures were limited in assessing the process and outcomes of referrals. For example, we identified few measures that assess the adequacy and appropriateness of coordination of referrals, an important process measure intended to capture the allocation of tasks and assignment of roles among primary care and specialty providers in coordinating care for patients.
Table 2 Characteristics of Referral Metrics by Domain
Characteristic Need1 Initiation2 Entry3 Coordination Type4 Tasks5 Resources6 Quality Outcomes All
N = 14 N = 73 N = 53 N = 60 N = 3 N = 19 N = 13 N = 57 N = 9 Domains
N = 244
Donabedian Domains n (%) 14(100) 73(100) 53(100) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 147(60.2)
Structure 0(0) 0(0) 0(0) 60(100) 3(100) 19(100) 13(100) 0(0) 0(0) 84(34.4)
Process 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 57(100) 9(100) 46(18.9)
Outcome 14(100) 70(95.9) 52(98.1) 59(98.3) 3(100) 19(100) 12(92.3) 57(100) 9(100) 239(98.0)
Validity, n(%)
Content 2(14.3) 22(30.1) 24(45.3) 53(88.3) 2(66.7) 12(63.2) 1(7.7) 25(43.9) 3(33.3) 103(42.2)
Criterion 1(7.1) 5(6.8) 4(7.5) 1(1.6) 0(0) 3(15.8) 2(15.4) 4(7.0) 0(0) 17(7.0)
Construct 14(100) 67(91.8) 49(92.5) 40(58.6) 3(100) 14(73.7) 10(76.9) 54(94.7) 7(77.7) 215(88.1)
Reliability, n(%) 0(0) 15(20.5) 7(10.1) 17(27) 0(0) 4(21.1) 1(7.7) 10(17.5) 1(11.1) 38(15.6)
Inter-Rater 0(0) 2(2.7) 2(2.9) 7(11.1) 0(0) 1(5.3) 0(0) 3(7.9) 0(0) 12(4.9)
Test-Retest 0(0) 1(1.4) 0(0) 3(4.8) 0(0) 1(5.3) 0(0) 0(0) 0(0) 4(1.6)
Internal 0(0) 12(16.4) 5(7.2) 13(20.6) 0(0) 3(15.8) 1(7.7) 8(21.1) 1(11.1) 28(11.5)
Consistency
Data Source, n(%)
Questionnaire 6(42.9) 51(69.9) 30(56.6) 46(76.7) 3(100) 11(57.9) 2(15.4) 42(73.7) 6(66.7) 152(62.3)
Administrative 4(28.6) 8(11.0) 24(45.2) 1(1.6) 0(0) 2(10.5) 10(76.9) 4(7.0) 1(11.1) 49(20.1)
Chart Review 2(14.3) 25 (34.2) 9(17.0) 22(34.9) 0(0) 10(52.6) 1(7.7) 18(31.6) 2(22.2) 75(30.7)
Not reported 2(14.3) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 0(0) 2(0.8)
1'Need' refers to a patient's evaluation of need for specialty care 2'Initiation' refers to the steps taken to refer a patient 3'Entry' refers to a patient's entry into specialty care
4'Type' refers to the relationship between a PCP and referred specialist
5'Tasks' refers to the clinical tasks performed during care
6'Resources' refers to the usage and costs of services
In addition, we identified few measures that assess the quality, resource consumption, and outcomes of referrals. Despite the availability of a number of validated measures designed to capture health services use or patient outcomes and well-being, few of these measures have been utilized in studies of specialty referrals [34-37]. Process measures help to tie structures to their intended outcomes. Clearly, measures that are intended to assess the process of referrals like care coordination and outcome measures are needed in order to better evaluate current referral quality.
The majority of measures we identified in this study require collection of new data through either questionnaires or chart reviews. Few (20%) are designed to rely on existing administrative data. This presents challenges in the ability of metrics to be used to compare referral data across provider groups and health plans. However, with the national movement toward electronic health records (EHR) in the U.S. and elsewhere, it may be conceivable to embed key metrics within the EHR and collect referral data for cross-system comparisons. This will require consensus on the selection of key metrics among the many that are available.
This study has limitations that should be addressed. First, as with all systematic reviews, there exists the possibility of publication bias in which we missed important specialty care measures. We took a number of steps to limit this bias by utilizing a previously validated search strategy for identifying papers on specialty referrals, a thorough review of the reference lists of all eligible papers, and contact with experts in the field who could nominate additional papers to supplement our list. Second, our review identified a large pool of available validated measures without providing an endorsement for a core set of measures. Future study is needed to identify a subset of these measures that can be endorsed for more general use.
Conclusions
Operational measures of the specialty referral process are available in the published literature. A majority of identified metrics were studied among adult patients and evaluated the frequency and reason for referrals, specialty accessibility and timeliness, communication, and satisfaction. In addition, a majority of the measures focused on structural components of the referral process; few measures assessed the process or outcomes of referrals. While measures in this review nearly universally included assessments of validity, few reported on reliability. Most measures relied on collection of new data rather than on existing administrative or claims data.
We believe that our study has important research implications. First, a set of referral measures such as
those identified in this study can be utilized to assess the performance of the current referral system and to evaluate the comparative effectiveness of interventions to improve the quality and efficiency of specialty referrals. A toolkit is available on-line at http://www.research.chop. edu/tools/psrt/index.php which lists searchable characteristics of these measures and provides their link to the conceptual framework. Identification and categorization of published measures of specialty care referral can be considered the first step toward assessment of the referral system. Consensus on a core set of measures derived from this study can then be developed to uniformly measure specialty referrals and benchmark referral practices. Second, given the limited number of measures in key domains, researchers should target the development of new measures to fill these gaps and better assess the processes and outcomes of referrals. Third, the dynamics between primary and specialty clinicians are evolving as technological advancements become an important part of medical practices and electronic records, and electronic communication is changing the way information is managed and transferred. Finding ways to incorporate measures of specialty care and other aspects of health care delivery into electronic information systems can facilitate evaluation of current practices and contribute to the redesign of future specialty care referral processes.
Additional material
Additional file 1: Referral Appendix. Reference list of all eligible papers from the Systematic Review
Acknowledgements and Funding
This study was funded by a grant from the California Healthcare Foundation, which provided input on the design of the study and interpretation of data. Information from this paper was presented at the AcademyHealth Annual Meeting in Boston, MA on June 27, 2010. We wish to thank members of our expert panelwho provided invaluable critiques of the study design, conceptualframework, and metrics toolkit:
(1) Michael Cabana, MD, MPH
Professor of Pediatrics, Epidemiology and Biostatistics; Chief, Division of GeneralPediatrics, University of California, San Francisco, San Francisco, CA, USA
(2) Alan Glaseroff, MD
Chief MedicalOfficer, Humboldt-DelNorte Independent Practice Association, Humboldt, CA, USA
(3) David Grembowski, PhD
Professor, Department of Health Services and DentalPublic Health Sciences, University of Washington; Adjunct Professor, Department of Sociology, University of Washington, Seattle, WA, USA
(4) Jeffery Guterman, MD
MedicalDirector, Los Angeles County, Department of Health Services; Professor of Medicine, UCLA Schoolof Medicine, Los Angeles, CA, USA
(5) Dave Pearson, PhD
Associate Director, Center for Community Health and Evaluation, Group Health Cooperative of Puget Sound, Seattle, WA, USA
(6) Karin Rhodes, MD, MS/Joanna Bisgaier, MSW
Karin Rhodes, Director, Division of Emergency Care Policy & Research, Department of Emergency Medicine, University of Pennsylvania Schoolof Medicine; Department of Psychiatry and Schoolof SocialPolicy & Practice, University of Pennsylvania Schoolof Medicine; Senior Fellow, Leonard Davis of Health Economics & Center for Public Health Initiatives, Philadelphia, PA, USA
Joanna Bisgaier, Ph.D. Candidate, Schoolof SocialPolicy & Practice, University of Pennsylvania, Philadelphia, PA, USA
(7) Martin Roland, MD
Professor of Health Services Research, GeneralPractice and Primary Care Research Unit, University of Cambridge, Cambridge, UK
(8) Neil Solomon, MD
Vice President and Senior MedicalDirector, Quality Improvement and MedicalPolicy, Health Net; ClinicalDirector, California Quality Collaborative, San Francisco, CA, USA
(9) Christopher Stille, MD, MPH
Associate Professor, Department of Pediatrics Department, University of Massachusetts Medical School, Worcester, MA, USA
(10) Hal Yee, MD, PhD
Chief MedicalOfficer and Chief of Gastroenterology and Hepatology, San Francisco General Hospital; San Francisco GeneralHospitaland Trauma Center William and Mary Ann Rice MemorialDistinguished Professor of Medicine, University of California, San Francisco, San Francisco, CA, USA
Author details
1PolicyLab: Center to Bridge Research, Practice, & Policy, The Children's Hospitalof Philadelphia, Philadelphia, PA, USA. 2Division of General Pediatrics, The Children's Hospitalof Philadelphia, Philadelphia, PA, USA. 3Center for ClinicalEpidemiology and Biostatistics, University of Pennsylvania, Philadelphia, PA, USA. 4Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA, USA.
Authors' contributions
JG conceived of the study, wrote the protocol, participated in the data collection, conducted phone interviews with expert panelmembers, drafted and revised alldrafts of the manuscript, and approved the finalversion. DH conducted the systematic review, abstracted eligible studies, helped conduct phone interviews with expert panelmembers, assisted in the drafting of the manuscript, and approved the finalversion. CF helped conceive of the study, participated in phone interviews with expert panelmembers, assisted in the drafting of the manuscript, and approved the finalversion.
Authors' information
James P. Guevara is at the Perelman Schoolof Medicine at the University of Pennsylvania and The Children's Hospitalof Philadelphia. Diane Hsu is at The Children's Hospitalof Philadelphia. Christopher B. Forrest is at the Perelman Schoolof Medicine at the University of Pennsylvania and The Children's Hospitalof Philadelphia.
Competing interests
The authors declare that they have no competing interests.
Received: 7 January 2011 Accepted: 13 July 2011 Published: 13 July 2011
References
1. Clancy CM, Franks P: Utilization of specialty and primary care: the impact of HMO insurance and patient-related factors. J Fam Pract 12/1997 1997,
45(6):500-508.
2. Bachman KH, Freeborn DK: HMO physicians' use of referrals. Soc Sci Med 2/1999 1999, 48(4):547-557.
3. Forrest CB, Glade GB, Baker AE, Bocian AB, Kang M, Starfield B: The pediatric primary-specialty care interface: how pediatricians refer children and adolescents to specialty care. Arch Pediatr Adolesc Med 7/ 1999 1999, 153(7):705-714.
4. Forrest CB: A typology of specialists' clinical roles. Arch Intern Med 2009, 169:1062-1068.
5. Ayanian JZ, Hauptman PJ, Guadagnoli E, Antman EM, Pashos CL, McNeil BJ: Knowledge and practices of generalist and specialist physicians regarding drug therapy for acute myocardial infarction [see comments].
N EnglJ Med 10/27/1994 1994, 331(17):1136-1142.
6. Cooper RA, Getzen TE, McKee HJ, Laud P: Economic and demographic trends signal an impending physician shortage. Health Affairs 2002, 21:140-154.
7. Organization for Economic Corporation and Development: OECD health data; 2007 [http://www.oecd.org/document/10/03343, en_2649_33729_38976778_1_1_1_1,00.html], Accessed October 25, 2008..
8. Goodman DC: Expanding the medical workforce. BMJ 2007, 335:218-219.
9. Weiner JP: Expanding the U.S. medical workforce: global perspectives and parallels. BMJ 2007, 335:236-238.
10. Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ, Lucas FL, Pinder EL: The implications of regional variations in Medicare spending. Part 1: The content, quality, and accessibility of care. Ann Intern Med 2003, 138:273-287.
11. Weiner JP: Prepaid group practice staffing and U.S. physician supply: lessons for workforce. Hea th Affairs 2004, , Jan-Jun SupplWeb Exclusives:
W4-43.
12. Feachem RG, Sekhri NK, White KL: Getting more for their dollar; a comparison of the NHS with California's Kaiser Permanente. BMJ 2002, 324:135-141.
13. Cooley W: Redefining primary pediatric care for children with special health care needs: the primary care medical home. Curr Opin Pediatr 2004, 16:689-692.
14. Barr M, Ginsburg J: The Advanced Medical Home: A Patient-Centered, Physician-Guided Model of Health Care: A Policy Monograph of the American College of Physicians. Philadelphia, PA: American College of Physicians; 2006.
15. Donaldson MS, Yordy KD, Lohr KN, Vanselow NA, eds: Primary Care: America's Health in a New Era. Washington, DC: National Academies Press; 1996.
16. Starfield B: Primary Care: Balancing Health Needs, Services, and Technology. New York, NY: Oxford University Press; 1998.
17. Rosenthal TC: The Medical Home: growing evidence to support a new approach to primary care. J Am Board Fam Med 2008, 21:427-440.
18. Starfield B, Shi L, Macinko J: Contribution of primary care to health systems and health. Milbank Q 2005, 83:457-502.
19. Smetana GW, Landon BE, Bindman AB, Burstin H, Davis RB, Tjia J, Rich EC: A comparison of outcomes resulting from generalist vs. specialist care for a single discrete medical condition: a systematic review and methodologic critique. Arch Intern Med 2007, 167:10-20.
20. Donohoe MT: Comparing generalist and specialty care: discrepancies, deficiencies, and excesses. Arch Intern Med 1998, 158(15):1596-1608.
21. Harrold LR, Field TS, Gurwitz JH: Knowledge, patterns of care, and outcomes of care for generalists and specialists. J Gen Intern Med 1999, 14(8):499-511.
22. Weingarten SR, Lloyd L, Chiou CF, Braunstein GD: Do subspecialists working outside of their specialty provide less efficient and lower-quality care to hospitalized patients than do primary care physician? Arch Intern Med 2002, 162:527-532.
23. Willison DJ, Soumerai SB, McLaughlin TJ, Gurwitz JH, Gao X, Guadagnoli E, Pearson S, Hauptman P, McLaughlin B: Consultation between cardiologists and generalists in the management of acute myocardial infarction: implications for quality of care. Arch Intern Med 1998, 158(16):1778-1783.
24. Lafata JE, Martin S, Morlock R, Divine G, Xi H: Provider type and the receipt of general and diabetes-related preventive health services among patients with diabetes. Med Care 2001, 39(5):491-499.
25. Ayanian JZ, Landrum MB, Guadagnoli E, Gaccione P: Specialty of ambulatory care physicians and mortality among elderly patients after myocardial infarction. N Engl J Med 2002, 347(21):1678-1686.
26. Earle CC, Neville BA: Under use of necessary care among cancer survivors. Cancer 2004, 101(8):1712-1719.
27. Centers for Medicare and Medicaid Services, Office of the Actuary, National Health Statistics Group: 2007 National health Care Expenditures Data; 2009.
28. Kaiser Family Foundation, Health Research and Education Trust: Employer Health Benefits 2008 Annual Survey; 2008.
29. Davis K, Stremlkis K: The costs of failure: economic consequences of failure to enact Nixon, Carter, and Clinton health reforms. Commonwealth Fund Blog; 2009.
30. Grembowski DE, Diehr P, Novak LC, Roussel AE, Martin DP, Patrick DL, Williams B, Ulrich CM: Measuring the "managedness" and covered benefits of health plans. Health Serv Res 2000, 35(3):707-734.
31. Grembowski DE, Cook K, Patrick DL, Roussel AE: Managed care and physician referral. Med Care Res Rev 3/1998 1998, 55(1):3-31.
32. Shortell SM, Anderson OW: The Physician Referral Process: A Theoretical Perspective. HSR 1971, 39-48, 1971;Spring:.
33. Donabedian A: The quality of care: how can it be assessed? JAMA 1988, 260:1743-1748.
34. Forrest CB, Lemke KW, Bodycombe DP, Weiner JP: Medication, diagnostic, and cost information as predictors of high-risk patients in need of care management. Am J Manag Care 2009, 15(1):41-48.
35. Fries JF, Krishnan E: What constitutes progress in assessing patient outcomes? J Clin Epidemiol 2009, 62(8):779-780.
36. Revicki DA, Kawata A, Harnam N, Chen WH, Hays RD, Cella D: Predicting EUROQOL (EQ-SD) scores from the Patient Reported Outcomes Measurement Information System (PROMIS) global items and domain item banks in a United States sample. Quality Life Research 2009, 18(6):783-791.
37. Ader D: Developing the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care 2007, 45(5):S1-2.
38. Forrest CB, Reid RJ: Passing the baton: HMOs' influence on referrals to specialty care. Health Aff (Millwood) 11/1997 1997, 16(6):157-162.
39. Ludke RL: An Examination of the Factors That Influence Patient Referral Decisions. Med Care 1982, 20:782-796, 1982.
40. Armstrong D, Britten N, Grace J: Measuring General Practitioner Referrals: Patient, Workload, and List Size Effects. J R Coll Gen Pract 1988, 38:494-497, 1988;.
41. Gonzalez J WJJ, Noel PH, Lee S: Adherence to mental health treatment in a primary care clinic. J Am Board Fam Pract 2005, 18(2):87-96.
42. West J, Kohout J, Pion GM: Mental health practitioners and trainees.Edited by: Mandercheid RW, Henderson MJ. Mental Health, United States, 2000. Washington, DC: Superintendent of Documents, US Government Printing Office; 2001.; 2001:.
43. Stille CJ, Primack WA, Savageau JA: Generalist-subspecialist communication for children with chronic conditions: a regional physician survey. Pediatrics 2003, 112(6 Pt 1):1314-1320.
44. Forrest CB, Nutting PA, Starfield B, von Schrader S: Family physicians' referral decisions: results from the ASPN referral study. J Fam Pract 2002, 51(3):215-222.
45. Pullen IM YA: Is communication improving between general practitioners and psychiatrists? Br Med J 1985, 290(6461):31-33.
46. Allison JJ, Kiefe CI, Cook EF, Gerrity MS, Orav EJ, Centor R: The association of physician attitudes about uncertainty and risk taking with resource use in a Medicare HMO. Med Decis Making 1998, 18(3):320-329.
47. Kapur K, Joyce GF, Van Vorst KA, Escarce JJ: Expenditures for physician services under alternative models of managed care. Med Care Res Rev 2000, 57(2):161-181.
48. Bird JC, Beynon GJ, Prevost AT, Baguley DM: An analysis of referral patterns for dizziness in the primary care setting. Br J Gen Pract 1998, 48(437):1828-1832.
49. Brunner-La Rocca HP CJ, Kiowsk W: Compliance by referring physicians with recommendations on heart failure therapy from a tertiary center. J Cardiovasc Pharmacol Ther 2006, 11(1):85-92.
50. Cowen ME, Zodet MW: Methods for analyzing referral patterns. J Gen intern Med 8/1999 1999, 14(8):474-480.
51. Roland MO, Green CA, Roberts SO: Should general practitioners refer more patients to hospital? J R Soc Med 1991, 84(7):403-404.
52. Ware JEJ, Hays RD: Methods for measuring patient satisfaction with specific medical encounters. Med Care 1988, 1988(26)4.
53. Redlick FRB, Gomez M, Fish JS: An initial experience with telemedicine in follow-up burn care. J Burn Care Rehabil 2002, 23(2):110-115.
54. Parloff MB KH, Frank JD: Comfort, effectiveness, and self-awareness as criteria of improvement in psychotherapy. Am J Psychiatry 1954, 111(5):343-352.
55. Mommsen SAJ, Sell A: Presenting symptoms, treatment delay and survival in bladder cancer. Scand J Urol Nephrol 1983, 17(2):163-167.
56. Grembowski DE, Martin D, Patrick DL, Diehr P, Katon W, Williams B, Engelberg R, Novak L, Dickstein D, Deyo R, Goldberg HI: Managed care, access to mental health specialists, and outcomes among primary care patients with depressive symptoms. J Gen intern Med 2002, 17(4):258-269.
Pre-publication history
The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1472-6963/11/168/prepub
doi:10.1186/1472-6963-11-168
Cite this article as: Guevara et al.: Performance measures of the specialty referral process: a systematic review of the literature. BMC
Health Services Research 2011 11:168.
Submit your next manuscript to BioMed Central and take full advantage of:
• Convenient online submission
• Thorough peer review
• No space constraints or color figure charges
• Immediate publication on acceptance
• Inclusion in PubMed, CAS, Scopus and Google Scholar
• Research which is freely available for redistribution
Submit your manuscript at www.biomedcentral.com/submit
Central
