Scholarly article on topic 'Failure to Follow-Up Test Results for Ambulatory Patients: A Systematic Review'

Failure to Follow-Up Test Results for Ambulatory Patients: A Systematic Review Academic research paper on "Clinical medicine"

Share paper
Academic journal
OECD Field of science

Academic research paper on topic "Failure to Follow-Up Test Results for Ambulatory Patients: A Systematic Review"



Failure to Follow-Up Test Results for Ambulatory Patients: A Systematic Review

Joanne L. Callen, PhD, Johanna I. Westbrook, PhD, Andrew Georgiou, PhD, and Julie Li, BAppSc (HIM) (Hons1)

Centre for Health Systems and Safety Research, Faculty of Medicine, The University of New South Wales, Sydney, NSW, Australia,

BACKGROUND: Serious lapses in patient care result from failure to follow-up test results. OBJECTIVE: To systematically review evidence quantifying the extent of failure to follow-up test results and the impact for ambulatory patients. DATA SOURCES: Medline, CINAHL, Embase, Inspec and the Cochrane Database were searched for English-language literature from 1995 to 2010. STUDY SELECTION: Studies which provided documented quantitative evidence of the number of tests not followed up for patients attending ambulatory settings including: outpatient clinics, academic medical or community health centres, or primary care practices.

DATA EXTRACTION: Four reviewers independently screened 768 articles.

RESULTS: Nineteen studies met the inclusion criteria and reported wide variation in the extent of tests not followed-up: 6.8% (79/1163) to 62% (125/202) for laboratory tests; 1.0% (4/395) to 35.7% (45/126) for radiology. The impact on patient outcomes included missed cancer diagnoses. Test management practices varied between settings with many individuals involved in the process. There were few guidelines regarding responsibility for patient notification and follow-up. Quantitative evidence of the effectiveness of electronic test management systems was limited although there was a general trend towards improved test follow-up when electronic systems were used. LIMITATIONS: Most studies used medical record reviews; hence evidence of follow-up action relied upon documentation in the medical record. All studies were conducted in the US so care should be taken in generalising findings to other countries. CONCLUSIONS: Failure to follow-up test results is an important safety concern which requires urgent attention. Solutions should be multifaceted and include: policies relating to responsibility, timing and process of notification; integrated information and communication technologies facilitating communication; and consideration of the multidisciplinary nature of the process and the role of the patient. It is essential that evaluations of interventions are undertaken and

Received July 29, 2011 Revised September 28, 2011 Accepted November 4, 2011 Published online December 20, 2011

solutions integrated into the work and context of ambulatory care delivery.

KEY WORDS: patient safety; test result follow-up; medical errors: primary care; quality improvement, J Gen Intern Med 27(10):1334-48 DOI: 10.1007/s11606-011-1949-5

© The Author(s) 2011. This article is published with open access at


Failure to follow up test results is a critical safety issue which has been identified as a major problem in ambulatory settings.1-4 The practices and processes currently used are

2 5 6 7 8 9

varied and unsystematic2,5 and physicians6,7 and patients8,9

acknowledge that this needs to improve. The testing process

in ambulatory settings is complex and can be divided into

three broad phases, pre-analytic, analytic and post-analytic

(Fig. 1), each involving multiple steps and multiple

personnel including clinicians, patients, office and laboratory


Most primary care practices are not using electronic health records,11 and most are communicating with multiple laboratories often not electronically connected. Increased volumes of tests and the time consuming nature of test follow-up places further pressures on physicians. , , Failure to follow-up can lead to missed or delayed diagnoses which impact on

patient care


and can also have medico-legal implica-

tions for health services and health professionals.18-21

Without knowledge of the size of the problem, many clinicians may underestimate its extent and therefore fail to take any action to improve the process. Feedback on medical errors is essential to negate overconfidence in decision making in relation to diagnostic accuracy. -Ambulatory settings pose specific challenges for effective test management in addition to many of those present in acute care settings.26 There have been no systematic reviews in this setting, and therefore the aim of this paper was to review evidence which quantifies the size and impact on patient outcomes of failure to follow-up test results for ambulatory patients.

Figure 1. Conceptual framework of the testing process (Source Hickner et al. 2008, p.195).


Data Sources and Searches

Medline, CINAHL, Embase, Inspec and the Cochrane Database were systematically searched for English language articles published between 1995 and November 2010 which quantified diagnostic tests not followed-up (Fig. 2). Search terms were identified from keyword lists of core articles related to the topic. Reference lists of articles which met the inclusion criteria were hand searched. A web search using the Google Scholar search engine was completed to locate un-indexed publications.

Study Selection

Included were studies which quantified failure to follow-up laboratory and radiology tests for patients attending ambulatory settings including: outpatients, patients treated at academic medical and community health centres and attending primary care practices. Failure to follow-up a test result was defined as the ordering physician or another provider neglecting to document follow-up action(s) relating to a test result. Excluded were studies reporting physicians' or patients' perceived rates of failure, time from placement of test order to treatment, analysis and reporting of test follow-up for inpatients and outpatients combined27'28 and studies on individual patients.

Data Extraction

Four reviewers (JC, AG, JL, JW) each independently reviewed all articles for inclusion. Discrepancies among reviewers were resolved by discussion until a consensus was reached. Data extracted included: test type, indication of failure to follow-up, systems used, extent and patient outcomes.


The literature search yielded 768 references (including four from hand searching) from which 19 articles were eligible (Table 1).

Study Characteristics

All studies were conducted in the United States. The majority of study designs used retrospective medical record reviews (n=16)2'29-43 to provide documentary evidence of test follow-up. Other methods were: retrospective linkage of databases (n=1),44 retrospective review of malpractice claims for missed or delayed diagnoses (n=1),45 and a prospective longitudinal medical record review (n=1).46 Most studies focused on lack of follow-up of

29 32 35_38 41_45

abnormal laboratory tests only (n=11). ' ' ' Four studies investigated lack of abnormal radiology follow-up

only31,39,40,46 and another four studies included lack of

follow-up for both laboratory and radiology test


Extent of Failure to Follow-up

There was wide variation in the rates of missed abnormal laboratory results ranging from 6.8% (79/1163)41 of alerts displayed through a computerised provider order entry system which were not followed-up within 30 days to 62% (125/202) of abnormal glucose tests not followed-up.32 Similarly, for abnormal radiology, lack of follow-up ranged from 1.0% (4/395)31 of patients with suspected malignancy to 11% (131/1196) of critical imaging alerts. The two studies on mammograms reported 11% (9/82)30 and 35.7% (45/126)46 with no evidence of patient follow-up. Even studies which focused on the same diagnostic tests showed a broad range in the extent of lack of follow-up: glucose



; abnormal FOBT,2943 and TSH

Impact on Patient Outcomes

The impact of missed test results on patient outcomes were reported in seven of the 19 studies.29'31'32'38'40'42'45 Missed cancer diagnoses were reported in four.29'31'40'45 Other reported outcomes were increased visits to hospital as a result of hyperkalaemia related to missed abnormal serum potassium levels38 and adverse drug events related to insufficient supplementation with levothyroxine due to missed follow-up of abnormal TSH results.42 Patients

Figure 2. Search flow for failure to follow-up test results for ambulatory patients, including keywords and MeSH terms used in search


who received appropriate follow-up of diabetes screening were more likely to have been scheduled for follow-up appointments than those who were not (92% versus 66%, P=0.001) and were also more likely to have kept the appointments (90% versus 58%, P=0.001).32

People and Policies

Test management involves information communication between many individuals across care settings including: physicians, nurses, clerks, laboratory staff, and patients. Studies showed that test follow-up practices varied between individuals ' and practice settings. Individuals other than physicians were involved in the test follow-up phase,

OQ '-fc 1 ^^ 1A

including nurses and practice managers. ' '

Several studies reported an absence of guidelines regarding who has responsibility, and how and when to notify patients of results.2'29'33'34 Singh et al. in 20 0739 showed lower rates of missed abnormal imaging results than similar studies which they concluded was associated with having standardized processes and procedures for abnormal test result follow-up in combination with an electronic test result notification system.

Systems Used for the Test Management Process

Those studies which described the systems used to deliver test results to physicians and how follow-up was documented ranged from paper-based medical records

O OQ ^fl ^^ "XA.

with test results delivered in hard copy2,29,30,33,34 to

electronic medical records (EMR) with results transmitted

2 33 34 39_41

electronically, , , , or a combination of paper-based medical records and electronic or part-electronic test

2 31 37 38

management systems. , , , Evidence of the effectiveness of electronic test management systems to reduce missed follow-up was limited to five studies.2,34,39-41 Although these evaluations were not pre/post control studies they showed a general trend towards improved test follow-up in ambulatory settings which used electronic systems.

Three studies evaluated whether an automatic alert system for notification of abnormal results resulted in timely follow-up of radiology39,40 and laboratory tests.41 Even though these studies have some features which may limit generalisability, due to the unique characteristics of the Veterans Affairs (VA) population (predominantly male veterans), the home-grown EMR VA system, and the lack of pre EMR data for comparison, results showed that the rates of loss to follow-up appeared lower than those

reported in sites that do not use information technology. However, even with the sophisticated electronic alert system in place a proportion of results were still missed: 4% (45/1017) of critical imaging results39; 11% of alerts for abnormal imaging (acknowledged and unacknowledged)40; and 6.8% of specified abnormal laboratory tests.41 Interestingly, in the latter study Singh et al. found that 10.2% of alerts were unacknowledged by physicians (provider did not click on and open the message within two weeks of transmission) and timely follow-up was statistically not different for acknowledged and unacknowledged alerts (6.4% versus 10.1%; P=0.13).

Having test management processes supported by hybrid paper and electronic systems has also been shown to create problems with test follow-up. Casalino et al. reported that the use of a partial electronic medical record (paper based progress notes and electronic tests or vice versa) was associated with higher follow-up failure rates compared to relying entirely on paper-based systems (OR 1.92; P=0.03) or compared to having a complete electronic record that included both progress notes and test results (OR 2.37; P=0.007).2 Another study found that results managed in the EMR were significantly better documented than those with paper (40%, 33/82, compared to 64%, 57/88, p=0.001)34 and documentation of follow-

up actions has been shown to impact positively on test management.38,46

DISCUSSION Extent and Impact on Patient Outcomes

Failure to follow-up test results occurs frequently in ambulatory settings and evidence of its impact demonstrates that it is an important patient safety issue which needs urgent attention. There was wide variability reported regarding the extent of the problem ranging from 6.8%41 to 62%32 for missed laboratory tests and 1.0%31 to 35.7%46 for radiology. This variability was also present in studies which examined follow-up of the same test types which highlights the heterogeneity of the studies and the complexity of the problem.29,32,35,41-44 Similar results relating to extent and variability of test follow-up have previously been reported for inpatients (20.04%-61.9%) and patients attending emergency departments (1%-75%).26 Impact on ambulatory patient outcomes were considerable and included missed cancer diagnoses29,31,40,45

underpinning the urgent nature of the problem and the need to evaluate solutions.

Given that all studies were conducted in the US care should be taken in generalising the findings to other

countries although anecdotal evidence suggests that test management practices are similar worldwide. Eight of the 19 studies were based in three sites and undertaken by the same lead researcher in each site.33,34,36-41 A limitation of 12 of 19 studies was that they were conducted in single sites. However the six studies which included multiple primary care practices with large samples showed consistently high rates of lack of follow-up.2,30,32-34,46 Most studies used retrospective medical record reviews which may result in an overestimation of the problem as some results could have been followed-up even though the physician failed to document the action. It is possible that there may be some publication bias whereby papers which reported higher rates of missed follow-up of test results are more likely to be published than studies which found low rates.

Missed test results in ambulatory settings are attributable to multiple factors including: the paucity of governance principles related to test management; the lack of integrated information systems around test management; the multidisciplinary nature of test management processes, and the need to consider the role of the patient in test result follow-up.

Test Management Governance Principles

There are few standard policies or procedures for test result management in the literature or within ambulatory practice organisations. This contributes to the variability of responsibility and the diversity of work practices around each of the steps in the test management

process.2,29,33,34,39,40 Singh et al.39 concluded that if

policies and procedures were in place, combined with the use of an electronic alert notification of abnormal radiology, then follow-up of results would be improved. However they found that missed results were not eliminated altogether39 indicating the complex nature of the problem and the need to consider a combination of interventions.

The lack of clear lines of responsibility for follow-up is further complicated by the number of people involved. Blurred lines of responsibility were evident in one study which evaluated the practice of dual-alert notification: this measure was adopted as a safeguard and entailed communicating the alert of an abnormal radiology result to two recipients, for example the hospital doctor who ordered the test and the primary care physician.40 Results showed that dual-alert notification significantly increased the odds that the alert would not be read and receive timely follow-up action (OR, 1.99; 95% CI, 1.06-3.48).40 This highlights the need for explicit

guidelines regarding responsibility and timeframes for follow-up.

The Role of Information and Communication Technology in the Process

Although studies of evidence regarding the link between EHRs and ambulatory quality of care have shown mixed results,47-51 a number have shown that information and communication technology (ICT) can play an important role in ensuring a safer and more systematic test management process.1,52,53 The five evaluations of the impact of ICT on missed test results, included in the review, provided evidence of a favourable trend towards reduced missed results.2,34,39-41 The use of ICT facilitates communication flow between individuals across care settings and has been shown to improve documenta-tion38,46 all of which are crucial for the test management process. Other studies have shown that ICT can support

and enhance the test management process14,54-57 with

acknowledgement of receipt of results and documentation of follow-up action and patient notification recorded in an EMR which can be securely accessed by multiple health professionals.

Timely communication is vital and can be particularly challenging for ambulatory patients given the many individuals and settings involved. Information transfer about tests ordered and results received can have multiple points of communication breakdown: between inpatient and outpatient settings , ; between laboratories and physicians58,59; between physicians and other member of their team33,34; and between physicians and patients.9,60 Electronic systems are an essential component of any solutions, however, they need to be integrated into the context of ambulatory settings and the way health professionals work.61,62 Financial issues are also considered to be a key barrier to primary care physicians' use of EHRs and financial incentives will be a key lever for improving uptake.11

Technological solutions are not the entire answer as studies show that even with the use of an EMR with computerised notification of abnormal results using an alert every time the provider logs-on, abnormal results can still be missed.39-41 Even though physicians can electronically acknowledge that they have opened an alert message this does not necessarily indicate they have read and acted upon the abnormal result.41 Appropriate follow-up requires the physician to review the result, communicate this to the patient, decide on an appropriate plan and discuss this with the patient and help them with that follow-up plan.46

Electronic discharge summaries have been shown to improve quality and timeliness of information including test results.63 However, there can also be problems as electronic discharge summaries may not always reach the family doctor64 and may not include all the necessary clinical information required for the ongoing treatment of the patient.65,66 There can also be an overreliance on ICT. One study reported that staff who used an EHR believed that electronically reported results did not get lost and were unaware that several tasks still depended on individuals.33 Thus, even with an EHR there can still be weaknesses in test tracking.56

Multidisciplinary Nature of Laboratory and Radiology Test Management Processes

Test result follow-up and patient notification in ambulatory settings are generally undertaken by the physician treating the patient, with involvement of other physicians, nurses, laboratories, and practice managers. A number of studies recommend the need to take account of the multidisciplinary nature of the test management process and include not just the physician who ordered the test but other ambulatory practice staff30,40 and also laboratory staff,67 in the design of solutions.

The involvement of nurses in the test notification process is

29 31 33 34 37

evident in a number of studies29'31'33'34'3, and in busy ambulatory settings, where follow up of results is time consuming,13,14 this would seem to be a valuable approach. Nurses were reported as having a positive impact in one study where there was no significant difference between physicians and nurse practitioners in appropriate documentation of follow-up (p=0.61).31 Studies have shown that documentation was associated with the delivery of appropriate46 and with timely follow-up of test results. Clerical staff in ambulatory settings can also assist in the follow-up of test results particularly in relation to ensuring systems are in place to alert physicians to results returned from laboratories and in the patient notification stage of the process.33

An innovative work practice change is the suggestion that radiology staff should become more involved in the patient test notification process.67,68 The process of direct patient notification by the radiologist has been described by one Imaging Service in the US whereby patients' are given a card on arrival asking if they would like to receive preliminary results of their examination.69 If the patient agrees the radiologist documents the preliminary results on the card which is given to the patient before they leave the Imaging Centre.69 Currently there are guidelines for direct communication of critical imaging findings by radiology staff to ordering physicians.70 However, problems arise in ensuring critical results are communicated to the appropriate person. , 7 Singh et al. found that verbal communication by radiology staff in addition to electronic communication of results strongly predicted timely response and follow-up,

however this was probably because the radiologist called only for life threatening findings. Solutions which acknowledge the inter-dependence between physicians, nurses, and radiology/laboratory staff are important to ensure the safe communication of abnormal test results.7

The Role of the Patient in Test Result Follow-up

There have been suggestions that patients can play a role in detecting and preventing medical errors73 and could therefore have an enhanced role in the test result follow-up process.27-29 Direct notification of test results to patients could serve as a safety net for providers and empower patients to be partners in their care.8,9 It is possible for electronic systems, linked to test result databases, to generate automatic letters, email or telephone messages to notify patients directly of positive and negative test results9 or to allow patients' direct access to results via a patient portal.74 The issue of direct patient notification however, is complex and many factors need to be considered. Physicians have expressed concerns including: whether an abnormal result will alarm a patient unnecessarily; receipt of an avalanche of contact from worried patients; practice variations in timing of patient notification; malpractice risks; and an increase in un-reimbursed tasks.7 There are few published studies in this area, however, a feasibility study to introduce on-line laboratory results to patients in two primary care practices with 10 physicians reported that physicians had no increase in messages from their patients about inconsequential results showing that fears of unnecessary patient alarm may be unwarranted.7 If all patients are directly notified of their test results, they need to be able to read and understand the result and its implications. Patient understanding of test results could be enhanced by access to relevant online information sources with one study reporting 50% of patients who viewed results via a patient portal accessed reference information linked to the result.74

Whether patients hear their results verbally or receive them in hard copy can also impact on their understanding and the emotional state of patients receiving significant results may also impair their ability to digest information provided verbally.7 7 Poon et al. found that a significant proportion of women whose physician documented that they had discussed the test result with the patient did not recall the discussion, highlighting the need for patients to have a written record of their results. Direct access to results via patient portals where patients could print a copy of the result would ensure they had an accurate hard-copy of the result which could be used as a memory aid. It is now a US FDA requirement under the Mammography Quality Standards Act (MQSA 2004) that a summary of the written report in terms understandable by a layperson is sent directly to the patient within 30 days,77 however studies have shown that giving

patients their results is not in itself enough to ensure that patients follow-up on the result.46,78 Physicians need to have an active role in follow-up to ensure the plans are understood by patients. Further work needs to be undertaken in this area to test the feasibility and effectiveness of strategies which enable patients to receive their test results automatically.


There are significant safety issues in the management of laboratory and radiology test results for ambulatory patients. Studies show that factors associated with failure to follow-up test results in ambulatory settings are complex and often it is a combination of elements, systems, people, organisational factors and work practices interacting that leads to important results being missed. This concords with patient safety research which has progressed to acknowledge the multi-factorial nature of medical errors and the need to address people and system factors if improvements are to be realised.79-81 Solutions need to be multipronged and include: explicit policies, procedures and responsibilities for test follow-up; consideration of the role of others in the process including the patient and laboratory and radiology staff; evaluation of ICT solutions; and integration of solutions into the work practices of health professionals and into the context of health care delivery.

Acknowledgements: This study is part of an Australian Research Council Linkage Grant (LP0989144) funded project to investigate the use of information and communication technologies to support effective work practice innovation in the health sector. The authors have no relevant financial interest in this manuscript.

Conflicts of Interest: None disclosed.

Open Access: This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Corresponding Author: Joanne L. Callen, PhD; Centre for Health Systems and Safety Research, Faculty of Medicine, The University of New South Wales, Sydney, NSW 2052, Australia (e-mail:


1. Hickner JM, Fernald DH, Harris DM, Poon EG, Elder NC, Mold JW.

Issues and initiatives in the testing process in primary care physician offices. Jt Comm J Qual Patient Saf. 2005;31(2):81-9.

2. Casalino LP, Dunham D, Chin MH, Bielang R, Kistner EO, Karrison

TG, et al. Frequency of failure to inform patients of clinically significant outpatient test results. Arch Intern Med. 2009;169(12):1123-9.

3. World Alliance for Patient Safety. World Health Organisation. Summary of the Evidence on Patient Safety: Implications for Research. In: Jha A, ed; 2008:1-118.

4. Schiff GD. Getting Results—Reliably Communicating and Acting on Critical Test Results: Joint Commission Resources; 2006.

5. Elder NC, Graham D, Brandt E, Dovey S, Phillips P, Ledwith J, et al.

The Testing Process in Family Medicine: Problems Solutions and Barriers as Seen by Physicians and Their Staff. A study of the American Academy of Family Physicians' National Research Network. J Patient Saf. 2006:2 (1):25-32.

6. Boohaker EA Ward RE, Uman JE, McCarthy BD. Patient notification and follow-up of abnormal test results. A physician survey. Arch Intern Med. 1996;156(3):327-31.

7. Murff HJ, Gandhi TK, Karson AK, Mort EA, Poon EG, Wang SJ, et al.

Primary care physician attitudes concerning follow-up of abnormal test results and ambulatory decision support systems. Int J Med Inform. 2003;71(2-3):137-49.

8. Meza JP, Webster DS. Patient preferences for laboratory test results notification. Am J Manag Care. 2000;6(12):1297-300.

9. Keren R, Muret-Wagstaff S, Goldmann DA, Mandl KD. Notifying emergency department patients of negative test results: pitfalls of passive communication. Pediatr Emerg Care. 2003;19(4):226-30.

10. Hickner J, Graham DG, Elder NC, Brandt E, Emsermann CB, Dovey S, et al. Testing process errors and their harms and consequences reported from family medicine practices: a study of the American Academy of Family Physicians National Research Network. Qual Saf Health Care. 2008;17(3):194-200.

11. Bates DW. Physicians and ambulatory electronic health records. Health Aff (Millwood). 2005;24(5):1180-9.

12. Elder NC, Hickner J, Graham D. Quality and safety in outpatient laboratory testing. Clin Lab Med. 2008;28(2):295-303. vii.

13. Poon EG, Gandhi TK, SequistTD, MurffHJ, KarsonAS, Bates DW. "I wish I had seen this test result earlier!": Dissatisfaction with test result management systems in primary care. Arch Intern Med. 2004:164 (20):2223-8.

14. Poon EG, Wang SJ, Gandhi TK, Bates DW, Kuperman GJ. Design and implementation of a comprehensive outpatient Results Manager. J Biomed Inform. 2003;36(1-2):80-91.

15. Wahls TL, Cram PM. The frequency of missed test results and associated treatment delays in a highly computerized health system. BMC Fam Pract. 2007:8:32.

16. Roy CL, Poon EG, Karson AS, Ladak-Merchant Z, Johnson RE, Maviglia SM, et al. Patient safety concerns arising from test results that return after hospital discharge. Ann Intern Med. 2005;143(2):121-8.

17. Gandhi TK. Fumbled handoffs: one dropped ball after another. Ann Intern Med. 2005:142(5):352-8.

18. Bird S. Missing test results and failure to diagnose. Aust Fam Physician. 2004:33(5):360-1.

19. Berlin L. Malpractice and radiologists. AJR Am J Roentgenol. 1980:135 (3):587-91.

20. Berlin L. Malpractice issues in radiology: res ipsa loquitur. AJR Am J Roentgenol. 2009;193(6):1475-80.

21. Potchen EJ, Bisesi MA, Sierra AE, Potchen JE. Mammography and malpractice. AJRAm J Roentgenol. 1991;156(3):475-80.

22. Graber M. Diagnostic errors in medicine: a case of neglect. Jt Comm J Qual Patient Saf. 2005;31(2):106-13.

23. Leape LL. Error in medicine. JAMA. 1994;272(23):1851-7.

24. Parker D, Lawton R. Psychological contribution to the understanding of adverse events in health care. Qual Saf Health Care. 2003;12(6):453-7.

25. Hodges B, Regehr G, Martin D. Difficulties in recognizing one's own incompetence: novice physicians who are unskilled and unaware of it. Acad Med. 2001:76(10 Suppl):S87-9.

26. Callen J, Georgiou A, Li J, Westbrook JI. The safety implications of missed test results for hospitalised patients: a systematic review. BMJ Qual Saf. 2011;20(2):194-9.

27. Cram P, Rosenthal GE, Ohsfeldt R, Wallace RB, Schlechte J, Schiff GD. Failure to recognize and act on abnormal test results: the case of screening bone densitometry. Jt Comm J Qual Patient Saf. 2005;31(2):90-7.

28. Schiff GD, Kim S, Krosnjar N, Wisniewski MF, Bult J, Fogelfeld L, et al. Missed hypothyroidism diagnosis uncovered by linking laboratory and pharmacy data. Arch Intern Med. 2005;165(5):574-7.

29. Chen ZJ, Kammer D, Bond JH, Ho SB. Evaluating follow-up of positive fecal occult blood test results: lessons learned. J Healthc Qual: Promot Excell Healthc. 2007;29(5):16.

30. Chen ET, Eder M, Elder NC, Hickner J. Crossing the finish line: follow-up of abnormal test results in a multisite community health center. J Natl Med Assoc. 2010;102(8):720-5.

31. Choksi VR, Marn CS, Bell Y, Carlos R. Efficiency of a semiautomated coding and review process for notification of critical findings in diagnostic imaging. AJRAm J Roentgenol. 2006;186(4):933-6.

32. Ealovega MW, Tabaei BP, Brandle M, Burke R, Herman WH. Opportunistic screening for diabetes in routine clinical practice. Diabetes Care. 2004;27(1):9-12.

33. Elder NC, McEwen TR, Flach JM, Gallimore JJ. Management of test results in family medicine offices. Ann Fam Med. 2009;7(4):343-51.

34. Elder NC, McEwen TR, Flach J, Gallimore J, Pallerla H. The

management of test results in primary care: does an electronic medical record make a difference? Fam Med. 2010;42(5):327-33.

35. Kern LM, Callahan MA, Brillon DJ, Vargas M, Mushlin AI. Glucose testing and insufficient follow-up of abnormal results: a cohort study. BMC Health Serv Res. 2006;6:87.

36. Moore C, Wisnivesky J, Williams S, McGinn T. Medical errors related to discontinuity of care from an inpatient to an outpatient setting. J Gen Intern Med. 2003;18(8):646-51.

37. Moore CR, Lin JJ, O'Connor N, Halm EA. Follow-up of markedly elevated serum potassium results in the ambulatory setting: implications for patient safety. Am J Med Qual. 2006;21(2):115-24.

38. Moore C, Lin J, McGinn T, Halm E. Factors associated with time to follow-up of severe hyperkalemia in the ambulatory setting. Am J Med Qual. 2007;22(6):428-37.

39. Singh H, Arora HS, Vij MS, Rao R, Khan MM, Petersen LA. Communication outcomes of critical imaging results in a computerized notification system. J Am Med Inform Assoc. 2007;14(4):459-66.

40. Singh H, Thomas EJ, Mani S, Sittig D, Arora H, Espadas D, et al. Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential? Arch Intern Med. 2009;169(17):1578-86.

41. Singh H, Thomas EJ, Sittig DF, Wilson L, Espadas D, Khan MM, et al. Notification of abnormal lab test results in an electronic medical record: do any safety concerns remain? Am J Med. 2010;123(3):238-44.

42. Stelfox HT, Ahmed SB, Fiskio J, Bates DW. An evaluation of the adequacy of outpatient monitoring of thyroid replacement therapy. J Eval Clin Pract. 2004;10(4):525-30.

43. Etzioni DA, Yano EM, Rubenstein LV, Lee ML, Ko CY, Brook RH, et al. Measuring the quality of colorectal cancer screening: the importance of follow-up. Dis Colon Rectum. 2006;49(7):1002-10.

44. Edelman D. Outpatient diagnostic errors: unrecognized hyperglycemia. EffClin Pract. 2002;5(1):11-6.

45. Gandhi TK, Kachalia A, Thomas EJ, Puopolo AL, Yoon C, Brennan TA, et al. Missed and delayed diagnoses in the ambulatory setting: a study of closed malpractice claims. Ann Intern Med. 2006;145(7):488-96.

46. Poon EG, Haas JS, Louise Puopolo A, Gandhi TK, Burdick E, Bates DW, et al. Communication factors in the follow-up of abnormal mammo-grams. J Gen Intern Med. 2004;19(4):316-23.

47. Romano MJ, Stafford RS. Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Arch Intern Med. 2011;171(10):897-903.

48. Linder JA, Ma J, Bates DW, Middleton B, Stafford RS. Electronic health record use and the quality of ambulatory care in the United States. Arch Intern Med. 2007;167(13):1400-5.

49. Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223-38.

50. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, et al. Systematic review: impact of health information technology on quality, efficiency, and costs ofmedical care. Ann Intern Med. 2006;144(10):742-52.

51. O'Connor PJ, Sperl-Hillen JM, Rush WA, Johnson PE, Amundson GH, Asche SE, et al. Impact ofelectronic health record clinical decision support on diabetes care: a randomized trial. Ann Fam Med. 2011;9(1):12-21.

52. Bates DW, Gawande AA. Improving safety with information technology. N Engl J Med. 2003;348(25):2526-34.

53. Bates DW. Using information technology to reduce rates of medication errors in hospitals. BMJ. 2000;320(7237):788-91.

54. Kuperman GJ, Teich JM, Tanasijevic MJ, Ma'Luf N, Rittenberg E, Jha A, et al. Improving response to critical laboratory results with automation: results of a randomized controlled trial. J Am Med Inform Assoc. 1999;6(6):512-22.

55. Greenes DS, Fleisher GR, Kohane I. Potential impact of a computerized system to report late-arriving laboratory results in the emergency department. Pediatr Emerg Care. 2000;16(5):313-5.

56. Callen J, Paoloni R, Georgiou A, Prgomet M, Westbrook J. The rate of missed test results in an emergency department: an evaluation using an electronic test order and results viewing system. Methods Inf Med. 2010;49(1):37-43.

57. Singh H, Naik AD, Rao R, Petersen LA. Reducing diagnostic errors through effective communication: harnessing the power of information technology. J Gen Intern Med. 2008;23(4):489-94.

58. Piva E, Sciacovelli L, Zaninotto M, Laposata M, Plebani M. Evaluation of effectiveness of a computerized notification system for reporting critical values. Am J Clin Pathol. 2009;131(3):432-41.

59. Plebani M. Interpretative commenting: a tool for improving the laboratory-clinical interface. Clin Chim Acta. 2009;404(1):46-51.

60. Couchman GR, Forjuoh SN, Rascoe TG, Reis MD, Koehler B, Walsum KLv. E-mail communications in primary care: what are patients' expectations for specific test results? Int J Med Inform. 2005;74(1):21-30.

61. Berg M, Langenberg C, vd Berg I, Kwakkernaat J. Considerations for sociotechnical design: experiences with an electronic patient record in a clinical context. Int J Med Inform. 1998;52(1-3):243-51.

62. Callen JL, Braithwaite J, Westbrook JI. Contextual implementation model: a framework for assisting clinical information system implementations. J Am Med Inform Assoc. 2008;15(2):255-62.

63. O'Leary KJ, Liebovitz DM, Feinglass J, Liss DT, Evans DB, Kulkarni N, et al. Creating a better discharge summary: Improvement in quality and timeliness using an electronic discharge summary. J Hosp Med. 2009;4(4):219-25.

64. van Walraven C, Seth R, Austin PC, Laupacis A. Effect of discharge summary availability during post-discharge visits on hospital readmission. J Gen Intern Med. 2002;17(3):186-92.

65. Bolton P, Mira M, Kennedy P, Lahra MM. The quality ofcommunication between hospitals and general practitioners: an assessment. J Qual Clin Pract. 1998;18(4):241-7.

66. Callen J, McIntosh J, Li J. Accuracy of medication documentation in hospital discharge summaries: A retrospective analysis of medication transcription errors in manual and electronic discharge summaries. Int J Med Inform. 2010;79(1):58-64.

67. Berlin L. Communicating results of all outpatient radiologic examinations directly to patients: the time has come. AJR Am J Roentgenol. 2009;192


68. Berlin L. Communicating results of all radiologic examinations directly to patients: has the time come? AJR Am J Roentgenol. 2007;189(6):1275-82.

69. Hammerman HJ. Communicating imaging results to patients: OnSite results. AJR Am J Roentgenol. 2009;192(4):852-3.

70. American College of Radiology. ACR Practice Guideline for Communication of Diagnostic Imaging Findings. 2010:1-6. Accessed 12 May 2011. guidelines/dx/comm_diag_rad.aspx. 2010.

71. Kilpatrick ES. Use of computer terminals on wards to access emergency test resutls: a retrospective audit. Br Med J. 2001;322:3.

72. Graber M. The physician and the laboratory: partners in reducing diagnostic error related to laboratory testing. Am J Clin Pathol. 2006;126(Suppl 1):S44-7.

73. Unruh KT, Pratt W. Patients as actors: the patient's role in detecting, preventing, and recovering from medical errors. Int J Med Inform. 2007;76(Suppl 1):S236-44.

74. Wald JS, Burk K, Gardner K, Feygin R, Nelson E, Epstein M, et al. Sharing electronic laboratory results in a patient portal-a feasibility pilot. Stud Health Technol Inform. 2007;129(Pt 1):18-22.

75. Ley P. Memory for medical information. Br J Soc Clin Psychol. 1979;18 (2):245-55.

76. Kessels RP. Patients' memory for medical information. J R Soc Med. 2003;96(5):219-22.

77. US Department of Health and Human Services, US Food and Drug Administration. 2004. Mammography Quality Standards Act (MQSA) MAMMOGRAPHY QUALITY STANDARDS ACT (MQSA) (AS AMENDED BY MQSRA of 1998 and 2004).Accessed September, 26, 2011. http://www.fda. gov/Radiation-EmittingProducts/MammographyQualityStandards ActandProgram/Regulations/ucm110823.htm.

78. Priyanath A, Feinglass J, Dolan NC, Haviley C, Venta LA. Patient satisfaction with the communication of mammographic results before and after the Mammography Quality Standards Reauthorization Act of 1998. AJR Am J Roentgenol. 2002;178(2):451-56.

79. Shekelle PG, Pronovost PJ, Wachter RM, Taylor SL, Dy SM, Foy R, et al. Advancing the science of patient safety. Ann Intern Med. 2011;154 (10):693-6.

80. Leape LL, Berwick DM, Bates DW. What practices will most improve safety? Evidence-based medicine meets patient safety. JAMA. 2002;288


81. Wachter RM. Why diagnostic errors don't get any respect-and what can be done about them. Health Af (Millwood). 2010;29(9):1605-10.


Table 1. Studies Showing the Extent of Failure to Follow-up Test Results for Ambulatory Patients and Impact on Patient Outcomes

Study design & population

Test type and indication of failure to follow-up

System used to follow-up results

Extent of failure to follow-up

Patient outcomes

Casalino et al. ^

Chen et al.

Retrospective medical record review and physician notification form (PNF) (survey).

Population: 5,434 randomly selected patients aged 50-69 years who had been seen 90-360 days prior to the date of the review (between June 2005 and February 2006) in 19 community-based and 4 academic medical centre primary care practices (n=23).

Retrospective medical record review

Population: 423 positive FOBT were identified for a 10 month period (December 1 2003 to September 30 2004) by the laboratory service of the medical centre of a 347 bed tertiary care facility of the Department of Veteran Affairs

Test type: 11 blood tests and 3 screening tests (mammography, Papanicolaou smear, Fecal occult blood) commonly performed in the OP setting with clinically significant results (values defined by physician consultation).

Indicators: Apparent failure to inform defined as abnormal results for which the reviewer could not find evidence within the medical record that the patient had been informed within the defined time interval (usually 90 days). The PNF was mailed to confirm failure to inform.

Failure to document defined as apparent failure to inform for which the physician stated in the PNF that the patient had been informed but this had not been documented.

Failure to inform defined as apparent failures to inform for which the physician did not return the PNF or stated on the form that the patient had not been informed.

Test type: positive fecal occult blood test (FOBT)

Indicator: Outcomes of positive FOBT cases which had no documented follow-up plan and no explanation for this (such as 'patient declined' or 'patient opted to go elsewhere'). % reported per test and per patient (one test per patient)

Varied systems across practices: 14 with no EMR (paper based progress notes and test results), 5 with EMR (for outpatient progress notes and test results), 4 with partial EMR (electronic progress notes or test results but not both).

Centralised nurse based notification system. A designated nurse reviewed all electronic medical records of positive FOBT cases from each preceding month. The nurse then provided a monthly spreadsheet to notify appropriate clinic managers of cases with no indication of follow-up. The clinic managers then notified the patients' physician.

Full electronic medical record system with access to patient medical data from all VA medical centers and clinics.

Rate of apparent failures to inform or document was 7.1% (135 failures divided by1889 abnormal results) (Range 0-26.2%)

Use of a partial EMR was associated with higher failure rates for follow-up compared to not having an EMR (OR, 1.92; P=0.03) or with having a total EMR (OR, 2.37; P=0.007).

Patient outcomes not examined

15% (63/423) of positive FOBT cases notified by the nurse lacked a documented follow-up plan within 4 weeks of the positive result.

15/63 (24%) had no appropriate follow-up plans; 30/63 (48%) had follow-up plans delayed beyond 4 weeks; 13/63 (21%) had plans within 4 weeks which were not documented; 5/63 (8%) had colonoscopy not indicated (example patient with known active Crohn's disease with colonoscopy 105 days prior).

In seventeen of the 30 cases which had follow-up plans delayed beyond 4 weeks, 2 had colorectal cancers found 215 and 233 days after the initial positive FOBT (1 included T3N0M0 rectal cancer and 1 included T4N0M0 transverse colon cancer)

Study design & population

Test type and indication of failure to follow-up

System used to follow-up results

Extent of failure to follow-up

Patient outcomes

Retrospective medical record review

Population: 344 abnormal results from 11 clinics (non random selection) within a large urban multisite federally qualified community health center. Abnormal results were identified in two ways: patients with abnormal results from 2004 to 2007 using administrative databases and, secondly manual review of paper logs of abnormal results maintained at family health centres from January 2007 to August 2008.

Retrospective electronic medical record audit

Population: 395 possible malignant cases between April 2003 and March 2004 in ER/urgent care, primary care medicine, non-primary care medicine and surgery in a University affiliated hospital.

Test type: Abnormal results from 4 tests: pap smears; mammograms; international normalized ration (INR); prostate-specific antigen (PSA) (called high risk abnormal results)

Indicator: No documentation that the result was: filed in the medical record, the provider signed and responded to the result, the patient was notified of the result, and the appropriate follow-up occurred in a timely manner (or patient refusal for follow-up). Timely follow-up for each test type clearly defined in study (example, pap smear — colposcopy performed within 3 months; mammogram — diagnostic procedure performed within 3 months). % reported per test

Paper logs of abnormal test results are maintained by staff at each community health clinic as an internal record of tracking patients with abnormal results.

Manual medical records used in each community health clinic.

Test type: Suspected malignancy in radiology examinations (including conventional radiology, sonography, CT, MRI, barium exams, excretory urography, myelography, angiography and excluding mammography).

Indicator: No documentation of follow-up in electronic medical record.

% reported per patient (each patient had only one test) with analysis separated based on referring service.

34% (116/344) of abnormal results did not have documentation that that appropriate follow-up had occurred: 11% (9/82) for mammography; 26% (16/61) for INR; 45% (47/105) for Pap smears; 46% (44/96) for PSA.

Range of failure to follow-up between clinics was 13% to 59%.

Not examined

A semi-automated process involving electronic, paper and telephone reporting was employed for the coding and review of potentially malignant findings.

If a malignancy was found the radiologist contacted the referring clinician or appropriate member of the clinical team by telephone or rarely by secure email. The radiologist documented the contact in the report. Radiologist also assigned a code 8 to the report to signify possible malignancy and documented this in the electronic record. On a weekly basis the cancer registrar (a nurse practitioner) retrieved a list of all suspected malignancy patients from the database of all oncology patients at the hospital. The registrar monitored the electronic clinical record (patient notes, patient appointments, additional radiology or pathology visits scheduled) for documentation of appropriate follow-up. The registrar notified the hospital's tumour board and/or ordering provider. Then the responsibility for tracking additional patient evaluation was delegated to the nurse practitioner in charge of active cancer cases.

The last step of the results management process, ensuring appropriate follow-up, accounted for most

of the errors in follow-up.

For patients receiving follow-up care 49% of the time the follow-up did not occur in a timely manner

Provider unaware of imaging finding in 4/395 (1.0%) of

primary medicine clinic patients with suspected malignancy

In all referring services (IPs and OPs) provider unaware of imaging findings in 8/395 (2.0%) code 8 cases.

The type of imaging test did not predict appropriate follow-up (p=0.18).

In the primary medicine service, no difference was seen between physicians and nurse practitioners in appropriate documentation of follow-up (p=0.61).

5 of the 8 patients (IP or OP) where the provider was unaware of the findings had a final diagnosis of cancer.

Ealovega et al. 32 Retrospective medical record review

Population: 8,286 patients 45 years and over who had continuous enrolment in the M-CARE between 1.1.199 8 and 30 June 2001. 5,752 (69%) were screened for diabetes over this three year period with 202 (4%) people having an abnormal result.

M-CARE health maintenance organisation members assigned to the University of Michigan Health System (UMHS) for primary care (184 primary care physicians in 22 locations). M-CARE is a member managed care organisation wholly owned by the UMHS.

Edelman 44 Retrospective analysis of merged

databases: managed care administrative database; billing data from the MCO; and the DUMC laboratory database (96% of Duke Managed Care laboratory tests). Review of medical records of selected patients.

Population: All patients over 30 years enrolled in the MCO of Duke University Medical Centre between April 1996 and March 1999. 1426 patients during this period had a laboratory test result suggestive of diabetes

Test type: Abnormal fasting plasma glucose levels or oral glucose tolerance tests (OGGTs) (Diabetes screening) Indicator: No documentation of recognition of the abnormal screening in the medical record, including a comment about the result being abnormal, that is, a diagnosis of impaired fasting glucose, impaired glucose tolerance or diabetes referral to a dietician and/or performance of a definitive diagnostic test (fasting glucose, OGTT or HbA lc) within 6 months of screening. % reported per test

Test type: Abnormal glycemic testing. Laboratory tests used were plasma glucose tests and haemoglobin Aic (HbAic) measurements. Abnormal defined by any one of three criteria: HbAic 7.0% or higher, plasma glucose 200 mg/dL or higher, or plasma glucose 126 mg/dL or higher if the laboratory value was denoted as fasting in the database. Indicator: Number of patients with abnormal glycemic test results suggestive of diabetes who had no evidence of recognised diabetes documented in their medical record or included in the billing database. A patient had recognised diabetes if they had the appropriate ICD-9 code in the administrative database or if diabetes was mentioned in their medical record. They identified those without ICD-9 codes as candidates for medical record review and then reviewed a 30% random sample of those charts for mention of the word diabetes or DM in the progress notes. % reported per patient

Not stated in paper but use of electronic laboratory database and electronic medical records.

125/202 (62%) of abnormal results received no appropriate follow-up

Those who received appropriate follow-up were more likely to have been scheduled for follow-up appointments than those who did not (92% versus 66%, P=0.001) and were more likely to have kept the scheduled follow-up appointment (90% versus 58%, P=0.001)

36/304 without ICD-9 codes had mention of diabetes on medical record review.

Not stated in the paper but use of electronic laboratory database and manual medical records.

At least 9% (when restricted to findings that are most suggestive of diabetes (high HbAic or 2 high plasma glucose tests) and as many as 18% who have a positive lab. test suggesting diabetes have no evidence in the billing data or medical record that the condition was recognised.

1426 patients had laboratory tests suggestive of diabetes.

Not examined

ra a C

i 5* G

1122/1426 (79%) had ICD-9 codes for diabetes in the database.

Study design & population

Test type and indication of failure to follow-up

System used to follow-up results

Extent of failure to follow-up

Patient outcomes

Elder et al. "

Retrospective medical record review.

Retrospective medical record

Test type: Abnormal laboratory and imaging

Used qualitative methods to explore test management practices in each of the 4 family medicine offices.

Population: 4 family medicine offices in the Ohio region. 25 random charts from each office (n=100) that contained laboratory or imaging results in a 12 month period. 160 test results reviewed.


a) no documentation of clinician signature on result; or

b) no documentation in the chart of clinicians' response to the result.

% reported by test type

Test type: Clinically significant abnormal laboratory and imaging

Population: 8 purposefully chosen Family Medicine Offices to provide variation around geographic location (rural, suburban, urban), practice size, patient insurance status, technology level (EMR, no EMR), and residency program (program, no program).

25 charts at each office (n=200) that contained laboratory or imaging results in a 12 month period. Total of 461 test results reviewed in the 200 charts in the 8 offices.

Indicator: No documentation in chart of follow-up plans.

% reported by test type (commonly grouped tests (complete blood counts, metabolic profile etc) were considered a single test

Detailed description of different ways each step of the results management process was undertaken in each of the four family medicine offices, 1 of which had an EHR. Most processes were manual (phone, fax, written response) involving physicians and other staff. One practice had no physician acknowledgement or response to result nor patient notification nor documentation nor follow-up process.

Examples of patient notification of results included:

• Copy of test result mailed to patient

• Standard test result checklist mailed to patient

• Letter or note written to patient

• Physician calls patient

• Staff calls patient

• At a future patient visit

• Patient not notified

Examples of varied processes listed from the four family medicine offices for the follow-up of results needing further care step were:

• notes made to self by physicians;

• notes made to self by staff;

• ticker file maintained by staff or physician;

• registry of single test type (e.g. Pap smears);

• no follow-up.

4 paper based and 4 EMR systems. A total of 274 results were managed by an EMR. Results managed with an EMR were more often in the right place in the chart (100% versus 98%), had more clinician signatures (100% versus 86%), interpretations (73% versus 64%) and patient notifications (80% versus 66%) documented.

Four offices had written protocols and/or adhered to office-wide practices in two or three test result management steps (patient notification, clinician signature and test tracking). The other four offices had only one or no steps with written protocols or results management practices that were well followed. No offices had standardized processes for documenting interpretation of test results or follow-up for abnormal results.

For the subset of abnormal results (n=170) 64% of results managed with an EMR had a follow-up plan documented, compared to 40% for paper managed results.

42/160 (26%)abnormal laboratory or radiology t ests had no documentation in the chart of the clinicians response to the abnormal result

Not examined

17/160 (10%) of abnormal radiology or laboratory results had no clinician signature; 58/102 (57%) had no documentation that advice, recommendations or information were given to the patient about the abnormal result.

EMR systems: 36% (21/88) of abnormal results had no follow-up plan documented in an EMR.

Not examined

Paper based systems: 60% (49/82) of abnormal results had no follow-up plan documented in the manual system

Retrospective medical record review after data linkage of patients who received colorectal cancer screening in the VA quality improvement program linked with the VA Electronic Outpatient Clinic File.

Population: All patients aged over 52 years sampled from a QI review for colorectal cancer screening attending a primary clinic in a VA hospital between 1st October 2001 to 3041 September 2002.

Retrospective review of malpractice claims

Test type: Positive fecal occult blood test (FOBT)

Indicator: Patients had no follow-up if one of two criteria were not met: 1) documentation of colonoscopy or barium enema that was dated after the positive FOBT or 2) if the patient was referred for colonoscopy.

Test type: abnormal diagnostic or laboratory test result

Population: 181 closed malpractice claims involving diagnostic errors in the ambulatory setting which resulted in patient harm between 1984 and 2004 4 malpractice insurance companies (north eastern, south western, western United States) insuring approximately 21,000 physicians, 46 acute care hospitals and 390 outpatient facilities: including a variety of primary care and outpatient specialty practices. Retrospective medical record review

Indicator: Evidence of missed diagnosis resulting from provider not receiving abnormal diagnostic or laboratory test results abstracted from documentation in claim file from each insurer and background information from relevant medical record

Test type: abnormal value for glucose test for diabetes

Population: 301 randomly selected patients over 20 years of age with no known diabetes who received care at the study site over a one year period with a minimum of 2 further visits in the next 3 years Study site was a large outpatient general internal medicine practice affiliated with an academic health centre (New York USA) (Cornell Univ.) Retrospective medical record review

Population: 86 patients hospitalised in one medicine service who were subsequently seen by their primary care physicians in the affiliated outpatient practice within 2 months of discharge Participants selected by retrospective record review of 366 randomly selected patients chosen from 2139 discharged from 950 bed urban teaching hospital (Mount Sinai School of Medicine, New York, USA) between July 2000 and June 2001.

Indicator: No documented follow-up for abnormal value of glucose test for diabetes, or no indication in record of subsequent action in response to abnormal result

Test type: Laboratory tests pending at discharge (for example pleural fluid cytology, thyroid function tests, stool culture, HIV test)

Indicator: Laboratory test was pending at discharge and was not acknowledged (ie: documented) in the outpatient chart by the outpatient primary care physician

% reported per patient

Not stated in the paper however, VA have a quality improvement program which monitors rates of preventative care across a broad range of clinical domains, including colorectal cancer screening and VA has had electronic medical record system since 1985.

Of those patients with positive fecal occult blood tests (n=313) 41% (128/313) did not receive follow-up testing.

Not examined

Not stated in paper

23/181 (13%) cases of missed diagnoses were due to 'responsible provider did not receive diagnostic or laboratory test results'.

In 22/181 (12%) of cases, the diagnostic or laboratory test results were not transmitted to the patient

Of the 23 cases where the responsible provider did not receive the test result, 17 cases were for missed cancer diagnoses.

Of the 22 cases where the test result was not transmitted to the patient, 15 were for missed cancer diagnoses.

The practice has an EMR which includes appointments, test orders, laboratory results and progress notes. At the time of the study there was no systematic screening program for diabetes and the EMR did not have specific flags for episodes of diabetes screening.

466 glucose tests performed and 65/466 were abnormal. The proportion of abnormal values depended on the cut-off: 65 (14%) >101 mg/dl; of these, 26 (6%) >110 mg/dl; and 15 (3%) >125 mg/dl For glucose values:

Not examined

Not stated in paper but use of separate manual inpatient and outpatient medical records and outpatient PCPs had no access to the patients' discharge summaries.

101-109 mg/dl, 100% (24/24) had no follow-up; 110-125 mg/dl, 57% (15/26) had no follow-up: >126 mg/dl, 46% (7/15) had no follow-up

7/86 (8%) (95% CI, 2 to 14) of all participants and 41.2% of participants with tests pending at discharge

had at least 1 test follow-up error No association between test follow-up errors and rehospitalisation

Not examined

Study design & population

Test type and indication of failure to follow-up

System used to follow-up results

Extent of failure to follow-up

Patient outcomes

Moore et al. '

Retrospective medical record review Test type: abnormal serum potassium

Poon et al.

Singh et al.

Population: 86 patients with abnormal potassium levels (hyperkalaemia or hypo) seen at a large hospital based general internal medicine/primary care practice (Mount Sinai Medical Centre, New York USA) Between September 2003 and August 2004.

Retrospective medical record review

Population: 190 adult patients with elevated serum potassium levels over a 4-year period (January 2002 to December 2005) seen at a large primary care practice at a 1100 bed tertiary care academic teaching hospital (Mount Sinai School of Medicine, New York USA)

259 serum potassium tests that were non-haemolyzed and non-contaminated samples with values >6.0 mEq/L

Prospective longitudinal study involving medical record review and patient survey

Population: 126 (out of 181 with abnormal mammogram requiring 6 month follow-up) English or Spanish speaking women with abnormal mammograms (between June 1996 and June 1997) requiring short term (6 months) follow-up imaging from ten academically affiliated ambulatory medical practices in the Boston metropolitan area. The practices were diverse in location, structure and degree of academic affiliation.

Retrospective medical record review plus telephone contact with providers

Population: 1,017 electronically transmitted alerts for abnormal imaging results over an 83 day study period from patients attending multispecialty ambulatory clinics of a Veterans Affairs medical centre (Houston, Texas USA) between March 2 2006 and May 28 2006.

Indicator No documented response /action to abnormal serum potassium result in the medical record or computerised laboratory system

% reported per test and per patient

Test type: abnormal serum potassium

Indicator No documented attempt to contact patient for follow-up of abnormal result, or no documentation that subsequent patient visit addresses the elevated serum potassium result, or no documented repeat serum potassium testing

Test type: "Marginally abnormal" mammograms for which short term follow-up imaging is required in 3-6 months Indicator: no documentation in their medical record that they had received a follow-up mammogram, a surgical consult or a breast biopsy within 7 months of the index abnormal mammogram.

OP practice has a paper based chart system and laboratory test results are retrieved on a computerised information system via computers located in each patient care room. Paper copies of test results are placed in attending preceptors' mail-boxes within 1-7 days of the results being available. All results are reviewed by attending preceptors who are expected to act on abnormal test results. Critical lab results are communicated from the lab to a nurse in the primary care practice. The nurse then communicates the critical lab result to an attending physician.

Paper based chart system with results of laboratory tests retrievable via the computerised information system. Critical lab results are directly communicated from the lab to physicians in the practice.

Not stated in paper.

Test type: Critical imaging results (abnormal radiographs, CT scans, MRI scans, mammograms, ultrasonograms)

Indicator: No documented evidence of response to abnormal imaging results in patient chart, and no provider awareness of abnormal imaging results upon telephone contact with provider. Failure to acknowledge an alert was based entirely on the provider clicking on the alert in the electronic View Alert system in the EMR.

No documented evidence of follow-up at all in 11 /109 cases of hyperkalaemia (10.1%).

25% of patients repeat potassium levels were not performed until they were seen at routine follow-up visits or when they visited the clinic for problems unrelated to hyperkalemia such as medication refills.

After 30 days 26/259 elevated serum potassium

tests (10%) had no follow-up. After 30 days 18% of the cases had no repeat testing.

Not examined.

VA uses CPOE which relies on computerised notification of abnormal test results (alerts) displayed prominently through a view alert window displayed in the EMR every time the provider logs on.

With the exception of life threatening findings, communicated via telephone, the radiologist alerts the provider electronically to the presence of'significant unexpected findings' using codes specific to the institution. When the provider clicks on the alert this indicates that the provider received the imaging result and this is the only mechanism for the EMR to record the providers' acknowledgement However, providers can ignore alerts and bypass them.

45/1,017 abnormal result alerts (4%) were completely lost to follow-up (after checking with provider if they knew of result) 4 weeks after the date of the study.

Providers failed to acknowledge 368/1,017 (0.36%) transmitted alerts (ie: failed to click on the alert in the EMR).

Overall 10.8% of the episodes of hyperkalemia resulted in hospitalisations, and another 8.5% of episodes resulted in ED visits. There were no deaths resulting from the episodes of hyperkalemia.

45/126 (36%) of women with abnormal mammogram requiring short-term follow-up did not receive appropriate or timely follow-up.

Key finding was that documentation of follow-up plan by the treating doctor was associated with the appropriate follow-up of test results (adjusted odds ration 2.79; 95% CI, 1.11 to 6.98; P=0.029)

No adverse effects for those who did not received appropriate follow-up (one who did receive appropriate follow-up was diagnosed with breast cancer during the course of the study)

Not examined

Singh et al. 40 Retrospective medical record review

plus telephone contact with providers

Test type: Abnormal imaging results (abnormal radiographs, CT scans, MRI scans, mammograms, ultrasonograms)

Singh et al.

Stelfox et al.

Population: Of 123,638 imaging studies, 1,196 (0.9%) generated alerts for abnormal imaging results from November 2007 to June 2008 from patients attending multispecialty ambulatory clinics of a Veterans Affairs medical centre (Houston, Texas USA)

Retrospective medical record review plus telephone contact with providers

Population: Between May and December 2008, 78,158 tests (Hb Ale, Hep c, TSH, PSA) \were performed of which 1163 (1.48%) were transmitted as alerts. Study conducted in a large multispecialty ambulatory clinic of VA Medical Center and 5 satellite clinics located in Southeast Texas.

Retrospective medical record review

Population: 363 outpatients from a large tertiary care hospital receiving levothyroxine therapy over a 1 -year period - 2000 to 2001 (randomly selected)

Indicator: Alerts lacking electronic acknowledgement—Tracking software determined if the alert was acknowledged (that is, opened for viewing within 2 weeks of transmission).

Alerts lacking timely follow-up: Medical records were reviewed for unacknowledged electronic alerts within 4 weeks of transmission to determine if there was any documented response to the alert (defined as any documented evidence of follow-up action). Providers were telephoned to exclude failure to document. Test type: Abnormal laboratory tests which generated a 'high priority' mandatory (provider does not have the option to turn off the alert) automated notification to the provider without concomitant verbal notification. 4 laboratory test alerts met these criteria: haemoglobin >=15; positive hep C antibody; prostate specific antigen >=15; thyroid stimulating hormone (TSH) >=15 Indicator: Alerts lacking electronic acknowledgement—alert tracking system determined whether the alert was acknowledged (provider clicked on and opened the message) within 2 weeks of transmission.

Alerts lacking timely follow-up: Within 30 days of result transmission, record review and provider contact determined follow-up actions (e.g. Ordering a follow-up test, prescribing or changing treatment, contacting the patient) Test type: Abnormal thyroid stimulating hormone (TSH) for monitoring patients with hypothyroidism on levothyroxine medication Indicator: No appropriate surveillance response within 2 months of a newly abnormal surveillance result

The Computerised Patient Record System (the EMR at VA) includes a notification system (the View Alert system) for alerting clinicians to clinically significant events such as abnormal diagnostic test results. When the PCP is out of the office, they can assign covering surrogate HCPs to receive their alerts. When a trainee, a subspecialist, a surrogate or a substitute HCP orders a test, a second alert is automatically transmitted to the PCP (dual-alert communication). In this study the HCP who received an alert was considered responsible for timely follow-up (if 2 HCPs were alerted, they were both responsible for timely follow-up). HCPs have the option of bypassing or ignoring the View Alert window.

Staff radiologists electronically code abnormal imaging that requires action as alerts which are then transmitted to the View Alert window. Alerts stay active in the window for 2 weeks

For all alerts (acknowledged and unacknowledged^ no evidence of documented follow-up action was found in 131/1196 (11%) of the alerts.

In the 92 cases that did not receive timely follow-up, more than one fourth (n=26 (28%)) led to a new diagnosis being established Cancer was determined to be a new diagnosis in 11 of these cases.

For all alerts (acknowledged and unacknowledged) timely follow-up was lacking in 92/1196 (7.7%) of

all alerts.

217/1196 (18.1%) alerts were unacknowledged

The Computerised Patient Record System (the EMR at VA) includes a notification system (the View Alert system) for alerting clinicians to clinically significant events such as abnormal diagnostic test results.

Timely follow-up (within 30 days) was lacking in 79/1163 (6.8%) (79 out of all alerts) and was statistically not different for acknowledged and unacknowledged alerts.

Not reported

Timely lack of follow-up for specific tests: HCV Ab (Elisa) 13.2%; PSA 0.9%; TSH 6.3%; Haemoglobin A1C 6.9%.

10.2% of alerts (119/1163) were unacknowledged.

Not stated in paper

37/96 patients (39%) had no appropriate surveillance

response within 2 months to high or low TSH results.

24 patients (7%) experienced an adverse drug event (ADE) during the study period and 12 patients (3%) developed an ADE related to levothyroxine and 6 of these were secondary to insufficient supplementation