Scholarly article on topic 'Educational interventions to train healthcare professionals in end-of-life communication: a systematic review and meta-analysis'

Educational interventions to train healthcare professionals in end-of-life communication: a systematic review and meta-analysis Academic research paper on "Psychology"

Share paper
Academic journal
BMC Medical Education
OECD Field of science

Academic research paper on topic "Educational interventions to train healthcare professionals in end-of-life communication: a systematic review and meta-analysis"

Chung et al. BMC Medical Education (2016) 16:131 DOI 10.1186/s12909-016-0653-x

BMC Medical Education

Educational interventions to train healthcare professionals in end-of-life communication: a systematic review and meta-analysis


Han-Oh Chung1*, Simon J. W. Oczkowski2, Louise Hanvey3, Lawrence Mbuagbaw1,4 and John J. You1,4


Background: Practicing healthcare professionals and graduates exiting training programs are often ill-equipped to facilitate important discussions about end-of-life care with patients and their families. We conducted a systematic review to evaluate the effectiveness of educational interventions aimed at providing healthcare professionals with training in end-of-life communication skills, compared to usual curriculum.

Methods: We searched MEDLINE, Embase, CINAHL, ERIC and the Cochrane Central Register of Controlled Trials from the date of inception to July 2014 for randomized control trials (RCT) and prospective observational studies of educational training interventions to train healthcare professionals in end-of-life communication skills. To be eligible, interventions had to provide communication skills training related to end-of-life decision making; other interventions (e.g. breaking bad news, providing palliation) were excluded. Our primary outcomes were self-efficacy, knowledge and end-of-life communication scores with standardized patient encounters. Sufficiently similar studies were pooled in a meta-analysis. The quality of evidence was assessed using GRADE.

Results: Of 5727 candidate articles, 20 studies (6 RCTs, 14 Observational) were included in this review. Compared to usual teaching, educational interventions to train healthcare professionals in end-of-life communication skills were associated with greater self-efficacy (8 studies, standardized mean difference [SMD] 0.57;95 % confidence interval [CI] 0.40-0.75; P <0.001; very low quality evidence), more knowledge (4 studies, SMD 0.76;95 % CI 0.40-1.12; p < 0.001; low quality evidence), and improvements in communication scores (8 studies, SMD 0.69; 95 % CI 0.410.96; p < 0.001; very low quality evidence). There was insufficient evidence to determine whether these educational interventions affect patient-level outcomes.

Conclusion: Very low to low quality evidence suggests that end-of-life communication training may improve healthcare professionals' self-efficacy, knowledge, and EoL communication scores compared to usual teaching. Further studies comparing two active educational interventions are recommended with a continued focus on contextually relevant high-level outcomes. Trial registration: PROSPERO CRD42014012913

Keywords: End of life care, Communication, Advance care planning, Advance directives, Communication training, Medical education

* Correspondence:

11280 Main Street West, HSC-2C8, Hamilton, Ontario L8S 4K1, Canada

Full list of author information is available at the end of the article

(3 BioMed Central

© 2016 Chung et al. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.


Advances in medical care and the aging population have highlighted the need for good end-of-life (EoL) communication and decision-making, in order to ensure that invasive medical treatments are not administered to patients who would prefer less aggressive forms of care at the end-of-life [1]. Unfortunately, health care providers (HCPs) often fail to engage patients in EoL discussions and to document patient wishes in the medical chart [1, 2]. This puts many patients at risk of having unwanted aggressive and potentially futile medical care during their last days of life, which is associated with worsened patient and caregiver quality of life and psychological burden [3].

An important strategy for improving the quality of EoL discussions is to improve EoL communication skills amongst HCPs [4]. The educational need for this skill has been well described for both trainee and practitioners alike: medical graduates currently are entering practice ill-prepared to discuss the important EoL issues with patients and families [5-7]. In a multicenter Canadian survey, internal medicine residents at five universities identified that EoL communication skills were a high learning priority [8] as resident physicians are often responsible for facilitating EoL discussions with hospitalized patients in academic centers [7, 9, 10]. This need persists in even practicing HCPs such as physicians and nursing staff who continue to have discomfort in facilitating EoL discussions [11-15]. There are numerous EoL communication skills training programs described in literature, however the cumulative evidence on the impact of such an educational intervention remain unclear. Therefore, we conducted a systematic review to evaluate the effectiveness of educational interventions to train HCPs in EoL communication skills compared to usual teaching (i.e. standard curriculum). The effectiveness was measured based on the Kirkpatrick training evaluation model (Reaction, Learning, Behaviour and Results) [16] which was represented by outcomes of self-efficacy, knowledge, communication skills and patient-level effects.


Protocol and registration

The protocol for the complete review is available in the PROSPERO database at:

Eligibility criteria

Studies were eligible for our systematic review if they included adult patients over age 18 years, healthcare providers, or trainees, and if they evaluated a communication tool to assist adult patients in EoL decision-making, in comparison to a control group. Our definition of a communication tool included traditional decision aids in any

format (paper, video, computer, etc.), and other structured approaches to help with decision-making, including organized meeting plans, reminders to complete advance directives (AD) or educational interventions for patients or healthcare providers. Interventions designed solely for information-sharing (e.g. breaking bad news, providing emotional support) were excluded, because although such interventions may affect EoL decision-making, it is not their sole or explicit purpose to do so. We included randomized controlled trials (RCTs) and prospective observational studies with a control group (including cohort studies and uncontrolled before-after studies in which participants acted as their own control). We restricted the review to studies published in peer-reviewed journals in the English language (See Additional file 1).

Eligible studies were then divided into a subgroup of studies of educational interventions for health care providers, and a subgroup of studies used as clinical intervention. In this paper, we specifically review only the studies of educational interventions directed to health care providers, whether trainees in a health professional training program (e.g. medical or nursing student, postgraduate training), or practicing providers receiving continuing medical education. Reviews of the patient directed end-of-life communication tools will be analyzed and reported elsewhere.

Outcome measures

Based Kirkpatrick model of evaluation Reaction measures the learners' value they perceive in the educational intervention. Learning measures improvements in their knowledge, Behaviour measures their capability applied in context, and Result measures the impact the training on the target outcome - in this case, patient level outcomes [16].

With this framework, primary outcome measures were:

1) Self-efficacy (participant's confidence or estimate of their ability to perform a task) [17].

2) Knowledge test scores on EoL communication and decision-making.

3) Communication scores using a standardized checklist during a standardized patient encounter.

Secondary outcome measured patient-level outcomes such as completion of AD, health care utilization, patient satisfaction with EoL planning, and patient assessment of clinician communication skills.

Search strategy

A comprehensive search was performed for papers available for search from database inception to July 2014 from Medline (1946 - July 2014), Excerpta Medica database (EMBASE 1980 - July 2014), Cumulative Index to Nursing

and Allied Health Literature (CINAHL 1982 - July 2014), Cochrane Database of Controlled Clinical Trials (2005 -July 2014) and Education Resources Information Center (ERIC 1966 - July 2014). Searches were conducted using terms related to EoL decision-making and communication, including: "communication, " "decision-making, " "end-of-life" and "cardiopulmonary resuscitation!'. A snowball technique was used to hand search references for additional papers for review. This review is a subset of a larger systematic review on EoL decision-making interventions. Only those studies relevant to medical education are reviewed here. Reviews of communication interventions evaluated in the clinical setting will be analyzed and reported elsewhere.

Study selection

Title and abstracts were screened for relevance independently and in duplicate by two reviewers (SO, HC). Articles that passed initial screening by either reviewer underwent full-text review independently and in duplicate by the same two reviewers. Standardized, piloted eligibility forms were used for both title and abstract screening, and for full-text review. Disagreements about study eligibility were resolved through consensus discussion or resolved by a third reviewer (JY) in the case of ongoing disagreement. Kappa statistics were calculated to assess the inter-rater reliability of title and abstract screening and full-text review [18].

Data collection and data items

Data extraction was done using standardized, piloted, online forms by two reviewers (HC, SO), including publication information, study dates and population characteristics, interventions, outcomes, and study methods required to assess the risk of bias. We contacted study authors to obtain missing information relevant to outcomes or risk of bias.

Study quality and risk of bias of individual studies

Educational study quality and risk of bias were assessed in duplicate by two reviewers using the MERSQI (Medical Education Research Study Quality Instrument) Scale and Newcastle-Ottawa Scale Education (NOS-E) [19, 20]. The two instruments were used as they assess different aspects of quality and risk of bias acting in a complementary fashion. We considered a score above the sample median MERSQI score (12.5) and NOS-E score (2.5) as the threshold for high methodological quality as described in other literature [20].

The risk of bias for RCTs was additionally assessed using the Cochrane risk of bias tool which includes assessments of random sequence generation, allocation concealment, blinding of participants and personnel, incomplete outcome data, and selective reporting. Each

domain was assessed independently by both reviewers and reported as being at 'high,' 'low,' or 'uncertain' risk of bias. Studies were considered to be at overall 'high' risk of bias if judged to be at 'high' risk of bias at any domain; 'uncertain' risk of bias if judged to be at uncertain risk of bias in any one domain, with no domains at high risk of bias; and at overall 'low' risk of bias, if not judged to be at 'high' or 'uncertain' risk of bias in any domains [21]. All risk of bias assessment was judged at the outcome level.

Publication bias

Publication bias was assessed using visual inspection of funnel plots, where sufficient numbers of studies existed to permit interpretation [22].

Data synthesis, summary measures and sensitivity analysis

Reviewers assessed studies for clinical heterogeneity by investigating study populations, interventions, and comparisons before considering whether to pool data. We assessed statistical heterogeneity for each of the outcomes of interest using the I2 statistic, with values greater than 50 % indicating significant heterogeneity [23].

We used Review Manager 5.3 software to calculate pooled estimates of effects using all relevant studies employing the generic inverse variance method. A random-effects model was used to pool weighted outcomes of standardized mean differences (SMD). The magnitude of effect was interpreted in accordance with the Cohen effect size classification (small 0.2-0.5, moderate 0.5-8, large >0.8) [24]. Estimates for standard deviation for change scores were calculated when not reported or obtainable from study authors [25]. To see if the methodologic quality would materially affect our findings, sensitivity analyses were conducted post-hoc by restricting pooling of studies to those of higher methodologic quality such as RCTs only, or high MERSQI or NOS-E scores.

Ratings of quality of evidence

We used the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach to assess the quality of evidence for each outcome of interest. To rate the quality of evidence, the GRADE approach considers, for each outcome of interest, risk of bias within each study; risk of bias across studies (e.g. publication bias); imprecision of results; inconsistency of results; and indirectness of the evidence [26]. Summary of finding tables were generated using GradePRO software [27].


Study selection

Initial database searches retrieved 5727 articles. After exclusion of duplicate references, conference abstracts, and title and abstract screening, 424 articles were selected for full-text review (k = 0.65, 95 % CI [0.60, 0.70]). A total of 166 articles were found to be eligible for inclusion after full-text review and additional manual reference screening. Of these, 20 were studies of educational interventions and were reviewed in this paper (Fig. 1).

Study characteristics Study setting & populations

Most of the studies were completed in the USA (80 %), and 17 (90 %) were aimed toward medical trainees (14 postgraduate level, 3 undergraduate medical school), one to postgraduate medical trainees and nurse practitioners in acute care programs, and two studies were open to all acute care HCP (Table 1).

Study interventions

Review of the instructional design showed the majority of the studies used a combination of didactic lectures (17 studies), small group discussions (16 studies) and role-play with direct observation and feedback (16 studies). A minority of studies included self-study modules (3 studies), video or audio transcript analysis (2 studies), exemplar demonstration (5 studies) and reflective portfolio (1 study). One study used review of a multimedia Advance Directive decision aid with a patient as the educational intervention. Ten studies distributed the learning over more than a day, ten were workshops or tutorials done within a day or less.

Study quality and risk of bias

Six studies were RCTs, three of which were considered overall 'high' risk of bias, and 3 considered to be at 'unclear' risk of bias. Of the 14 observational studies, 10 studies were uncontrolled before-after design and 4 studies were double-arm cohort studies with a control group.

CO i j

MEDLINE, Embase, Cochrane Registry, CINAHL, ERIC (n=5727)

Additional articles from manual search of references (n=58)

Excluded (n=5361)

• Not relevant to EoL decision communication interventions

• Duplicates

Excluded (n=256)

• Not relevant to EoL decision communication interventions (156)

• Conference abstract only (44)

• Study protocol only (7)

• Duplicates (7)

• No outcome of interest (5)

• Pediatric studies (3)

• No comparator arm (2)

• Other® (32)

Qualitative only studies for future review (n=47)

Inpatient, non-ICU setting (n=15)b Intensive Care setting (n=19)b Ambulatory settings (n=67)

Fig. 1 Flow diagram of study screening and eligibility. aMarked 'other' due to unclear documentation whether study excluded due to duplication or non-relevance. bOne educational study overlapped with the inpatient studies and two overlapped with the ICU studies

Study ID Design Intervention Control Learner description group, n group, n

Alexander Cohort 37 2006 [38]

Ambulatory care rotator residents in Duke University Medical Centre

2007 [39]

Pre/Post 100 Participants

Oncology fellows from 62 different institutions who applied for the specialized workshop

Brist owe 2014 [30]

Pre/Post 16 Participants

Hemodialysis Nurses/Physicians

Clayton Pre/Post 21 Participants

2012 [31]

Residents at large tertiary-referral Australia

teaching-hospital in Sydney, Australia, voluntary participation

Fischer 2007 [35]

Furman 2006 [5]

Green 2011 [34]

Pre/Post 29 Participants Pre/Post 8 Participants RCT 60 56

Primary care internal medicine residents, USA voluntary participation

nternal Medicine and medicine- USA

paediatric Residents at Louisville Veterans Affairs Medical Center

Second year medical student in Penn USA State College of Medicine

Green berg RCT 46 1993 [28]

Medical Students beginning their USA



Comparator Outcome

The two-day retreat involving small-group lecture/ discussion, audio-visual materials, recordings of actual physician-patient encounters, and scripted "trigger tapes." Learners practiced through supervised role-play.

The Oncotalk curriculum involved a 4 day retreat, taught in small groups of 5 participants and 1 faculty facilitator. The curriculum was organized around 5 simulated patients. Learning activities included overviews, skills practice sessions, and reflective discussions.

Communication workshop with following sessions: fact session; patient and carer experience session; professional and personal experience session; communication, role-playing and feedback

Three 1 hour, onsite teaching sessions and a follow-up telephone call, spread out over 4 weeks. Sessions included interactive presentation of evidence-based strategies for conversations with patients expected to die within days/weeks and their caregivers and practice with standardized patients and feedback from an expert facilitator. Follow-up phone cal one week after final session reinforced and extended learning, and offered further support and feedback. Written and audio take-home learning materials provided.

Workshop included a brief lecture, group discussions, role-playing, and videotape review. Topics included included breaking bad news, advance care planning, ethics.

One morning report session consisting of both didactic training and three-person role-played discussion.

No Communication

intervention Behaviour

Pre- Communication

intervention Behaviour

No Self-Efficacy


Pre- Self-Efficacy


Communication Behaviour


P re-intervention

Pre- Patient AD

intervention Document

Students help patients create advance directives using a Students multimedia decision-aid which helps patients clarify their values, using

explain end-of-life conditions, help users choose and commun!- Standard

cate with their surrogate decision makers and translate their AD wishes and goals.

The high-intervention received the same reading as low inter- Provided

vention group as well as a small group seminar on topics of Self-study

historical development of advance directives, students' experi- readings

ences with death and dying, contents of a durable power of

attorney for health care (DPAHC). Students also viewed a

videotape illustrating important aspects of discussing the

DPAHC. They were finally assigned to initiate a discussion

about advance directives with a patient, family member, or



Satisfaction with EoL and EoL Care planning

Student Knowledge



Hales Pre/Post 18 Particiants Multidisciplinary critical care Canada

2008 [4] practitioners (MD, RN, SW, RT)

Holloran Cohort Sample size of learners Surgical residents rotating through the USA

1995 [42] not recorded, outcomes Surgical ICU (SICU). Outcomes collected

were patient based for patients who spent more than 30 days in SICU.

Lorin 2006 Cohort 53 53 All 4th year Medical Students rotating USA

[40] through mandatory ICU rotation

Pekmezaris Cohort 77 73 Residents rotating through internal USA

2011 [6] medicine inpatient service in New York

Perron 2002 Pre/Post 9 participants Residents in Internal Medicine Ward of Switzer-

[36] university affiliated community hospital land

Schell 2013 Pre/Post 22 Participants Nephrology Fellows at Duke University USA

[32] and University of Pittsburgh

Sharma RCT 23 28 Residents rotating through internal USA

2014 [37] medicine inpatient service

Smith 2013 Pre/Post 38 Participants [33]

nternal Medicine residents at UCSF USA

Workshop participants were assigned to practice groups of three to six members of varying disciplines and institutions. Groups rotated through the six 45-minute stations, enacting scenarios with standardized colleagues and families on topics ranging from the role of the substitute decision maker to approaching families about organ and tissue donation

Four weekly 60 minute case study discussion groups. Groups led by SICU attending physician and nursing director. Cases designed to force discussion of issues of withholding or withdrawing treatment, eliciting patient and family wishes, incompetent patients and conflict with families

Didactic teaching session on ICU communication framework, followed by practice with standardized patients

The training was composed of six sessions discussing importance of advance care planing, palliation and life sustaining therapies and interventions. The learners and were involved in role playing with standardized patients and received feedback from expert moderators.

Detailed information was given to all physicians in the department about the meaning of a 'Do not resuscitate' (DNR) order, its ethical dimension, the right of patients to make their own decision , and the concept of medical futility. Ethical aspects addressed only DNR measures and did not cover other measures such as life sustaining treatments.

Session consisting of large group didactic session to highlight the communication skills for breaking bad news and eliciting patient preferences, faculty role-play demonstrating these skills, then the fellows were divided into small groups of five to six members each for skills practice using standardized patients. Fellows had an opportunity to be a practicing fellow or an active observer.

The intervention group residents completed a multimodality code status discussion (CSD) educational intervention including didactic content, deliberate skills practice and self-study (e.g., online modules and maintenance of a log). In a follow-up intervention, residents received a 2-hour CSD skills "booster" session where they discussed themes from CSD logs, reviewed the CSD framework, and again observed a role play

The curriculum consisted of two one-hour lunch conference sessions and six one-hour morning reports at each hospital site, integrated into the regularly scheduled teaching sessions for residents on inpatient rotations. Residents explored challenging patient interactions and to discuss ways for conflict resolution and respond to their own emotional reactions to these scenarios.

Pre- Self-Efficacy


No Patient AD

intervention document

Health Care Utilization

No Communication

intervention Behaviour

No Self-Efficacy




Pre- Self-Efficacy



Communication Behaviour



Szmuilowicz RCT 2010 [29]

Szmuilowicz RCT 2012 [7]

Williams 2011 [41]

Pre/Post 24 Participants

Second year internalmedicine residents USA at Brigham and Women's Hospital

InternalMedicine Residents at USA

Northwestern University

All first year internalresident at Thomas USA Jefferson University Hospital

One day retreat covering conversations of 'Breaking Bad No

News" and "Discussing the Direction of Care", and skills intervention

related to responding to emotions. Every participant had the opportunity to interview a standardized patient and receive feedback from trained faculty at least once during the retreat.

Intervention included a 2 hour seminar discussing advance No

care planning and framework for EoL conversations, intervention

observing a code-status discussion modeled by faculty, and

exploration of past experiences. Intervention also included

self-study materials, internet communication skills teaching

modules and reflective portfolios.

3-hour workshop began with a review of the evidence Pre-

behind good communication skills, a discussion of barriers to intervention proper communication, and an in-depth explanation of the SPIKES protocol. Critique and discussion of a communication transcript of actualencounters and 5 minute video highlighting poor communication.


Communication Behaviour

Communication Behaviour

Communication Behaviour

Pre/Post refers to pre-intervention/post-intervention (i.e. before-after) studies where participants serve as their own control

Mean NOS-E was 3.35 (SD 2.13) out of maximum 6 points. The mean quality of all studies by MERSQI was 11.97 (SD 2.1) out of maximum 18 points. None of the RCTs were rated low overall risk of bias. 11 and 10 studies met the median threshold for high quality by MERSQI or NOS-E criteria respectively (See Additional file 2).

Synthesis of results

Ratings for the overall quality of evidence and effectiveness of the educational interventions can be seen in the GRADE summary of findings table (See Additional file 3).

1. Self-Efficacy

Eight studies (2 RCTs [28, 29], 6 Observational [4, 6, 30-33]), including 522 participants found that EoL communication skills training was associated with improved self-efficacy compared to usual training (SMD 0.57; 95 % CI 0.40-0.75; p <0.001; very low quality evidence). There was no evidence of heterogeneity (I2 = 0 %) (Fig. 2a).

2. Knowledge

Four studies (2 RCTs [28, 34] and 2 Observational [35, 36]), including 290 participants, reported knowledge outcomes. EoL communication skills training was associated with an increase in knowledge scores compared to usual training (SMD 0.76; 95 % CI 0.40-1.12 p <0.001, low quality evidence) with moderate heterogeneity (I2 = 47 %) (Fig. 2b).

3. Communication score

Eight studies (3 RCTs [7, 29, 37] and 5 Observational [31, 38-41]), including 590 participants found that EoL communication skills training was associated with improvement in communication scores rated during standardized patient encounters (SMD 0.69; 95 % CI 0.41-0.96; p < 0.001; very low quality evidence) with appreciable heterogeneity (I2 = 57 %). Heterogeneity could not be easily explained by qualitative examination of learner demographics, study quality or instructional design;

Study or Subgroup Sid. Mean Difference

Std. Mean Difference SE Weight IV. Random. 95% CI

Std. Mean Difference IV, Random, 95% CI

Bristowe 2014 Clayton 2012 Greenberg 1993 Hales 2008 Pekmezarls 2011 Schell 2013 Smith 2013 Szmuilowicz 2010

Total (95% CI) Heterogeneity: Tau! = Test for overall effect:

0.2818 0.3555 6.4%

1.0752 0.3323 7.3%

0.6995 0.2139 17.6%

0.6751 0.3438 6.8%

0.4823 0.1658 29.4%

0.9329 0.319 7.9%

0.4747 0.2328 14.9%

0.2164 0.2896 9.6%


0.00; Chi' = 6.66, df = 7 (P . Z = 6.38 (P < 0.00001)

0.47); I1

1.08 [0.42, 1.73] 0.70 [0.28, 1.12] 0.68 [0.00, 1.35] 0.48 [0.16, 0.81] 0.93 [0.31, 1.56) 0.47 [0.02, 0.93] 0.22 [-0.35, 0.78]

0.57 [0.40, 0.75]

Favours no intervention Favours intervention

Study or Subgroup Std. Mean Difference

Std. Mean Difference SE Weight IV, Random, 95% CI

Std. Mean Difference IV, Random, 95% CI

Fischer 2007 Green 2011 Greenberg 1993 Perron 2002

Total (95% CI)

Heterogeneity: Tau! = Test for overall effect:

0.8998 0.2765 24.4%

0.473 0.1844 35.6%

0.6995 0 2139 31.6%

1.8026 0.5812 8.3%


0.06; Chi1 = 5.63, df = Z = 4,18 (P < 0.0001)

3 (P = 0.13); l!

0.90 [0.36, 1.44] 0.47 [0.11, 0.83] 0.70 [0.28, 1.12] 1.80 [0.66, 2.94]

0.76 [0.40, 1.12)

Favours no intervention Favours intervention

Study or Subgroup Std. Mean Difference

Std. Mean Difference SE Weight IV, Random, 95% CI

Std. Mean Difference IV, Random, 95% CI

Alexander 2006 Back 2007 Claylon 2012 Lorin 2006 Sharma 2014 Szmuilowicz 2010 Szmuilowicz 2012 Williams 2011

Total (95% CI)

Heterogeneity Tau2 = 0.08; Chi2 = 16.26, df = Test for overall effect: Z = 4.89 (P < 0.00001)

0.1674 0.2827 12.0%

0.4706 0.1434 18.8%

1.3082 0.3431 9.8%

0.506 0.1975 16.0%

0.9047 0.2965 11.5%

0.2412 0.2898 11.7%

1.1378 0.3525 9.5%

1.2096 0.3161 10.7%


0.17 [-0.39, 0.72] 0.47 [0.19, 0.75] 1.31 [0.64, 1.98] 0.51 [0.12, 0.89] 0.90 [0.32, 1.49] 0.24 [-0.33, 0.81] 1.14 [0.45, 1.83] 1.21 [0.59, 1.83]

0.69 [0.41, 0.96]

7 (P = 0.02); I2 = 57%

Favours no intervention Favours intervention

Fig. 2 Effect of educationalinterventions on a self-efficacy b knowledge and c communication scores with standardized patient encounters

however in all studies, point estimates of effect were in the direction of benefit for EoL communication skills training (Fig. 2c).

4. Patient outcomes

There were four studies (2 RCT and 2 Observational) that reported patient-important outcomes. Outcome measures were heterogeneous, precluding pooling of data across studies. Overall, the interventions were neutral to positive: one study found no statistically significant change in the overall proportion of AD completed after a morning educational session [5], whereas another Intensive Care Unit (ICU) based intervention showed a beneficial effect on earlier completion of AD and decreased non-beneficial care in the ICU [42]. One study showed improved patient satisfaction in advance care planning [34], while conversely, another showed no improvement in patient-reported quality of EoL care or quality of communication [43].

Publication bias

No asymmetry was detected with a visual inspection of the funnel plot for self-efficacy or communication score outcomes, while knowledge outcomes showed some asymmetry. However, due to the limited number of studies, we could not conclusively comment on possible publication bias (See Additional file 4).

Sensitivity analysis

Sensitivity analyses were conducted in which we restricted our pooled analyses to those studies with higher methodological quality: RCTs only, high MERSQI scores only, or high NOS-E scores only. In these sensitivity analyses, the overall direction and magnitude of the effect remained similar after restricting to studies of higher methodological quality (Fig. 3).


In this systematic review, we found very low to low quality evidence from a modest number of studies suggesting that EoL communication training for HCP may improve self-efficacy, knowledge and communication scores compared to no formal training. Our confidence in the effect of these interventions on self-efficacy, knowledge, and communication scores is very low to low primarily because of the high overall risk of bias of individual studies included in the review, as well as imprecision in the pooled results due to small sample sizes. Several studies used uncontrolled pre-post designs, which may overestimate effects due to concurrent co-interventions and maturation effects [44].

Self-efficacy was found to be a common outcome measure as it is easy to measure. However the outcome has important limitations. At best, improving self-

efficacy may be beneficial as lack of confidence or negative expectancy may decrease the likelihood that the HCP will voluntarily utilize beneficial communication behaviours [17]. Otherwise, self-assessed performance measures are generally a poor surrogate marker for competence [44]. Earlier studies show physician confidence and actual ability in EoL discussions showed a large disconnect [10, 45]. Similarly, knowledge outcomes do not serve as surrogates to adequate communication skills, as learners may cognitively understand what is important in these discussions, but lack the appropriate skills to carry them out.

We found communication skills outcome the most relevant in capturing the construct of EoL decision-making communication. Our review found evidence of improvement in these measures in the training group, however of very low quality. As well, estimates of effect were quite heterogeneous, which was not easily explained by known study characteristics. There are also potential issues with using a reductionist approach to assessing competence, as expertise may not be adequately captured by binary checklist scores done in these studies [46]. While this way of assessing the outcome measure may be adequate for novice learners, an addition of a subjective global rating may provide a better understanding of their skill in future studies.

We did not find much data on our secondary patientlevel outcomes. We found four studies with conflicting effects on the overall benefit of the intervention. This was not unexpected; although patient-level outcomes are an important measure, educational studies rarely have sufficient power or long-term follow-up to detect these high-level outcomes. There are significant confounding variables in between the effect of an educational intervention to finally the effect on patient behavior. This dilution of effect makes it difficult to design the study for adequate power or follow-up length [47]. We must be careful to judge the value of an educational intervention on patient outcomes alone.

Limitations and strengths

This study is not without limitations. The broad inclusion criteria required to capture a thorough review of the field may have led to some additional heterogeneity and inconsistency in our data, resulting in low quality of evidence according to GRADE, however we suspect that even with more narrow inclusion criteria, the overall quality of the evidence for our outcomes of interest would still be low due to the limited size and quality of studies in this area. We also restricted our search to studies published in the English language. This may limit the applicability of our results to predominantly English-speaking regions.

Std. Mean Difference

RCTs only

Greenberg 1993 Szmuilowicz 2010 Total (95% CI) Heterogeneity Tau2

Std. Mean Difference SE Weight IV. Random. 95% CI

Std. Mean Difference IV. Random. 95% CI

0.6995 0.2139 58.2% 0.70 [0.28, 1.12] 0.2164 0.2896 41.8% 0.22 |-0.35, 0.78] 100.0% 0.50 [0.03, 0.96] 0.05; Chi2 - 1.80, df - 1 (P - 0.18); I2 - 44%

Test for overall effect: Z = 2.09 (P = 0.04) Quality: MERSQI > 12.S

Clayton 2012 1.0752 0.3323 26.9% 1.08 [0.42, 1.73]

Greenberg 1993 0.6995 0.2139 41.6% 0.70 [0.28, 1.12]

Szmuilowicz 2010 0.2164 0.2896 31.5% 0.22 [-0.35, 0.78]

Total (95% CI) 100.0% 0.65 [0.21, 1.08] Heterogeneity Tau2 - 0.07; Chi2 - 3.93, df - 2 (P - 0.14); I2 - 49% Test for overall effect: Z - 2.92 (P = 0.004)

Quality: NOS-E> 2.5

Greenberg 1993 0.6995 0.2139 58.2% 0.70 [0.28, 112]

Szmuilowicz 2010 0.2164 0.2896 41.8% 0.22 [-0.35, 0.78]

Total (95% CI) 100.0% 0.50 [0.03, 0.96]

Heterogeneity Tau2 - 0.05; Chi2 - 1.80, df - 1 (P - 0.18); I2 - 44% Test for overall effect: Z = 2.09 (P = 0.04)

Std. Mean Difference

Std. Mean Difference SE Weight IV. Random. 95% CI

Favours no Intervention Favours intervention

Std. Mean Difference IV. Random. 95% CI

RCTs only

Green 2011 Greenberg 1993 Tolal (95% CI) Heterogeneity Tau2

0.473 0.1844 57.4% 0.47 [0.11, 0.83] 0.6995 0.2139 42.6% 0.70 [0.28, 1.12] 100.0% 0.57 [0.30, 0.84] 0.00; Chi2 - 0.64, df = 1 (P - 0.42); I2 - 0%

Test for overall effect: Z = 4.08 (P < 0.0001) Quality: MERSQI > 12.5

Green 2011 0.473 0.1844 57.4% 0.47 [0.11, 0.83]

Greenberg 1993 0.6995 0.2139 42.6% 0.70 [0.28, 1.12]

Total (95% CI) 100.0% 0.57 [0.30, 0.84]

Heterogeneity Tau2 - 0.00; Chi2 - 0.64, df - 1 (P - 0.42); I2 - 0% Test for overall effect: Z = 4.08 (P < 0.0001)

Quality: NOS-E> 2.5

Green 2011 0.473 0.1844 45.0% 0.47 [0.11, 0.83]

Greenberg 1993 0.6995 0.2139 41.3% 0.70 [0.28, 112]

Perron 2002 1.8026 0.5812 13.7% 1.80 [0.66, 2.94]

Total (95% CI) 100.0% 0.7S [0.27, 1.23]

Heterogeneity Tau2 - 0.10; Chi2 - 4.90, df - 2 (P - 0.09); I2 - 59% Test for overall effect: Z = 3.06 (P = 0.002)

Favours no Intervention Favours intervention

Std. Mean Difference

Std. Mean Difference SE Weight IV, Random. 95% CI

0.9047 0.2965 0.2412 0.2898 1.1378 0.3525

34.8% 35.5% 29.7% 100.0%

RCTs only

Sharma 2014 Szmuilowicz 2010 Szmuilowicz 2012 Subtotal (95% CI)

Heterogeneity Tau2 - 0.12; Chi2 - 4.52, df - 2 (P - 0.10); I2 - 56% Test for overall effect: Z = 2.73 (P = 0.006)

Quality: MERSQI > 12.5

Alexander 2006 Back 2007 Clayton 2012 Lorin 2006 Sharma 2014 Szmuilowicz 2010 Szmuilowicz 2012 Williams 2011 Total (95% CI)

Heterogeneity Tau2 = 0.08; Chi Test for overall effect: Z = 4.89 (P < 0.00001)

Quality: NOS-E > 2.5

Alexander 2006 Cla\ton 2012 Lorin 2006 Sharma 2014 Szmuilowicz 2010 Szmuilowicz 2012 Total (95% CI) Heterogeneity Tau2 = 0.11, Test for overall effect: Z = 3.75 (P = 0.0002)

0.90 [0.32, 1.49] 0.24 [-0.33, 0.81] 1.14 [0.45, 1.83] 0.74 [0.21, 1.27]

0.1674 0.2827 12.0% 0.17 [-0.39, 0.72]

0.4706 0.1434 18.8% 0.47 [0.19, 0.75]

1.3082 0.3431 9.8% 1.31 [0.64, 1.98]

0.506 0.1975 16.0% 0.51 [0.12, 0.89]

0.9047 0.2965 11.5% 0.90 [0.32, 1 49]

0.2412 0.2898 11.7% 0.24 [-0.33, 0.81]

1.1378 0.3525 9.5% 1.14 [0.45, 1.83]

1.2096 0.3161 10.7% 1.21 [0.59, 1 83]

100.0% 0.69 [0.41, 0.96]

16.26, df - 7<P- 0.02); I2 - 57%

0.1674 0.2827 17.1% 0.17 [-0.39, 0.72]

1.3082 0.3431 14.2% 1.31 [0.64, 1.98]

0.506 0.1975 21.8% 0.51 [0.12, 0.89]

0.9047 0.2965 16.4% 0.90 [0.32, 1 49]

0.2412 0.2898 16.7% 0.24 [-0.33, 0.81]

1.1378 0.3525 13.8% 1.14 [0.45, 1.83]

100.0% 0.67 [0.32, 1.02]

11.71, df - 5 (P - 0 04); I2 = 57%

Std. Mean Difference IV, Random, 95% CI

-4-5 2

Favours no intervention Favours intervention

Fig. 3 Sensitivity analysis restricting to studies of higher methodologic quality (RCTs only, higher quality MERSQIand NOS-E) for a self-efficacy b knowledge and c communication score

Other limitations were intrinsic to the available data. We found that the terminology used in the area of EoL communication and decision making is still not well established and is not uniform, which made our literature search difficult. We believe our manual searching of references adequately mitigates this limitation and improves the comprehensiveness of our search, although it is possible that we still missed some relevant studies.

The strength of our review is in the comprehensive literature search with no restrictions with time, inclusion criteria of a broad range of learners, outcomes, and study design; our independent, duplicate screening, eligibility, and quality assessment with rigorous data collection and secondary verification. We also used multiple measures to assess the quality of evidence, using the Cochrane tool, MERSQI or the NOS-E, and conducted sensitivity analyses based on these measures. Finally, we performed comprehensive quality assessments of the totality of evidence using the rigorous GRADE approach.

To our knowledge, this is the only systematic review of educational interventions to train healthcare providers in EoL communication skills that has assessed the quality of evidence using GRADE and conducted meta-analyses to obtain pooled estimates of effect. Other reviews only included a narrative summary of the interventions, with less comprehensive search criteria, and no assessment of study quality [48, 49]. As well, we looked specifically at interventions aimed at improving communication of facilitating and supporting patient decision making on EoL treatment goals, whereas these other reviews looked at a broader skills in palliative care symptom management and breaking bad news.


These results generally support the use of structured communication training to improve HCP's ability to discuss and facilitate EoL decision-making, since they suggest that such training may be effective in improving HCP communication skills. Unfortunately, justification studies such as these that compare against no intervention do not tell us anything aside from the fact that an intervention works [50]. We cannot infer the comparative effectiveness of the different teaching methods, nor can we determine which interventions might be most suitable for a given educational setting or learner population. More studies of higher quality and sound instructional design need to be performed with contextually relevant outcome data and against other active educational comparators.


In this systematic review, we found consistent, but low-quality evidence that structured communication training, compared to usual curricula, may increase HCP self-

efficacy, knowledge, and communication skills for EoL decision-making. While awaiting more robust evidence in this area, educators of health professionals electing to introduce EoL communication skills curricula should continue to design their interventions according to best practice guidelines and base them on a solid theoretical framework.

Ethics approval and consent to participate

Not Applicable.

Consent for publication

Not Applicable.

Availability of data and materials

The datasets supporting the conclusions of this article are included within the article and its additional files. All raw data used in this systematic review are extracted from available published articles. Extracted raw data and Revman analysis files available upon request.

Additional files

Additional file 1: Search Strategy - Describes our search strategy for various electronic databases. (DOCX 25 kb)

Additional file 2: Table S1. Assessment of Study Quality - The rating of individualstudy quality. (PDF 30 kb)

Additional file 3: Table S2. GRADE Summary of Findings for Primary Outcomes - Overallquality of evidence by GRADE criteria. (PDF 47 kb)

Additional file 4: Figure S1. FunnelPlot - Funnelplot used to assess for potentialpublication bias. (PDF 38 kb)


AD: advance directives; CI: confidence interval; EoL: end of life; GRADE: grading of recommendations assessment, development, and evaluation; HCP: health care professionals; ICU: intensive care unit; MERSQI: medicaleducationalresearch study quality instrument; NOSE: Newcastle Ottawa scale education; RCT: randomized controlled trials; SMD: standardized mean difference.

Competing interests

The authors declare that they have no competing interests. Authors' contributions

HC, SO, and JY conceived the study, conducted the analysis, interpreted the data and drafted the manuscript. LH and LM contributed to the analysis and interpretation of the data and edited the manuscript. Allauthors approved the finalsubmitted version of the manuscript.

Authors' information

SO is a Clinical Scholar with the Department of Medicine McMaster University.

JY is Associate Professor in the department of Medicine, and Clinical Epidemiology & Biostatistics in McMaster University. LH is the Director of Advance Care Planning Canada/Canadian Hospice Palliative Care Association.

LM is an assistant professor in the department ClinicalEpidemiology & Biostatistics, McMaster University and Research Methodologist for the Biostatistics Unit, St Joseph's Healthcare Hamilton.


The study and authors were supported by a grant from the Technology Evaluation in the Elderly Network (Grant #KS 2014 - 06). The funding body was not involved in the design, analysis and manuscript generation.

Author details

11280 Main Street West, HSC-2C8, Hamilton, Ontario L8S 4K1, Canada. 2Hamilton General Hospital, McMaster Clinic 4th floor, Room 434, 237 Barton St East, Hamilton, Ontario L8L2X2, Canada. 3Canadian Hospice Palliative Care Association, Annex D, Saint-Vincent Hospital, 60 Cambridge Street North, Ottawa, ON K1R 7A5, Canada. 4St Joseph's Healthcare, 50 Charlton Avenue East, 3rd Floor Martha Wing, Room H321, Hamilton, Ontario L8N4A6, Canada.

Received: 28 October 2015 Accepted: 26 April 2016 Published online: 29 April 2016


1. Heyland DK, Barwich D, Pichora D, Dodek P, Lamontagne F, You JJ, et al. Failure to engage hospitalized elderly patients and their families in advance care planning. JAMA Intern Med. 2013;173:778-87.

2. You JJ, Fowler RA, Heyland DK. Canadian Researchers at the End of Life Network (CARENET). Just ask: discussing goals of care with patients in hospital with serious illness. CMAJ. 2014;186:425-32.

3. Wright AA, Zhang B, Ray A, Mack JW, Trice E, Balboni T, et al. Associations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA. 2008;300:1665-73.

4. Hales BM, Hawryluck L. An interactive educational workshop to improve end of life communication skills. J Contin Educ Health Prof. 2008;28:241-8. quiz249-55.

5. Furman CD, Head B, Lazor B, Casper B, Ritchie CS. Evaluation of an educational intervention to encourage advance directive discussions between medicine residents and patients. J Palliat Med. 2006;9:964-7.

6. Pekmezaris R, Walia R, Nouryan C, Katinas L, Zeitoun N, Alano G, et al. The impact of an end-of-life communication skills intervention on physicians-in-training. Gerontol Geriatr Educ. 2011;32:152-63.

7. Szmuilowicz E, Neely KJ, Sharma RK, Cohen ER, McGaghie WC, Wayne DB. Improving residents' code status discussion skills: a randomized trial. J Palliat Med. 2012;15:768-74.

8. Schroder C, Heyland D, Jiang X, Rocker G, Dodek P. Canadian researchers at the end of life network. Educating medical residents in end-of-life care: insights from a multicenter survey. J Palliat Med. 2009;12:459-70.

9. Gorman TE, Ahern SP, Wiseman J, Skrobik Y. Residents' end-of-life decision making with adult hospitalized patients: a review of the literature. Acad Med. 2005;80:622-33.

10. Tulsky JA, Chesney MA, Lo B. See one, do one, teach one? House staff experience discussing do-not-resuscitate orders. Arch Intern Med. 1996;156:1285-9.

11. Curtis JR, Wenrich MD, Carline JD, Shannon SE, Ambrozy DM, Ramsey PG. Understanding physicians' skills at providing end-of-life care perspectives of patients, families, and health care workers. J Gen Intern Med. 2001;16:41-9.

12. Powazki R, Walsh D, Cothren B, Rybicki L, Thomas S, Morgan G, et al. The care of the actively dying in an academic medical center: a survey of registered nurses' professional capability and comfort. Am J Hosp Palliat Care. 2014;31:619-27.

13. White KR, Coyne PJ. Nurses' perceptions of educational gaps in delivering end-of-life care. Oncol Nurs Forum. 2011;38:711-7.

14. Wenrich MD, Curtis JR, Ambrozy DA, Carline JD, Shannon SE, Ramsey PG. Dying patients' need for emotional support and personalized care from physicians: perspectives of patients with terminal illness, families, and health care providers. J Pain Symptom Manage. 2003;25:236-46.

15. Dunlay SM, Foxen JL, Cole T, Feely MA, Loth AR, Strand JJ, et al. A survey of clinician attitudes and self-reported practices regarding end-of-life care in heart failure. Palliat Med. 2015;29:260-7.

16. Kirkpatrick DL, Kirkpatrick JD. Evaluating training programs the four levels. San Francisco, CA: Berrett-Koehler;2006. Print.

17. Parle M, Maguire P, Heaven C. The development of a training model to improve health professionals' skills, self-efficacy and outcome expectancies when communicating with cancer patients. Soc Sci Med. 1997;44:231-40.

18. Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med. 2005;37:360-3.

19. Reed DA, Beckman TJ, Wright SM, Levine RB, Kern DE, Cook DA. Predictive validity evidence for medical education research study quality instrument

scores: quality of submissions to JGIM's Medical Education Special Issue. J Gen Intern Med. 2008;23:903-7.

20. Cook DA, Reed DA. Appraising the quality of medical education research methods: the medical education research study quality instrument and the Newcastle-Ottawa scale-education. Acad Med. 2015;90:1067-76.

21. Higgins JPT, Altman DG, Sterne JAC. Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S,editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from Accessed 6 Sep 2015.

22. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ Group. 1997;315:629-34.

23. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. Br Med J Publishing Group. 2003;327:557-60.

24. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale: Lawrence Erlbaum Associates, Inc; 1988.

25. Higgins JPT, Deeks JJ. Chapter 7: Selecting studies and collecting data. In: Higgins JPT, Green S,editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011).The Cochrane Collaboration, 2011. Available from Accessed 6 Sep 2015.

26. Schünemann H, Brozek J, Guyatt G, Oxman A. GRADE handbook for grading quality of evidence and strength of recommendations. [Internet]. GradePRO. [cited 2015 Sep 6]. Available from: handbook/. Accessed 6 Sep 2015.

27. GRADEpro GDT: GRADEpro Guideline Development Tool [Software]. McMaster University, 2015 (developed by Evidence Prime, Inc.). Available from

28. Greenberg JM, Doblin BH, Shapiro DW, Linn LS, Wenger NS. Effect of an educational program on medical students' conversations with patients about advance directives: a randomized trial. J Gen Intern Med. 1993;8:683-5.

29. Szmuilowicz E, El-Jawahri A, Chiappetta L, Kamdar M, Block S. Improving residents' end-of-life communication skills with a short retreat: a randomized controlled trial. J Palliat Med. 2010;13:439-52.

30. Bristowe K, Shepherd K, Bryan L, Brown H, Carey I, Matthews B, et al. The development and piloting of the REnal specific Advanced Communication Training (REACT) programme to improve Advance Care Planning for renal patients. Palliat Med. 2014;28:360-6.

31. Clayton JM, Butow PN, Waters A, Laidsaar-Powell RC, O'Brien A, Boyle F, et al. Evaluation of a novel individualised communication-skills training intervention to improve doctors' confidence and skills in end-of-life communication. Palliat Med. 2013;27:236-43.

32. Schell JO, Green JA, Tulsky JA, Arnold RM. Communication skills training for dialysis decision-making and end-of-life care in nephrology. Clin J Am Soc Nephrol. 2013;8:675-80.

33. Smith L, O'Sullivan P, Lo B, Chen H. An educational intervention to improve resident comfort with communication at the end of life. J Palliat Med. 2013;16:54-9.

34. Green MJ, Levi BH. Teaching advance care planning to medical students with a computer-based decision aid. J Cancer Educ. 2011;26:82-91.

35. Fischer GS, Arnold RM. Feasibility of a brief workshop on palliative care communication skills for medical interns. J Palliat Med. 2007;10:19-23.

36. Junod Perron N, Morabia A, De Torrente A. Evaluation of do not resuscitate orders (DNR) in a Swiss community hospital. J Med Ethics. 2002;28:364-7.

37. Sharma RK, Jain N, Peswani N, Szmuilowicz E, Wayne DB, Cameron KA. Unpacking resident-led code status discussions: results from a mixed methods study. J Gen Intern Med. 2014;29:750-7.

38. Alexander SC, Keitz SA, Sloane R, Tulsky JA. A controlled trial of a short course to improve residents' communication with patients at the end of life. Acad Med. 2006;81:1008-12.

39. Back AL, Arnold RM, Baile WF, Fryer-Edwards KA, Alexander SC, Barley GE, et al. Efficacy of communication skills training for giving bad news and discussing transitions to palliative care. Arch Intern Med. 2007;167:453-60.

40. Lorin S, Rho L, Wisnivesky JP, Nierman DM. Improving medical student intensive care unit communication skills: a novel educational initiative using standardized family members. Crit Care Med. 2006;34:2386-91.

41. Williams DM, Fisicaro T, Veloski JJ, Berg D. Development and evaluation of a program to strengthen first year residents' proficiency in leading end-of-life discussions. Am J Hosp Palliat Care. 2011;28:328-34.

42. Holloran SD, Starkey GW, Burke PA, Steele G, Forse RA. An educational intervention in the surgical intensive care unit to improve ethical decisions. Surgery. 1995;118:294-8. discussion298-9.

43. Curtis JR, Back AL, Ford DW, Downey L, Shannon SE, Doorenbos AZ, et al. Effect of communication skills training for residents and nurse practitioners on quality of communication with patients with serious illness: a randomized trial. JAMA. 2013;310:2271-81.

44. Norman G, Eva KW. Quantitative research methods in medical education. In: Swanwick T, editor. Understanding medical education. Oxford, UK: Wiley-Blackwell; 2010. pp. 301 -22.

45. Tulsky JA, Chesney MA, Lo B. How do medical residents discuss resuscitation with patients? J Gen Intern Med. Springer. 1995;10:436-42.

46. Hodges B, Regehr G, McNaughton N, Tiberius R, Hanson M. OSCE checklists do not capture increasing levels of expertise. Acad Med. 1999;74:1129-34.

47. Cook DA, West CP. Perspective: reconsidering the focus on "outcomes research" in medical education: a cautionary note. Acad Med. 2013;88:162-7.

48. Kottewar SA, Bearelly D, Bearelly S, Johnson ED, Fleming DA. Residents' end-of-life training experience: a literature review of interventions. J Palliat Med. 2014;17:725-32.

49. Lloyd-Williams M, Macleod RDM. A systematic review of teaching and learning in palliative care within the medical undergraduate curriculum. Med Teach. 2004;26:683-90.

50. Cook DA. Randomized controlled trials and meta-analysis in medical education: what role do they play? Med Teach. 2012;34:468-73.

Submit your next manuscript to BioMed Central and we will help you at every step:

• We accept pre-submission inquiries

• Our selector tool helps you to find the most relevant journal

• We provide round the clock customer support

• Convenient online submission

• Thorough peer review

• Inclusion in PubMed and all major indexing services

• Maximum visibility for your research

Submit your manuscript at

BioMed Central