CC BY-NC-SA
0Contents lists available at ScienceDirect
Social Science & Medicine
journal homepage: www.elsevier.com/locate/socscimed
The use of citizens' juries in health policy decision-making: A systematic review
CrossMark
Jackie Streeta*, Katherine Duszynskib, Stephanie Krawczyka, Annette Braunack-Mayera
a School of Population Health, The University of Adelaide, Australia
b School of Paediatrics & Reproductive Health, Discipline of Paediatrics, The University of Adelaide, Australia
ARTICLE INFO
Article history: Received 22 May 2013 Received in revised form 30 January 2014 Accepted 6 March 2014 Available online 6 March 2014
Keywords: Citizens' juries Public participation Community engagement Deliberative democracy Health policy Decision making
ABSTRACT
Deliberative inclusive approaches, such as citizen juries, have been used to engage citizens on a range of issues in health care and public health. Researchers engaging with the public to inform policy and practice have adapted the citizen jury method in a variety of ways. The nature and impact of these adaptations has not been evaluated.
We systematically searched Medline (PubMED), CINAHL and Scopus databases to identify deliberative inclusive methods, particularly citizens' juries and their adaptations, deployed in health research. Identified studies were evaluated focussing on principles associated with deliberative democracy: in-clusivity, deliberation and active citizenship. We examined overall process, recruitment, evidence presentation, documentation and outputs in empirical studies, and the relationship of these elements to theoretical explications of deliberative inclusive methods.
The search yielded 37 papers describing 66 citizens' juries. The review demonstrated that the citizens' jury model has been extensively adapted. Inclusivity has been operationalised with sampling strategies that aim to recruit representative juries, although these efforts have produced mixed results. Deliberation has been supported through use of steering committees and facilitators to promote fair interaction between jurors. Many juries were shorter duration than originally recommended, limiting opportunity for constructive dialogue. With respect to citizenship, few juries' rulings were considered by decision-making bodies thereby limiting transfer into policy and practice.
Constraints in public policy process may preclude use of the 'ideal' citizens' jury with potential loss of an effective method for informed community engagement. Adapted citizens' jury models provide an alternative: however, this review demonstrates that special attention should be paid to recruitment, independent oversight, jury duration and moderation.
© 2014 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-SA
license (http://creativecommons.org/licenses/by-nc-sa/3.0/).
1. Background
Deliberative inclusive approaches, as a vehicle for citizen engagement, have particular appeal both because of the fiscal importance of health policy and because health matters touch the lives of citizens very personally. Such approaches aim to bring together diverse citizens, supported by a range of resources, to discuss issues of public concern (Davies et al., 2006, p.4). Some deliberative inclusive approaches methods have been well described in the theoretical literature including citizens' juries (Parkinson, 2004; Pickard, 1998; Smith and Wales, 2000),
* Corresponding author. School of Population Health, The University of Adelaide, Mail Drop 650 550, Level 7,178 North Tce, Adelaide 5005, Australia. E-mail address: Jackie.street@adelaide.edu.au (J. Street).
consensus conferences (Dryzek and Tucker, 2008; Hendriks, 2005), planning cells (Hendriks, 2005) and deliberative polling (Fishkin, 1991). Others, such as World Cafe (Brown, 2001), remain primarily outside academic peer-review and critique. Some deliberative methodologists advocate combining methods in order that "the weaknesses of one would be overcome by the strengths of another" (Carson and Hartz-Karp, 2005, p.121), while others argue that their application, as originally described, is unworkable in real-world settings (Pickard, 1998).
Citizens' juries, in particular, have undergone a process of evolution and adaptation. Developed in the 1970s, the term is a registered trademark of the Jefferson Centre (2004, p10) purportedly to "preserve the integrity of the process". The Centre has described, essential characteristics of a citizens' jury and, within the USA, the term has been tightly regulated. Elsewhere, it has been used much less precisely, as researchers have variously adapted the
http://dx.doi.org/10.1016/j.socscimed.2014.03.005
0277-9536/© 2014 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-SA license (http://creativecommons.org/licenses/by-nc-sa/3.0/).
citizens' jury approach. However, the nature and impact of these changes has not been documented.
Citizens' juries offer a useful tool for engaging citizens in health policy decision-making: they are small enough to permit effective deliberation, relatively inexpensive compared to the larger deliberative exercises of planning cells and consensus conferences, yet sufficiently diverse that the citizens engaged are exposed to a broad range of public experience and perspectives. In this paper, we review use of citizens' juries for community engagement in health research, focussing on methodological aspects. We have taken an interpretation of citizens' juries that accepts that the term is now used more loosely and covers a broader array of activities than originally described. We examine ways in which researchers have adapted the citizens' jury model and explore the fit between these methodological adaptations of the historic citizens' jury and principles associated with deliberative democracy: inclusivity, deliberation and active citizenship. In this comparison, we draw on the conceptualisation of these principles described by Smith and Wales (2000).
2. Methodology
2.1. Sources
Published documents identifying studies under the broad heading of deliberative inclusive methods in health-related areas, including health care and services, were sourced for the years 1995-2010. The choice of the year 1995 reflects the onset of health authorities' and researchers' use of deliberative processes for citizen engagement on health issues (Abelson et al., 2003; Parkinson, 2004). Databases, Medline (PubMED), CINAHL and Scopus, were selected for their coverage of health-oriented research, as well as political and social science materials.
2.2. Search strategy development
Searches were carried out by one researcher (SK) based on criteria developed by all authors (Table 1). Search terms were selected to identify deliberative inclusive methods deployed in health research. Terms centred on public participation, as opposed to expert discussion, with the term, citizen, and its synonyms included. Terms related to deliberative processes were included, specifically names given to variations of deliberative methods (Abelson et al., 2003). The final search strategy was revised with the assistance of a research librarian. The full search is provided online (Appendix 1).
2.3. Article screening and criteria
Relevant search results (Fig. 1) were combined in a citations database. Abstracts were scanned by one author (SK) using Table 1
Table 1
Criteria used to select material for analysis.
Inclusion criteria:
• English language
• Article describes use of a deliberative forum (e.g. citizen jury, citizen panel, planning cell, consensus conference)which aimed for deliberation, inclusivity and influence in policy or practice
• Forum was applied to topics, activities or projects that impacted on public health, health care and health services
• Participants were lay citizens OR lay citizens were included in the deliberative forum
Exclusion criteria:
• Insufficient detail provided to gauge nature of the forum
• Forums not explicitly addressing a health issue
as a selection guide. Full-text review of selected articles was performed by two research assistants. A deliberative inclusive method was identified as being a citizens' jury if it contained all or most of the elements of the citizens' jury model (Coote and Lenaghan, 1997; Smith and Wales, 2000; The Jefferson Center, 2004). Specifically, a jury was characterised by: 12-25 participants selected to reflect the community and acting as independent citizens rather than experts or representatives; a charge or research question(s) provided by organisers; deliberation informed by evidence provided by expert witnesses and a verdict delivered by jurors. Studies not meeting this description were excluded. In this paper, we use the term 'jury' to describe any deliberative forum conducted in the style of a citizens' jury.
Articles lacking methods description were excluded since informed comment could not be made on the nature of the jury. This followed cursory inspection of the article, its references and a Google search for relevant grey literature.
2.4. Analysis
Data were managed with a Microsoft Access database using a framework comprising eight domains (Table 2).
The jury research questions were classified into categories (Table 3). Where questions addressed more than one category, the primary category (as assessed by the authors), was selected.
Analysis was informed by the description of citizen juries by Smith and Wales (2000). This framework was chosen for its focus on how key tenets of deliberative democracy, namely inclusivity, deliberation and active citizenship, play out in the implementation of the citizens' jury model. Smith and Wales describe how citizens' juries approximate the inclusivity ideal "by aiming for a broadly representative jury selection" (Smith and Wales, 2000, p.56) where inclusivity describes participation of "all citizens' in public dialogue", with all viewpoints given "equal right to be heard"(Smith and Wales, 2000, p.53). Furthermore, citizens' juries have implemented deliberation by establishing "rules of conduct" between jurors (Smith and Wales, 2000, p.58), with this tenet described as discussion that "encourages mutual recognition and respect and is orientated towards the public negotiation of the common good" (Smith and Wales, 2000, p.53). Citizens' juries have advanced an active value of citizenship by encouraging citizen participation in decision-making processes. Realisation of active citizenship may be bolstered by good facilitation and pre-jury contracts binding commissioning organisations to respond to jury recommendations (Smith and Wales, 2000, p.60). Attention to fulfilling the tenets of inclusivity and "egalitarian, uncoerced, competent" deliberation "free from delusion, deception, power and strategy" permits the practice of active citizenship where individuals can engage with the diverse "knowledge, experience and capabilities" of others (Smith and Wales, 2000, p.53-54). Such engagement "has the potential to transform the values and preferences of citizens in response to encounters with others" (Smith and Wales, 2000, p.54).
We thus examined overall process, recruitment, evidence presentation, documentation and outputs (e.g. reports), to explore the relationship between deliberative democratic tenets, as described by Smith and Wales, and the practice of citizens' juries in health research.
In particular, we wished to know where and how the citizens' jury model has been adapted to meet the needs or restrictions of a research or policy context and how such methodological changes have impacted on the outcome. The word 'outcome' describes the findings or verdict of the jury, reached by consensus or vote, including any record of dissent and underlying reasons for the
Duplicates, book reviews, magazine articles and editorials removed
12,105 articles excluded: Reasons for exclusion: ♦Involved experts rather than public/patients *Not directly health related *No deliberative inclusive method used
995 articles excluded on basis of criteria
2 articles not found
3 additional articles sourced through pearling including 1 not indexed by databases (Chen & Deng, 2007) Additional supplementary material for 6 articles sourced through Google search
Excluded 38 papers describing other deliberative methods: 20 consensus conferences; 1 deliberative poll; 1 deliberative mapping ; 2 electronic juries; 2 citizens' councils; 1 deliberative planning cells; 1 citizens' parliament; 1 citizens' workshop; 6 deliberative studies which could not be categorised
Fig. 1. Study search and selection process flow chart.
verdict reached. We used the following questions to inform our
analysis.
1. What topic areas have been considered?
2. Where have juries been conducted?
3. What recruitment strategies have been used and how did these strategies impact on the representativeness of the demographic composition of the jury?
4. What was the duration of the juries and how were timelines structured? How did jury duration and timing impact on deliberation and the ability to provide a usable outcome in the policy or research context?
5. How were the juries conducted particularly with respect to steering groups, moderation and structured deliberation?
6. How were expert witnesses selected and expert testimony presented?
7. How have jury recommendations been formulated, reported and disseminated?
3. Findings
Our findings indicate that, although the citizens' jury method originally described a tightly prescribed method, this precision has been lost over time and currently the term describes a broad array of methodological approaches. In all, 37 papers were identified describing 66 citizens' juries within 28 studies. The term, 'citizens' 'jury', or a variant thereof was explicitly used to describe the method in 22 studies. Of the remaining six, the methods employed were similar, although four (Abelson et al., 2007a; Huston, 2004; Maxwell et al., 2003; Willison et al., 2008) also drew on the work ofYankelovich (1991). Of 66 juries, 30 involved examining the same question at multiple sites, 17 related to a single jury and question, 11
involved multiple questions conducted at multiple sites in a single study and 8 were jury pairs conducted at the same site examining the same question.
Two thirds of studies (n = 18) were overseen by a steering committee or advisory group although the composition and influence of these groups varied widely. Usually the group consisted of key stakeholders, particularly policy-makers, but could also include discipline experts, advocacy group representatives, clinical practitioners, deliberative methodologists, patients and caregivers. The role of the groups was variously described as: to prevent bias in expert presentation (Pidgeon and Rogers-Hayden, 2007), to guide question development and evidence presentation (Dunkerley and Glasner, 1998; Parkin and Paul, 2011; Pickard, 1998; Rogers et al., 2009; Toni & von Braun, 2001), general oversight (Dunkerley and Glasner, 1998; Maxwell et al., 2003), to engage stakeholder representatives (Haigh and Scott-Samuel, 2008; Huston, 2004; Lena-ghan, 1999; Lenaghan et al., 1996) and to disseminate or implement findings (Gooberman-Hill et al., 2008; Kashefi and Mort, 2004).
Five juries reported jury costings (Einsiedel, 2002; Elwood and Longley, 2010; Iredale et al., 2006; Lenaghan, 1999; Maxwell et al., 2003).
3.1. Topic areas
A large proportion of the 66 juries (9 studies comprising 25 juries) addressed population based ethical issues but a range of other issues were discussed (Table 3).
3.2. Location
More than half (38/66) of the juries were conducted in Canada, 16 in the UK, seven in Australia or New Zealand, three in the USA
Table 2
Framework for information retrieval.
Domain
Topics
Overall process Recruitment
(experts and participants) Output
Documentation Funding
Nature of process Surveys
Additional issues
Description of jury issue; research objective; research questions; number of juries; location; venue; jury length; steering group. Recruitment strategies and uptake; selection criteria; assessment of conflict of interest; number of participants; number of drop-outs; bias; participant honorarium or payment.
Nature of outcomes e.g. consensus, voting, dissent; Nature of reporting; authors. Purpose of reporting; target audience; audience response; publicity.
Funding source including organisation(s) names and organisation type(s); Lead agency including organisation(s) names and organisation type(s).
Facilitation/moderation; nature of evidence provided; method(s) of synthesis; nature of evidence presentation; nature of deliberation; observers; division of tasks over jury length.
Nature and timing of surveys (if any) conducted with participants, stakeholders or researchers. Ethical approval; participant feedback; cost; evaluation.
and one each in Brazil and Italy. Nearly half were conducted within four Canadian studies characterised by multiple juries addressing a single question (Einsiedel, 2002; Huston, 2004; Maxwell et al., 2003; Willison et al., 2008).
3.3. Recruitment strategies
Around 20 different recruitment strategies were used with the most common (11 studies/29 juries) being stratified random sampling through the electoral roll (Bennett and Smith, 2007; Haigh and Scott-Samuel, 2008; Oliver-Weymouth, 2000; Parkin and Paul, 2011; Paul et al., 2008), random digit dialling (Secko et al., 2009) or commercial database of registered telephone numbers (Menon and Stafinski, 2008) or national polling institute (Carson, 2006). Three studies (9 juries) used non-stratified random sampling through electoral roll, random digit dialling or survey response (Kim et al., 2009; Paul et al., 2008; Willison et al., 2008). Recruitment by a market research company was used in six studies (11 juries) (Dunkerley and Glasner, 1998; Elwood and Longley, 2010; Huston, 2004; Kashefi and Mort, 2004; Lenaghan, 1999; Rogers et al., 2009). One jury (Kashefi and Mort, 2004), employed a professional recruiter who directly recruited individuals at public sites. Newspaper advertisements were used to recruit women with incontinence (Herbison et al., 2009) while word-of-mouth or advertising through networks was used to recruit youth, aged,
caregiver or marginalised population groups (Iredale et al., 2006; Kim et al., 2010; Pidgeon and Rogers-Hayden, 2007; Timotijevic and Raats, 2007). Other juries recruited through community organisations, government departments or existing citizens' council (Abelson et al., 2007a; Gooberman-Hill et al., 2008; Mort and Finch, 2005; Toni & von Braun, 2001).
Most studies explicitly stated that they aimed to recruit a jury, descriptively representative of the community (Button and Mattson, 1999; Carson, 2006; Dunkerley and Glasner, 1998; Einsiedel and Ross, 2002; Elwood and Longley, 2010; Haigh and Scott-Samuel, 2008; Huston, 2004; Lenaghan, 1999; Lenaghan et al., 1996; Menon and Stafinski, 2008; Rogers et al., 2009; Secko et al., 2009), providing a cross-section of community perspectives (Einsiedel and Ross, 2002; Maxwell et al., 2003) or incorporating diverse voices (Bennett and Smith, 2007; Gooberman-Hill et al., 2008). Where criteria were stated (15 studies), both age and sex were used as stratifying variables in all studies, bar one (Haigh and Scott-Samuel, 2008), which used geographic area only. Race/ ethnicity and education were each used in five studies; and at least one of employment status, housing tenure, religion and occupation was used in three studies. Geographic location, socioeconomic status, income, social class, car access, health parameters, children and language were used in only one or two studies.
No juries specifically recruited patients although Herbison et al. (2009) recruited women with incontinence: half had
Table 3
Topic areas for included juries.
Topic area Studies(n =) Juries (n =) 1 Specific topics References
Ethical issues 9 25 Consent (Kim et al., 2010, 2009; Parkin and Paul, 2011; Secko et al., 2009; Willison et al., 2008)
in population health Genetic testing (Bennett and Smith, 2007; Dunkerley and Glasner, 1998; Glasner and Dunkerley, 1999;
Iredale et al., 2006)
Xenotransplantation (Einsiedel, 2002; Einsiedel and Ross, 2002)
Placebo use (Huston, 2004)
Bio-banks (Burgess et al., 2008; Longstaff and Burgess, 2010; O'Doherty and Burgess, 2009;
Secko et al., 2009)
Priority-setting 6 15 Resource allocation (Abelson et al., 2007a; Lenaghan, 1999; Lenaghan et al., 1996; Menon and Stafinski,
2008; Mooney and Blackwell, 2004)
Setting research priorities (Gooberman-Hill et al., 2008; Herbison et al., 2009)
Health policy 7 19 Mammography screening (Paul et al., 2008)
Pandemic communication (Rogers et al., 2009)
Food retailing (Timotijevic and Raats, 2007)
Use of medicines (Elwood and Longley, 2010)
Telemedicine (Mort and Finch, 2005)
Health system reform (Button and Mattson, 1999; Maxwell et al., 2003)
Environmental health 3 3 Nanotechnology (Pidgeon and Rogers-Hayden, 2007; Rogers-Hayden and Pidgeon, 2006)
GM foods (Toni & von Braun, 2001)
Road traffic volume (Carson, 2006)
Community wellbeing 3 4 Anti-social behaviour (Haigh and Scott-Samuel, 2008)
Community health (Kashefi and Mort, 2004)
and wellbeing
Community mental (Pickard, 1998)
health services
sought medical help and therefore could be considered patients. Four studies (five juries) recruited particular age groups (Iredale et al., 2006; Kim et al., 2009; Paul et al., 2008; Timotijevic and Raats, 2007), two studies (six juries) representatives from community organisations (Abelson et al., 2007a; Mort and Finch, 2005) and one each recruited carers (Kim et al., 2010), poor/unemployed citizens (Toni & von Braun, 2001) and 'marginalised' persons (Pidgeon and Rogers-Hayden, 2007). In one jury, the recruitment profile, developed by a steering group to provide diverse community voices, resulted in a jury composed of primarily unemployed and disadvantaged participants (Kashefi and Mort, 2004).
In six studies, conflicts of interest were gauged during recruitment and particular groups actively excluded: on the grounds of previous or current employment (Maxwell et al., 2003; Menon and Stafinski, 2008; Mooney and Blackwell, 2004; Timotijevic and Raats, 2007; Toni & von Braun, 2001), previous disease experience (Elwood and Longley, 2010; Paul et al., 2008), prior topic knowledge, or membership of a patient advocacy group (Menon and Stafinski, 2008). In one jury, to minimise recruitment of participants with strongly held prior beliefs, participants were not told of the topic until the day of the jury (Gooberman-Hill et al., 2008).
Honorariums were offered in 12 studies (28 juries). Thirteen studies (31 juries) did not state whether an honorarium was offered and three studies (7 juries) stated that honorariums were not provided.
Although 19 studies (50 juries) aimed for jury recruitment representative of the community only 15 studies (33 juries) described the jury's demographic composition in sufficient detail to enable evaluation of representativeness. We considered recruitment to be biased if the jury composition was skewed, particularly with respect to age, gender, education or income. Only six studies (nine juries) (Bennett and Smith, 2007; Elwood and Longley, 2010; Lenaghan, 1999; Lenaghan et al., 1996; Menon and Stafinski, 2008; Rogers et al., 2009; Secko et al., 2009), were found to be unbiased. Absence of bias in jury composition was judged either by report from the study's authors (6 juries) or through examination of described participant characteristics (3 juries).
Studies without evidence of bias all used:
- stratified sampling with broad range of criteria;
- recruitment through market research company or by telephone,
letter or doorknocking; and
- a substantial honorarium
The seven studies (23 juries) (Dunkerley and Glasner, 1998; Einsiedel and Ross, 2002; Haigh and Scott-Samuel, 2008; Huston, 2004; Parkin and Paul, 2011; Timotijevic and Raats, 2007; Willison et al., 2008) reporting biased participant recruitment lacked one or more of these recruitment strategies. Bias, where identified, generally favoured participation of women over men and a more highly-educated, higher-income group than the general population (Einsiedel, 2002; Haigh and Scott-Samuel, 2008; Huston, 2004; Willison et al., 2008). Stratified random sampling with direct-contact methods did not guarantee unbiased recruitment. Haigh and Scott-Samuel (2008) recruiting via random sampling from the electoral register, stratified by geographic area with no honorarium, convened a jury with no young people and 70% aged over 45 years. One jury which aimed to enlist politically and socially active citizens through local organisations recruited a group predominantly female, college educated and middle-aged (Abelson et al., 2007a). Of note, advertising, including advertising through networks, was only used to recruit specific population samples such as younger (Iredale et al., 2006), older (Timotijevic and Raats, 2007), marginalised and poor (Pidgeon and Rogers-Hayden, 2007;
Toni & von Braun, 2001), caregivers (Kim et al., 2010) and women with symptoms of incontinence (Herbison et al., 2009).
Only three studies reported juries with more than 25 participants, each with around 40 participants (Carson, 2006; Kim et al., 2010; Maxwell et al., 2003). Most studies recruited to achieve the recommended composition of 12-25 participants and most succeeded. Six studies reported juries with 9-11 participants (Huston, 2004; Mort and Finch, 2005; Parkin and Paul, 2011; Paul et al., 2008; Timotijevic and Raats, 2007; Toni & von Braun, 2001), primarily due to late drop-out of participants. Four studies did not report individual jury numbers (Haigh and Scott-Samuel, 2008; Huston, 2004; Mooney and Blackwell, 2004; Willison et al., 2008).
Most studies reported neither the number of individuals contacted for participation, nor the size of the population from which the sample was drawn. Therefore, in most cases, it was not possible to assess participation or attrition rates and relate these to recruitment strategy. Nine studies did report withdrawal numbers, which ranged from 1 to 10 individuals. Attrition is an important factor in bias since those dropping out tend to reflect those harder to recruit initially (e.g. Timotijevic and Raats, 2007).
3.4. Duration and timing
Nearly two-thirds of the juries took place over the equivalent of 1 -2 days, usually on a weekend, considerably fewer than the 4-5 days recommended by the Jefferson Centre (2004) or UK Institute for Public Policy Research (Coote and Lenaghan, 1997). The longest jury (Kashefi and Mort, 2004) was conducted over five consecutive weekdays with all but two participants unemployed or retired. In most, the days were consecutive but one jury was held over two weekends (Secko et al., 2009), in another, through brief sessions over five weeks (Pidgeon and Rogers-Hayden, 2007) and one jury met 11 times over 16 weeks (Gooberman-Hill et al., 2008).
In shorter juries, some participants complained of insufficient time to explore the issues but brief daylong juries still delivered outcomes (Abelson et al., 2007a; Carson, 2006; Huston, 2004; Kim et al., 2010; Maxwell et al., 2003; Mooney and Blackwell, 2004; Mort and Finch, 2005; Willison et al., 2008). Jury length did not appear to impact on recruitment bias; that is, longer juries were balanced in terms of the selection criteria, providing measures were taken to support recruitment of hard-to-engage groups. Longer juries did permit participants greater control over the ensuing report (Dunkerley and Glasner, 1998; Iredale et al., 2006; Lenaghan, 1999; Parkin and Paul, 2011) and provided opportunity to engage with different forms of evidence. For example, one five-day jury (Kashefi and Mort, 2004) included video, witnesses and a creative writing exercise. Juries held in staggered fashion encountered problems associated with the disjointed nature of multiple sessions held over a long period. For example, in one case, experts presenting at different times created confusion when their evidence conflicted (Pidgeon and Rogers-Hayden, 2007).
3.5. Moderation
Only three studies indicated that the jury was not facilitated (Herbison et al., 2009; Pickard, 1998; Willison et al., 2008): in two, jury members lead discussion (Herbison et al., 2009; Pickard, 1998) whereas the other was guided by a workbook (Willison et al., 2008). Three studies provided insufficient detail to determine if facilitation occurred (Haigh and Scott-Samuel, 2008; Mooney and Blackwell, 2004; Mort and Finch, 2005). Even where facilitation was described, the nature of the facilitation - particularly the facilitators' independence, training and experience - was not always clear. Nine studies described using trained, skilled or experienced facilitators (Dunkerley and Glasner, 1998; Huston, 2004; Iredale et al.,
2006; Kim et al., 2010; Lenaghan et al., 1996; Maxwell et al., 2003; Menon and Stafinski, 2008; Rogers et al., 2009; Secko et al., 2009). The facilitator role was variously defined as: drafting a proposal for common ground (Huston, 2004); being neutral in content but active in process (Kim et al., 2010, 2009); ensuring discussion stayed on-topic (Dunkerley and Glasner, 1998; Lenaghan, 1999; Lenaghan et al., 1996) and assisting question formulation and reaching for consensus (Toni & von Braun, 2001). In most cases, however, the facilitator role was undefined. Two juries included a resource person whose role was to investigate questions raised by the jury (Kashefi and Mort, 2004; Willison et al., 2008).
Structured elements, used to stimulate and guide discussion, were rarely discussed: 17 studies described small group work and 10 used scenarios or hypotheticals. Other techniques included: scoring methods (Menon and Stafinski, 2008), priority setting (Herbison et al., 2009; Menon and Stafinski, 2008), workbooks (Maxwell et al., 2003; Secko et al., 2009; Willison et al., 2008), dialogue guide (Huston), voting (Bennett and Smith, 2007; Pickard, 1998), physical model (Burgess et al., 2008) and a courtroom format (Bennett and Smith, 2007).
3.6. Selection of expert witnesses and nature of expert testimony
Most studies included a range of expert testimony, described either the nature of the expert evidence or the presenters' expertise and indicated that participants could engage with presenters and challenge the evidence (See online Appendix 2). In most cases, neither the mode nor the reasons for expert selection were discussed although four studies described involving the steering group in these decisions (Dunkerley and Glasner, 1998; Gooberman-Hill et al., 2008; Iredale et al., 2006; Parkin and Paul, 2011). Three Canadian studies (Huston, 2004; Maxwell et al., 2003; Willison et al., 2008) and possibly Abelson et al. (2007a) did not use expert witnesses but instead utilised a workbook to provide balanced relevant information. Two studies undertook local research to present to the jury (Haigh and Scott-Samuel, 2008; Kashefi and Mort, 2004). Written material was provided to jurors in 14 studies with seven juries provided with material in advance. One study required participants to prepare questions (Kim et al., 2010, 2009). Seven studies included ethical analysis (Bennett and Smith, 2007; Einsiedel, 2002; Huston, 2004; Kim et al., 2010, 2009; Parkin and Paul, 2011; Rogers et al., 2009; Secko et al., 2009). Very few studies specifically described aiming for balanced evidence presentation (Iredale et al., 2006; Mooney and Blackwell, 2004; Paul et al., 2008; Pidgeon and Rogers-Hayden, 2007; Rogers et al., 2009), although this may be surmised from the discussion in other studies.
3.7. Output formulation, reporting and dissemination of jury recommendations and findings
Various strategies were used to collect data. Proceedings were audio-recorded in half the studies. Other methods included: contemporaneous notes by organisers (Abelson et al., 2007a; Rogers et al., 2009; Willison et al., 2008) or participants (Einsiedel, 2002) workbooks (Huston, 2004; Maxwell et al., 2003), video-recording (Button and Mattson, 1999; Iredale et al., 2006), whiteboard scribing (Herbison et al., 2009; Rogers et al., 2009), flip charts (Dunkerley and Glasner, 1998; Herbison et al., 2009; Huston, 2004; Secko et al., 2009), voting (Pickard, 1998), participant diaries (Iredale et al., 2006), participant hand-held video-recording (Iredale et al., 2006), questionnaires (Iredale et al., 2006) and interviews (Iredale et al., 2006). Data were analysed qualitatively in six studies using 'content analysis' (Menon and Stafinski, 2008; Timotijevic and Raats, 2007), 'close and repeated reading' (Bennett and Smith, 2007), discourse analysis (O'Doherty and
Burgess, 2009), 'coding' (Willison et al., 2008) or without method description (Haigh and Scott-Samuel, 2008).
Recommendations were developed through consensus (Huston, 2004; Menon and Stafinski, 2008; Parkin and Paul, 2011; Rogers et al., 2009), consensus with minority opinion (Bennett and Smith, 2007; Kim et al., 2010, 2009; O'Doherty and Burgess, 2009; Pidgeon and Rogers-Hayden, 2007) or voting (Paul et al., 2008). In one study, no decision choice dominated (Willison et al., 2008). Some studies failed to describe how recommendations were reached and whether decisions were unanimous. Jury reports were variously written by researchers based on participants' recommendations (Rogers et al., 2009), by jurors alone (Parkin and Paul, 2011) or with facilitator assistance (Dunkerley and Glasner, 1998; Elwood and Longley, 2010; Herbison et al., 2009) or by researchers in consultation with participants (Elwood and Longley, 2010; Kashefi and Mort, 2004; Lenaghan, 1999).
In addition to peer reviewed articles, 13 described a grey-literature report. Dissemination through other outlets (13 studies) included: media coverage (Abelson et al., 2007a; Dunkerley and Glasner, 1998; Haigh and Scott-Samuel, 2008; Iredale et al., 2006), direct presentation to decision makers (Abelson et al., 2007a; Dunkerley and Glasner, 1998; Iredale et al., 2006), direct community engagement (Kashefi and Mort, 2004) and through planning meetings (Abelson et al., 2007a; Haigh and Scott-Samuel, 2008; Kashefi and Mort, 2004). Only three described commitment by a decision-making body to consider recommendations (Lenaghan, 1999; O'Doherty and Burgess, 2009; Pickard, 1998). One jury fell within formal health technology assessment (Menon and Stafinski, 2008).
4. Discussion
Citizen juries permit citizens to engage with evidence, deliberate and deliver recommendations on a range of complex and demanding topics. This review demonstrates that, provided the research question is tightly defined and concrete in nature, even brief one-day citizens' juries can deliver useable outcomes to inform policy and practice.
Many juries described in this review were conducted to meet instrumental aims, that is, to deliver recommendations usable in policy and practice decision-making. It was often not possible, in our review, to establish the extent to which juries succeeded in this respect, primarily because the final link between the jury and decision-makers is not well established. However, in the tradition from which citizens' juries arise, instrumental goals are less important and goals related to empowering citizens have greater prominence.
Smith and Wales' (2000) work on citizens' juries supports further inquiry into the relationship between health research and philosophical aims. Given there is no consensus on what constitutes a deliberative process 'in theory or in practice' (Blacksher et al., 2012, pp14), Smith and Wales provide a framework to consider the extent to which health researchers engage with deliberative theory in their work on citizens' juries. By examining how citizens' juries implement inclusivity, deliberation and citizenship, the following discussion reveals a tension between research aims and deliberative ideals. We further suggest that improvements in reporting the practice of citizens' juries could produce insights relevant to the 'macro-political uptake' of deliberative processes (Goodin and Dryzek, 2006), and strengthen dialogue between deliberative practitioners and theoreticians.
4.1. Inclusivity
Inclusivity refers to efforts to ensure that citizens' juries represent a wide variety of experience and backgrounds (Smith and
Wales, 2000). Most of the juries aimed to recruit a jury descriptively representative of the community from which the jurors were drawn, suggesting that, at least in intent, inclusivity was valued. However, a significant minority of studies did not sufficiently report on the jury's composition to allow a judgement about inclusivity to be made, implying either that inclusivity was taken for granted or, more likely, concern for inclusivity was not built into the study design.
The extent to which recruitment strategies succeeded in creating an inclusive environment varied considerably. In general, studies systematically seeking to include people from a wide variety of backgrounds - by using stratification, recruitment using market researchers, and honoraria, tended to be more successful in recruiting diverse voices than those relying on random sampling. Ironically, this finding suggests that the rationale for random sampling - the idea that each person from a population has an equal chance of being selected, with the randomly-selected group understood to represent the population in microcosm (Carson and Hartz-Karp, 2005) - appears not to work for the small sample size of a jury. Even in studies which appear to have recruited a jury intuitively representative of a broader population, the lack of information about the populations from which jurors were drawn, makes it difficult to draw conclusions about the extent to which these juries were truly inclusive. In addition, having a jury of fewer than 12 persons, may impact on the ability of the jury to support the diverse range of community perspectives and experience required for inclusivity.
The comments above presume that inclusivity turns only on jury composition. However, scholars have suggested that meeting the criterion of inclusivity also requires attention to other aspects. For example, inclusivity may be implemented through presentation of diverse witnesses (Smith and Wales, 2000), both for content and diverse viewpoints, through participants' presentation of narrative that captures relevant personal experiences (Burkhalter et al., 2002; Young, 1996) and by allowing juries opportunity to challenge presented evidence and request additional information. Our findings indicate that at least some studies attempted to meet these conditions but it is also apparent that relatively little attention was paid to this criterion, such that, in many cases, it was difficult to judge if the criterion of inclusivity had been met.
4.2. Deliberation
Deliberation refers to "communication that induces reflection on preferences, values and interests in a non-coercive fashion"(-Mansbridge et al., 2010, p.2) but which in the deliberative democracy context reaches for consensus. The concept is drawn from deliberative democratic theory, which explores the possibility of democracy through the mode of the "public forum. oriented towards the common good" (Hendriks, 2002, p8) with forum proceedings equating to "reasoned public discussion amongst equals" who are in pursuit of "reaching understanding" such that "legitimate decisions" can be made (Hendriks, 2002, p8).
The citizens' juries reported in this paper created a positive environment for deliberation in a range of ways. First, two-thirds of the studies were overseen by a steering or advisory committee. Smith and Wales (2000) suggest that "the very integrity" of the process is dependent on decisions made early in the development process and that such decisions must be seen to be unbiased. However, even when steering group composition was described in detail, bias was difficult to judge since the group could replicate imbalances in society.
Second, 21 of the 28 studies indicated that their juries were facilitated by a person other than a jury member and only three explicitly indicated the juries were not facilitated. However, not all
studies described the facilitators as 'trained' or 'independent', both factors which may be important for providing an unbiased and supportive process. In addition, the role of the moderator was generally not well defined.
Thirdly, all but three of the juries described a jury size of fewer than 25. Keeping group size small appears to provide greatest opportunity for participants to receive equal and adequate speaking time (Smith and Wales, 2000). The way in which decisions were reached and communicated also helped to support deliberative dialogue. The dominant method through which recommendations were developed was consensus or consensus with minority opinion, suggesting that considerable care had been given to ensuring that jurors were able to participate effectively in the jury.
Smith and Wales (2000) note that the short length of juries can limit the opportunity for jurors to develop and maintain open and constructive dialogue. Almost two-thirds of the juries reported in this paper were conducted over 1 -2 days, arguably, too short a time for jurors to develop the collaborative environment needed for deliberative dialogue.
4.3. Active citizenship
Citizens' juries have been promoted as vehicles to "reassert the importance of a more active form of citizenship" (Smith and Wales, 2000, p60). It is clear that the term, citizen, as used by Smith and Wales, and interpreted as such in most of the studies, more closely approximates the Athenian ideal rather than its popular use synonymous with nationality. In only six of the juries was juror selection dependent on formal requirement for 'citizenship' of the country in which the study took place. Recognition of the active citizenship role, within a jury, might require that, at a minimum, there be formal recognition of jurors' perspectives or recommendations by appropriate authorities.
Defined in this way, only a small number of the identified studies met the citizenship criterion. Only three studies described commitment by a decision-making body to consider recommendations (Lenaghan, 1999; O'Doherty and Burgess, 2009; Pickard, 1998). Given the relationship researchers have with governments and other decision-makers, this is not surprising. Most researchers recognise that it is beyond their power to enact participants' recommendations (Carson and Hartz-Karp, 2005). However, they also recognise the importance of addressing juror-policy maker interactions: citizens have reported the need for greater 'accountability' by decision-makers who support deliberative forums so that citizens can feel that "their input is wanted and is going to be needed" (Abelson et al., 2004, p209-10). This review shows that researchers use a range of techniques to promote the findings of the citizens' juries including academic literature, media channels and direct engagement with decision-makers.
There are other ways in which juries can enhance the citizen role, particularly in capacity building for empowered citizens (Abelson et al., 2007b). From our own experience (Braunack-Mayer et al., 2010; Rogers et al., 2009; Watt et al., 2012), jurors are often impressive, both in their commitment and their capacity to grasp complex topics. Our experience was mirrored in some included juries: for example, Gooberman-Hill et al. (2008, p.280) comment that "deep engagement" of the jurors "manifested itself in commitment to the process and continued interest in research".
5. Conclusion
This review demonstrates that the citizens' jury model has been extensively adapted from its earliest forms. Inclusivity has been operationalised with sampling strategies that aim to recruit representative juries, although these efforts have produced mixed
results. Various strategies have been implemented to support deliberation, including enlistment of steering committees and facilitators to promote fair interactions between jurors. However, citizens' juries generally appear to be conducted over a shorter length of time than originally recommended, thereby limiting opportunity for reflection on the preferences, values and interests of others. Finally, fewer studies have honoured the citizenship criterion in jury designs, with only a handful of juries' rulings considered by decision-making bodies.
In light of these findings, the 'ideal' process promulgated by Smith and Wales (2000), Crosby (The Jefferson Center, 2004) and others might be considered to be undermined by deliberative experiments in health research. Such a view pays inadequate attention to the administrative, financial and political constraints that shape the design and implementation of many community engagement strategies in real world settings. Indeed, restricting the use of citizens' juries to a narrowly defined set of parameters may preclude their use in policy processes or to inform practice reform. Purity about the nature of the ideal deliberative process, such that the methods are untenable within the constraints of the decision-making process, may impose a considerable loss to community engagement. Strict adherence to and, in particular, legal regulation of a methodology, through patent or trademark, could be counterproductive for knowledge production since it is through testing and adapting methodologies that new ideas are developed and our understanding expands.
Regardless of the gap between ideal and more pragmatic approaches to the use of citizens' juries, it is still important to understand how various methodological decisions can shape jury processes and outcomes. It is clear that some factors are essential to provide an unbiased inclusive deliberative process. This review indicates that, in adapting the citizen jury to instrumental aims, particular care and attention should be paid to recruitment methods, independent oversight by a steering committee, jury duration, moderation and respect for the participant volunteer. The use of adapted deliberative inclusive processes for instrumental means, and, in particular, to inform health policy, is a developing field. Careful attention to reporting and improved evaluation of process and outcomes can only assist in ensuring that these methods are best designed to meet both democratic and instrumental goals.
Acknowledgements
We gratefully acknowledge the work of Monira Osman and Yee Ling Kok in undertaking the full-text review of selected articles and Vicki Xafis for discussions assisting the review. Jackie Street is supported by an NHMRC Capacity Building Grant 565501 and an Australian National Preventive Health Agency Fellowship.
Appendix A. Supplementary data
Supplementary data related to this article can be found at http:// dx.doi.org/10.1016/j.socscimed.2014.03.005.
References
Abelson, J., Forest, P.-G., Eyles, J., Casebeer, A., Mackean, G., Effective Public Consultation Project Team, 2004. Will it make a difference if I show up and share? A citizens' perspective on improving public involvement processes for health system decision-making. Journal of Health Service Research and Policy 9, 205-212.
Abelson, J., Forest, P.G., Eyles, J., Smith, P., Martin, E., Gauvin, F.P., 2003. Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. Social Science & Medicine 57, 239-251. Abelson, J., Forest, P., Eyles, J., Casebeer, A., Martin, E., Mackean, G., 2007a. Examining the role of context in the implementation of a deliberative public
participation experiment: results from a Canadian comparative study. Social Science & Medicine 64, 2115-2128.
Abelson, J., Giacomini, M., Lehoux, P., Gauvin, F., 2007b. Bringing 'the public' into health technology assessment and coverage policy decisions: from principles to practice. Health Policy 82, 37-50.
Bennett, P., Smith, S.J., 2007. Genetics, insurance and participation: how a Citizens' Jury reached its verdict. Social Science & Medicine 64, 2487-2498.
Blacksher, E., Diebel, A., Forest, P., Dorr Goold, S., Abelson, J., 2012. What is public deliberation. Hastings Centre Report 42,14-17.
Braunack-Mayer, A., Street, J., Rogers, W., Givney, R., Moss, J., Hiller, J., et al., 2010. Including the public in pandemic planning: a deliberative approach. BMC Public Health 10, 501.
Brown, J., 2001. The World Cafe (PhD dissertation). The Fielding Institute.
Burgess, M., O'Doherty, K., Secko, D., 2008. Biobanking in British Columbia: discussions of the future of personalized medicine through deliberative public engagement. Personalized Medicine 5, 285-296.
Burkhalter, S., Gastil, J., Kelshaw, T., 2002. A conceptual definition and theoretical model of public deliberation in small face-to-face groups. Communication Theory 12, 398-422.
Button, M., Mattson, K., 1999. Deliberative democracy in practice: challenges and prospects for civic deliberation. Polity 31, 608-636.
Carson, L., 2006. Improving public deliberative practice: a comparative analysis of two Italian citizens' jury projects in 2006. Journal of Public Deliberation 2.
Carson, L., Hartz-Karp, J., 2005. Adapting and combining deliberative designs. In: Gastil, J., Levine, P. (Eds.), The Deliberative Democracy Handbook. Jossey-Bass, San Francisco, pp. 120-138.
Coote, A., Lenaghan, J., 1997. Citizens' Juries: Theory into Practice. Institute for Public Policy Research, London.
Davies, C., Wetherell, M., Wetherell, M., Barnett, E., 2006. Citizens at the Centre: Deliberative Participation in Healthcare Decisions. Policy Press, Bristol.
Dryzek, J.S., Tucker, A., 2008. Deliberative innovation to different effect: consensus conferences in Denmark, France, and the United States. Public Administration Review 68, 864-876.
Dunkerley, D., Glasner, P., 1998. Empowering the public? Citizens' juries and the new genetic technologies. Critical Public Health 8,181-192.
Einsiedel, E.F., 2002. Assessing a controversial medical technology: Canadian public consultations on xenotransplantation. Public Understanding of Science 11,315331.
Einsiedel, E.F., Ross, H., 2002. Animal spare parts? A Canadian public consultation on xenotransplantation. Science and Engineering Ethics 8, 579-591.
Elwood, P., Longley, M., 2010. My health - whose responsibility: a jury decides. Journal of Epidemiology and Community Health 64, 761 -764.
Fishkin, J., 1991. Democracy and Deliberation. Yale University Press, New Haven Connecticut.
Glasner, P., Dunkerley, D., 1999. The new genetics, public involvement, and citizens juries: a welsh case study. Health, Risk & Society 1, 313-324.
Gooberman-Hill, R., Horwood, J., Calnan, M., 2008. Citizens' juries in planning research priorities: process, engagement and outcome. Health Expectations 11, 272-28 .
Goodin, R., Dryzek, J., 2006. Deliberative impacts: the macro-political uptake of mini publics. Politics & Society 34, 219-244.
Haigh, F.A., Scott-Samuel, A., 2008. Engaging communities to tackle anti-social behaviour: a health impact assessment of a citizens' jury. Public Health 122, 1191 -1198.
Hendriks, C., 2002. The ambiguous role of civil society in deliberative democracy. In: Australasian Political Studies Association Conference. ANU, Canberra.
Hendriks, C., 2005. Consensus conferences and planning cells. In: Gastil, J., Levine, P. (Eds.), The Deliberative Democracy Handbook. Jossey-Bass, San Francisco, pp. 80-110.
Herbison, P., Hay-Smith, J., Paterson, H., Ellis, G., Wilson, D., 2009. Research priorities in urinary incontinence: results from citizens' juries. BJOG 116, 713718.
Huston, P., 2004. What does the public think of placebo use? The Canadian experience. Science and Engineering Ethics 10, 103-117.
Iredale, R., Longley, M., Thomas, C., Shaw, A., 2006. What choices should we be able to make about designer babies? A Citizens' Jury of young people in South Wales. Health Expectations 9, 207-217.
Kashefi, E., Mort, M., 2004. Grounded citizens' juries: a tool for health activism? Health Expectations 7, 290-302.
Kim, S.Y., Uhlmann, R.A., Appelbaum, P.S., Knopman, D.S., Kim, H.M., Damschroder, L., et al., 2010. Deliberative assessment of surrogate consent in dementia research. Alzheimers & Dementia 6, 342-350.
Kim, S.Y., Wall, I.F., Stanczyk, A., De Vries, R., 2009. Assessing the public's views in research ethics controversies: deliberative democracy and bioethics as natural allies. Journal of Empirical Research on Human Research Ethics 4, 3-16.
Lenaghan, J., 1999. Involving the public in rationing decisions. The experience of citizens juries. Health Policy 49, 45-61.
Lenaghan, J., New, B., Mitchell, E., 1996. Setting priorities: is there a role for citizens' juries? BMJ 312, 1591-1593.
Longstaff, H., Burgess, M.M., 2010. Recruiting for representation in public deliberation on the ethics of biobanks. Public Understanding of Science 19, 212224.
Mansbridge, J., Bohman, J., Chambers, S., Estlund, D., Follesdal, A., et al., 2010. The place of self-interest and the role of power in deliberative democracy. Journal of Political Philosophy 18, 64-100.
Maxwell, J., Rosell, S., Forest, P.G., 2003. Giving citizens a voice in healthcare policy in Canada. BMJ 326,1031-1033.
Menon, D., Stafinski, T., 2008. Engaging the public in priority-setting for health technology assessment: findings from a citizens' jury. Health Expectations 11,282-293.
Mooney, G.H., Blackwell, S.H., 2004. Whose health service is it anyway? Community values in healthcare. MJA 180, 76-78.
Mort, M., Finch, T., 2005. Principles for telemedicine and telecare: the perspective of a citizens' panel. Journal of Telemedicine and Telecare 11 (Suppl. 1), 66-68.
O'Doherty, K.C., Burgess, M.M., 2009. Engaging the public on biobanks: outcomes of the BC biobank deliberation. Public Health Genomics 12, 203-215.
Oliver-Weymouth, M., 2000. Citizens' juries - a model for involving the community in health policy. Health Issues 64, 32-35.
Parkin, L., Paul, C., 2011. Public good, personal privacy: a citizens' deliberation about using medical information for pharmacoepidemiological research. Journal of Epidemiology and Community Health 65, 150-156.
Parkinson, J., 2004. Why deliberate? The encounter between deliberation and new public managers. Public Administration Review 82, 377-395.
Paul, C., Nicholls, R., Priest, P., McGee, R., 2008. Making policy decisions about population screening for breast cancer: the role of citizens' deliberation. Health Policy 85, 314-320.
Pickard, S., 1998. Citizenship and consumerism in health care: a critique of citizens' juries. Social Policy and Administration 32, 226-244.
Pidgeon, N., Rogers-Hayden, T., 2007. Opening up nanotechnology dialogue with the publics: risk communication or 'upstream engagement'? Health, Risk & Society 9,191-210.
Rogers, W.A., Street, J.M., Braunack-Mayer, A.J., Hiller, J.E., 2009. Pandemic influenza communication: views from a deliberative forum. Health Expectations 12, 331342.
Rogers-Hayden, T., Pidgeon, N., 2006. Reflecting upon the UK's citizens' jury on nanotechnologies: NanoJury UK, Nanotechnology Law & Business 3, 167-178.
Secko, D.M., Preto, N., Niemeyer, S., Burgess, M.M., 2009. Informed consent in biobank research: a deliberative approach to the debate. Social Science & Medicine 68, 781-789.
Smith, G., Wales, C., 2000. Citizen juries and deliberative democracy. Political Studies 48, 51-65.
The Jefferson Center, 2004. Citizens Jury Handbook (Washington DC).
Timotijevic, L., Raats, M.M., 2007. Evaluation of two methods of deliberative participation of older people in food-policy development. Health Policy 82, 302-319.
Toni, A., Braun, von J., 2001. Poor citizens decide on the introduction of GMOs in Brazil. Biotechnology and Development Monitor 47, 7-9.
Watt, A.M., Hiller, J.E., Braunack-Mayer, A.J., Moss, J.R., Buchan, H., Wale, J., et al., 2012. The ASTUTE Health study protocol: deliberative stakeholder engagements to inform implementation approaches to healthcare disinvestment. Implementation Science 7, 10 .
Willison, D.J., Swinton, M., Schwartz, L., Abelson, J., Charles, C., Northrup, D., et al., 2008. Alternatives to project-specific consent for access to personal information for health research: insights from a public dialogue. BMC Med Ethics 9, 18.
Yankelovich, D., 1991. Coming to Public Judgement: Making Democracy Work in a Complex World. Saracus University Press, NY.
Young, I.M., 1996. Communication and the other: beyond deliberative democracy. In: Benhabib, S. (Ed.), Democracy and Difference: Contesting the Boundaries. Princeton University Press, West Sussex.
