Scholarly article on topic 'Scrutinizing the impact of CCS communication on opinion quality: Focus group discussions versus Information-Choice Questionnaires: Results from experimental research in six countries'

Scrutinizing the impact of CCS communication on opinion quality: Focus group discussions versus Information-Choice Questionnaires: Results from experimental research in six countries Academic research paper on "Economics and business"

CC BY-NC-ND
0
0
Share paper
Academic journal
Energy Procedia
OECD Field of science
Keywords
{"CO2 capture and storage (CCS) technologies" / "Public perception" / "Opinion quality" / "Focus group discussions" / "Information-Choice Questionnaire (ICQ)"}

Abstract of research paper on Economics and business, author of scientific article — Dancker D.L. Daamen, Bart W. Terwel, Emma ter Mors, David M. Reiner, Diana Schumann, et al.

Abstract Previous research has shown that public knowledge and awareness of carbon dioxide capture and storage (CCS) is very limited. As a result, traditional surveys designed to collect public opinions about CCS do in fact assess so-called pseudo opinions. Pseudo-opinions are of very low quality because they are mostly unstable and inconsistent. Therefore, they are not predictive for actual and future public support for or opposition against CCS. As compared to pseudo opinions, opinions expressed after the public has been provided with factual information about CCS are likely to be of higher quality. Focus group discussions and Information-Choice Questionnaires (ICQs) are two research techniques frequently used in the CCS literature that aim to collect such informed public opinions. In this study, we examined which of these two research technique leads to the highest quality opinions (i.e., to opinions that are consistent, stable, and that people are confident about). Our results showed that ICQs yielded higher-quality opinions than focus group discussions. Practical implications and recommendations are discussed.

Academic research paper on topic "Scrutinizing the impact of CCS communication on opinion quality: Focus group discussions versus Information-Choice Questionnaires: Results from experimental research in six countries"

Available online at www.sciencedirect.com

ScienceDirect

Energy Procedía 4 (2011) 6182-6187

Energy Procedía

www.elsevier.com/locate/procedia

GHGT-10

Scrutinizing the impact of CCS communication on opinion quality:

Focus Group Discussions versus Information-Choice Questionnaires: Results from experimental research in six countries

Dancker D. L. Daamena, Bart W. Terwela, Emma ter Morsa 1*, David M. Reinerb, Diana Schumannc, Sorin Angheld, Ioanna Bouloutab, Diana M. Cismarue, Carmencita Constantinf, Chris C. H. de Jagera, Alexandra Dudud, Rebecca M. Firthb, Vassiliki Gemenig, Chris Hendriksh, Nikolaos Koukouzasg, Angelos Markosi, Robert N^ssj, Oana C. Nihfidovf, Katja Pietznerk, Irene R. Samoilaf, Constantin S. Savad, Mike H. Stephenson^ Claudia E. Tomescuf, Hans Y. Torvatnj, Sturle D. Tvedtj, Daniel Vallentink, Julia M. Westl, Fotini Ziogoug

aLeiden University, the Netherlands bUniversity of Cambridge, UK cForschungszentrum Jülich GmbH, Germany dNational Institute of Marine Geology and Geoecology - GeoEcoMar, Romania eNational School of Political Studies and Public Administration, Romania fInstitute for Studies and Power Engineering (ISPE), Romania gInstitute for Solid Fuels Technology and Applications/Centre for Research and Technology Hellas (ISFTA/CERTH), Greece hEcofys, the Netherlands University of Macedonia, Greece jSINTEF, Norway kWuppertal Institute for Climate, Environment and Energy, Germany 1 British Geological Survey (BGS), UK

Abstract

Previous research has shown that public knowledge and awareness of carbon dioxide capture and storage (CCS) is very limited. As a result, traditional surveys designed to collect public opinions about CCS do in fact assess so-called pseudo opinions. Pseudo-opinions are of very low quality because they are mostly unstable and inconsistent. Therefore, they are not predictive for

* Corresponding author. Tel.: +31 (0) 715274075. E-mail address: Emors@fsw.leidenuniv.nl.

doi:10.1016/j.egypro.2011.02.629

actual and future public support for or opposition against CCS. As compared to pseudo opinions, opinions expressed after the public has been provided with factual information about CCS are likely to be of higher quality. Focus group discussions and Information-Choice Questionnaires (ICQs) are two research techniques frequently used in the CCS literature that aim to collect such informed public opinions. In this study, we examined which of these two research technique leads to the highest quality opinions (i.e., to opinions that are consistent, stable, and that people are confident about). Our results showed that ICQs yielded higher-quality opinions than focus group discussions. Practical implications and recommendations are discussed.

© 2011 Published by Elsevier Ltd.

Keywords: CO2 capture and storage (CCS) technologies; public perception; opinion quality; Focus Group Discussions; Information-Choice Questionnaire (ICQ)

1. Introduction

A considerable number of surveys have been conducted to examine public attitudes toward and knowledge of carbon dioxide capture and storage (CCS). Indeed, surveys constitute a proper way to assess levels of knowledge and awareness among the public. It has consistently been found that members of the general public have very little knowledge about CCS and are hardly aware of its existence [1]. For example, Curry et al. [2] found that in 2006 only about 5% of respondents in the United States had heard of CCS. As explained by De Best-Waldhober et al. [3,4] and Malone et al. [1], the fact that knowledge of CCS is very limited implies that surveys are not very suited to assess public opinions about CCS. That is, most people simply do not have an opinion about CCS. Yet, even when people indicate to have never heard of CCS, they often do not refrain from giving their opinion about the technology when asked. Such uninformed opinions are referred to as pseudo opinions or non-attitudes [3,4], which are low-quality opinions that are not predictive for actual support for or opposition against CCS.

Given the low public awareness of CCS, there is a need to educate people before asking them to express their opinions about CCS. Indeed, researchers have used several different techniques that examine public opinions after people have been informed about CCS, resulting in an assessment of informed rather than uninformed opinions. As compared to uninformed opinions, informed public opinions are of relatively high quality (rather stable over time and predictive of future attitudes and behavior) [5-7]. Among the most frequently used techniques to examine informed public opinions about CCS are focus group discussions (or related participatory group-based techniques)[8,9] and surveys of informed opinion, most notably the so-called Information-Choice Questionnaire (ICQ) repeatedly administered by De Best-Waldhober, Daamen and colleagues [3,4,10].

Both these communication methods are established methods in research on public perceptions of CCS and have in common that they address informed public opinions. That is, even though these methods differ in many regards (for a discussion of differences, see Ref. 11) in both these methods people are provided with relevant information about CCS before they form an opinion about the technology. However, it has remained unclear which of these communication methods is most effective in terms of the quality of opinions formed. The higher the quality of informed opinions, the better they can be used to predict and explain these people's future level of acceptance of CCS. Therefore, the question that this study aims to answer is: Which of these two methods (focus group discussions versus ICQs) results in the highest quality opinions? Before we describe the experiment that we designed to answer this question, we briefly discuss the main indicators of opinion quality proposed in literature on opinion quality that we used in the study.

1.1. Indicators of opinion quality

Price and Neijens [12] proposed opinion consistency and opinion stability as indicators of opinion quality (also see Ref. 6). Opinion consistency can be defined as "the extent to which opinions are consistent with evaluations of related variables". In the context of CCS, opinion consistency can be assessed by looking at the strength of relationships between people's evaluations of the consequences of a CCS option and their general attitude towards (or choice for) this CCS option [4]. For example, one would expect that a person who believes that CCS is very dangerous and expensive also evaluates CCS as undesirable. Opinion stability can be defined as "the extent to which opinions are stable over time". One way to determine opinion stability is by assessing opinions

about CCS at two different points in time and to examine whether shifts in opinions occur. A third indicator of opinion quality distinguished in the opinion strength literature [13] is opinion confidence, which can be defined as a subjective sense of conviction or validity about one's opinion [14, 15]. Opinion confidence in the context of CCS can be assessed by asking people whether they have been able to form an accurate impression about CCS and whether they are certain about their opinion about the technology. Based on the above, we define the effectiveness of CCS communication methods as "the extent to which CCS communications result in well-informed, well-considered and high quality opinions which are a) consistent, b) stable, and c) that people are confident about. Accordingly, should either of the two communication methods—focus group discussions or ICQs—prove to be superior to the other in terms of opinion quality, we would expect this to be reflected in higher opinion consistency, higher opinion stability, and higher opinion confidence for this method.

1.2 Focus group discussions and opinion quality

Focus groups are small groups of people who are invited to discuss a certain topic, in this case CCS. An expert presents the group members with information about CCS, after which the groups discusses the topic. A moderator usually guides the group discussion [16-18]. Focus groups have frequently been used to inform people about CCS and to measure their perceptions of the technology [8,9]. However, despite the widespread use of focus group discussions, the literature offers only few empirical evaluations of the method. Focus group discussions may yield high quality opinions due to accountability effects. That is, members expect having to take a stand on CCS at the end of a meeting and to defend their point of view (i.e., they feel accountable). Accountability promotes more systematic information processing and information integration [19]. Moreover, the face-to-face interaction of group members and experts provides the opportunity to directly ask for clarification when information is not clear. Both the opportunity to ask questions and accountability may lead to higher quality opinions. On the other hand, disruptive group processes may inhibit the quality of opinions. For instance, group dynamics such as group think [17,20] may occur, or group members may fail to exchange all unique (unshared) information that they have (and focus on shared knowledge only). Further, with a complex topic as CCS, group members can easily confuse each other so that their ability to integrate different pieces of information is reduced.

1.3 Information-Choice Questionnaires and opinion quality

The Information-Choice Questionnaire is the most elaborate form of surveys that aim to collect informed opinions [21-24]. However, the aim of an ICQ is not only to provide respondents with the necessary information to reach a well-informed opinion, but also to individually help them process and integrate this information in order to reach a well-considered opinion. To stimulate information processing and to help respondents reach a decision, they are requested to give a quantitative evaluation of each consequence (a rating on a scale with 19 response categories ranging from -9 ''a very big disadvantage'' via 0 ''totally irrelevant'' to +9 ''a very big advantage''). On the basis of these quantitative evaluations, the subjective utility of each option may be determined to evaluate each option overall and to choose which option is preferred and which option(s) is (are) unacceptable. Based on experimental studies by Neijens [23], there are reasons to expect that an ICQ results in high quality opinions. For instance, Neijens showed that information presented in statements about consequences which respondents should evaluate (as in an ICQ) is better processed than when the same information is presented in story form. Neijens [23] further proved that other elements of the ICQ procedure (e.g., a book-keeping system for respondents' evaluations of consequences) add to better information processing and information integration. De Best-Waldhober et al. [4] showed that opinions formed by means of an ICQ are relatively consistent and stable over time. At the same time, compared to focus group discussions, in an ICQ the opportunity to ask questions for clarification is non-existent, and accountability effects that are likely to occur in focus group discussions are not to be expected.

2. Method and results of the present research1

The present research systematically compared the effectiveness (in terms of opinion quality) of focus group discussions to that of information-choice questionnaires. An experimental study was conducted in six European countries simultaneously: Germany, Greece, the Netherlands, Norway, Romania, and the UK. In each country,

participants received identical factual information about CCS in general and country-specific information about two CCS options. This information was developed by British Geological Survey (BGS) in collaboration with the partners involved in the research. In developing the information, we also used previous research using ICQ methodology in which comparable information comprehensible for laypeople was developed [4]. We subsequently systematically varied the way the CCS information was revealed to participants. In each country half of the participants received and processed the CCS information in a focus group, whereas for the other half of participants this was done by means of an ICQ. Participants in both conditions were subsequently asked to state their overall opinion about each of the two CCS options. The quality of their opinions was determined by looking at three indicators of opinion quality: opinion consistency, opinion stability, and opinion confidence (cf. Ref. 6 and Ref. 12). Should either of the two communication methods under examination prove to be superior to the other in terms of opinion quality, we expected this to be reflected in higher opinion consistency, higher opinion stability, and higher opinion confidence for this method. Important to note, for a meaningful comparison between both communication methods, we made sure that each focus group participant was paired with an identical ICQ participant (i.e., identical in education level, occupational category, age group, and gender) and that within each country the information presented in both experimental conditions was identical. Added over the six countries we had 183 participants in the focus group condition and 183 participants in the ICQ condition. Aggregation of the data collected in each country was necessary to carry out meaningful analyses

The results of the present study confirmed that participants in both experimental conditions indeed considered the CCS information provided to be factual and of high quality; they considered it to be unbiased, not one-sided, useful, comprehensible and valid. More important though, the results for the three indicators of opinion quality—opinion consistency, opinion stability, and opinion confidence—convincingly demonstrated that opinion quality was significantly higher after completing an ICQ than after participating in a focus group discussion. Hence, the present cross-national study shows that the ICQ is the more effective communication method of the two when it comes to communication about CCS, as it leads to significantly higher quality opinions that can be expected to be more stable over time and predictive of future opinions, intentions, and behavior. It should be noted, however, that focus groups were still doing quite well, which implies that if focus groups are well-managed and high-quality information (information that is relevant, valid, balanced, and comprehensible for lay people) is provided, focus groups can also lead to opinions of relatively high quality.

3. Implications

The results of this cross-national study show that the ICQ leads to higher quality opinions, as compared to focus group discussions. Of course, this is not to say that ICQs are preferred over focus group methodology at all times. Important to realize is that both methods require that high-quality information is developed (i.e., information that is relevant, valid, balanced, and comprehensible for lay people). Developing this information takes a lot of time and money. However, which method is most suitable depends on factors such as the target group (e.g., local communities, the general public, policymakers) and what one wants to establish (e.g., education, assess the general public's opinions about CCS, stakeholder engagement).

For example, researchers whose aim is to study informed opinions among a representative sample of a country's population (which would require a sample size of about N = 1000) may prefer to develop an ICQ. That is, the current study made use of rather small samples, but up scaling can be done against relatively low additional cost (i.e., costs additional to the costs of developing the CCS information). Hence, the significant costs associated with the development and deployment of the ICQ can be offset by the greater potential for large-scale applicability of this method. By contrast, it seems almost undoable in terms of both financial and personal resources to study informed opinions among a representative sample of a country's population by means of focus groups in that at least a hundred focus groups need to be run. For each group meeting, there are costs involved for hiring a well-trained moderator, selecting a well-established expert to present information during the focus group meetings, and locations for the focus groups to take place etc. etc. Further, to inform the general population about actual or planned CCS activities, an Internet version of the ICQ could be easily used without high additional costs compared to the large-scale implementation of focus groups. The design and development of web-based ICQs on CCS is already in progress (in a project funded by the Global CCS Institute).

On the other hand, a focus group could be used to collect opinions of small samples. For example, in countries with actual or planned CCS activities (especially in the case of onshore CO2 geological storage), there may be the need for real interaction with people living nearby these activities (for example to identify common misperceptions and to understand people's reactions). In that case, it would be useful to conduct a certain number of focus groups at that specific location to promote dialogue and engagement of local stakeholders. Indeed, face-to-face dialogue appears to be an effective way not only to communicate CCS but also to acknowledge the concerns of the local community.

As mentioned in the introduction, 'negative' intra-group processes may hinder information processing and integration in focus groups and, as a result, reduce the quality of opinions. To reduce disruptive group dynamics and to enhance information processing and integration, an interesting possibility could be to introduce a shorter version of an ICQ within a focus group meeting, this way combining oral and written information. One could for instance introduce a personal quiz during the expert presentations to avoid group processes such as group think and to limit the possibility that group rather than individual opinions are reported. In addition, combining quantitative and qualitative methodologies may offer a deeper understanding of public perceptions of CCS.

4. Acknowledgements

The research reported here has been conducted as part of the FENCO-ERA project "Scrutinizing the impact of CCS communication on the general and local public (Impact of communication)". We are grateful to the national funding agencies of Germany, Greece, the Netherlands, Norway, Romania, and the United Kingdom for their financial support for conducting the study.

1 Explaining the rather complex design and detailed results of this extensive cross-national study requires considerable space. For those readers interested in the specifics of the study, we refer to the research report "Scrutinizing the impact of CCS communication on opinion quality: An experimental comparison between Focus Group Discussions versus Information-Choice Questionnaires (ICQs): Results from cross-national analyses" by Terwel et al. [11] (also see Ter Mors et al. [25]).

5. References

[1] Malone, E. L. Dooley, J. J., & Bradbury, J. A. (2010). Moving from misinformation derived from public attitude surveys on carbon dioxide capture and storage towards realistic stakeholder involvement. International Journal of Greenhouse Gas Control, 4, 419-425.

[2] Curry, T.E., Ansolabehere, S., & Herzog., H., 2007. A Survey of Public Attitudes towards Climate Change and Climate Change Mitigation Technologies in the United States: Analyses of 2006 Results. Publication No. LFEE 2007-01 WP. Massachusetts Institute of Technology, Laboratory for Energy and the Environment, Cambridge.

[3] De Best-Waldhober, M., Daamen, D. D. L., & Faaij, A. P. C. (2006). Public perceptions and preference regarding large-scale implementation of six CO2 capture and storage technologies: Well-informed and well-considered opinions versus uninformed pseudo-opinions of the Dutch public. Research report.

[4] De Best-Waldhober, M., Daamen, D. D. L., & Faaij, A. P. C. (2009). Informed and uninformed public opinions on CO2-capture and storage technologies in the Netherlands. International Journal of Greenhouse Gas Control, 3, 322-333.

[5] Eagly, A. H., & Chaiken, S. (1993). The Psychology of Attitudes. Forth Worth, TX: Harcourt, Brace, Jovanovich.

[6] Krosnick, J. A., & Petty, R. E. (1995). Attitude Strength: An Overview. In: Richard E. Petty and Jon A. Krosnick, editors. Attitude strength: Antecedents and Consequences (1-24). Hillsdale, NJ; Erlbaum.

[7] Petty, R. E., & Cacioppo, J. T. (1986). The Elaboration Likelihood Model of persuasion. In: Leonard Berkowitz, editor. Advances in Experimental Social Psychology, (123-205). New York: Academic Press.

[8] Shackley, S., McLachlan, C., & Gough, C. (2005). The public perception of carbon capture and storage in the UK: Results from focus groups and a survey. Climate Policy, 4, 377-398.

[9] Sharp, J. D., Jaccard, M. K., & Keith, D. W. (2009). Anticipating public attitudes toward underground CO2 Storage. International Journal of Greenhouse Gas Control, 3, 641-651.

[10] De Best-Waldhober, M., Daamen, D. D. L., Hendriks, C., de Visser, E., Ramirez Ramirez, A., & Faaij, A. P. C. (2008). How the Dutch evaluate CCS options in comparison with other CO2 mitigation options: Results of a nationwide Information Choice Questionnaire survey. Research report.

[11] Terwel, B. W., Ter Mors, E., Daamen, D. D. L., Reiner, D. M., Schumann, D., et al. (2009). Scrutinizing the impact of CCS communication on opinion quality: An experimental comparison between Focus Group Discussions versus Information-Choice Questionnaires (ICQs): Results from cross-national analyses. Research report.

[12] Price, V., & Neijens, P. (1997). Opinion quality in public opinion research. International Journal of Public Opinion Research, 9, 336-360.

[13] Gross, S. R., Holtz, R., & Miller, N. (1995). Attitude Certainty. In: Richard E. Petty and Jon A. Krosnick, editors. Attitude strength: Antecedents and Consequences (215-242). Hillsdale, NJ: Erlbaum.

[14] Festinger, L. A. (1950). Informal social communication. Psychological Review, 57, 271-282.

[15] Festinger, L. A. (1954). A theory of social comparison processes. Human Relations, 7, 117-140.

[16] Morgan, D. L. (1996). Focus Groups. Annual Review of Sociology, 22, 129-152.

[17] Morgan, D. L. (1997). Focus Groups as Qualitative Research. Thousand Oaks, CA: Sage.

[18] Stewart, D. W., & Shamdasani, P. N. (1990). Focus Groups: Theory and Practice. Newbury Park, CA: Sage.

[19] Tetlock, P. E. (1983). Accountability and complexity of thought. Journal of Personality and Social Psychology, 45, 74-83.

[20] Janis, I. L. & Mann, L. (1977). Decision making: A psychological analysis of conflict, choice, and commitment. New York: Free Press.

[21] Saris, W. E., Neijens, P., & De Ridder, J. A. (1983a). Keuze-enquete [Choice-questionnaire]. Vrije Universiteit, Amsterdam.

[22] Saris, W. E., Neijens, P., & De Ridder, J. A. (1983b). Kern energie: Ja of nee? Een weloverwogen mening van de Nederlandse bevolking [Nucleair energy: Yes or no? A well-considered opinion of the Dutch general public]. SSO, Amsterdam.

[23] Neijens, P. (1987). The choice questionnaire: Design and evaluation of an instrument for collecting informed opinions of a population. Amsterdam, Free University Press.

[24] Neijens, P., De Ridder, J. A., & Saris, W. E. (1992). An instrument for collecting informed opinions. Quality and Quantity, 26, 245-258.

[25] Ter Mors, E., Terwel, B. W., Daamen, D. D. L., Reiner, D. M., Schumann, D., et al. (2010). A comparison of techniques aimed at collecting informed public opinions: Opinion quality in focus group discussions versus information-choice questionnaires (manuscript submitted for publication).