Scholarly article on topic 'Voting and civic engagement: Results from an online field experiment'

Voting and civic engagement: Results from an online field experiment Academic research paper on "Media and communications"

0
0
Share paper
Academic journal
Research & Politics
OECD Field of science
Keywords
{""}

Academic research paper on topic "Voting and civic engagement: Results from an online field experiment"

Research Note

Research and Politics

a* 1 • • a n Ij- lanuary-March 2017: 1-3

Voting and civic engagement: Results ©^^2017

#»ll • DOI: 10.1177/2053168017690736

from an online field experiment

Seth H. Werfel

Abstract

How does voting interact with civic engagement outside the electoral process? An online field experiment on more than 140,000 registered voters in San Francisco yielded two main results. Subjects who voted in the 2016 primary elections were nearly three times more likely to open a survey from a nonprofit organization than those who did not vote in the primary election. However, explicitly priming voter identity and gratitude made all subjects far less likely to engage in this form of civic participation.

Keywords

Civic engagement, field experiment, nonprofits, survey response, voting

Introduction

Citizens have two main types of actions in their political toolkits: electoral and non-electoral. Recent work at the macro-level has observed substitution between these two categories over time (Dalton, 2008), while individual-level analysis has suggested a positive correlation (Oser, 2016). However, the causal relationship between these two forms of participation remains unclear. Does engaging in one type cause more or less of the other type? As a practical matter, can organizations leverage prior activity in one category to induce the other?

Nonprofit organizations provide a useful context for studying causal relationships within the political toolkit (Han, 2016). These organizations can facilitate electoral participation through voter mobilization efforts (LeRoux and Krawczyk, 2014). In addition, nonprofits often experiment with different kinds of appeals to induce a wide range of non-electoral actions (Han, 2009). Furthermore, some of the most effective appeals establish a relational context by linking current requests to prior behavior (Han, 2016).

This study leverages a sample of more than 140,000 registered voters to better understand who participates in modern forms of civic engagement and what kinds of appeals are most effective. The results of an online field experiment yielded two main results. Subjects who voted in the 2016 primary elections were nearly three times more likely to engage with a survey distributed by a civic organization

than those who did not vote in the primary election. However, explicitly priming voter identity and gratitude made all subjects far less likely to engage in this form of civic participation. These results may offer some practical wisdom to nonprofit organizations and improve our understanding of how voters manage their political toolkits.

Experimental design

A field experiment was conducted in collaboration with a pro-housing civic organization in San Francisco, California. The organization designed a survey to measure public opinion about issues affecting the community and wanted to maximize participation. The sample for the experiment included 140,189 registered voters who provided their email address during the registration process. This design did not control for information spillovers between multiple units in the same household (Coppock, 2014), although interference should bias against significant treatment effects.

The survey was distributed via email using a fully crossed 2 x 2 between-subjects factorial design. The voter

Stanford University, USA Corresponding author:

Seth H. Werfel, Department of Political Science, 616 Serra Street, Room 100, Stanford, CA 94305, USA. Email: swerfel@stanford.edu

Creative Commons CC-BY: This article is distributed under the terms of the Creative Commons Attribution 3.0 License (http://www.creativecommons.org/licenses/by/3.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).

Research and Politics

identity manipulation occurred in the subject line of the email: "SF [voter/resident], what are your priorities for the city?" This manipulation was inspired by prior research on the effects of priming voter identity (Bryan et al., 2011), although it does not rely on subtle linguistic cues (Gerber et al., 2016). The voter gratitude manipulation occurred in the first line in the body of the email: treated subjects were thanked for being a registered voter in San Francisco (Panagopoulos, 2011), while subjects in the control condition were again asked what was important to them. Crucially, this control condition did not include any additional language that was not already contained by the treatment condition.

The outcome variable measured whether the subject clicked the link embedded in the email to open the survey. Since survey responses were anonymous, it was not possible to match this click rate with survey completion rates. However, it is expected that treatment assignment had no effect on the probability of completing the survey given that the subject had already clicked to open the survey.

Results

The voter file contained information about subjects' party identification, age, and whether they voted in the 2016 primary election that occurred two months prior to the experi-ment.1 Figure 1 explores which of these characteristics were associated with higher click rates. The most striking result is that subjects who voted in the primary election were nearly three times as likely to open the survey than non-voters. This result may reflect the fact that participatory citizens tend to embrace both non-institutionalized and electoral-oriented activities (Oser, 2016), or that voters are more likely to contribute to public goods (Bolsen et al., 2014).

Figure 2 reports the results associated with the experimental manipulations. Surprisingly, subjects in all three treatment conditions were about 40% less likely to open the survey than subjects in the control condition (p = 0.000).2 This result is even more remarkable given the relatively low baseline, as floor effects should have biased against finding significant treatment effects. Floor effects may, however, explain why the identity and gratitude manipulations did not appear to have an additive effect on click rates.

What explains this decrease in civic engagement? Since the manipulations conflated voter identity and gratitude with politics more generally, one explanation could be that subjects are simply averse to politics. This mechanism seems unlikely since all subjects are registered voters who volunteered their email addresses during the registration process. Furthermore, if the main effect was driven by an aversion to politics, then this effect should be smaller for subjects with higher levels of interest in politics. Instead, treatment effects were significant and just as large for subjects who voted in the primary election as those who did not vote. In fact, the voter identity treatment effect was slightly larger for primary voters.

Figure 1. Predictors of online civic engagement. Each line represents the partial correlation between a particular demographic factor and the likelihood of opening the survey, along with 95% confidence intervals. These point estimates were derived from a logistic regression of the outcome variable on all covariates and condition fixed effects.

Figure 2. Treatment effects on online civic engagement. Each bar represents the proportion of subjects in each condition who clicked on the survey link, along with 95% confidence intervals.

A separate but related mechanism is voter fatigue. The experiment was conducted three months before a general election in a state that include several propositions on the ballot. Therefore, treated subjects may have exhausted their capacity to engage in electoral politics and potentially misconstrued the email as one about the general election. This mechanism is consistent with the larger treatment effect observed among primary voters.

Alternatively, the priming of past voting behavior may have caused a boost in self-concept that subsequently enabled treated subjects to feel "licensed" to dismiss additional requests to take costly civic action (Merritt et al., 2010). This mechanism would also be consistent with larger treatment effects among primary voters, although

Werfel

licensing should apply only to the voter gratitude treatment. These data cannot fully distinguish between the voter fatigue and licensing mechanisms because self-concept was not measured directly.

Discussion

This paper explored whether voting influences civic engagement outside the electoral process. The results of an online field experiment on more than 140,000 registered voters in San Francisco yielded two main results. First, subjects who voted in the recent primary election were far more likely to open a survey from a nonprofit organization than those who did not vote in the primary election. However, at least one attempt to explicitly leverage prior voting behavior had negative unintended consequences. Priming voter identity and gratitude made subjects far less likely to engage in this form of non-electoral participation.

The results of this experiment may offer practical wisdom to nonprofit organizations looking to improve the effectiveness of their appeals. Of course, these data were generated by a unique sample of registered voters at a particular time in the electoral cycle. In addition, interacting with a survey distributed by a nonprofit organization is hardly representative of all forms of non-electoral civic engagement. Therefore, additional research is needed to evaluate the generalizability of these results. Future work should continue to explore the conditions under which citizens construe the various items in their political toolkits as complements or substitutes.

Acknowledgements

I am grateful to Sonja Trauss for the work that she does and for the work that she enabled me to do.

Funding

This work was supported by the National Science Foundation (grant number DGE-114747). The ideas presented in this paper do not necessarily reflect the views of the National Science Foundation.

Supplementary material

The Supplementary material and replication files are available at http://journals.sagepub.com/doi/suppl/10.1177/2053168017690736 and http://dx.doi.org/10.7910/DVN/8K9BBR respectively.

1. Gender was also available for a non-random subset of 58,758 subjects, but including this covariate did not substantively

change the results (see "Supplemental information" for more details).

2. The four experimental conditions were slightly imbalanced on levels of primary voting. However, covariate adjustment has no substantive effect on the magnitude or significance of average treatment effects (see "Supplemental information" for more details).

Carnegie Corporation of New York Grant

This publication was made possible (in part) by a grant from

Carnegie Corporation of New York. The statements made and

views expressed are solely the responsibility of the author.

References

Bolsen T, Ferraro PJ and Miranda JJ (2014) Are voters more likely to contribute to other public goods? Evidence from a large-scale randomized policy experiment. American Journal of Political Science 58(1): 17-30.

Bryan CJ, Walton GM, Rogers T, et al. (2011) Motivating voter turnout by invoking the self. Proceedings of the National Academy of Sciences 108(31): 12653-6.

Coppock A (2014) Information spillovers: Another look at experimental estimates of legislator responsiveness. Journal of Experimental Political Science 1(2): 159-169.

Dalton RJ (2008) Citizenship norms and the expansion of political participation. Political Studies 56(1): 76-98.

Gerber AS, Huber GA, Biggers DR, et al. (2016) A field experiment shows that subtle linguistic cues might not affect voter behavior. Proceedings of the National Academy of Sciences 113(26): 7112-7117.

Han HC (2009) Does the content of political appeals matter in motivating participation? A field experiment on self-disclosure in political appeals. Political Behavior 31(1): 103-116.

Han HC (2016) The organizational roots of political activism: Field experiments on creating a relational context. American Political Science Review 110(2): 296-307.

LeRoux K and Krawczyk K (2014) Can nonprofit organizations increase voter turnout? Findings from an agency-based voter mobilization experiment. Nonprofit and Voluntary Sector Quarterly 43(2): 272-292.

Merritt AC, Effron DA and Monin B (2010) Moral self-licensing: When being good frees us to be bad. Social and Personality Psychology Compass 4(5): 344-357.

Oser J (2016) Assessing how participators combine acts in their "political tool kits": A person-centered measurement approach for analyzing citizen participation. Social Indicators Research. Epub ahead of print 30 May 2016. D0I:10.1007/s11205-016-1364-8.

Panagopoulos C (2011) Thank you for voting: Gratitude expression and voter mobilization. Journal of Politics 73(3): 707-717.