Scholarly article on topic 'Examining user surprise as a symptom of algorithmic filtering'

Examining user surprise as a symptom of algorithmic filtering Academic research paper on "Media and communications"

CC BY
0
0
Share paper
Keywords
{"Algorithmic curation" / "Socio-technical systems" / Relationships / "Facebook News Feed"}

Abstract of research paper on Media and communications, author of scientific article — Emilee Rader

Abstract The Facebook News Feed prioritizes posts for display by ranking them more prominently in the News Feed, based on users’ past interactions with the system. This study investigated constraints imposed on social interactions by the algorithm, by triggering participants’ awareness of “missed posts” in their Friends’ Timelines that they did not remember seeing before. If the algorithm prioritizes posts from people that users feel closer to and want to stay in touch with, participants should be less likely to report missed posts from close Friends. However, the results showed that relationship closeness had no effect on the likelihood of noticing a missed post, after controlling for how many Facebook Friends participants had and the accuracy of participants’ memories for their Friends’ Facebook activity. Also, missed posts from close Friends were more surprising, even when participants believed that the actions of the system caused the missed posts, indicating that these instances represent participants’ unmet expectations for the behavior of their News Feeds. Because Facebook posts present opportunities for feedback important for social support and maintaining social ties, this could indicate bias in the way the algorithm promotes content that could affect users’ ability to maintain relationships on Facebook. These findings have implications for approaches to improve user control and increase transparency in systems that use algorithmic filtering.

Academic research paper on topic "Examining user surprise as a symptom of algorithmic filtering"

Contents lists available at ScienceDirect

Int. J. Human-Computer Studies

journal homepage: www.elsevier.com/locate/ijhcs

Examining user surprise as a symptom of algorithmic filtering

Emilee Rader

Department of Media and Information, Michigan State University, United States

CrossMark

ARTICLE INFO

Keywords: Algorithmic curation Socio-technical systems Relationships Facebook News Feed

ABSTRACT

The Facebook News Feed prioritizes posts for display by ranking them more prominently in the News Feed, based on users' past interactions with the system. This study investigated constraints imposed on social interactions by the algorithm, by triggering participants' awareness of "missed posts" in their Friends' Timelines that they did not remember seeing before. If the algorithm prioritizes posts from people that users feel closer to and want to stay in touch with, participants should be less likely to report missed posts from close Friends. However, the results showed that relationship closeness had no effect on the likelihood of noticing a missed post, after controlling for how many Facebook Friends participants had and the accuracy of participants' memories for their Friends' Facebook activity. Also, missed posts from close Friends were more surprising, even when participants believed that the actions of the system caused the missed posts, indicating that these instances represent participants' unmet expectations for the behavior of their News Feeds. Because Facebook posts present opportunities for feedback important for social support and maintaining social ties, this could indicate bias in the way the algorithm promotes content that could affect users' ability to maintain relationships on Facebook. These findings have implications for approaches to improve user control and increase transparency in systems that use algorithmic filtering.

1. Introduction

Many socio-technical systems, including search engines, news aggregators, and social media sites, employ personalization algorithms to rank and filter the content displayed to users. The Facebook News Feed Algorithm is one example, designed to "deliver the right content to the right people at the right time, so they don't miss the stories that are important to them" (Backstrom, 2013). The algorithm acts as a constraint on the information available for users to pay attention to, by generating a personalized ranking of posts for each user. The ranking is based on quantified signals such as how often Facebook Friends interact with each others' posts, overall post-level engagement, and the type of content in the post. For example, a post can be "surfaced" by the algorithm for a given user—moved higher up in the ranking—based on how much other users are interacting with it (Backstrom, 2013). Personalization algorithms are designed to reduce information overload and improve the user experience by connecting users with the information the system predicts they are likely to want to see, based on their past interactions with the system.

The Facebook News Feed presents a unique opportunity to study the potential effects of a filtering algorithm for end users in an extremely popular1 socio-technical system in which complex interactions between user and algorithm behavior determine the constraints

on content visibility (Rader and Gray, 2015). Algorithms are often thought of as neutral gatekeepers because they are computer code, assumed to be free from human bias (Bozdag, 2013). However, rules designed to promote some information necessarily make other information less visible, imparting a very real "threat of invisibility" upon those whose contributions are not evaluated favorably by the algorithm (Bucher, 2012). Personalized content filtering may restrict the subset of Friends whose posts appear in a user's News Feed to only those who they interact with frequently, or who post information similar to what they have read in the past (Hill et al., 1992). Friedman and Nissenbaum (1996) defined algorithmic bias as systematically and inappropriately denying opportunities or assigning undesirable outcomes. By constraining which posts and thereby which Friends users can most easily interact with, choices made by the system which are invisible to users may be just as important to study and understand as the visible aspects, especially for the identification of possible bias.

Kitchin (2016) recently argued that the work an algorithm does looks very different to the end user experiencing the effects of this work than it does to the system designers and operators responsible for creating, maintaining, and updating the algorithm. It is therefore important to investigate users' experiences interacting with algorithmic filtering, to understand systematic, unexpected patterns of system behavior. This paper presents a study in which Facebook users engaged

E-mail address: emilee@msu.edu. 1 On average, 1.09billion daily active users as of March 2016. Source: http://newsroom.fb.com/company-info/

http://dx.doi.org/10.1016/j.ijhcs.2016.10.005

Received 22 December 2015; Received in revised form 17 June 2016; Accepted 19 October 2016 Available online 21 October 2016

1071-5819/ © 2016 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.Org/licenses/by/4.0/).

in a task where they might experience algorithm effects firsthand, by visiting a Friend's Timeline and noticing posts that they did not remember seeing in their own News Feeds. Because Facebook users can navigate directly to their Friends' Timelines and view a reverse-chronological list of another user's past posts, they can use the system to find posts that may not have appeared in their News Feeds. This task triggered an "infrastructural inversion" (Bowker et al., 2010), making invisible aspects of the way the system works visible to participants (Hamilton et al., 2014), in order to measure their expectations for what they should see in their News Feeds. The study investigated whether the prevalence of missed posts might vary according to the closeness of participants' relationship with each Friend, and whether that would have any bearing on how surprised participants were about missing posts from particular people. Surprise is evidence of an unmet expectation (Burgoon, 1993), indicating that a particular missed post was one the user would expect to see in the News Feed.

Results show that individuals are likely to have missed posts from Friends in their News Feeds regardless of how close the relationship is, after controlling for factors like participants' memories of their Friends' past posting behavior and their perceptions of how recently the Friends were active. Even frequent Facebook users who accurately remembered when Friends' latest posts were created still encountered missed posts, indicating that some of the missed posts participants identified were likely due to the algorithm and not to their own attention and memory. Participants were more surprised about missed posts from close Friends, and from Friends they felt like they saw often in their News Feeds. In addition, believing that the system caused missed posts, as opposed to their own behavior, was related to more surprise.

The infrastructural inversion method used in this study created an opportunity for users to notice aspects of the system's behavior that would have been invisible otherwise. The conditions under which participants experienced surprise reveal that participants believed the system would prioritize posts from close friends, and these beliefs were strongest for those who thought the system took an active role in choosing which posts to display. This study highlights a possible consequence of offloading the work of choosing which posts are attention-worthy onto the algorithm, by identifying a pattern of opportunities for interaction that users did not know they were missing. Because even passive consumption of posts on Facebook can strengthen ties between Friends (Burke and Kraut, 2014), these results suggest that choices the system makes regarding visibility and invisibility of posts could have consequences for real relationships.

2. Related work

2.1. Measuring algorithm effects

Systems like the Facebook News Feed that use personalization algorithms to filter content are conceptually and technically similar to recommender systems, with one important difference. In a social media system, there is no recommendation, and no obvious moment of choice or evaluation by the user. This means that biased performance and impact of a filtering algorithm is difficult to measure. Nguyen et al. (2014) analyzed data from 1405 MovieLens users to look for evidence of a "filter bubble" (Pariser, 2011) effect of the recommendation algorithm over time, operationalized as decreasing diversity in the set of movies either recommended to a user, or rated by the user. They found a reduction in content diversity of the recommended and rated movies over the length of a user's participation in the site, although the change was small. This supports the idea that filter bubbles do exist in some recommender systems. Hosanagar et al. (2014) also looked for evidence of decreasing diversity of music consumption over time in a large dataset from a music recommender system that worked as a plugin to Apple's iTunes software. They found that people who used the plugin became more similar to each other in terms of the artists they listened to, than people who did not use the plugin. Despite this, they

found no evidence of clusters or fragmentation of music interests among the users of the recommendation system, indicating that filter bubbles did not seem to be forming among the system's users. However, it isn't clear whether the effect these researchers identified is because of the algorithm, or because the users themselves narrowed their actual preferences, or some combination of both. In addition, because these studies used log data only, it is not possible to know what the users were thinking when they chose movies to watch or artists to listen to, or how they reacted to the recommendations when they received them.

There are few studies of the effects of the News Feed Algorithm, because outsiders cannot obtain access to the necessary data to conduct this type of research (Lazer, 2015). Eslami et al. (2015) recruited 40 Facebook users for a study that was designed to explore the effects of users' awareness of the algorithm on their satisfaction and preferences for what they wanted to see in their News Feeds. They created a tool that used the Facebook Graph API to access and compare the output of the /user-id/home2 function, which returns the participant's News Feed posts, against the/friend-id/feed3 function, which they used to pull all posts created in the past week by the participant's Friends. However, this illustrates one of the difficulties of trying to operationa-lize the extent of the influence of the algorithm: the Facebook API does not provide information about which posts were actually seen by participants in their News Feeds. In fact, the documentation for Graph API 1.0 used by Eslami et al. for /user-id/home includes a cautionary note: "The posts returned by this API may not be identical to the posts a person would see on facebook.com or in Facebook's mobile apps". To verify whether the posts had appeared or been filtered by the algorithm, Eslami et al. "asked participants if they remembered seeing randomly selected stories". Despite variability introduced by the API, this manipulation was accurate enough to trigger awareness of the presence of the filtering algorithm by showing users missed posts, but not accurate enough to measure and estimate the magnitude of its effects. Studies like this one are unfortunately no longer possible for researchers unaffiliated with Facebook, due to changes to the Graph API that removed the ability to request the read_stream permission necessary to access users' News Feed posts.4

Bakshy et al. (2015) undertook an analysis of a dataset consisting of data from 10.1 million active US Facebook users, to measure the impact of individual choice on what "hard news" links users click on. As employees of Facebook, they had access to information not available to other researchers via the API: a measure of how long posts have displayed on the user's screen. This allowed them to determine which posts users were "exposed" to. They reported that ideological conservatives are exposed to 5% fewer cross-cutting links (links that tend to be shared by people who hold an opposing political ideology) than are actually posted by their Friends, whereas liberals are exposed to 8% fewer. However, this metric cannot be used to determine the extent to which the user noticed and attended to the posts that were displayed, or whether viewing the content had other user-level effects, like attitude change. This illustrates that even with access to system logs and behavioral trace data, it is still challenging to conclusively measure the impact of filtering algorithms.

2.2. Missed posts, memory, and expectations

The only way for a Facebook user to find out via the system about missed posts—those posts the algorithm has assigned a low rank so that they are unlikely to be seen by the user—is to visit each Friend's Timeline individually and look for them (Rader and Gray, 2015). Identification of a missed post requires two kinds of information:

2 https://developers.facebook.com/docs/graph-api/reference/vL0/user/home

3 https://developers.facebook.com/docs/graph-api/reference/vL0/user/feed

4 https://developers.facebook.com/docs/facebook-login/permissions/v2.4

whether or not the post was displayed to the user, and whether the user read the post. At this time, information about post exposure is only available internally to Facebook, or by building client-side software that automatically records which posts are displayed, which violates the Facebook terms of Service.5 Information about whether or not the user read the post depends on the user's memory for posts he or she has seen before, and can be obtained by asking users directly. Asking users to identify missed posts is an approximate measure, likely to be correlated with the truth but also biased in predictable ways. Upon encountering a missed post, users might experience a surprised reaction which would indicate that their expectations for the system's behavior had not been met. Measuring users' reactions to encountering system behavior that does not meet their expectations is one way to characterize the impact of a filtering algorithm.

2.2.1. Memory

People sometimes feel a sense of recognition for items or events that they have not actually seen or experienced. False recognition has been widely studied using cognitive psychology lab experiments, and is more likely when people are shown a false cue that is very similar to the item that was actually seen or studied, than when they see a cue that is different from the real item (Schacter et al., 1998). However, even if people are given an incomplete but accurate cue (e.g., fragments of memorized words (Cleary and Greene, 2004)) people can accurately recognize and distinguish between items they have previously seen and items they have not (Cleary and Greene, 2000; Nomi and Cleary, 2012). There are many properties of both the remembered item itself and the conditions surrounding participants' exposure to the item that affect whether or not the participant recognizes it at all. For example, the emotional valence of the content (Ochsner, 2000) and how deeply the participant processes the information (Gardiner et al., 1996) are both important factors. In general, the more time a person spends processing a piece of information, more likely they are to accurately recognize it (Toglia, 1999); but, the more distracted a person is during the initial exposure, the less able they are to remember the information later (Fernandes and Moscovitch, 2000); this might cause failure to recognize errors. In addition, memory for past events decays and becomes less accurate over time (Koriat et al., 2000). And, the farther into the past something happened, the more uncertain people are about when it happened (Bradburn et al., 1994).

Memory for social media posts is likely to be different than for cues studied in cognitive psychology lab experiments, because the content is more personally relevant to users. This could influence things like emotional valence and depth of processing, and make memory for social media posts more accurate than for cues in the lab. False recognition might be more likely if, for example, a particular Facebook Friend creates posts that are very similar to each other. However, participants in cognitive psychology memory experiments are able to accurately distinguish between falsely similar cues and actual items when they see all of them at once (Guerin et al., 2012). This suggests that seeing a Friend's posts all together in the Timeline would lessen the occurrence of false recognition, as compared to seeing past posts one at a time, out of context and not in reverse chronological order.

There has been little research on memory accuracy for social media posts. Counts et al. (2011) studied recognition memory for tweets in a lab experiment, in which they created a Twitter app which pulled 100 tweets from participants' own feeds and then showed them half of the tweets. In the recall phase of the experiment they presented participants with all 100 tweets in random order, and asked them which tweets they remembered seeing. Participants were 69% accurate at identifying which they had seen before and which they had not. They were also more accurate at recognizing the tweets they rated as

5 https://www.facebook.com/apps/site_scraping_tos_terms.php

interesting, and the ones they spent more time looking at. Unfortunately, this paper did not report the false recognition rate. Based on what is known about recognition memory, it is reasonable to expect that some false recognition and failure to recognize errors will occur when visiting Friends' Timelines, but also that participants should be able to identify some missed posts accurately.

2.2.2. Unmet expectations

The cognitive function of surprise is to help people "develop and maintain accurate representations of the world" (Maguire et al., 2011). Surprise is an automatic mechanism (Schützwohl, 1998) that provides feedback essential for recognizing when expectations do not match reality, so people can learn to better comprehend and predict future events (Teigen and Keren, 2003). Expectations are powerful influences on interaction patterns and behavior, and experiencing surprise is a signal that an expectation has not been met, and that there is something new to be learned about the situation at hand (Burgoon, 1993). Recent work suggests that surprise is initially a slightly negative experience (Noordewier and Breugelmans, 2013), but this reaction is quickly followed by an emotional response that depends on the interpretation of the event in relation to other possible outcomes (Mellers et al., 2013). Surprise is different from other emotions like happiness or sadness, because the same surprising event can bring about positive or negative emotion depending on the situational context (Mellers et al., 2013).

Many users are not aware of the News Feed algorithm (Eslami et al., 2015), which makes it hard to directly ask them about which posts they expect to see in their News Feeds. However, when a user notices a missed post, this presents an opportunity to assess their reaction and thereby indirectly learn about their expectations for system behavior. Previous research has shown that if a Facebook user notices a missed post from a Friend, they frequently react with surprise, which can sometimes lead to frustration, and anger (Eslami et al., 2015, Rader and Gray, 2015). Points of unexpected or surprising behavior like this within a system often occur in conjunction with aspects of the system's structure or behavior that people do not understand (Meadows, 2008). In this case, surprise is an indication that the user's expectations for what the News Feed would show them have not been met.

2.3. Facebook and relationships

Online social networks like Facebook support "pervasive awareness" of one's social ties which preserves a continuing connection between people, even through life events like moving to a new city or graduating from college (Hampton, 2015). Lu and Hampton (2016) found that Facebook status update posts provide visibility and opportunities for interaction with Friends, and creating posts was positively associated with receiving more social support even after controlling for the size and diversity of a user's network. They suggest that for post creators, monitoring the feedback they receive on their posts helps them to know who is paying attention to them on Facebook, and therefore who is available to provide future social support.

Research by Burke and Kraut (2014) also found that communicating with others on Facebook is related to increases in reported relationship closeness, even when controlling for other kinds of non-Facebook communication, such as face-to-face interaction, phone calls, and email. Text-based, written communication on Facebook, such as direct posts, had a larger impact than other kinds of signals such as "Likes". Ellison et al. (2014) conducted a survey which measured Facebook relationship maintenance behaviors, focusing on participants' responses to posts from their Friends sharing good or bad news, or asking for advice. Engaging in more relationship maintenance behaviors was positively related to increased bridging and bonding social capital. Other recent research has also found that more frequent communication on Facebook, as well as the use of more kinds of Facebook communication modes, predicts "relational escalation," or an

increase in tie strength (Sosik and Bazarova, 2014). And Eslami et al. (2015) reported that after being made aware of particular missed posts, some participants indicated that the post would have triggered a supportive response, had they seen it.

People's perceptions and memories of interactions with members of their social networks are important aspects relationship maintenance, because they constrain their awareness of others and their understanding of the social environment (Brashears and Quintane, 2015). A filtering algorithm that ranks posts for display provides a solution to problems of information overload, but may also constrain the user's active, visible social network by affecting which posts are available for feedback and comments from other users. This happens for users as consumers of posts, in that if they do not see posts from certain friends they miss opportunities to stay informed about those Friends' lives and to engage with their posts. It also affects users as producers of their own posts, because they will not receive social support from people who do not see the posts they create. In other words, an algorithm that ranks posts for display might create opportunities for giving and receiving social support between certain people while systematically removing those opportunities from others.

2.4. Research questions

In most systems that use algorithmic filtering, users are not aware of the bounds of the set of possible items that might be presented to them so they do not have a way to know what items they're not seeing. However, Facebook users typically have an existing relationship with the people they are Friends with (Subrahmanyam et al., 2008) and may have a general sense of what and who should be represented in their News Feeds. This study triggered an infrastructural inversion by exposing participants to potential invisible effects of the News Feed Algorithm: Friends' posts that they may have missed in their News Feeds. This made hidden aspects of the filtering work done by the algorithm visible to users, in order to identify patterns in the circumstances under which such missed posts are evident, and unexpected. Missed posts are a socio-technical outcome that arise from the combined influences of user and algorithm behavior, and participants' reactions to the infrastructural inversion may illustrate areas in which the system is biased. In addition, because posts from Friends are opportunities for relationship maintenance that are important for feedback and social support, there could be implications for users' real-world relationships. To investigate this possibility, I asked the following research questions:

1. How is relationship closeness related to the likelihood of noticing a missed post on a Facebook Friend's Timeline? Because people use Facebook to maintain relationships, they should be more likely to notice missed posts from those whom they feel closer to, and from people who are in general more visible or noticeable to them in their News Feeds. Analyzing the characteristics of the set of missed posts is one way to identify possible algorithm effects and biased system behavior.

2. Is encountering a missed post more surprising from Facebook Friends that users feel closer to? Missed posts from closer, more visible Friends should be more surprising than posts from distant acquaintances that users do not often see in their News Feeds. Patterns of surprised reactions to being exposed to invisible aspects of system behavior may be a symptom of unmet expectations and system-wide bias.

3. Method

3.1. The survey

I used the Qualtrics platform to conduct a survey in April 2014 that asked participants questions about themselves and their experiences on

Facebook, their relationships with eight of their Facebook Friends, and general questions about those Friends' Facebook activity. The survey required participants to select four Facebook Friends they felt closest to6; another four Friends were automatically selected at random from each participant's Friend List. Because people typically have many more weak ties in their social networks than strong ties (Roberts et al., 2009), I explicitly sampled for close Friends to ensure that relationships of this type were included in the survey.

After consenting to the study, participants were prompted to log in to Facebook (if they were not logged in already) and authorize an app developed for the survey. The app used Facebook's Graph API v1.0 to access the names, profile photos and user ids of the participant's Facebook Friends. This data was all classified in v1.0 of the API as "basic information" available by default once the user authorized the app. Using only this data was an intentional research design choice; if the app asked for additional permissions, this would increase the dropout rate and exacerbate selection effects in ways that would negatively impact the sampling frame. The names and profile photos of all eight selected Friends, as well as links to the Friends' Timelines, were incorporated into questions in the survey. All Friend information was discarded automatically when each participant finished the survey, and participants were provided with instructions for de-authorizing the app upon completion.

The survey started by displaying a "Friend selector" that allowed participants to scroll through the names and profile photos of all of their Facebook Friends, and select the initial four close Friends. They were instructed not to select Friends under the age of 18 (per the IRB), anyone they would feel uncomfortable answering questions about (participants were allowed to opt out of visiting particular Timelines that might cause them distress), and any "Friends" that were not actually people (e.g. pets, organizations, etc.). After selecting close Friends, the Friend selector appeared again, this time populated with 20 people randomly selected from the participant's Friend list. The 20 randomly selected Friends did not include the close Friends they had already selected. Participants de-selected any of these Friends per the instructions described above, and then four of the randomly selected Friends were retained by survey. They were not allowed to proceed with the survey until they had completed the Friend selection process. After Friend selection, the survey ordered all eight Friends randomly, and the same random Friend order was used throughout the survey.

Participants first answered questions about what they recalled about each Friend's recent activity on Facebook, how close they felt to each Friend, and how often in the past month they had used various media to communicate with each Friend. Then, they were asked to visit each Friend's Facebook Timeline via a customized profile link for each Friend, generated by the Facebook app and embedded in the survey question along with a clickable thumbnail of the Friend's profile photo. They were instructed to skim the posts each Friend made just within the past week, and answer questions about each Friends' actual posting behaviors. The survey specifically instructed participants to: "Scroll down through (Friend Name)'s Facebook Timeline until you get to (Date), which is one week into the past. As you scroll, pay attention to the dates of the posts, and the different kinds of posts (status, photo, etc.) that (Friend Name) created in the past week". The date was automatically customized by the survey for each participant to be one week prior to when the participant started the survey, and inserted into the question. This was intended to constrain participants' responses to a timeframe they would reasonably be able to remember. Because this was an online survey, direct evidence that each participant visited all eight Timelines was not available; however, if more participants in the dataset were cheaters than honest participants, the data would be too noisy to see any results.

After visiting each Friend's Timeline, participants were asked

6 The instructions were based on the survey in Burke and Kraut (2014).

questions that make up the two main dependent variables in this study, about whether they had noticed a missed post, and if they had, how surprised they felt by it. The final section of the survey asked about participants' own Facebook activity, their beliefs and inferences about how the News Feed selects content to display (including whether they are aware of the News Feed algorithm), and demographics.

3.2. Participants

Participants were recruited from a panel provided by Qualtrics. Eligible participants lived in the United States, were 18 or older, had more than 20 Facebook Friends, and reported visiting Facebook once per week or more often. The sample included an age quota: 30% of participants had to be older than 50. 1576 participants started the survey; 530 completed it. Participants could complete the survey using multiple platforms, including mobile devices. Participants received an incentive specified by Qualtrics equivalent to $1.50 for completing the entire survey. The median completion time was 23 minutes (M=35, SD=106).

To ensure data quality, potential participants who answered attention-check questions incorrectly were not allowed to complete the survey. Of those who completed the survey, I excluded cases that reported "Good" or "Full" familiarity with a fake word that was part of the Internet Literacy index variable.71 also used an edit distance metric to identify cases that used the same response for many questions (e.g., choosing the middle category for every question), and excluded cases with low edit distance scores across multiple sections of the survey.

An issue with the way the Qualtrics survey platform's client-side JavaScript works in Internet Explorer resulted in some otherwise eligible participants being unable to complete the survey correctly. These cases were excluded from the analysis. Finally, three participants selected four Friends as "close", but later in the survey reported that they did not remember who the Friend was. These Friends were excluded from the analysis, but the remaining Friends for these three participants were retained. The final number of participants was 410, which is 26% of those who began the survey.

Online survey panels from Qualtrics tend to be more female than male, and this sample was no exception. The sample had 260 women and 149 men, with one person reporting "other". Participants were mostly white, with a median age range of 35-50. A majority reported some education after high school, and 55% of the sample reported that they had attended or graduated from college. The median participant reported visiting Facebook several times per day, posting less than once per week, and having 300 or fewer Facebook Friends. The mean Internet Literacy score was 2.64 out of 5 (SD=0.87).

The sample resembles the population of US Facebook users in several ways. According to a report released in January 2015 by the Pew Research Center, the majority of American Facebook users visit the site at least once a day (70%) with at least 45% checking the site several times per day (Pew Research Center, 2014). Also, an analysis of Nielsen data from a nationally representative US household audience panel collected in March 2011 that involved measured computer usage (not self-report) found that Facebook users are more likely than nonusers to be female, young (13-17 years old) , white, and to have at least a high school diploma (Wells and Link, 2014) . This sample is a reasonable approximation of these characteristics, with the caveat that users younger than 18 were ineligible for the survey.

3.3. Measures

As with any survey, the measures described below are self-report.

7 The questions that comprise the Internet Literacy variable are based on the Web Use Skills survey reported in Hargittai and Hsieh (2011). It consists of the average of participants' assessments of their familiarity with a list of Internet-related terms (Cronbach's a = 0.80).

Some questions asked about participants' perceptions and reactions, which cannot be objectively measured. Other questions asked them to retrospectively estimate the frequency of their own past actions or experiences, or to report what they remembered about the Facebook activity of their Friends.

3.3.1. Noticing a missed post

A user who reads a Facebook Friend's Timeline to identify whether there are any past posts that he or she did not see in the News Feed is engaged in a recognition task. A recognition task was included in the survey to measure the prevalence of missed posts, and characterize aspects of the circumstances in which they occur. The dependent variable notice.missed is a Friend-level variable, meaning that participants answered the same question once for each Friend: "When you were scrolling through (Friend Name)'s Timeline, did you notice posts he or she created that you don't remember seeing in your News Feed? [No, Yes]". Seventy-six percent of participants (311 of 410) had at least one Friend for whom they noticed a missed post; 99 participants reported no missed posts, 120 reported one Friend with a missed post, and 14 participants reported missed posts from all 8 Friends. The frequency histogram of number of Friends with at least one missed post, ranging from 0 (none) to 8 (all) can be found in Fig. 1.

Each participant was asked, "Approximately how many posts did you see on (Friend Name)'s Timeline that you don't recall seeing in your News Feed?" [1 post, 2-3 posts, 4-5 posts, More than 5 posts]. This is a Friend-level measure that asked participants to estimate how many missed posts they saw, and choose a categorical response. Out of all 3276 Friends included in the survey, participants noticed at least one missed post for 871 of them (27%). Looking at just the Friends for whom participants noticed missed posts, for 25% participants reported only one missed post, 44% estimated they had seen 2-3 missed posts, 14% had 4-5, and participants reported more than 5 missed posts for 16% of the Friends for whom there were missed posts.

3.3.2. Surprise about missed posts

If a participant reported that she had missed a post on a Friend's Timeline, the survey presented a question asking her to indicate how surprised she was about this. The question was, "I am surprised that I did not see (Friend Name)'s post(s) in my News Feed". Responses on the dependent variable surprise ranged from Strongly Disagree (1) to Strongly Agree (7). The mean level of surprise was 4.43 (SD=1.7). One hundred ninety-two participants (47%) reported Somewhat Agree or higher for at least one Friend.

The survey question did not ask participants to assign an emotional valence to surprise (e.g., positive or negative). The experience of surprise is evidence of a reaction from each participant about whether, for this particular relationship, the system behavior has met their expectations. Whether a single missed post is a net positive or negative experience would be different for each user based on their own unique circumstances. However, independent of valence, a relationship between participants' surprise about noticing a missed post on a Friend's Timeline and other variables, like relationship closeness, identification of patterns across users in their expectations about system behavior.

3.3.3. Relationship closeness

The survey measured relationship closeness using the "Inclusion of Other in the Self Scale" (Aron et al., 1992), which is a 7-point scale measuring emotional closeness that uses images of circles that do not overlap on the low end (coded as 1) and move closer to each other until they almost completely overlap on the high end (coded as 7). The mean closeness for the "close" Friends participants had specified via the Facebook app at the beginning of the survey was 4.57 (SD=2.11), and for the "randomly selected" Friends was 2.31 (SD=1.76).

The sample may over-represent the proportion of closer relationships to distant ones because the distribution of Friend closeness was not sampled to be representative of the proportion of close and distant

1 2 3 4 5 6 7

Number of Friends with Missed Posts

Fig. 1. Bar chart showing the count of participants who experienced at least one missed post from 0 to 8 Friends when they visited their Timelines.

relationships for the participants. However, participants answered questions about fewer close than distant relationships which is an approximation in the same direction as these effects in the real world. In the sample, the median number of high closeness Friends (5 or higher on the relational closeness scale) was 3 Friends out of the 8 they answered questions about.

3.3.4. Frequency of communication on facebook

A survey question from Burke and Kraut (2014) was included to measure communication frequency between the participant and each of their Friends selected for the survey, using four different channels: "Over the PAST MONTH, about how often have you and (Friend Name)... Talked in person, Talked on the phone (including calls and texts), Talked online or by email (NOT including Facebook), Talked on Facebook?" [Not at all, Once, A few times, Several times, Daily or almost daily].

This question was used to assess whether frequency of recent communication with each Friend was related to the likelihood of noticing missed posts, or surprise.

Previous research has found that stronger social network ties use more different types of media to communicate than weaker ties (Haythornthwaite, 2005), and if participants used Facebook less often than other channels for closer relationships, this could have an impact on how the News Feed Algorithm prioritizes posts from those Friends. If close Friends do not interact on Facebook, the system could infer that they are weak ties rather than strong ties.

Fig. 2 displays communication frequency in the past month broken down by whether the Friend was selected by the participant (close) or chosen at random by the Facebook app, and communication channel. Communication frequency via all channels, including Facebook, is positively correlated with relationship closeness.8 This means that close Friends do not communicate less on Facebook than they do using other channels. The data also confirm that closer Friends communicate through more channels; the bars in Fig. 2 are all around the same height for the close Friends.

< 4—

o -t—' tz a) o

Not at All

A Few Times

Several Times

8 Pearson's r=0.53 (In Person), 0.63 (Phone), 0.42 (Online), 0.43 (Facebook); p < 0.001 for all four correlations.

Communication Frequency in the Past Month

Fig. 2. Percent of all Friends participants reported talked with in the past month via four channels: Facebook, Online or by Email, Phone (including calls and texts), and In Person.

Talking on Facebook is the only channel included in the survey that would generate metadata that the system could incorporate into the News Feed algorithm, providing relationship-level information that might impact post ranking; therefore, Facebook communication frequency was included as a variable in the analyses. In the sample, 88% of participants (359) had talked on Facebook a few times or more in the last month with at least one Friend they were asked about in the study. Overall, this was 47% percent of Friends (1540).

3.3.5. Recall accuracy

The accuracy of participants' recall about their Facebook Friends' posting activity could be a source of variability in their responses about whether or not they noticed or were surprised by missed posts. For example, if a participant's recall for a particular Friend's behavior is inaccurate, it is reasonable to expect that her judgment about whether the Friend's posts had appeared in her News Feed would be less reliable than if she had remembered her Friend's posting behavior accurately.

To assess the accuracy of participants' recall for details about the Friend's posting activity, the survey asked participants before visiting each Friend's Timeline to recall how long ago they thought each Friend had created a post: "How recently would you estimate was the last time (Friend Name) posted on Facebook, without looking at his or her Timeline?" Then, after visiting each Friend's Timeline, they reported how recently each Friend had actually created a post: "How recently did (Friend Name) create a post? Please note: this does not include instances when other friends posted on (Friend Name)'s Timeline; only when (Friend Name) him or herselfposted or instances in which (Friend Name) changed their profile or cover photo".

The responses to both questions were categorical, and asked participants about relative time intervals [Today, yesterday, within the past week but not as recently as yesterday, not at all within the past week]. The recall accuracy measure was calculated by taking the difference between the before and after responses for each Friend, and centering them at the median. I then coded the differences as a categorical variable with two levels: "Yes", meaning that there was a

match between participant's memory and when the Friend was most recently active and therefore the participant's recall was accurate (62% of cases); or "No", the answers to these questions were not the same, and therefore the participant's recall was inaccurate (38% of cases). This measure is used in the analyses as a control for memory accuracy.

3.3.6. Recency of friend activity

The survey instructions directed participants to visit each Friend's Timeline and scroll back one week into the past. It is possible that some of the Friends that participants were asked about had not created any posts in the past week. If so, it is reasonable to expect that if participants followed instructions they would not have noticed any posts created in the past week from inactive Friends. To account for this variability, I used the responses to the question about how recently the Friend had created a post, asked after participants visited their Friends' Timelines, to create a binary variable representing whether or not the Friend had created a post in the past week. Responses were coded as "No" if the participant reported that the Friend's most recent activity was "Not at all within the past week", and "Yes" otherwise. Sixty-seven percent of all Friends in the study (2197) were active, and 407 participants (99%) reported having at least one Friend that had created post(s) in the week before they took part in the study.

3.3.7. Friend visibility

Posts from some Friends may appear more often in a user's News Feed than posts from others. This could represent a prioritization of a particular Friend's posts by the News Feed Algorithm, or that a particular Friend simply creates more posts. Both of these explanations could result in some Friends appearing more often—being more visible—in a user's News Feed than their other Friends. Regardless of the cause, the more a user sees a particular person represented in his or her News Feed, the more the user might believe they are seeing most or all of the Friend's posts. I anticipated that for more frequently-seen, high visibility Friends, missed posts might be more noticeable and also more surprising.

However, participants' Friends were not participants in the study themselves, and the survey did not have permission to access their posting histories for a direct measure of their posting frequency. Although this information could have been accessed via the Facebook API, the survey did not do so because it would have required additional permissions beyond "basic information," which would have discouraged privacy-sensitive users from participating. I also determined through experimentation with the Facebook API that it was not possible to re-create an accurate representation of which posts had actually displayed in a user's News Feed over the past week, or the precise order in which they appeared as determined by the News Feed Algorithm. So it was not possible to construct an objective measure of how often posts from a particular Friend appeared in participants' News Feeds.

For more frequently-seen, high salience Friends, missed posts might be more noticeable and also more surprising. Therefore, to measure the overall visibility of each Friend in a participant's News Feed relative to the other Friends, participants were asked the following question: "Think about the top 5 people whose activity you see the most in your news feed during an average week. To what extent do you consider (Friend's Name) one of those top 5? [Definitely Not (1) - Definitely Yes (5)]". Seventy-six percent of participants answered this question "Probably Yes" or "Definitely Yes" for at least one Friend that they were asked about (25% of Friends). This self-reported measure is a proxy for Friends' overall News Feed visibility as it is experienced by the participant, and captures the participants' subjective impression of how much they see a particular Friend in their News Feed in relation to their other Friends.

3.3.8. Facebook activity, number of friends, and demographics

Three variables describe participants' self-reported level of activity

on Facebook in terms of how often they visit, post, and how many Facebook Friends they have. These variables are important controls. Users who visit Facebook multiple times per day are likely to have seen more different Facebook posts than those who visit only about once per week, and users who post to Facebook often may attend to the content of others' posts differently than those users who do not create posts. Visit frequency was measured with the question, "How often do you usually VISIT Facebook?" Participants were also asked, "How often do you usually POST to Facebook?" Two additional variables are demographic controls in the analyses: internet literacy and participant age. Age is particularly important as a control for memory performance differences across the lifespan (Yonelinas, 2002).

4. Results

I used two multi-level regression analyses to answer questions about how important closeness with a Friend is for whether or not a participant noticed a missed post when visiting the Timeline of a Facebook Friend (notice.missed), and how much the missed post surprised him or her (surprise). Both models use the same predictors, including self-report measures controlling for subject-level differences like individual Facebook activity and demographics, and Friend-level differences like frequency of Facebook communication between the participant and the Friend, and visibility of the Friend in the participant's News Feed. The subject-level questions were answered once by each participant, and the Friend-level measures were answered eight times per participant (once for each Friend). Because the dataset consists of multiple responses from each participant, both models include a random effects control for participant. Both models are specified as follows; (F) denotes a Friend-level variable, and (P) denotes a participant-level variable:

DV = f(closeness(F) X talked. facebook(F),

accurate. recall(F) X active. recently(F), friend. visibility(F),

visit. freq(P), post. freq(P), howmany. friends(P), internet. literacy(P),

age(P), randomeffect(P))

4.1. Closeness did not matter for missed posts

I expected that participants would be less likely to notice missed posts from close Friends, because the News Feed Algorithm prioritizes posts it predicts users will want to see, and because participants should be most interested in posts from people they feel close to. I used a mixed-effects binary logistic model to identify effects related to the likelihood of noticing a missed post when visiting the Timelines of specific Friends. I compared this model with a null model containing only the intercept and random effects term (x2(2, 14) = 242.72, p < 0.001). This is similar to the F-test in an OLS regression, and statistical significance indicates that the more fully specified model does a better job of explaining the data than the null model. The regression results are presented in Table 1 under notice.-missed.

The intercept (-2.33, SE=0.24) represents the log odds of noticing a missed post from a Friend who was not active on Facebook in the past week and whose activity the participant does not remember accurately, with closeness, frequency of communication on Facebook in the last month, Facebook visit and post frequency, age, and number of Facebook Friends centered at the median, and internet literacy centered at the mean. In other words, the model predicts that there is a 9% probability that a participant with 101-300 Friends who is 3550 years old and visits Facebook a few times per day and posts a few times per week would notice a missed post from a Friend that has not created a post in the past week, when the participant is not that close to the person and doesn't remember the Friend's past posting activity very clearly.

Table 1

Mixed effects regression results for notice.missed and surprise, using data from the Qualtrics sample. Closeness, frequency of communication on Facebook in the last month, Friend visibility, and participant Facebook visit and post frequency and number of Friends are centered at the median; internet literacy is centered at the mean. OR=Odds Ratio. (F)=Friend level variable. R2 values were calculated based on Nakagawa and Schielzeth (2012) and Johnson (2014).

Model Term DV=notice.missed DV= surprise Variable Info

No=2405 M=4.43

Yes=871 SD=1.7

Coef. SE OR Coef. SE

Intercept -2.325*** 0.236 0.098 4.414*** 0.206

closeness (F) -0.033 0.032 0.968 0.085** 0.027 Median=3, Range=1-7

talked on Facebook (F) 0.060 0.059 1.061 0.080 0.050 Median="Once" (in past month)

accurate recall: Yes (F) -0.784** 0.254 0.457 -0.517* 0.233 all=2037, missed=471

active recently: Yes (F) 1.300*** 0.228 3.658 -0.102 0.198 all=2197, missed=739

friend visibility (F) 0.029 0.053 1.030 0.113* 0.044 Median="Probably Not"

FB visit frequency -0.325* 0.158 0.722 -0.244^ 0.131 Median="Several times per day"

FB post frequency -0.021 0.070 0.980 0.164** 0.060 Median="A few times per week"

# of Facebook Friends -0.023 0.010 0.977 0.037 0.083 Median="101-300 Friends"

internet literacy 0.230* 0.112 1.253 -0.054 0.095 Mean=2.64, SD=0.87

age -0.115 0.083 0.891 0.021 0.071 Median="35-50"

closeness * talked on FB 0.008 0.019 1.008 -0.032* 0.016

recall * recent activity 0.563^ 0.292 1.756 -0.464^ 0.260

Marginal R2 0.13 0.07

Conditional R2 0.51 0.61

SD of Random Effect (1.60) (1.22)

Dataset All Friends Only Missed Posts

Total N 3271 Friends 870 Friends

•p < 0.10; *p < 0.05; **p < 0.01; ***p < 0.001.

In the model, closeness had a slightly negative relationship with the likelihood of noticing a missed post, but was not statistically significant (OR=0.968, p=0.30; an odds ratio of 1 means the likelihood of both outcomes is the same). However, two other measures in the model had statistically significant relationships with the likelihood of noticing a missed post: whether the participant's recall was accurate, and if the Friend had posted on Facebook in the past week. Participants were more likely to notice missed posts from Friends who were active in the past week (OR=3.66, p < 0.001), and less likely to notice them when their memory was accurate (OR=0.46, p < 0.01).

Two variables that might have an impact on noticing a missed post on a Friend's Timeline both had very small effect sizes and were not statistically significant: how often the participant reported he or she had talked on Facebook with the Friend in the past month (OR=1.06, p=0.31), and the perceived visibility of that Friend in the participant's News Feed (OR=1.03, p=0.58). I included an interaction between closeness and frequency of communication on Facebook in the model, because previous research has shown that communicating on Facebook with strong ties has less of an impact on tie strength than communicating with weak ties (Burke and Kraut, 2014). This suggests that the effect of Facebook communication frequency on the likelihood of noticing a missed post could be different at different levels of closeness. However, this was not the case in the data: the effect size of the interaction term in the model was also very small and was not statistically significant (OR=1.008, p=0.66).

I also included an interaction in the model between whether the participant's recall was accurate, and whether the Friend had created a post in the past week. If the Friend had not been active, there wouldn't be any posts for the participant to have missed, regardless of what he or she remembered. The effect size for this interaction was positive but not very large (OR=1.76, p=0.05), and not statistically significant, so this is evidence that there is no reliable interaction between these two variables. Two control variables were, however, statistically significant: Internet literacy (OR=1.25, p < 0.05) and how often the participant visits Facebook (OR=0.72, <0.05). This means that participants who were more Internet-savvy were more likely to notice missed posts, and

who reported visiting Facebook more often were less likely to notice missed posts. This could illustrate an overall effect of time spent online.

The pattern of the results is easiest to understand as the likelihood of a missed post being noticed based on different combinations of values of the predictors in the model, illustrated in Fig. 3. The bar chart illustrates the impact in the model of closeness, recall accuracy and whether the Friend posted in the past week, for participants who talked on Facebook with their Friend frequently in the past month and for whom their Friend is highly visible their News Feed. (The remaining controls in the regression are held at their median or mean; see Table 1 for those values.) There is little difference in the likelihood of noticing a missed post between the levels of closeness (the three bars); however, participants were much more likely to notice a missed post when their

■ Low Closeness Median Closeness! High Closeness Recall: Inaccurate Recall: Accurate

No Yes No Yes

Friend Posted in the Past Week?

Fig. 3. Predicted probability of noticing a missed post by whether the participant was able to accuracy recall the Friend's past posting behavior, and whether the Friend had posted to Facebook in the past week.

Friend had created posts in the past week (Yes on the x-axis).

In summary, the most important factors in terms of both effect size and statistical significance for predicting whether a participant would notice a missed post were how recently the Friend was active on Facebook and how well the participant remembered the Friend's activity. Consider a participant with accurate recall for the posting behavior of a Friend she is not very close to, but who she has talked with on Facebook a few times in the past month. If that Friend has not created a post in the past week, the model indicates there is a 4% chance that upon visiting that Friend's Timeline she would encounter a missed post. However, if the Friend is someone she talks to daily on Facebook and who has created a post in the past week, the probability is 27%—a greater than one in four chance of noticing a missed post.

I had expected the data to show evidence that participants are more aware of—in essence, that they "keep up with"—more of the posts created by Friends they feel closer to. This would happen if the News Feed algorithm prioritizes closer Friends' posts, and would manifest in the data as noticing fewer missed posts from closer Friends. But the results suggest that this is not the case. After controlling for participants' recall accuracy, characteristics of the Friend's posting behavior such as recency and visibility in the participant's News Feed, and the participant's own Facebook use, closeness had no impact on the likelihood of noticing a missed post. Noticing a missed post happens equally across Facebook users' close and distant relationships, indicating that the algorithm may not be prioritizing close Friends' posts for display.

4.2. Greater closeness is related to more surprise

Recent studies of user reactions to algorithmic curation in the Facebook News Feed have found that noticing a missed post can be an unpleasant surprise for Facebook users (Rader and Gray, 2015; Eslami et al., 2015). I expected that it would be more surprising for a participant to find a post she had missed on a close Friend's Timeline than a distant acquaintance's, because people use Facebook to stay in touch with and feel connected to others they care about (Joinson, 2008). I used a second mixed-effects regression with the same predictors as the previous model, on a subset of the data that included only the Friends for whom each participant had noticed at least one missed post. The dependent variable is how surprised participants were when they noticed a missed post on a Friend's Timeline. I compared this model with a null model containing only the intercept and random effects term (x2(3, 15) = 68.64, p < 0.001); statistical significance means the more fully specified model does a better job of explaining the data than the null model. The regression results are presented in Table 1 under surprise.

The intercept in this model is 4.41 (SE=0.21, 4.0 is "Neutral"), and represents the level of surprise experienced by a participant about a missed post from a Friend who had not created a post in the past week, whose activity the participant did not remember accurately, with frequency of communication on Facebook in the past month and the two Facebook activity variables (participant visit and post frequency, number of Facebook Friends) held at their median, and internet literacy (mean), and age (median) also centered. In other words, the model predicts that a participant with 101-300 Friends who is 3550 years old and visits Facebook a few times per day and posts a few times per week would be slightly surprised to notice a missed post on the Timeline of a Friend that has not created a post in the past week, when the participant is not that close to the person and doesn't remember the Friend's past posting activity very accurately.

As expected, closeness was positively associated with surprise. Surprise increased by 0.09 for each level of closeness (p < 0.01). Participants also expressed more surprise about missed posts from Friends that were more prominent in their News Feeds (coef =0.11, p <0.01). Both of these results are in contrast to the notice.missed model, where these predictors were very small and not

statistically significant.

How often the participant and the Friend talked on Facebook in the past month, as reported by the participant, also had a positive relationship with surprise, although it was not statistically significant (coef=0.08, p=0.11). However, I again included an interaction between closeness and frequency of communication on Facebook in the model, and this time it was statistically significant (coef = - 0.03, p < 0.05). This means that talking with a Friend on Facebook more times in the past month has a different effect on surprise, depending on the levels of closeness. For low closeness relationships, more frequent communication on Facebook increased surprise about missed posts; but for high closeness, it decreased the amount of surprise. Nevertheless, despite the interaction, surprise was highest overall for high-closeness relationships.

Recall accuracy (coef = - 0.52, p < 0.05) is an important control in this model, like the notice.missed model. The effect is negative and large compared with the other predictors: accurate recall meant a half point less surprise. Unlike the notice.missed model, whether or not a Friend had created a post in the past week was not statistically significant (coef =-0.10, p=0.60). This is likely because 85% of the Friends for whom participants said they had noticed a missed post had been active on Facebook in the past week. I also included the interaction between recall accuracy and recent activity, which was not statistically significant in this model either (cof=0.47, p=0.07).

The low end of the range of predicted values is 3.46 out of 7, for a participant who is not close to a particular Friend that he has not talked with on Facebook in the last month or seen any posts from recently, with accurate participant memory, when the Friend has not posted in the past week. This means that a missed post from a low closeness relationship with a Friend that is not very active on Facebook would not be very surprising. Predicted surprise increases to 4.68, more surprised than not, for a Friend that the participant is very close to but that otherwise has the same characteristics as the previous example. On the high end of the predicted values for surprise is 4.98, when the participant is very close to a Friend that was recently active and is typically very visible in the participant's News Feed, but the participant has not talked with them on Facebook lately. Greater closeness is associated with more surprise, even when past interactions are infrequent and the Friend is not visible in the participant's News Feed.

The predicted values generated from the model and displayed in Fig. 4 illustrate these patterns. The graph presents three factors: frequency of communication on Facebook (x-axis), Friend visibility (left and right panels), and closeness (lines). To generate these values,

1234512345 Talked on FB in the last month (1=Not at All, 5=Daily)

Fig. 4. Predicted value of participant surprise about a missed post, by how often the participant talked with the Friend on Facebook in the last month, Friend visibility, and relationship closeness.

Surprised: No (241) Surprised: Neutral (175) Surprised: Yes (455)

4 5 5 7

4 1 13 11 1

5 4 18 4 2 11 3 3 2 I 16 23 9 3

2 12 5 2

4 14 9 2

10 3 4 1 2

2 5 7 1 1

60 19 19 6 3

8 3 15 8 31

5 9 15

12 2 2 5 5 5 2

■ 44 44 31 14

Talked on Facebook (a) Surprise by communication Face to Face and via Facebook.

Percent

110.0 75

Surprised: No (241) Surprised: Neutral (175) Surprised: Yes (455)

o o es

cu o CO LL ■o

1 2 18

2 11 15 2

4 3 15 11

12 3 2 2

119 9 3 7

2 5 14

1 2 6 7 4

7 3 7 2 1

6 5 4 1

90 5 7 4 1

2 3 9 15 36 6 4 18 41 11 12 7 28 12 3

3 9 3 3 1

H 12 18 18 3

Percent

■ is 10 5

Talked on the Phone/SMS (b) Surprise by communication Face to Face and via Phone/SMS.

Fig. 5. Heatmaps with three facets for surprise: No (values 1-3), Neutral (values=4), and Yes (values 5-7). The x- and y-axes of each facet represent increasing frequency of communication in the past month via the given channel, ranging from Never to Daily. For example, in (a), there are 95 cases in the dataset where the participant Never talks to the Friend Face to Face or directly on Facebook, but was surprised by a missed post. (a) Surprise by communication Face to Face and via Facebook. (b) Surprise by communication Face to Face and via Phone/SMS.

all other variables in the model were held at their median or mean. The closeness x Facebook communication frequency interaction is clearly visible in the graph. For low and median closeness, more frequent communication on Facebook was associated with more surprise at noticing a missed post. But, for high closeness Friends, there was actually slightly less surprise about missed posts as frequency of communication increased.

A large number of the surprising missed posts in this study came from Friends that participants reported they had not recently communicated with via any channel. Fig. 5a and b present two heatmaps with three facets for surprise: No (values 1-3), Neutral (4), and Yes (5-7). Each facet shows a comparison between two channels of communication. The x- and y-axis of each facet show increasing frequency of communication in the past month via the given channel, ranging from "Never" to "Daily". For example, there are 177 cases in the dataset where the participant "Never" talks to the Friend Face to Face or via Phone or SMS, and was surprised by a missed post. This means that Facebook could be a primary channel of communication for some participants and their Friends, and for these relationships missed posts might have greater impact than for those who communicate via multiple channels. Relationships that are primarily maintained on Facebook and through no other channels would be most sensitive to being made invisible by the algorithm.

4.3. No effect for algorithm awareness

Users who were aware of the News Feed algorithm might be more likely to notice missed posts, and also less surprised by them, than users who were not. Therefore, in addition to the above results, I also ran both the notice.missed model and the surprise model with an additional variable controlling for participants' awareness of the News Feed Algorithm. At the end of the survey, right before the demo-

graphics questions, participants were asked the following: "Do you feel like Facebook uses computer programs or algorithms to automatically choose what stories to show you in your News Feed?" [No (61 participants), Maybe (203), Yes (146)].

I compared the models with the algorithm awareness variable against the models reported above. In both cases, there was no statistical difference between them (notice.missed: x2(14, 16) = 3.50, p=0.17; surprise: x2(15, 17) = 2.03, p=0.362). This means that the additional variable does not change the results, and awareness of the algorithm was not meaningfully related to the likelihood of noticing a missed post on a Friend's Timeline or the experience of surprise about missed posts.

4.4. Causal beliefs about missed posts

There is some evidence from Rader and Gray (2015) and Eslami et al. (2015) that Facebook users would not be surprised by posts they believe they missed because of their own actions, such as not visiting Facebook often enough, or because they were skimming and not fully reading every post, or did not scroll down far enough to see the missed post. However, the survey discussed thus far did not include questions about participants' causal beliefs regarding why they had missed particular posts. To address this, I conducted a second survey, also in April 2014. The two surveys will be referred to from now on as the "initial survey" and the "follow-up survey".9

4.4.1. Method, participants and measures

The follow-up survey instrument was very similar to the initial survey, with a few differences. A set of questions were added that

Rader and Gray (2015) reported on two questions from the follow-up survey; however, those two questions Rader and Gray (2015) are not included in this paper.

displayed only if participants reported noticing a missed post, which asked, "Below are possible reasons why you might have missed post(s) on Facebook from (Friend Name). Please indicate your level of agreement from Strongly Disagree (1) to Strongly Agree (5)". The closed-ended items were developed after summarizing responses to an exploratory open-ended question included in the initial survey, about why participants believed they had missed posts. The new items were presented in random order for each missed post a participant reported, and included:

1. I don't spend a lot of time going through my News Feed

2. I don't scroll down to see older posts in my News Feed

3. I don't always read every post when I browse my News Feed

4. I do not interact (comment, like, share) with (Friend Name)'s posts very often

5. Facebook must think I don't want to see (Friend Name)'s posts

6. (Friend Name) is not popular enough on Facebook for me to see all of his or her posts

7. Facebook thinks (Friend Name) and I are not good friends

Items 1-3 above were combined into a composite variable representing participants' beliefs about how their own behaviors can cause them to miss posts (user.beliefs, M=2.61, SD=1.01, Cronbach's a = 0.80), and items 4-7 into a second composite variable representing beliefs about how the filtering algorithm can cause missed posts to occur (filter.beliefs, M=2.90, SD=0.89, Cronbach's a = 0.72). There was no meaningful correlation between the two variables (Pearson's r = - 0.06, p < 0.05). The scale of the surprise dependent variable question was modified to use the same 5-point agreement scale described above. And, participants in the follow-up survey were shown 5 close Friends and 5 randomly selected Friends, instead of 4 of each as in the initial survey. These additions made the survey significantly longer, so the questions about communication frequency via different media with each Friend and Friend visibility in the News Feed were removed from the follow-up survey, to keep the total completion time about the same in both surveys. The median completion time for the follow-up survey was 19 min (M=29, SD=95). Table 2 shows a comparison of the items included in the two surveys.

Participants for the follow-up survey were recruited using Amazon Mechanical Turk, and were required to meet the same recruiting criteria as the initial survey, with two differences: only workers from the USA who had a 90% or higher approval rating after completing at least 500 tasks were eligible, and there was no age quota. Participants received $5 for completing the entire survey. A total of 505 respondents finished the follow-up survey, and after following the same data cleaning procedure as the initial survey, 464 remained in the dataset. There were some demographic differences between the Qualtrics panel and the Amazon Mechanical Turk workers who participated. The sample from Qualtrics had more women than men, while the MTurk sample was the opposite. Qualtrics participants had a lower mean Internet Literacy score, and were older overall than MTurk participants. Finally, the MTurk sample had twice as many college graduates as the Qualtrics sample. Table 3 contains demographic information about both samples.

4.4.2. Greatest surprise with strong filter beliefs

The purpose of the follow-up survey was to learn more about why people were surprised by missed posts, and whether the correlation between closeness and surprise observed in the initial survey would still be present after controlling for participants' causal beliefs. It is possible that surprise about missed posts is a function of how participants think the system works or their own behavior when reading posts, more than the closeness of their relationships. If this is the case, it implies that surprise might be reduced with better transparency about cause-and-effect in filtered systems. To investigate this, I conducted a mixed-effects regression using data from the follow-

Table 2

Comparison of which items appeared in the initial survey and follow-up survey, presented in the order in which they appeared in each survey. (F) indicates a friend level variable, asked once about each Friend. (*) indicates a question that was asked in both surveys, but in a different place in the sequence of questions. (**) indicates a question that was only asked if the participant had noticed a missed post.

Variable Name and Survey Item Initial Follow-up Section

Survey Survey Mentioned

(Qualtrics) (MTurk)

(*) howmanyfriends, visitfreq, and X 3.3.8

postfreq: Questions about the

participant's use of Facebook

Allow access to Friend List, Select X X 3.1

close and random Friends

(F) closeness: Inclusion of Other in X X 3.3.3

the Self Scale

(F) talkedfacebook: Over the PAST X 3.3.4

MONTH, about how often have

you and (Friend Name)... Talked

on Facebook?

(F) accurate.recall: How recently X X 3.3.5

would you estimate was the last

time (Friend Name) posted on

Facebook, without looking at his

or her Timeline?

(F) friend.visibility: Think about the X 3.3.7

top 5 people whose activity you

see the most in your news feed

during an average week.

(*) howmanyfriends, visitfreq, and X 3.3.8

post.freq: Questions about the

participant's use of Facebook

Do you feel like your Facebook News X n/a

Feed always shows you every post

created by your Facebook

friends? Please explain your

answer. (Results reported in

(Rader and Gray, 2015).)

(F) accurate.recall and X X 3.3.5, 3.3.6

active.recently: How recently did

(Friend Name) create a post?

(F) notice.missed: When you were X X 3.3.1

scrolling through (Friend

Name)'s Timeline, did you notice

posts he or she created that you

don't remember seeing in your

News Feed?

(F) Approximately how many posts X 3.3.1

did you see on (Friend Name)'s

Timeline that you don't recall

seeing in your News Feed?

(F**) surprise: I am surprised that I X X 3.3.2

did not see (Friend Name)'s post

(s) in my News Feed

(F**) user.beliefs and filter.beliefs: X 4.4.1

Below are possible reasons why

you might have missed post(s) on

Facebook from (Friend Name).

Do you ever feel like you are missing X 4.4.1

out on posts by Friends in your

News Feed? If so, why do you feel

this way, and why do you think

this happens? If not, why not?

(*) algorithm.aware: Do you feel X 4.3

like Facebook uses computer

programs or algorithms to

automatically choose what stories

to show you in your News Feed?

internet.literacy: Internet Literacy X X 3.2

variable

(*) algorithm.aware: Do you feel X n/a

like Facebook uses computer

programs or algorithms to

automatically choose what stories

to show you in your News Feed?

(Results reported in (Rader and

Gray, 2015).)

Demographics questions X X 3.2, 3.3.8

Table 3

A comparison of demographics between the two samples.

Qualtrics MTurk

Total N 410 464

Internet Literacy M=2.5 M=3.5

(0.92) (0.83)

Men 149 274

Women 260 190

Other 1 0

FB visit freq

About once/wk 5 8

A few times/wk 28 32

About once/day 44 63

Several times/day 333 361

FB post freq

Never 6 10

Less than once/wk 107 170

About once/wk 54 89

A few times/wk 98 98

About once/day 69 54

Several times/day 76 43

# FB friends

21-100 friends 154 114

101-300 friends 126 208

301 -500 friends 74 83

501+ friends 56 59

Friends in Survey 8 10

Number of Friends M=2.13 M=2.95

with missed posts (2.15) (2.22)

18-25 88 157

26-34 97 191

35-50 98 103

51-65 101 12

66-75 26 1

Education

Some HS 12 9

HS grad 108 63

Vocational 33 31

Some college 144 170

College grad 82 161

Post-grad 31 30

Ethnicity

Caucasian/White 358 372

African American 21 41

Native American 8 7

Asian 18 40

Pacific Islander 3 3

Hispanic/Latino 26 24

Other 5 1

up survey with surprise as the dependent variable, restricted to the Friends for whom each participant had noticed at least one missed post. Thirty-one participants reported zero missed posts, so the total number of participants included in this model was 433. The model was specified as follows; (F) indicates a Friend-level variable, and (P) indicates a participant-level variable:

Table 4

Mixed effects regression results for surprise using data from the Mechanical Turk sample. Closeness, Friend visibility, participant Facebook visit and post frequency, and number of Friends are centered at the median; user- and filter-oriented beliefs about why posts were missed and internet literacy are centered at the mean. (F)=Friend level variable. R2 values were calculated based on Nakagawa and Schielzeth (2012) and Johnson (2014).

DV=surprise; M=2.79, SD=1.22, Range=1-5; N=1370 Friends

Model Term Coef. SE Variable Info

Intercept 3.154 0.106

closeness (F) 0.190*** 0.019 Median=3, Range=1-7

accurate recall: Yes (F) -0.216 0.132 all=2719, missed=724

active recently: Yes (F) -0.110 0.107 all=3005, missed=1103

FB visit frequency 0.139* 0.069 Median="Several times per day"

FB post frequency 0.004 0.033 Median="Less than once per week"

# of Facebook Friends -0.023 0.046 Median="101-300 Friends"

internet literacy -0.159** 0.053 Mean=3.48, SD=0.83

age 0.041 0.053 Median="26-34"

recall * recent activity 0.135 0.149

user.beliefs (F) -0.165*** 0.041 Mean=2.61, SD=1.01

filter.beliefs (F) -0.035 0.043 Mean=2.90, SD=0.89

closeness* user beliefs -0.049** 0.016

closeness* filter beliefs 0.125*** 0.019

Marginal R2=0.14, Conditional R2=0.40, SD of Random Effect=0.629 •p < 0.10; *p < 0.05; **p < 0.01; ***p < 0.001.

Surprise = f(closeness(F) X user. beliefs(F), closeness(F) X filter. beliefs(F),

accurate. recall(F) X active. recently(F), visit. freq(P), post. freq(P), howmany. friends(P), internet. literacy(P), age(P), randomeffect(P))

I compared the above model with a null model containing only the intercept and random effects term, and also with a model containing all of the above variables except user.beliefs and filter.beliefs. The more fully specified model does a better job of explaining the data than the null model (x2(3,16)=206.99, p < 0.001) or the model without the beliefs variables (x2(12,16)=64.80, p < 0.001). The intercept is 3.15 (SE=0.10, 3.0="Neither Agree nor Disagree"), and represents the level of surprise experienced by a participant about a missed post from a Friend who had not created a post in the past week, whose activity the participant did not remember accurately, with the remaining variables in the model centered at the median or mean. As in the initial survey, closeness was positively correlated with surprise (coef=0.19, SE=0.02, p < 0.001). The regression results are presented in Table 4. Fig. 6a and b show the predicted values from the both the initial and follow-up surprise models side-by-side, using a standardized dependent variable. From these graphs it is clear that the relationship between closeness and surprise is very similar in both samples.

I included separate interactions in the model between closeness and each of the causal belief variables, to find out if the relationship between surprise and closeness varied based on users' causal beliefs. These interactions are illustrated in the predicted values generated from the model, displayed in Fig. 7a and b. The difference in the level of surprise between close and distant Friend relationships increases as participant beliefs implicating filter behavior as the cause of the missed post get stronger (Fig. 7a). When filter beliefs are weak, predicted levels of surprise are neutral or low at all levels of closeness. This means that when participants did not assign responsibility for the missed post to the system, they were not surprised when they noticed a missed post. But, when filter beliefs are strong, predicted surprise is very high for close friends and low for distant friends. This indicates that participants with filter-related beliefs about how the News Feed works expect the system to take action to make posts from people they feel close to more visible, but not those posts from people they do not feel close to.

Participants' beliefs that their own behavior caused them to miss posts had a different relationship with closeness and surprise. The

No Yes

Friend Posted in the Past Week?

(a) Qualtrics sample.

No Yes

Friend Posted in the Past Week?

(b) MTurk sample.

Fig. 6. A comparison of the standardized predicted values for the surprise regression in both the Qualtrics and MTurk samples. All variables not included in the graphs are held at the mean or median. Note that the predicted surprise variable (y-axis) was standardized in each dataset so that the mean is 0 and the standard deviation is 1. (a) Qualtrics sample. (b) MTurk sample.

difference in predicted surprise about missed posts between high and low closeness Friends decreases as user behavior beliefs become stronger (Fig. 7b). When user behavior beliefs are strong, surprise about missed posts is low. This means that when participants believed their own behaviors caused them to miss posts, they were not very surprised about it. But when user behavior beliefs are weak, predicted surprise is very high for close friends and low for distant friends. This indicates that in the absence of user behavior to the contrary, participants expected that posts from close friends would be highly ranked by the News Feed.

5. Limitations

One limitation of this research is that it relies on self report which can be biased. Some of the variables of interest can only be measured by asking participants about their perceptions, like closeness of the relationship between the participant and their Friends, noticing a

missed post, and surprise. However, others, such as the frequency of communication on Facebook, number of Facebook Friends, and how often the participant posts to Facebook ask participants to remember or estimate details about their behavior and their Friends' behavior on Facebook. These responses could be biased if participants answer inaccurately, either intentionally or unintentionally.

There are several sources of bias that often appear in surveys. For example, participants' responses might be inaccurate if they did not follow the instructions carefully regarding their selection of Friends, whether they actually visited their Friends' Timelines, or if they did not stop scrolling when they reached posts more than a week old. Not following instructions would cause variability in the data, because the responses from participants who were more careful about following directions would be more accurate than from participants who did not. However, there is no reason to expect these inaccuracies to vary systematically with the variables of interest within the sample. This means that, while it is harder to statistically detect a true effect than it

Filter Behavior Beliefs

(a) Surprise by system-focused beliefs.

User Behavior Beliefs

(b) Surprise by user-focused beliefs.

Fig. 7. Predicted value of surprise about a missed post when the participant had accurate memory and the Friend had posted in the past week, by participant beliefs about why the missed posts occurred and relationship closeness. All other variables in the model were held at the mean or median. (a) Surprise by system-focused beliefs. (b) Surprise by user-focused beliefs.

would be if participants all followed directions perfectly, the results presented here are still valid.

Another source of bias in surveys is social desirability, which causes participants to purposefully answer incorrectly in situations where true responses could be uncomfortable, embarrassing, or risky in some way (Krumpal, 2011). There is little reason to expect this is a problem in this survey; the questions do not ask about sensitive topics, nor is there a clear socially acceptable response or reason why participants might answer in a particular way to avoid embarrassment.

A third source of bias is imperfect memory. Some of the survey questions ask participants to remember things about their own or their Friends' past behavior, and recognize whether they had already seen their Friends' posts. They might not remember these things accurately, and there are a few ways in which this might have systematically biased the results. For example, participants may have overstated or understated how often they communicate with their Friends on Facebook. When participants were asked to estimate the frequency of past occurrences, they may unintentionally discount particular instances that are less salient or memorable, or a particularly memorable or recent instance might cause them to make their estimates too high (Huttenlocher et al., 1993). It is reasonable to guess that this might vary based on the closeness of the relationship, which is why I included the interaction between talked on Facebook and closeness in the models.

Also, there are a couple of ways that imperfect memory might result in incorrect estimates about the incidence of missed posts. The older the posts are on Friends' Timelines, the more likely participants are to have forgotten them. This might artificially increase the number of missed posts. I attempted to minimize the effects of this by directing participants to only scroll back one week in their Friends' Timelines, but as mentioned above some participants may not have followed this instruction. This means there could be an interaction between the age of the Friends' posts and whether the participant followed instructions that the analysis did not directly account for. Some of this within-participant variability should be captured by the random effects term in the models, however. Also, some of the posts that participants said they missed may have been created since the participant's last visit to Facebook before starting the survey, and so they could not reasonably remember them. (Participants reported that 32% of the Friends' most recent posts had been created "Today").

In addition, participants may falsely recognize a post they did not see in their News Feeds, or fail to recognize a post that they had actually previously seen. There are likely some instances of both kinds of errors in the data, but since data about the conditions under which participants first encountered the information in their News Feeds or the characteristics of the posts was not available using this method, it is very difficult to to substantively characterize how common these errors might be. However, recognition accuracy is highest for more recent events (Kristo et al., 2009), and when relevant contextual details are activated in memory as part of the recognition task (Guerin et al., 2012). Participants viewed posts from people they have an existing relationship with, that were one week old or less, in reverse chronological order. The format and content of posts viewed on one's Facebook Timeline are identical to how they appear in the News Feed, including details such as photos, link summaries, comments, etc. These characteristics of the task should reduce the incidence of both kinds of errors. Also, the recall accuracy measure should control for some of the factors that might affect both kinds of errors, because recall is correlated with recognition (Yonelinas, 2002). Other measures, like surprise, are not susceptible to imperfect recall. The survey asked about surprise immediately after participants were asked about missed posts, which yields more accurate results than if they had been asked to remember previous feelings of surprise (Thomas and Diener, 1990).

6. Discussion

Using an online survey, Facebook users were prompted to visit the Timelines of some of their Friends where in many cases they were exposed to posts that they did not remember seeing in their News Feeds. This experience made aspects of the socio-technical system's behavior visible that participants might not otherwise have been aware of. After controlling for variability in participants' memories of their Friends' past Facebook posts, noticing a missed post on a Friend's Timeline was not related to the closeness of the relationship between the participant and the Friend. However, the amount of surprise about missed posts was: greater closeness was associated with more surprise in data collected from two different samples. In other words, the system behaved more unexpectedly for participants when they missed posts from Friends they felt closer to, and also from those Friends they remembered seeing most often in their News Feeds.

When participants believed they missed a post due to their own actions, such as not scrolling down far enough or not visiting Facebook often enough to see the posts, they were less surprised about missed posts. But, when they believed they missed posts because of actions taken by the system, they found it more surprising that they had missed posts created by close Friends. Even after controlling for these causal beliefs, missing a close Friend's post was nearly always more surprising than missing a post created by a distant Friend. These findings show that because users have offloaded the work of curating their News Feeds to the algorithm, they have effectively delegated curating and maintaining their relationships as well, without being aware of it. The algorithm may not prioritize relationships the same way a user would; in fact, the feeling of surprise upon seeing some missed posts is some evidence of this. The findings also suggest that there are systematic patterns regarding which opportunities for interaction are being promoted versus demoted that make algorithmically curated relationships different from how people manage their "real world" interactions.

6.1. Visibility and invisibility

I have been careful so far to refer to posts as "missed" rather than "missing", to avoid causal judgments in reporting the results. A missed post might be the result of a user's selective attention, inaccurate memory, or it could have been made effectively invisible by the algorithm. However, the analyses included controls for alternative explanations including visibility of Friend activity, participant recall accuracy, and the number of Friends participants have. This means that "missed" and "missing" posts can reasonably be conceptualized differently from each other. Posts participants were surprised about can be thought of as "missing" rather than missed from the perspective of the user, because that feeling is a symptom that their expectations have not been met. In this case, it signals that participants expected to see posts from these closer, more visible Friends in their News Feeds.

It is important to separate what users "expect" from what they "want" from a content filtering algorithm. "Expect" implies an outcome that is different from what the user thought would happen, while "want" implies satisfaction with the outcome that occurred. Facebook has argued that if users were to see an unranked version of their News Feed, they would actually be more likely to miss posts they want to see than they are with the News Feed Algorithm in place (Backstrom, 2013). In fact, Eslami et al. (2015) found something similar to this in their study. When they gave participants an opportunity to move posts they had missed higher in a representation of their News Feeds, many of them concluded that the algorithm had actually done a decent job deciding which posts to show them. These are both arguments in favor of filtering as a mechanism to increase individual satisfaction with the News Feed, by offloading the work of categorizing desirable from

undesirable content.

Despite the importance of user satisfaction, optimizing for it does not preclude other kinds of systemic algorithm effects that are harder to measure, due to content that remains invisible. Focusing on user satisfaction as a metric for evaluating the effects of algorithms limits the attention of research and design to only a small part of the work filtering algorithms do in socio-technical systems. By focusing on expectations and surprise rather than satisfaction, this study identified an effect of invisibility at the relationship level. In essence: if I don't see your posts, I won't be reminded to communicate with you on Facebook. If I don't communicate with you on Facebook, the News Feed will think we aren't close friends. If the News Feed thinks we are not close friends, it won't show me your posts. The behavioral traces, which are all the evidence the News Feed Algorithm has to use to infer the strength of friendships, disappear. This pattern is suggestive of how relying on an algorithm to direct attention can have unexpected consequences beyond satisfaction that it would be difficult for individual users to identify on their own.

In addition to being potentially costly for individuals, large-scale patterns across users regarding whose posts are seen and whose are not can also be costly for the system overall. For example, the algorithm could act as a constraint on a user's visible network, artificially limiting the reach and impact of the user's posts. Facebook's reciprocal Friend connections may in reality behave more like Twitter's directional ties, if the constraints imposted by the ranking decisions of the filtering algorithm cause post visibility between two Facebook Friends to not be reciprocal. Whether constraints like these can be considered "bias" depends on the frame of reference (user vs. system operator, for example), and what kinds of outcomes each is hoping to achieve. In this case the underlying network structure of reciprocal Friend connections may be effectively altered by the visible and invisible output of the algorithm.

These findings raise a fundamental question about the power of algorithms and automation in general: if the algorithm exists to direct users' attention by showing them some posts and not others, and the system is biased in a particular direction, how will this ever be detected? Detection of bias is much easier when the system produces a clearly unfair or undesirable outcome, such as unintentionally reminding people about years-ago sad or painful past experiences (Meyer, 2014). But if the undesirable outcome is something that is not displayed and therefore interactions that do not happen, systemic bias is much harder to identify. Because filtering is invisible for many users without a triggering event like a Friend pointing out a missed post (Rader and Gray, 2015; Eslami et al., 2015), on a post-by-post basis it is impossible for users to notice and react to the fact that posts from some close Friends are less likely to be highly ranked, and certain relationships may become harder to maintain than others over time, whether or not the user is satisfied with the visible contents of the News Feed overall.

6.2. Implications for user control and algorithmic transparency

In July 2015, after the data collection for this paper took place, Facebook announced new "Updated Controls for News Feed" that allow users to "actively shape and improve the experience" by selecting particular Friends whose posts will be displayed at the top of the News Feed.10 Using these controls is advertised to change the way posts are ranked, and would thereby change the set of posts available for users to consume. Assuming a user is already spending as much time as he or she wants to on Facebook, choosing to see posts from some Friends first means other posts will receive less attention. This will shift the characteristics of the algorithm-imposed visibility constraints—but in what way? Consider two possible ways users might choose to prioritize

10 https://newsroom.fb.com/news/2015/07/updated-controls-for-news-feed/

people: based on a mental list of which friends they want to stay in touch with the most, or based on noticing they've been missing posts from some Friends in particular. At the level of each individual user, these two different approaches both could result in similar improvements in user satisfaction. Actually, any user-controlled prioritization will likely increase satisfaction with the News Feed; previous research in recommender systems has found that just giving users the appearance of control increases satisfaction whether or not their selections actually change any aspect of system ranking or performance (Solomon, 2014).

However, at the system level, the possible aggregate outcomes of these individual choices are harder to predict. Which posts will receive more interaction than before, aggregated across the entire system, and which will receive less? Person-level controls could plausibly cause users to see an increase in their News Feeds of posts that are less popular overall, as posts with fewer likes and comments (relative to before the introduction of the mechanism) might be ranked higher for individuals based on who created them. This could spread user attention in the system out over more posts than before resulting in more posts being seen by fewer people, and more diverse opportunities to give and receive social support. Depending on what criteria users base their choices on for which Friends to prioritize, perhaps this intervention will eliminate the correlation between closeness and surprise that observed in this study. It might also end up encouraging the formation of polarization and echo chambers by reducing the likelihood of serendipitous encounters with new perspectives from different people, as users' feeds fill up with posts from close Friends. But the missed posts will remain, as does the problem of measuring the invisible opportunity cost.

A different approach that has been proposed to help users better understand how the system works and promote more accurate expectations is greater algorithmic transparency: disclosure of information that would make details of the inner workings of the "black box" more visible, and thereby provide the necessary context for outsiders to understand cause and effect in the system (Diakopoulos, 2014). Transparency is one path to giving users more control by helping them understand the consequences of their actions in the system. The results from the follow-up survey showed that participants' causal beliefs about why they missed posts were important; surprise was high for close friends when filter beliefs were strong, but not when user beliefs were strong. This indicates that participants who thought Facebook was making choices about which posts to show them nevertheless had a mistaken impression of how those choices were made. There are many ways that transparency might be implemented, but no consensus on the most effective way to provide transparency information to users. Facebook already provides some transparency: users can see a chronologically ordered view of the News Feed by selecting the "Most Recent" view, which is ostensibly a less-filtered option. Also, the "facebook for business" blog (Backstrom, 2013) includes a description of the kinds of information the algorithm uses when making ranking choices. However, both of these ways of conveying transparency information require users to know about them and be able to imagine what might cause differences in the ranking; it seems unlikely that either would be effective.

Other options for increased transparency about relationship-level effects might include telling users which of their Friends have been active recently whose posts have not been displayed to them in their News Feeds, or characterizing which Friends' future posts are likely to be highly ranked. This would place the output of the algorithm in the context of users' experiences and relationships, and provide more information about cause and effect. But, this still requires the user to connect input (their own behavior) and output (algorithm ranking) on their own, and individual users typically do not have a broad enough view of the system to be able to do this accurately. Explicit feedback about cause and effect might look more like delivering information about how clicking on particular photos or Liking particular posts

would affect the post author's visibility in one's News Feed. This would allow users to imagine possible future scenarios and then make choices based on which actions would allow them to achieve the outcomes they prefer. It would also potentially reveal proprietary trade secrets and make it easier for users inclined to do so to "game" the system, which would be undesirable from the perspective of system designers (Diakopoulos, 2014). It is not a given that greater transparency is in the interest of system designers; however, it is also not universally true that transparency is better for users. Transparency adds additional information for users to attend to in a situation in which filters were added to reduce information overload.

7. Conclusion

There are public benefits to be had from the scientific study of socio-technical systems, like Facebook, that aren't in the public domain. It is valuable to study these interactions between human behavior, which is highly variable but not completely random, and the algorithms that attempt to separate signal from noise and make decisions on the user's behalf. Systems that involve algorithms and automation are becoming more complex, and more common. The constraints on what people consume on Facebook are a function of both their own behavior and the algorithm's rankings, and it is difficult to separate them. Findings and implications like these allow us to better understand how interdependencies between algorithm and user behavior affect the performance of socio-technical systems, and what designers must consider when working to produce desirable, emergent properties from the interactions between users and algorithms.

Acknowledgements

The author thanks Rick Wash and Scott Rucinski for assisting with integrating the Facebook app into the Qualtrics survey platform; Rebecca Gray for helping with the survey design; and Yumi Jung and the BITLab research group for providing feedback on the survey questions and results. This work was supported by the National Science Foundation under Grant No. IIS-1217212.

References

Aron, A., Aron, E.N., Smollan, D., 1992. Inclusion of other in the self scale and the structure of interpersonal closeness. J. Personal. Soc. Psychol. 63 (4), 596-612. Backstrom, L., 2013. News Feed FYI: A Window Into News Feed, URL <https://www.

facebook.com/business/news/News-Feed-FYI-A-Window-Into-News-Feed). Bakshy, E., Messing, S., Adamic, L., 2015. Exposure to ideologically diverse news and

opinion on facebook. Science 348 (6239), 1130-1132. Bowker, G.C., Baker, K., Millerand, F., Ribes, D., 2010. Toward information

infrastructure studies: ways of knowing in a networked environment. In: Hunsinger, J., Allen, M. (Eds.), International Handbook of Internet Research. Springer, London, New York, 97-117.

Bozdag, E., 2013. Bias in algorithmic filtering and personalization. Ethics Inf. Technol. 15 (3), 209-227.

Bradburn, N.M., Huttenlocher, J., Hedges, L., 1994. Telescoping and temporal memory. In: Schwarz, N., Sudman, S. (Eds.), Autobiographical Memory and the Validity of Retrospective Reports. Springer-Verlag, New York, 203-215. Brashears, M.E., Quintane, E., 2015. The microstructures of network recall: how social networks are encoded andrepresented in human memory. Soc. Netw. 41, 113-126. Bucher, T., 2012. Want to be on the top? Algorithmic power and the threat of invisibility

on facebook. New Media Soc. 14 (7), 1164-1180. Burgoon, J.K., 1993. Interpersonal expectations, expectancy violations, and emotional

communication. J. Lang. Soc. Psychol. 12 (1-2), 30-48. Burke, M., Kraut, R., 2014. Growing closer on facebook: changes in tie strength through social network site use. In: Proceedings of the CHI 2014 Conference on Human Factors in Computing Systems. ACM, New York, pp. 4187-4196. Cleary, A.M., Greene, R.L., 2000. Recognition without identification. J. Exp. Psychol.

Learn. Mem. Cogn. 26 (4), 1063-1069. Cleary, A.M., Greene, R.L., 2004. True and false memory in the absence of perceptual

identification. Memory 12 (2), 231-236. Counts, S., Fisher, K., 2011. Taking it all in? Visual attention in microblog consumption, In: Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media. pp. 97-104. Diakopoulos, N., 2014. Algorithmic accountability. Digit. J. 3 (3), 398-415. Ellison, N.B., Vitak, J., Gray, R., Lampe, C., 2014. Cultivating social resources on social

network sites: Facebookrelationship maintenance behaviors and their role in social capital processes. J. Comput. Mediat. Commun. 19 (4), 855-870.

Eslami M., Rickman, A., Vaccaro, K., Aleyasen, A., Vuong, A., Karahalios, K., Hamilton, K., Sandvig, C., 2015. I always assumed that I wasn't really that close to [her]: reasoning about invisible algorithms in the news feed. In: Proceedings of the CHI 2015 Conference on Human Factors in Computing Systems. ACM, New York, pp. 153-162.

Fernandes, M.A., Moscovitch, M., 2000. Divided attention and memory: evidence of substantial interference effects at retrieval and encoding. J. Exp. Psychol. Gen. 129 (2), 155-176.

Friedman, B., Nissenbaum, H., 1996. Bias in computer systems. ACM Trans. Inf. Syst. (TOIS) 14 (3), 330-347.

Gardiner, J.M., Java, R.I., Richardson-Klavehn, A., 1996. How level of processing really influences awareness in recognition memory. Can. J. Exp. Psychol./Rev. Can. Psychol. Exp. 50 (1), 114-122.

Guerin, S.A., Robbins, C.A., Gilmore, A.W., Schacter, D.L., 2012. Retrieval failure contributes to gist-based false recognition. J. Mem. Lang. 66 (1), 68-78.

Hamilton, K., Karahalios, K., Sandvig, C., Eslami, M., 2014. A path to understanding the effects of algorithm awareness. In: Proceedings of the CHI'14 Extended Abstracts on Human Factors in Computing Systems, CHI EA'14. ACM, New York, NY, USA, ISBN 978-1-4503-2474-8, pp. 631-642, http://dx.doi.org/10.1145/2559206.2578883, 2014.

Hampton, K.N., 2015. Persistent and pervasive community: new communication technologies and the future of community. Am. Behav. Sci. 60 (1), 101-124.

Hargittai, E., Hsieh, Y.P., 2011. Succinct survey measures of web-use skills. Soc. Sci. Comput. Rev. 30 (1), 95-107.

Haythornthwaite, C., 2005. Social networks and internet connectivity effects. Inf. Commun. Soc. 8 (2), 125-147.

Hill, W.C., Hollan, J.D., Wroblewski, D., McCandless, T., 1992. Edit wear and read wear. In: Proceedings of the CHI 1992 Conference on Human Factors in Computing Systems. pp. 3-9.

Hosanagar, K., Fleder, D., Lee, D., Buja, A., 2014. Will the global village fracture into tribes? Recommender systems and their effects on consumer fragmentation. Manag. Sci. 60 (4), 805-823.

Huttenlocher, J., Prohaska, V., 1993. Reconstructing the times of past events. In: Stein, N.L., Ornstein, P.A. (Eds.), Memory for Everyday and Emotional Events. pp. 165179.

Johnson, P.C.D., 2014. Extension of nakagawa & schielzeth's R2GLMMto random slopes models. Methods Ecol. Evol. 5 (9), 944-946.

Joinson, A.N., 2008. 'Looking at', 'looking up' or 'keeping up with' people? motives and uses of facebook. In: Proceedings of the CHI 2008 Conference on Human Factors in Computing Systems. ACM, New York, pp. 1027-1036.

Kitchin, R., 2016. Thinking critically about and researching algorithms. Inf. Commun. Soc., 1-16.

Koriat, A., Goldsmith, M., Pansky, A., 2000. Toward a psychology of memory accuracy. Annu. Rev. Psychol. 51, 481-537.

Kristo, G., Janssen, S.M.J., Murre, J.M.J., 2009. Retention of autobiographical memories: an Internet-based diarystudy. Memory 17 (8), 816-829.

Krumpal, I., 2011. Determinants of social desirability bias in sensitive surveys: a literature review. Qual. Quant. 47 (4), 2025-2047.

Lazer, D., 2015. The rise of the social algorithm. Science 348 (6239), 1090-1091.

Lu, W., Hampton, K.N., 2016. Beyond the power of networks: differentiating network structure from social media affordances for perceived social support. Technical Report. Rutgers University, URL http://dx.doi.org/10.7282/T3B27x7V.

Maguire, R., Maguire, P., Keane, M.T., 2011. Making sense of surprise: an investigation of the factors influencing surprise judgments. J. Exp. Psychol. Learn. Mem. Cogn. 37 (1), 176-186.

Meadows, D.H., 2008. In: Wright, D. (Ed.), Thinking in Systems: A Primer. Chelsea Green Publishing, White River Junction, VT.

Mellers, B., Fincher, K., Drummond, C., Bigony, M., 2013. Surprise: a belief or an emotion? In: Chandrasekhar Pammi, V.S., Srinivasan, N. (Eds.), Decision Making Neural and Behavioural Approaches 202. Elsevier, Amsterdam, 3-19.

Meyer, E., 2014. My Year Was Tragic. Facebook Ambushed Me With a Painful Reminder., URL (http://www.slate.com/blogs/future_tense/2014/12/29/ facebook_year_in_review_my_tragic_year_was_the_wrong_fodder_for_facebook. html).

Nakagawa, S., Schielzeth, H., 2012. A general and simple method for obtaining R2 from generalized linear mixed-effects models. Methods Ecol. Evol. 4 (2), 133-142.

Nguyen, T.T., Hui, P.-M., Harper, F.M., Terveen, L., Konstan, J.A., 2014. Exploring the filter bubble: the effect of using recommender systems on content diversity. In: Proceedings of the 23rd International World Wide Web Conference. pp. 677-686.

Nomi, J.S., Cleary, A.M., 2012. Judgments for inaccessible targets: comparing

recognition without identification andthe feeling of knowing. Mem. Cogn. 40 (8), 1178-1188.

Noordewier, M.K., Breugelmans, S.M., 2013. On the valence of surprise. Cogn. Emot. 27 (7), 1326-1334.

Ochsner, K.N., 2000. Are affective events richly recollected or simply familiar? The experience and process of recognizing feelings past. J. Exp. Psychol. Gen. 129 (2), 242-261.

Pariser, E., 2011. The Filter Bubble: What the Internet is Hiding from You. The Penguin Press, New York.

Pew Research Center, 2014. Social Media Update 2014, URL (http://www.pewinternet. org/2015/01/09/social-media-update-2014/).

Rader, E., Gray, R., 2015. Understanding User Beliefs About Algorithmic Curation in the Facebook News Feed. In: Proceedings of the CHI 2015 Conference on Human Factors in Computing Systems. pp. 173-182.

Roberts, S.G.B., Dunbar, R.I.M., Pollet, T.V., Kuppens, T., 2009. Exploring variation in active network size: Constraintsand ego characteristics. Soc. Netw. 31 (2), 138-146.

Schutzwohl, A., 1998. Surprise and schema strength. J. Exp. Psychol. Learn. Mem. Cogn. 24 (5), 1182-1199.

Schacter, D.L., Norman, K.A., Koutstaal, W., 1998. The cognitive neuroscience of constructive memory. Annu. Rev. Psychol. 49, 289-318.

Solomon, J., 2014. Customization bias in decision support systems. In: Proceedings of the CHI 2014 Conference on Human Factors in Computing Systems. pp. 3065-3074.

Sosik, V.S., Bazarova, N.N., 2014. Relational maintenance on social network sites: how facebook communicationpredicts relational escalation. Comput. Hum. Behav. 35, 124-131.

Subrahmanyam, K., Reich, S.M., Waechter, N., Espinoza, G., 2008. Online and offline

social networks. J. Appl. Dev. Psychol. 29 (6), 420-433.

Teigen, K.H., Keren, G., 2003. Surprises: low probabilities or high contrasts? Cognition 87 (2), 55-71.

Thomas, D.L., Diener, E., 1990. Memory accuracy in the Recall of Exploring variation in active network size: constraintsand ego characteristicshe recall of emotions. J. Personal. Soc. Psychol. 59 (2), 291-297.

Toglia, M.P., 1999. Recall accuracy and illusory memories: when more is less. Memory 7 (2), 233-256.

Wells, T., Link, M., 2014. Facebook user research using a probability-based sample and behavioral data. J. Comput. Mediat. Commun. 19 (4), 1042-1052.

Yonelinas, A.P., 2002. The nature of recollection and familiarity: a review of 30 years of research. J. Mem. Lang. 46 (3), 441-517.