Scholarly article on topic 'Flash Collabs: Collaborative Innovation Networks in Online Communities of Animators'

Flash Collabs: Collaborative Innovation Networks in Online Communities of Animators Academic research paper on "Psychology"

CC BY-NC-ND
0
0
Share paper
OECD Field of science
Keywords
{Animation / collaboration / "online creative collaboration" / "social computing" / leadership / amateurs / COINs / "collaborative innovation networks" / "computer-supported cooperative work" / mixed-methods}

Abstract of research paper on Psychology, author of scientific article — Kurt Luther, Amy Bruckman

Abstract Collaborative innovation networks (COINs) and swarm creativity is often studied in a professional business context. We propose that studying COINs of amateurs and non-business contexts provides a crucial and complementary perspective on these phenomena. By broadening the scope of COINs research to encompass these edge cases, we can begin to identify patterns and trends that persist across different contexts. For the past three years, we have been studying COINs in a novel context: online communities of Flash animators who collaborate over the internet to create animated movies and games called “collabs.” From a quantitative analysis of nearly 900 collabs on Newgrounds.com, we found that these projects can be highly successful, attracting hundreds of thousands of Internet audience members to download the completed animations. Our analysis also demonstrated that is possible to predict the success potential of a collab by examining specific factors, including attributes of the leader, organizational structures, and activity patterns within a collab. Our focus in this research has been on the social dynamics within collabs, especially the role of leadership. Through in-depth interviews with collab participants, we found that collabs are typically created by groups of amateurs, or in some cases, animation students, often located around the world and speaking different languages. Their motivations tend to be socialpsychological—such as learning, reputation, social support, and self-efficacy—rather than financial, and almost everyone contributes as an unpaid volunteer. We also found that a shared goal of almost every collab participants with whom we spoke was to create something original; that is, a movie or game that audiences perceived as unique. To this end, collab participants are constantly experimenting with new processes, team compositions, and artistic styles. We have identified notable similarities and differences between collabs and other COINs involved with entertainment production. Collab participants emulate the professional filmmaking community in that they recruit mainly through social networks; in contrast to movie studios, however, collabs are organized entirely online, via discussion forums, blogs, and instant messaging. This process is facilitated by online reputation markers, such as digital histories of past contributions. Another similarity between the film set and the collab production process is that both operate in a hierarchical fashion, with one individual at the top, the “benevolent dictator,” maintaining the ultimate creative direction and authority. This finding contradicts the received wisdom of online collaborations, which holds that flattened hierarchies make decisions in a democratic or meritocratic fashion. Finally, in contrast to professional filmmaking, we found that the division of labor in collabs is often modularized, rather than specialized. Collab leaders typically assign entire scenes to individual animators to be recombined upon completion. This modularization allows animators to work independently and in parallel to sidestep some of the challenges posed by distributed collaboration. The breadth and potential of COINs, as illuminated by our ongoing study of collabs, continues to surprise and inspire us. Our future work includes (1) developing software tools to support collabs and (2) relating these findings to other forms of online creative collaboration.

Academic research paper on topic "Flash Collabs: Collaborative Innovation Networks in Online Communities of Animators"

Available online at www.sciencedirect.com

ScienceDirect

Procedia Social and Behavioral Sciences 2 (2010) 6571-6581

COINs2009: Collaborative Innovation Networks Conference

Flash Collabs: Collaborative InnovationNetworks in Online Communities of Animators

Kurt Luther*, Amy Bruckman

Georgialnstitute ofTechnology, 85 Fifth Street NW, Atlanta, Georgia 30308 USA Elsevier use only: Received date here; revised date here; accepted date here

Abstract

Collaborative innovation networks (COINs) and swarm creativity is often studied in a professional business context. We propose that studying COINs of amateurs and non-business contexts provides a crucial and complementary perspective on these phenomena. By broadening the scope of COINs research to encompass these edge cases, we can begin to identify patterns and trends that persist across different contexts. For the past three years, we have been studying COINs in a novel context: online communities of Flash animators who collaborate over the internet to create animated movies and games called "collabs." From a quantitative analysis of nearly 900 collabs on Newgrounds.com, we found that these projects can be highly successful, attracting hundreds of thousands of Internet audience members to download the completed animations. Our analysis also demonstrated that is possible to predict the success potential of a collab by examining specific factors, including attributes of the leader, organizational structures, and activity patterns within a collab.

Our focus in this research has been on the social dynamics within collabs, especially the role of leadership. Through in-depth interviews with collab participants, we found that collabs are typically created by groups of amateurs, or in some cases, animation students, often located around the world and speaking different languages. Their motivations tend to be social-psychological—such as learning, reputation, social support, and self-efficacy—rather than financial, and almost everyone contributes as an unpaid volunteer. We also found that a shared goal of almost every collab participants with whom we spoke was to create something original; that is, a movie or game that audiences perceived as unique. To this end, collab participants are constantly experimenting with new processes, team compositions, and artistic styles. We have identified notable similarities and differences between collabs and other COINs involved with entertainment production. Collab participants emulate the professional filmmaking community in that they recruit mainly through social networks; in contrast to movie studios, however, collabs are organized entirely online, via discussion forums, blogs, and instant messaging. This process is facilitated by online reputation markers, such as digital histories of past contributions. Another similarity between the film set and the collab production process is that both operate in a hierarchical fashion, with one individual at the top, the "benevolent dictator," maintaining the ultimate creative direction and authority. This finding contradicts the received wisdom of online collaborations, which holds that flattened hierarchies make decisions in a democratic or meritocratic fashion. Finally, in contrast to professional filmmaking, we found that the division of labor in collabs is often modularized, rather than specialized. Collab leaders typically assign entire scenes to individual animators to be recombined upon completion. This modularization allows animators to work independently and in parallel to sidestep some of the challenges posed by distributed collaboration.

* Corresponding author. Tel.: +1-404-894-1558. E-mail address: luther@cc.gatech.edu.

1877-0428 © 2010 Published by Elsevier Ltd. doi:10.1016/j.sbspro.2010.04.067

The breadth and potential of COINs, as illuminated by our ongoing study of collabs, continues to surprise and inspire us. Our future work includes (1) developing software tools to support collabs and (2) relating these findings to other forms of online creative collaboration.

Keywords: Animation; collaboration; online creative collaboration; social computing; leadership; amateurs; COINs; collaborative innovation networks; computer-supported cooperative work; mixed-methods.

1. Introduction

Some of the most fascinating examples of COINs can be found outside the world of formal employment. Yet, the majority of COINs research seems to focus on professional business contexts (Gloor & Cooper, 2007; Gloor, 2006). In this paper, we focus on amateur COINs—collaborative innovation networks of amateurs, novices, students, and other non-professionals who leverage swarm creativity to produce knowledge and other valuable artifacts.

Amateur COINs are worthy of study for several reasons. First, by studying amateur COINs in non-business contexts, we provide a complementary perspective to the better-understood professional COINs. Comparing and contrasting them can help reveal patterns and trends and suggest which features of COINs are context-specific, or dependent on certain thresholds of expertise, and which aren't.

Second, some amateur COINs have been remarkably successful. Wikipedia, the free online encyclopedia, boasts over 10 million articles written and edited by amateur volunteers. Over three million of these articles are written in English, with the others being divided up among over 250 other languages. Studies comparing Wikipedia articles to those of professionally-written commercial encyclopedias show few differences in quality or tone (Emigh & Herring, 2005; Giles, 2005). Another example of amateur COINs, the open-source software movement, has produced some of the world's most popular software applications and programming languages. These include the Linux operating system, the Mozilla Firefox web browser, the Apache web server, and programming languages such as Perl, Python, and Ruby, among many others (Weber, 2004). Studying how these amateur COINs can succeed, and even beat out commercial competitors, is a useful endeavor.

Third, amateur COINs may encompass a wider range of creative contexts than professional ones because their motivations for existing are often fundamentally different. Amateur contributors to online communities are likely to be motivated by social or psychological motivations, such as learning or improving one's reputation within the community, rather than the economic motives that characterize most business-oriented COINs (Bryant, Forte, & Bruckman, 2005; Hertel, Niedner, & Herrmann, 2003; Kollock, 1999; Ye & Kishida, 2003). In fact, financial rewards can actually dampen amateurs' enthusiasm for participation in online collaborations (Benkler, 2002). As a result, amateurs can risk forming COINs around domains that may not seem profitable, and have therefore been largely ignored by commercial enterprises. These forays into new territory can yield valuable lessons to be applied to professional contexts and beyond.

In this paper, we investigate amateur COINs within the domain of online, collaborative animation projects called "collabs." As we will see, this domain exemplifies all three of the characteristics mentioned above. Although collabs show some similarities to the world of professional animation production, we found that they also demonstrate notable differences. We also found that while many collabs fail to produce a completed animation, those that do can be highly successful, attracting hundreds of thousands of views from Internet audiences. And finally, we found that the collab production process differs substantially from those observed in Wikipedia editing and open-source software development, in terms of originality, completion, subjectivity, and ownership.

Thus, collabs provide a compelling example of what we can learn from studying amateur COINs. However, our studies of collabs also revealed some promising opportunities for future research into understanding the potential and limitations of amateur COINs as a more general phenomenon. These challenges include involving more novices in the collaborative process, solving increasingly complex problems, providing appropriate non-monetary incentives, reducing the burden on leaders, and expanding to new contexts. We also describe how we are building software tools to explore some of these opportunities within the domain of computer-animated movie-making.

The rest of this paper is organized as follows: In section 2, we describe Newgrounds, the online community that served as the locus of our studies. In sections 3 and 4, we present the methods and results of our research. We

discuss high-level opportunities for amateur COINs research, and how the software tools we are building may address them, in section 5. We conclude with a summary in section 6.

2. Background

Adobe Flash is an interactive multimedia technology used for many purposes, including website design, educational software, video games, streaming video, and computer animation. The technology's expressiveness, flexibility, and visual orientation have attracted a huge following of programmers and artists alike. It is highly popular, in part because it is quick and easy to install the Flash player on almost any modern web browser. As of December 2009, it is estimated that mature market penetration for a recent version of Flash is 99.0% (Adobe, 2009).

Our study focused on Newgrounds (http://www.newgrounds.com), an online community of Flash animation enthusiasts. With over 1.8 million registered members, Newgrounds is the largest online community devoted to Flash animation on the Web. According to Alexa.com, Newgrounds is also among the 1,000 most-trafficked websites.

Newgrounds offers many features to its members, the most popular of which are the Flash Portal and the NGBBS. The Flash Portal is a section of the site where members can submit Flash games and movies to be considered for hosting on Newgrounds. Over 160,000 of these animations have been uploaded by Newgrounds members. When a Flash animation is submitted, members rate it while it is "under judgment." If the animation earns an acceptable rating, it is hosted on Newgrounds; otherwise, it is rejected. Audiences can continue to rate or provide written reviews of the hosted animation. This feedback is aggregated on the same page where the animation can be watched, along with additional information about it, such as the names of its creators and a space for creators to provide a written introduction. Highly-rated animations receive awards, including cash prizes from the Newgrounds staff, as well as increased exposure on the site via a listing on the high-traffic home page.

Besides the Flash Portal, the second most popular section of Newgrounds is the NGBBS, or Newgrounds Bulletin Boards. As the name implies, the NGBBS is an online discussion forum where Newgrounds members can discuss a wide range of topics. Of particular interest to this research is the Flash forum, which serves as a central meeting place for the COIN of amateur Flash animators. In the Flash forum, members present and solicit feedback on their works-in-progress, share the latest animation and programming techniques, and discuss and critique completed animations hosted on Newgrounds. Crucially, the Flash forum is also the main location where animators seek out other animators to recruit and organize collaborative animation projects called "collabs." In the following sections, we describe our studies involving collabs.

3. Study 1

3.1. Methods

First, we sought to understand how collabs are made—that is, the collab production process. How is it that people who may not previously know each other can make contact, agree on a shared vision for the project, coordinate their efforts, and produce a high-quality artifact? By understanding how this process works, we can begin to situate this type of amateur COIN against others that have been studied.

We employed in-depth, qualitative interviewing methods for this study (Seidman, 2006). We created a semi-structured interview guide, whose questions focused on the collab production process and the challenges involved with running a successful collab. Our recruitment strategies included posts and private messages to the NGBBS, as well as "snowball sampling," where participants suggest additional participants to us. In total, we conducted 17 interviews—14 via phone and 3 via email. The phone interviews lasted, on average, 60 minutes each, and were audio-recorded and fully transcribed with participants' permission. We analyzed these transcripts using a bottom-up approach based on grounded theory (Strauss & Corbin, 1998), in which we iteratively coded and refined ideas mentioned by participants, producing a detailed picture of the entire collab production process. We supplemented these data with our observations of the community over a one-year period of time.

Our participants were diverse in age and location, but not gender. The youngest was 16 and the oldest was 29, and six countries were represented: the United States, the United Kingdom, Spain, Australia, Holland, and Estonia. All participants were male; this gender bias seems to be representative of Newgrounds as whole, rather than an issue

with our recruitment procedures. Every participant had experience working on collabs, in a wide range of roles. Participants also spanned the gamut of expertise; we interviewed animation students in art school, professional animators who contributed to Newgrounds in their free time, and others in between these extremes.

A comprehensive report on our findings of this study is available elsewhere (Luther & Bruckman, 2008). For the purposes of this paper, we focus on the results that are relevant to the topic of amateur COINs.

3.2. Summary of Findings

Our study produced the following key findings:

• Leaders are crucial and pervasive, but overburdened.

All collabs mentioned by participants had a leader, who typically took on the largest burden of any collaborator. The leader was intimately involved with almost every aspect of the collab from start to finish. Leaders took on the roles of director, producer, editor, recruiter, among others. Additionally, many participants saw the leader's vision as inseparable from the collab. If a leader had to leave a collab for some reason, the collab rarely recovered, even if a substitute was found.

• Few collabs produce a completed animation.

Because our study was qualitative in nature, we weren't able to determine success rates for different types of collabs. However, all participants agreed that a majority of collabs failed—meaning they were started, but did not lead to a completed animation. Participants offered a variety of explanations for why so few collabs succeed, including overburdened or under-dedicated leaders, ill-conceived collab ideas, unmotivated artists, and poor timing.

• The workload is modularized, not specialized.

In the world of professional animation production, most roles on a project are specialized. Many responsibilities, such as lighting, concept artwork, film editing, sound design, and writing are usually divided up and assigned to separate teams or departments. Even character-centric tasks, such as modeling (creating the 3D model of a character), rigging (creating a digital 'skeleton' of the character to assist with animation), and animation (making the character move and react) are typically assigned to different people. Often, an animator is assigned to a single character, rather than a scene. In collabs, we found, modularization, rather than specialization, is the most common way to structure the division of labor. A typical collab organization has the leader deciding on a theme, such as a music video or story, and then dividing up the work into many short segments. Animators claim (or compete to claim) individual segments over which they have near-complete control of every aspect, from the dialog to the soundtrack to the artwork. In this way, the leader acts more like an editor assembling a collection of articles for a book, rather than a director of a feature animation.

• The collab production process is linear.

We found that every collab has a clearly defined beginning, middle, and end point. These three phases can be described as designing the project, managing the artists, and completing the project, respectively, and each brings with it a distinct set of challenges, especially for leaders. When designing the project, the beginning stage of the collab production process, the leader must first decide on the collab's structure. This include content guidelines (what is the animation supposed to be about?), technical specifications (what dimensions and frame rate should the segments be?), and a coordination style (will it be competitive, improvisational, or something else?). Then, the leader must find collaborators—artists who will do the actual animation work for the collab—either by proposing the idea to the entire community via the NGBBS, or by contacting friends and past collaborators. To manage these artists, the middle stage of the collab production process, leaders must act as chief motivators, pushing artists to complete their assignments and do their best work. When artists drop out, it is the leader's job to find replacements. The leader must balance his or her desire to achieve a particular creative vision with the artists' desire to have autonomy and express their unique styles and interpretations. Finally, to complete the project, the leader must assemble all the artists' segments into a coherent whole and integrate them into a single, completed animation. In this ending stage of the collab production process, the leader must also decide who is granted "co-author" status on

the Newgrounds page where the animation is hosted, which in turn determines which collaborators receive more exposure.

• Recruitment strategies depend on the leader's experience.

As we briefly mentioned, participants noted two main ways that leaders find collaborators. One option is to advertise to the entire community via the NGBBS, by posting a new thread in the Flash forum with all the relevant details artists need to make a decision. If the collab idea is popular, the thread will attract a lot of activity as animators post replies indicating their interest, ask questions, and start discussing their work. Otherwise, the thread will sink below more popular threads and is unlikely to get the exposure it needs to be successful. This method is popular among leaders who are new to the community, who need to recruit a large number of artists, or whose quality standards are relaxed. Alternatively, more experienced animators tend to reach out to their pre-existing social networks. They may skip the forum in favor of contacting past collaborators via more personalized communication tools like email. As with forum-based recruitment, however, these collaborators are often distributed across space and time zones, making synchronous communication difficult. A similar recruitment method is commonly used in the professional film industry, where social networks and reputations, rather than public job postings, are the primary mechanisms by which crews are assembled and people find work (Cattani & Ferriani, 2008; Faulkner & Anderson, 1987).

• Collab participants lack adequate technological support.

In many collabs, we found that communication and coordination happens chiefly through the NGBBS thread where the collab was initially proposed. However, the NGBBS was designed for conversation, rather than collaboration, and this mismatch often creates problems. For example, there are no affordances in the forum software for sharing files. Artists rarely solicit feedback on their collab works-in-progress because there is no easy way to share them with the group. Instead, artists typically submit their finished segments only to the leader, via email or instant message services. This workflow requires leaders to organize every submission, vet each for quality, and protect the files from loss or corruption—all additional burdens on an already overburdened leader. Another example of these technological limitations is the problem of group awareness. A collab thread started six months ago with fifty members, not an unusual occurrence, can see over a thousand replies distributed among fifty pages. Wading through this information can be challenging enough for participants who have been involved with the collab from the beginning, but for new or hopeful members, it is nearly impenetrable. As a result, leaders attempt to keep everyone up-to-date by posting regular announcements, but these often become infrequent as the leader's workload increases.

• Recognizing everyone's contributions is problematic.

A chief motivator for people to contribute to collabs is increasing one's exposure and reputation within the online community. To achieve this, however, one must receive appropriate credit for his or her contributions to a collab. Many websites which host user-generated videos and animated content (e.g., YouTube, Kongregate) only recognize one contributor—the person who uploaded the content. Newgrounds' multi-author system is more powerful, but still limited. When publishing a completed animation on Newgrounds, leaders can co-author up to nine other people besides themselves. If more than ten people contributed to the collab, the leader must make the tough decision regarding who gets left out. Contributors who aren't co-authored see little recognition, because their user accounts aren't linked to on the animation's viewing page, and their profiles do not list them as contributors to the collab.

These findings prompted us to conduct a follow-up study and more deeply investigate the issues raised by participants during our interviews. Specifically, we wanted to better understand why some collabs successfully produce a completed animation, but most do not. We also wanted to know how participants decided which collabs to join, and whether those criteria accurately predicted successful collabs. In the following section, we describe our second study geared towards addressing these open questions.

4. Study 2

4.1. Methods

To collect data for this quantitative study, we wrote a script that downloaded every thread on the Newgrounds Bulletin Boards (NGBBS) whose last reply was before January 1, 2008. This initial "scraping" captured approximately 137,000 threads. We then filtered these results for threads with the word "collab" in the title, a common convention used to signal collab threads. We divided this corpus into three groups: low activity (less than 10 replies), medium activity (10 to 50 replies), and high activity (50 or more replies). We then attempted to randomly select 300 threads from each group for further analysis. The entire high activity group contained just 292 threads, so we selected all of them. We also scraped profile data on the leaders of each of these collab threads.

We sought to determine which of these collab threads had successfully led to a completed animation, and which had not. Two judges categorized all 892 threads as either a success or a failure. If the thread was a success, the judge identified which completed animation hosted on Newgrounds was the product of the collab.

We then performed three types of additional analysis. First, we performed a content analysis on the first post of each collab thread, where leaders typically provide key details about the collab and try to attract collaborators. The purpose of this content analysis was twofold: (1) to provide us with descriptive statistics on the contents of the first posts of successful vs. failed collab threads; and (2) to allow us to see if we could predict collab outcomes from the first post of collab threads. Two judges coded the same set of test data to generate a coding scheme and reached an acceptable inter-rater reliability measure. Then, the judges split the corpus in half and coded the rest of the first posts individually. The final coding scheme had 47 items, including the items identified as important in the previous study, such as content guidelines, coordination styles, and technical specifications.

Second, we performed an exploratory logistic regression on the results of the content analysis to determine if it was possible to predict collab outcomes from their leaders' first posts. To reduce the number of predictor variables, we grouped the 47 items into four categories: content guidelines, coordination styles, technical specifications, and organization.

Finally, we ran two sets of two-tailed independent samples t-tests on the data we collected. One set of t-tests compared the attributes of leaders of successful collabs to leaders of failed ones. These attributes included the leader's posts on the NGBBS, his or her past experience with collabs and solo animation projects, and any awards the leader won. The other set of t-tests compared activity dynamics within successful vs. failed threads. Specifically, we compared how often, and when, collab leaders and artists posted replies in their collab threads.

More detailed discussion of the methods, results, and implications of this study can be found in (Luther, Ziegler, Caine, & Bruckman, 2009). We limit the discussion in this paper to a brief overview of the most relevant findings.

4.2. Summary of Findings

This study produced several key findings relevant to this paper:

• Most collabs fail.

We found that 780, or 87.4% of the collab threads we coded did not result in a completed animation. Clearly, the collab production process is at least as challenging as our interview study led us to believe, if not more so. This result underscores the need to understand why so many collabs fail.

• It is possible to predict a collab thread's outcome from its first post.

Despite all of the possible events that can occur over a collab's lifetime, the first post still holds many keys to determining its eventual outcome. More to the point, we have identified some attributes of collabs that are correlated with success, which we describe in more detail below. However, because our study is preliminary and looked at archival data, we can only suggest non-directional relationships rather than causations.

• Collabs with more organization and structure are more likely to succeed.

The exploratory logistic regression showed that content guidelines, coordination styles, technical specifications, and organization were all predictors of success. Technical specifications were the strongest predictors, with items

like frame rates and dimensions present in 90% and 81% of successful collabs, respectively. For comparison's sake, we also ran a t-test comparing the word count in high activity collab thread first posts and their outcomes, and found no significant difference. That is, the content of a first post, not the quantity of text, can predict the collab's success or failure.

• Collabs with leaders who are well-known in the community are more likely to succeed.

To operationalize "well-known," we looked at three factors for each leader and collab: (1) how much the leader had posted in the NGBBS; (2) how many Newgrounds awards the leader had won for his or her submitted animations; and (3) the leader's Batting Average, or average rating of the leader's three highest-rated submissions. We found that leaders of successful collabs had, on average, posted more often to the NGBBS (986 vs. 585 total posts), won more awards (1.6 vs. 0.5 awards), and had higher Batting Averages (3.2/5.0 vs. 2.9/5.0).

• Collabs with more activity, by both leaders and artists, are more likely to succeed.

We identified a significant difference in the posting frequency in successful vs. failed collab threads, even when we narrowed the comparison to just high activity threads (50 or more replies each). This trend was apparent for leaders (4.2 vs. 1.9 posts per day) and all collab participants (24.2 vs. 7.3 posts per day). We also noticed a pattern in posting frequency: failed threads see most posting in the first 10% of their life spans, and activity decreases consistently afterwards. Successful threads, however, see a more even distribution of posts over time, with activity reaching its nadir about halfway through the thread's life span, and then steadily increasing until completion.

In summary, our second, quantitative study revealed that members of this amateur COIN struggle to achieve the goal of completing a new animation, with just 112 of 892 collabs, or 12.6%, succeeding in this way. However, we also found evidence that it is possible to improve these numbers. Collab characteristics like the amount of organization and structure (particularly technical specifications); the attributes of the leader; and the patterns and amount of activity within the thread can all suggest that the collab has a high success potential. The next step, we propose, is to help members of this amateur COIN make use of this information when organizing future collabs.

The results of our previous, qualitative study suggest additional avenues for improvement. Interview participants articulated to us the many challenges that collab participants face. The burden was especially onerous for leaders, who take on a multitude of roles throughout the collab production process. By easing this burden, we may help members of this amateur COIN be more successful.

In the following section, we discuss the broader implications of the results of our two studies. We identify a number of opportunities for future research that may apply not only to the amateur COIN described in this paper, but perhaps amateur COINs more generally.

5. Discussion

We propose the following opportunities for future research involving amateur COINs. Where possible, we also describe some ways that we are attempting to take advantage of these opportunities. Because we work at a School of Interactive Computing, our proposed solutions center around the design of collaborative software tools. However, we can imagine many other types of research that could also provide highly useful.

• Opportunity 1: Involving more novices in the collaborative process.

Unlike the professional world, amateur COINs rely on novices to do much of the work. In general, there tend to be far more novices than experts, as there is often a steady stream of newcomers into the community who hope to learn from others and find social support. Thus, it is important for amateur COINs to find ways to take advantage of the collective efforts that novices can offer, despite their limited skills and experience. One way may be through scaffolding, a learning process in which novices are initially provided with extensive social and/or technical supports, but as they gain more experience, these supports are gradually removed, like scaffolding on a building under construction. Another way to take advantage of novices' contributions is to help them find tasks which are appropriate to their skills, interest level, and available time. Wikipedia researchers have taken this approach in developing SuggestBot, a tool which matches Wikipedia editors with articles they may want to edit, based on their

history of past contributions (Cosley, Frankowski, Terveen, & Riedl, 2006, 2007). This general concept may apply to a wide range of amateur COINs.

We are designing a collaborative software tool, tentatively called Pipeline, which offers several features geared towards helping novices contribute meaningfully. Pipeline provides scaffolding in the form of templates for popular collab designs. These templates have pre-set configurations for content guidelines, technical specifications, and coordination styles that are based on collab designs that have been successful. Novice leaders can use these templates initially, and as they become more comfortable, they may create their own designs. Additionally, Pipeline also tries to help novices find work via social networks, which, our interview study found, novices often lack. Novices can add people to their contact lists to receive updates about the collabs with which they are involved, including recruitment and requests for help.

• Opportunity 2: Solving increasingly complex problems.

Amateur COINs have created complex artifacts, including detailed encyclopedia articles, powerful software applications, and highly entertaining animated movies and games. One may ask, what is the limit, as far as how complex the artifacts made through amateur COINs can be?

This question can only be answered by additional research which seeks to test the boundaries, or understand how boundaries are already being tested, with respect to the creative potential of amateur COINs. For example, Mass Animation, a hybrid of a professional animation studio and student animators, has produced a short film, Live Music, that was screened before the feature film Planet 51 in movie theaters around the world. This early success has prompted Mass Animation to try its luck with producing a feature-length film (Barnes, 2009).

Newgrounds animators have not yet completed a feature-length animated movie, although several have been attempted. Yet, given how successful some collabs can be, this challenge seems worthy of additional research and exploration. Similar pursuits in the realm of other amateur COINs, striving towards reaching the maximum potential in each domain, may yield surprising and impressive results.

• Opportunity 3: Providing appropriate incentives.

Amateurs, by definition, participate out of their love for the practice, not to make a living. Research also shows that people often contribute to online collaborations for social and psychological, rather than monetary, rewards (Benkler, 2002; Kollock, 1999). The incentive mechanisms used to motivate members of amateur COINs must take into account these distinctions from professional COINs and business contexts.

Earlier in this paper, we described how most websites that host user-generated content, Newgrounds included, fail to appropriate recognize contributions to collaborative projects. Instead, these websites either credit just the person who uploaded the content, or, less often, allow the uploader to credit a limited number of co-authors. These restrictions are problematic. Collab leaders, for example, must negotiate the often-uncomfortable territory of acknowledging some artists' contributions, but not others. Worse, some artists whose work is included in the final animation may not be given credit. Ethical issues aside, artists are unable to present an accurate record of their online reputation and do not receive the desired recognition that motivated them in the first place.

Many alternatives to these models exist, some of which may prove to be easier, fairer, and more rewarding. In building Pipeline, we are divorcing the notions of attribution and commendation, which are often conflated in Newgrounds' co-author system. Every member of a Pipeline collab is automatically credited, thus ensuring attribution and influencing their online reputations. However, leaders can also single out some artists as having performed especially well. A similar combination of automated attribution and manual commendation has also been shown to work well on Wikipedia (Kriplean, Beschastnikh, & McDonald, 2008), suggesting that this approach may be suited to a wide range of amateur COINs.

• Opportunity 4: Reducing the burden on leaders.

Leaders play a major part in almost every online collaboration, including Newgrounds collabs, Wikipedia, and most mature open-source software projects (Forte, Larco, & Bruckman, 2009; Luther & Bruckman, 2008; O'Mahony & Ferraro, 2007). Yet, at least in the case of Newgrounds, leaders are overburdened. They must design the project, manage the artists, complete the project, and manage all of the challenges and sub-tasks involved in each of these phases. If leaders drop out or ask to be replaced, collabs usually fail. When we asked collab participants

why collabs fail, many pointed to the leader as the single most important factor. It is not difficult to imagine that other types of amateur COINs may face similar issues.

Two promising strategies for reducing the burden on leaders in amateur COINs are automation and decentralization. Wikipedia and the open-source software movement have taken advantage of both of these strategies, with excellent results. For example, Wikipedia uses "bots"—scripts that automatically handle tedious or repetitive editing tasks—so that human editors can focus on more creative, rewarding work. Wikipedia has also decentralized its leadership, creating a hierarchy of administrative levels and roles beneath Jimmy Wales, Wikipedia's founder (Forte et al., 2009). Similarly, many open-source software projects use bug trackers and version control software to automate tasks related to file sharing and asset tracking, and role specialization is commonplace in the larger projects (Gutwin, Penner, & Schneider, 2004; Shah, 2006; Yamauchi, Yokozawa, Shinohara, & Ishida, 2000).

These approaches to automation and decentralization may also apply to other types of amateur COINs. In our research, we are equipping Pipeline users with the ability to promote several leaders in a given collab. We are also providing a digital commons where animators can upload their submissions directly and solicit feedback. Pipeline automatically tracks the latest version and archives older ones, allowing leaders to focus on creative, rather than administrative, duties.

• Opportunity 5: Expanding to new contexts.

Currently, the bulk of research on amateur COINs focuses on Wikipedia and open-source software (OSS). Yet, the evidence suggests that amateur COINs may already exist, or can exist, in many other contexts, some radically different from encyclopedia writing or software development. Our studies of online collaboration in the domain of computer-animation speak to the breadth of amateur COINs and their great, yet untapped, potential.

We identified key differences between collabs and other types of online creative collaboration (namely, Wikipedia and OSS). For example, while collab participants strive to create something original, Wikipedia editors strives to make more accessible what is already known, and OSS developers often strive to recreate free versions of existing commercial software. Another notable difference is completion. While collab participants seek to complete an animated movie, Wikipedia and OSS projects are never truly completed; a newer version is often just around the corner.

These differences suggest that more research is needed to understand how amateur COINs members' needs change as the domain changes. More importantly, they require us to consider what new domains have not yet been explored, but might also yield successful amateur COINs. In addition to encyclopedias, software, and animations, what else might amateur COINs create that educates, entertains, or inspires us?

6. Conclusion

Amateur COINs are responsible for some of the greatest creative achievements in recent years, from the world's largest encyclopedia to some of its most widely-used software applications. Yet, most COINs research thus far has focused on professionals and business contexts. In this paper, we describe several studies of an amateur COIN within the domain of computer animation. The focus of our study, the popular online community Newgrounds, hosts over 1.8 million registered members and over 160,000 Flash-animated movies and games. In one study, we interviewed 17 members of online, collaborative animation projects, called "collabs," focusing on the collab production process and the challenges to completion, especially for leaders. Our findings include: (1) leaders are crucial and pervasive, but overburdened; (2) few collabs produce a completed animation; (3) the workload is modularized, not specialized; (4) the collab production process is linear; (5) recruitment strategies depend on the leader's experience; (6) collab participants lack adequate technological support; and (6) recognizing everyone's contributions is problematic.

These results prompted us to conduct a follow-up study to better understand why some collabs succeed and others fail. We analyzed 892 forum threads where collabs are organized, using quantitative methods such as content analysis, exploratory logistic regression, and t-tests. We found that most collabs (over 80%) fail. We also found that it is possible to predict which collabs have the most success potential. Successful collabs (1) are well-structured and organized, especially with sufficient technical specifications; (2) have leaders who are well-known in the

community; and (3) have leaders and artists who post frequently and consistently throughout the collab's life span. Because our data is archival rather than experimental, we only indicate correlations, not causations.

Our studies suggest that amateur COINs have great potential, and that this potential has not yet been reached. We propose several opportunities for future work to understand the power and limitations of amateur COINs. These are: (1) involving more novices in the collaborative process; (2) solving increasingly complex problems; (3) providing appropriate incentives; (4) reducing the burden on leaders; and (5) expanding to new contexts. We are currently building a collaborative software tool to explore some of these opportunities.

7. Acknowledgements

The authors wish to thank Kevin Ziegler for his help with data collection and coding, and Kelly Caine for her assistance with statistical analyses. The authors also wish to thank their participants for their valuable insights, and the Newgrounds staff for their time and resources. This research is supported by National Science Foundation CreativelT Award #0855952, "Pilot: Supporting and Transforming Leadership in Online Creative Collaboration."

8. References

Adobe. (2009, December). Adobe - Flash Player Version Penetration. Retrieved March 22, 2010, from http://www.adobe.com/products/player_census/flashplayer/version_penetration.html

Barnes, B. (2009, July 16). An Animated Film Is Created Through Internet Consensus. The York Times. Retrieved from http://www.nytimes.com/2009/07/16/movies/16mass.html?_r=l

Benkler, Y. (2002). Coase's Penguin, or Linux and the Nature of the Firm. Yale Law Journal, 112(3), 369-446.

Bryant, S. L., Forte, A., & Bruckman, A. (2005). Becoming Wikipedian: transformation of participation in a

collaborative online encyclopedia. In Proceedings of the 2005 international ACM SIGGROUP conference onSupporting group work (pp. 1-10). Sanibel Island, Florida, USA: ACM. doi:10.1145/1099203.1099205

Cattani, G., & Ferriani, S. (2008). A Core/Periphery Perspective on Individual Creative Performance: Social

Networks and Cinematic Achievements in the Hollywood Film Industry. ORGANIZATION SCIENCE, 19(6), 824-844. doi:10.1287/orsc,1070.0350

Cosley, D., Frankowski, D., Terveen, L., & Riedl, J. (2006). Using intelligent task routing and contribution review to help communities build artifacts of lasting value. In Proceedings of the SIGCHI conference on Human Factors in computingsystems (pp. 1037-1046). Montreal, Quebec, Canada: ACM.

Cosley, D., Frankowski, D., Terveen, L., & Riedl, J. (2007). SuggestBot: using intelligent task routing to help people find work in wikipedia. In Proceedings of the 12th international conference on Intelligent user interfaces (pp. 32-41). Honolulu, Hawaii, USA: ACM. doi:10.1145/1216295.1216309

Emigh, W., & Herring, S. C. (2005). Collaborative Authoring on the Web: A Genre Analysis of Online

Encyclopedias. In Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS'05) - Track 4 - Volume 04 (p. 99.1). IEEE Computer Society.

Faulkner, R. R., & Anderson, A. B. (1987). Short-Term Projects and Emergent Careers: Evidence from Hollywood. AmericanJournal of Sociology, 92(4), 879-909.

Forte, A., Larco, V., & Bruckman, A. (2009). Decentralization in Wikipedia Governance. Journal of Management Information Systems, 26(1), 49-72.

Giles, J. (2005). Internet encyclopaedias go head to head. Nature, 438(1010), 900-901. doi:10.1038/438900a

Gloor, P., & Cooper, S. (2007). The New Principles of a Swarm Business. MIT SloanManagement Review.

Retrieved from http://sloanreview.mit.edu/the-magazine/articles/2007/spring/48312/the-new-principles-of-a-swarm-business/

Gloor, P. A. (2006). Swarm Creativity: Competitive Advantage through Collaborative Innovation Networks (1st ed.). Oxford University Press, USA.

Gutwin, C., Penner, R., & Schneider, K. (2004). Group awareness in distributed software development. In

Proceedings of the 2004ACM conference on Computer supported cooperative work (pp. 72-81). Chicago, Illinois, USA: ACM. doi:10.1145/1031607.1031621

Hertel, G., Niedner, S., & Herrmann, S. (2003). Motivation of software developers in Open Source projects: an Internet-based survey of contributors to the Linux kernel. Research Policy, 32(7), 1159-1177. doi:10.1016/S0048-7333(03)00047-7

Kollock, P. (1999). The Economies of Online Cooperation: Gifts and Public Goods in Cyberspace. In M. A. Smith & P. Kollock (Eds.), Communities in Cyberspace: Perspectives on New Forms of Social Organization (pp. 220-239). New York, NY, USA: Routledge. Retrieved from http://www.sscnet.ucla.edu/soc/faculty/kollock/papers/economies.htm

Kriplean, T., Beschastnikh, I., & McDonald, D. W. (2008). Articulations of wikiwork: uncovering valued work in wikipedia through barnstars. In Proceedings of the ACM 2008 conference on Computer supported cooperative work(pp. 47-56). San Diego, CA, USA: ACM. doi:10.1145/1460563.1460573

Luther, K, & Bruckman, A. (2008). Leadership in online creative collaboration. In Proceedings of the ACM 2008 conference on Computer supported cooperative work (pp. 343-352). San Diego, CA, USA: ACM.

Luther, K, Ziegler, K, Caine, K. E., & Bruckman, A. (2009). Predicting successful completion of online

collaborative animation projects. In Proceeding of the seventh ACM conference on Creativity and cognition (pp. 391-392). Berkeley, California, USA: ACM. doi:10.1145/1640233.1640316

O'Mahony, S., & Ferraro, F. (2007). The emergence of governance in an open source community. Academy of Management Journal, 50(5), 1079-1106.

Seidman, I. (2006). Interviewing As QualitativeResearch: A Guide for Researchers inEducationAnd the Social Sciences (3rd ed.). Teachers College Press.

Shah, S. K. (2006). Motivation, Governance, and the Viability of Hybrid Forms in Open Source Software Development. Manage. Sci., 52(7), 1000-1014.

Strauss, A. C., & Corbin, J. M. (1998). Basics ofQualitativeResearch: Techniques andProceduresfor Developing Grounded Theory (Second Edition.). Sage Publications, Inc.

Weber, S. (2004). The Success of Open Source. Harvard University Press.

Yamauchi, Y., Yokozawa, M., Shinohara, T., & Ishida, T. (2000). Collaboration with Lean Media: how open-source software succeeds. In Proceedings of the 2000 ACM conference on Computer supported cooperative work (pp. 329-338). Philadelphia, Pennsylvania, United States: ACM. doi:10.1145/358916.359004

Ye, Y., & Kishida, K. (2003). Toward an understanding of the motivation Open Source Software developers. In Proceedings of the 25th International Conference on Software Engineering (pp. 419-429). Portland, Oregon: IEEE Computer Society.