Measuring Fandom: Social TV Analytics and the Integration of Fandom into Television Audience Measurement - Napoli & Kosterich
Measuring Fandom: Social TV Analytics
Introduction
Fan activities vary, some observable by media stakeholders, others less so.
Media industries want to observe, analyze, and monetize fan activities, from policing copyright to improving content.
The intersection of fan activity and industry interest leads to the measurement of fandom.
Measurement implies systematic observation and aggregation of fan activity for analysis.
Technological changes, especially social media, are making fan activities more measurable.
The measurement of fandom and its implications have received little attention in fan studies.
This chapter explores differences between traditional Nielsen ratings (sample-based) and Nielsen Twitter Television Ratings (social media activity).
Comparing representations of programs from these systems reveals how measuring fans differs from measuring consumers.
Distinguishing Among Audiences, Consumers, and Fans in Television Viewing
It's useful to distinguish between measuring audiences and measuring fans.
Fandom implies higher engagement, appreciation, and activity than being an audience member.
Fans are a subset of audiences; Lewis (1992) calls them the most visible and identifiable audiences.
Fans are the easiest to observe and measure because they are the most visible.
Abercrombie and Longhurst (1998) describe fans as "a form of skilled audience," placing them on an "audience continuum."
The continuum includes consumers, cultists, and enthusiasts, based on intensity of media usage.
Traditional measurement focuses on audiences as consumers.
Fans possess technical, analytical, and interpretive skills related to media content.
Fans engage more in communal activities.
Bielby, Harrington, and Bielby (1999) state that viewing TV is private, but being a fan involves public activities and emotional involvement.
Industry discourse distinguishes fans from audiences.
Mary Meeker: "An audience changes the channel when the show is over. A fan base shares, comments, creates content…magnifying the show’s reach" (Bloom 2014).
Key distinguishing characteristics are the volume and intensity of activities related to programming.
Measuring Television Audiences and Fans
Traditional "audience measurement" has only scratched the surface (Napoli 2011).
It has focused on superficial indicators of audience exposure.
Only the consumption dimension has mattered to measurement systems.
Attempts to measure audience appreciation have not been embraced (Napoli 2011).
Historically, "[i]n the business of television viewers matter more than fans" (Bielby, Harrington, & Bielby 1999: 35).
Industry mechanisms for understanding fandom have been ad hoc.
Early mechanisms included fan letters expressing enthusiasm.
The original Star Trek series was renewed due to fan letters (Cochran 2012; Collins 1997).
Fan activities have sustained programs facing cancellation (Brower 1992).
Executives used fan letters as an indicator to give programs more time (Sabal 1992).
Fan activism included supporting program sponsors and public displays of support (Savage 2014).
The Internet provided new outlets for fan support.
Fan web pages and online communities allowed fans to express opinions (Scardaville 2005).
Program producers monitored and responded to fan activity.
The key was to produce a measurable indicator of fan support.
Savage: "The evolution of fan campaigns taught fans that audience attention would often not be enough" (2014).
Gray: "Intentionally or not, audience research often equals fan research" (2003: 64).
Audience measurement differs from audience research.
Fandom measurement has been impressionistic and disconnected from audience measurement.
Social Media and Television Fandom
Social media has disrupted this state of affairs (Bore & Hickman 2013; D’heer & Verdegem 2015; Harrington & Bruns 2013).
Social media platforms make television fandom more visible and measurable.
Nielsen (2014): Over one million people per day discuss TV on Twitter, garnering over eleven million readers.
One might critique social media activity as a measure of fandom, arguing that it represents too low of an engagement threshold.
Network executives use social media activity as an indicator of fandom.
CW President Mark Pedowitz renewed a show with higher social engagement over a higher-rated show (Hibberd 2014).
Fandom contributed to the renewal of NBC’s Hannibal (Hall 2013).
Social media platforms have facilitated "newly visible" forms of fan activity (Hills 2013: 144).
Visibility creates incentive and opportunities for rigorous systems of measurement.
Social TV Analytics
This section discusses systems to measure, aggregate, report, and analyze social media activity.
The focus is on transforming fan activity into an "institutionally effective" form of audience information (Ettema & Whitney 1994).
Most social TV analytic services use "web scraping."
Conversations from social media platforms are aggregated and classified via language processing and hashtags.
Some services categorize sentiment (positive/negative).
Nielsen Twitter Television Ratings (NTTR) has emerged as the industry standard (Kosterich & Napoli forthcoming).
NTTR ranks the top ten most social episodes based on unique audience, impressions, unique authors, and tweets.
NTTR operates similarly to traditional audience ratings.
Many firms compete in the social TV analytics industry.
Traditional TV ratings require a sample; social TV analytics draw from the online population.
NTTR focuses on Twitter activity, raising questions about whether it reflects the broader social media space.
The demographics of Twitter users may not reflect the television viewing population (Duggan et al. 2015; Napoli 2014).
Social TV analytics does not seek to accurately represent the population as a whole.
The population under analysis is defined by expressive activities and platforms.
Fan communities may "game" the systems by hijacking hashtags (Highfield, Harrington, & Bruns 2013).
The motivations behind such activities may not always reflect genuine fandom.
Social TV analytics are a flawed measure of television fandom, similar to traditional ratings (Napoli 2003).
They have reached a point of "institutional effectiveness" and are embraced by the industry (Kosterich & Napoli forthcoming).
They serve as the primary means by which fandom is interpreted and acted upon.
Consumption- versus Fandom-Oriented Approaches to Audience Measurement: A Comparative Analysis
Social TV analytics may produce a different portrait of hit programs.
Programs with high social media conversation may not reach large audiences (Napoli 2014; Deller 2011).
Academic researchers have yet to analyze how program success differs across systems.
A stronger integration of fandom might influence perceptions of what constitutes a hit program.
This section presents a comparative analysis of Nielsen's traditional and Twitter ratings systems.
The analysis focuses on the top ten prime-time program lists from each system for twenty-five weeks (September 22, 2014, through March 15, 2015).
The dataset was constructed from weekly ratings information made publicly available by Nielsen.
Traditional ratings data were compiled to match the parameters provided by Nielsen’s social ratings.
NTTR provides a weekly report based on social activity three hours before, during, and three hours after broadcast.
Traditional ratings data were collected based on the live plus same-day metric.
The demographic skew of Twitter users is under fifty years old (Duggan et al. 2015); the dataset uses traditional ratings information from adults ages eighteen to forty-nine.
Repeat episodes and sports programming were discarded.
The unit of analysis is the ranking slot (N = 500 slotted shows, 250 slots from each system).
Programs were categorized according to genre and network source.
Genre classifications were obtained from an institutionalized genre classification system.
There were sixteen distinct genre categories.
Genre composition was measured with the Hirschman-Herfindahl Index (HHI). , where is the market share of firm i in the market and n is the number of firms. A diverse composition yields an HHI approaching zero; hits from one genre yield an HHI of 10,000.
T-tests were conducted to determine if the average weekly number of different genres represented by each system’s hits were statistically significant from one another.
This determined whether fandom results in a greater or lesser diversity of program types than traditional ratings.
Source was identified as the network on which the program aired.
There were thirty-two distinct sources.
HHI and t-tests were computed.
This determined whether fandom corresponds with an increase or decrease in the diversity of sources of hit programs.
Results
Under traditional ratings, 53 programs appeared in the top ten lists.
Under social TV analytics, 84 programs appeared.
Hit programs by traditional ratings fell across eleven genre categories with an HHI of 2201.92.
By social TV analytics, hits fell across fourteen genre categories with an HHI of 1923.2.
The average number of different genres by traditional ratings was 4.08.
The average number of different genres by social TV analytics was 4.58.
Significant at p < .10 (t_{25} = 1.73, p < .10).
Television hits produced by fandom are more evenly distributed across genre categories.
Traditional ratings portray a TV marketplace with eight networks responsible for top-ten programming (HHI of 2261.44).
Social TV analytics show thirty-two networks (HHI of 1168.70).
Traditional ratings present an average of 4.24 network sources each week.
Social TV analytics present an average of 7.28 channel sources each week.
Significant at p < .01 (t_{25} = 9.99, p < .01).
Conclusion
Embracing social TV analytics will lead to a different understanding of hit programming and sources.
Traditional audience measurement provides a constrained portrait of audience preferences.
Exposure-focused approaches produce a narrower set of hits from a limited range of sources.
Redefining a hit in terms of social media conversation expands the diversity of hit programs.
These patterns reflect the disparity between viewership and engagement.
This reminds us of least objectionable programming, where programmers sought acceptability over intense feelings.
Consumption dynamics differ from engagement dynamics.
A narrower range of programs produces large audiences than highly engaged audiences.
Programmers unable to succeed under traditional measurement may focus on social media.
Barriers to achieving large audiences are higher than barriers to social media activity.
Audiences on social media may differ from those in Nielsen's panels.
Social media analytics may tap into unobserved tastes and preferences.
If social TV analytics become influential in decision-making, this could diversify the industry.
This would encourage a greater array of content types and facilitate a more competitive landscape.
As decision-makers observe and respond, the programming and marketplace will evolve.
Institutionalizing social TV analytics represents a direct mechanism for fandom to influence television production.
There is increasing value placed on fans and their role.
A media system relying on fandom indicators is likely to serve a more diverse range of tastes and support programming from a greater diversity of sources.
Institutionalized representations of fandom can affect cultural production in ways that are desirable and beneficial.
This potential is limited by the fact that social TV analytics have settled into a secondary role.
However, disrupting the emphasis on exposure-based ratings could still have significant ramifications.