Matt Carlson, Associate Professor at St. Louis University

Matt Carlson, an associate professor of communication at Saint Louis University, was set to announce a collaborative research project that would “connect a lot of dots surrounding news metrics and digital distribution platforms.” He wanted to examine journalism’s embrace of real-time audience data by shining a spotlight on “all the different actors involved, from reporters and editors and news management to engineers and salespersons at data analytic firms to the audience on the other end.”

But first, he needed to find a term that tied everything together. “Measurable journalism” was the solution.

In a special issue of the academic journal Digital Journalism, “Measurable Journalism: Digital Platforms, News Metrics, and the Quantified Audience,” nine researchers explore the implications of these technological and cultural shifts. Carlson, who edited the special issue and wrote an introductory essay, “Confronting Measurable Journalism,” explained his interest in this topic in an e-mail to MediaShift: 

“When we think about measurable journalism, we need to keep in mind all the parts that go into it without privileging one over another. Often discussions of news metrics focus on technology, but it is equally about human actions that direct technology to do x and not y.

A concern I have with measurable journalism is when what can be measured takes precedent over what should be measured. These are sophisticated technologies, but they can only ever get to what people do. What we can’t know is what news audiences think or why they do what they do. I am always worried that user data becomes so fetishized that we forget it can only ever be a partial representation. We talk about such complex terms as impact or engagement but then we look for simple measurements.

The idea of measurable journalism can be both promising with the hope of creating journalism that is more accountable to the audience and frightening with the threat of journalists losing control over what is newsworthy. It may bring journalists and their audiences closer together or it might push economic imperatives ahead of journalism’s public service mission. Given these outcomes, what we need is vigilance and a solid understanding of all the forces in play.”

MediaShift discussed these forces in short interviews with the researchers who contributed to the special issue.

Quantified Audiences in News Production: A Synthesis and Research Agenda

Interview with Rodrigo Zamith, assistant professor, University of Massachusetts-Amherst

Why the interest in studying measurable journalism?

There are two main reasons. The first has to do with a professional observation: Journalism is becoming more sensitive to and powered by measurement. We see this in the rise of “data journalism” as well as the proliferation of audience analytics — the latter being the focus of much of my recent work. Those shifts have important implications for how journalism is constructed, performed and rewarded, yet the phenomenon is not yet well understood by scholars (or practitioners). The second reason is personal: I’m a nerd and think in terms of numbers. I find the tensions playing out as quantification and quantitatively oriented actors gain foothold in newsrooms to be fascinating.

What did your research show?

My contribution focused on synthesizing the current literature on audience analytics and metrics, offering new lenses for studying the phenomenon and identifying future research directions for the scholarship. Three arguments stand out in the piece: First, we are witnessing a new wave of audience measurement in journalism (following two waves in the 1930s and 1970s) that is driven by audience analytics (systems that automatically capture information about individuals’ media use). Second, while contemporary journalism is not being driven by quantifications of audiences (i.e., audience metrics), both audiences and quantification are playing far more prominent roles in news production than in the past. Third, scholars and practitioners have become less pessimistic about the impact of audience metrics and now recognize more nuanced impacts on news production as well as opportunities for using them to advance journalistic goals.

I have a new piece out (in Digital Journalism) arguing that we are witnessing a third wave of audience measurement in journalism and synthesizing the knowledge across 5 areas related to news production: https://t.co/G1Fzu3rGwo. Feedback welcome! #ddj #journalism #metrics

— Rodrigo Zamith (@rodzam) March 7, 2018

What are the main takeaways for journalists, journalism educators and others who are interested in media metrics?

Journalists and educators need to take analytics and metrics seriously. The measurement of audiences will only become more sophisticated and news organizations will face further pressures to make use of those data. Journalists should seek out training on how to use their newsrooms’ analytics suites (e.g., Chartbeat) and/or ask for permission to access the system. Educators need to ensure they incorporate analytics and metrics into their curricula and also provide students the opportunity to engage with those suites (e.g., integrating them into student media offerings, at minimum). In both cases, serious conversations need to be had about how to use those data sensibly — from influencing organizational coverage decisions to developing reward structures for individual journalists. I find the argument that metrics should neither be restricted to the business side nor the primary driver of journalism to be especially persuasive. Analytics are tools that can be put to good use, and that means trying to align journalistic aims like satisfying community information needs with the many data points that analytics can offer. At the same time, those tools can be misused and practitioners should therefore maintain a healthy skepticism and promote robust dialogue.

The Audience-Oriented Editor: Making Sense of the Audience in the Newsroom

Interview with Raul Ferrer-Conill, Ph.D. candidate, Karlstad University, Sweden, and Edson C. Tandoc, Jr., assistant professor, Nanyang Technological University, Singapore

Why the interest in studying measurable journalism?

In the last decades we’ve seen an increasing quantification of journalism, spurred by the use of metrics and analytics. These metrics are effectively the new way by which news organizations make sense of the audience. Combined with the commercial urgency, the need to entice and engage audiences makes the quantification of audience news consumption a key factor to understand the current news production process. Researchers and practitioners should pay more attention to how journalism production is quantified, measured and understood.

What did your research show?

First, the definition of engagement is almost entirely centered on different types of metrics. Second, while audience-oriented editors take part in the editorial process, their role is to help journalists negotiate between the information obtained by their metrics and their journalistic intuition to make editorial decisions. Third, there is a lack of cohesiveness regarding what these newsroom positions are and how they operate. We provide insight on the pervasiveness of metrics and quantification of journalistic processes by offering a more nuanced understanding of a new set of editorial roles.

What are the main takeaways for journalists, journalism educators and others who are interested in media metrics?

The more that metrics are part of news production and the increasing number of audience-oriented editors legitimize and institutionalize metrics. Therefore, understanding the impact of audience measurement on news work requires an analysis of these emerging roles, who act as intermediaries between audiences and the newsroom through their interpretation and valuation of audience data. The reliance on metrics and social media insights questions their capacity to capture the audience. The analytic tools are constrained by what they can measure and rely on likes, shares, number of comments and other audience metrics to define engagement. In this sense, it is user activity and behavior that becomes a proxy for the voice of the audience. This is a limited understanding of the audience, let alone having a dialog with the audience. Editors can assess the performance of their editorial choices as they scrutinize metrics in real time, but they are limited by and reliant on the technological affordances of the tools they use. We argue that this dialog is predominantly informed by metrics and therefore it needs to be understood as such. Metrics are not necessarily a valid way to measure audience engagement and should be used cautiously.

Selecting Metrics, Reflecting Norms: How Journalists in Local Newsrooms Define, Measure, and Discuss Impact

Interview with Elia Powers, assistant professor, Towson University

Why the interest in studying measurable journalism?

My interest is in examining the ways in which journalists think about and ultimately measure their work’s impact. Impact is a buzzword in newsrooms, but there are so many ways to define the term that little can be done to move the conversation forward until there’s more clarity about what journalists mean when they talk about impact. It’s also critical for journalists to feel comfortable publicly discussing their work’s impact, because newsrooms now more than ever need to make the case to audiences and funders that civic-oriented journalism makes a difference in local communities.

What did your research show?

Interviews with journalists from a range of local news organizations in one U.S. city found that they welcome the opportunity to inform audiences and effect change, and they had no issues discussing impact with newsroom colleagues and in promotional materials. However, journalists were generally more hesitant to discuss their work’s impact outside the newsroom — in follow-up news stories, social media posts, interviews, etc. Some journalists were concerned about being perceived as too self-congratulatory or being labeled advocates. There was a perception among some participants that publicizing impact violated journalistic standards of objectivity and detachment. Additionally, journalists had many ways of defining and measuring impact, one of which was audience analytics — although many felt these were more about engagement than enduring signs of impact. Effect-oriented metrics (audience awareness, public discourse, public policy, etc) were widely considered the best to assess impact but among the most difficult to systematically measure.

What are the main takeaways for journalists, journalism educators and others who are interested in media metrics?

First, I propose that measurable journalism should not be limited to quantitative metrics, often the standard by which news coverage is judged. Much of what journalists want to measure has little to do with audience analytics. Second, as I argue in the article, “publicizing the impact of journalism, when facts support such a claim, is central to the journalistic process and necessary for newsrooms to justify their funding…Newsroom policies and professional codes of conduct should clarify that engaging in public discourse on impact is central to the journalistic process, a necessary part of communicating with the public, and a way for newsrooms to justify their funding rather than a sign of self-promotion or
advocacy.”

Dimensional Field Theory: The Adoption of Audience Metrics in the Journalistic Field and Cross-field Influences

Interview with Qun Wang, Ph.D. Candidate, Rutgers University

Why the interest in studying measurable journalism?

Fifteen years ago, I was a TV news anchor and reporter in Beijing, China. I was able to access both quantitative and qualitative audience information due to the nature of the show that I covered: on the one hand, we kept a close eye on the ratings because the show was a signature show in Beijing’s competitive TV news market; on the other hand, we had our ears open to the three hotline phones on which the show relied for audience members to share news tips, feedback and comments like “I don’t like the anchor’s hair!” We sometimes learned a lot from the audience information and sometimes got lost in it. The show ceased years ago, but I have never stopped wondering how the team would deal with today’s audience information in the digital age if the show was still around. This work experience and this particular question have contributed to my interest in measurable journalism.

What did your research show?

In the years that I worked in the newsroom, I already felt that newsroom practices and norms were often a result of the negotiation of different forces. In my study, drawing on French sociologist Pierre Bourdieu’s field theory, I took a historical and relational approach to embed the journalistic field into a wider media ecosystem. Particularly, the study examines how the adoption of audience metrics in news media has been shaped by influences inside and outside the journalistic field in order to understand the origin, driving forces and implications of this trend. I looked at neighboring fields adjacent to the journalistic field, such as the online advertising and online audience research fields, as well as web analytics services working with news media that I identified as the intermediate field to trace the evolution and influence of these fields. I also looked into the journalistic field itself and identified three dimensions — the techno-economic means, the subject of journalism and the object of journalism — that construct the field and serve as sources of internal influence.

What are the main takeaways for journalists, journalism educators and others who are interested in media metrics?

The main takeaway of this study is that the journalistic field is not a static or monolithic arena. Rather, it is a social universe that is subject to and constantly interacts with cross-field influences. Therefore, to understand the root, development and effects of measurable journalism, we may need to inspect our own field and look elsewhere.

Boundary Work, Interloper Media, and Analytics in Newsrooms: An Analysis of Web Analytics Companies’ Role in News Production

Interview with Valerie Belair-Gagnon, assistant professor, University of Minnesota, and Avery Holton, assistant professor, University of Utah

Why the interest in studying measurable journalism?

Journalism has been undergoing a series of fascinating changes for several decades. In particular, social media has challenged the ways in which we measure the success of journalism. Much of that success still rests in financial growth, or at least stability, which itself is increasingly dependent on audience interactions with journalists and the content they produce. We’ve begun moving past a reliance on journalistic or editorial intuition and instead see tangible value in understanding complex web metrics and analytics. So if the latter are beginning to drive journalistic decision making, particularly in news production and professional identity, then they are critical to examine.

https://twitter.com/journoscholar/status/969222955677908992

What did your research show?

Our most recent research shows that web analytics companies seek to understand and address news production values and norms without assuming responsibility as journalists. We think of these companies, or their employees more specifically, as implicit media interlopers. These are journalistic outsiders, more or less, who are bit more welcome in the journalistic process than previous interlopers (e.g., citizen journalists, bloggers) because of the value they add to news products. These companies also foster profit-oriented norms and values in newsrooms by introducing web analytics as disruptive, connective and routinized in news production. By offering a product that needs to be modified on a continuous basis because of changes in the structure of the web and audience behaviors, web analytics companies foster a milieu of constant experimentation with old and new products. This helps place them squarely in the middle of evolving news organizations that are turning more to disruptors and innovators as they grapple for financial footing.

What are the main takeaways for journalists, journalism educators and others who are interested in media metrics?

Today, like many other technological innovations in newsroom, disruption increasingly comes from innovations from outside companies and individuals. As journalism evolves, scholars and practitioners need to understand more deeply what the values and practices are that these disruptive innovators bring to journalism. We’re not just talking about web analytics here, but rather programmers, app developers, drone hobbyists, and others who are interacting with journalists and news organizations in ways that are giving new meaning to what exactly journalism is and who exactly is doing it.

Engineering Consent: How the Design and Marketing of Newsroom Analytics Tools Rationalize Journalists’ Labor

Interview with Caitlin Petre, assistant professor, Rutgers University

Why the interest in studying measurable journalism?

Way back in 2010, Nick Denton, founder of the now-defunct Gawker Media, said, “probably the biggest thing in internet media isn’t the immediacy of it, or the low costs, but the measurability.” Superlatives are tricky, but Denton was surely right that the unprecedented ability to measure audience behaviors and demographics is a defining characteristic of digital media — one that has major implications for the working conditions in this industry and the kind of journalism that is produced. Media scholars have an urgent role to play in helping to interpret and explain the causes, manifestations and consequences of measurable media.

What did your research show?

In the early stages of my research on the role of analytics in journalism, I kept encountering the same puzzle. Journalists at a wide range of news outlets would profess a profound wariness or even hostility toward analytics tools, often seeing them as a threat to their professional autonomy and integrity. This by itself wasn’t all that surprising: Sociological research has found that workers (especially those, like journalists, who consider themselves to possess some kind of special knowledge or expertise) often resist the implementation of technologies that quantify their performance and rank them against each other.

But even as they regarded analytics tools with suspicion and resentment, journalists didn’t seem to be resisting them very much. On the contrary, many journalists would describe feeling “addicted” to real-time analytics tools, consulting them more frequently than was required or even encouraged by their managers, and scheming about how to boost their stats.

My article aims to figure out why that is. I find that a big part of the answer has to do with something that often gets overlooked in these discussions: the design and marketing of real-time newsroom analytics tools. Newsroom analytics companies engineer their dashboards to provide a user experience that is strongly habit-forming, flattering and emotionally compelling. The resulting products are so “sticky” that explicit managerial coercion to boost traffic (which many journalists would not take kindly to) becomes unnecessary. Once journalists get hooked on looking at real-time analytics tools, they begin to monitor themselves. They also push themselves to work harder and harder in hopes of gaining ever-higher traffic.

What are the main takeaways for journalists, journalism educators and others who are interested in media metrics?

When we have conversations about analytics in journalism, we tend to focus on which metrics are provided: time spent or page views? Scroll depth or uniques? In other words, we assess the merit of each metric and speculate about the kind of journalism it might incentivize. But my findings indicate that anyone seeking to make sense of the role of analytics in contemporary journalism should be just as attentive to the way the data are presented, and the daily experience of using these tools, as we are to the metrics themselves.

The Elusive Engagement Metric

Interview with Jacob Nelson, Ph.D. candidate, Northwestern University

Why the interest in studying measurable journalism? 

Over the past few years, a growing number of journalism stakeholders and researchers have argued that newsrooms should make “audience engagement” one of their chief pursuits. This term has many interpretations that stem from one underlying belief: Journalists better serve their audiences when they explicitly focus on how their audiences interact with and respond to the news in the first place. However, those who hope to make audience engagement a larger part of journalistic practice need to first settle an internal debate surrounding how audience engagement should be defined and evaluated. Because the term currently lacks an agreed upon meaning — let alone metric — it has become an object of contestation. The efforts to make audience engagement central to news production therefore present an opportunity to learn how journalism is changing, as well as who within the field have the power to change it.

My latest article looks at how @wearehearken persuades newsrooms to adopt audience engagement as a goal, and the institutional obstacles they navigate in the process. https://t.co/cAXzUP5vFf

— Jacob Nelson (@jnelz) March 6, 2018

What did your research show?

My study draws on an ethnographic case study of Hearken, a company that offers audience engagement tools and consulting to about 100 news organizations worldwide. Findings show that news industry confusion surrounding how audience engagement should be defined and measured has left Hearken unable to quantify the benefit of its offerings. The news industry currently privileges measures of audience size, so newsrooms face economic incentives to pursue audience growth (which they can measure) rather than audience engagement (which they can’t). Instead, Hearken’s pitch to newsrooms relies primarily on appeals to intuition. Its employees argue that their interpretation of audience engagement will lead to a better quality of journalism, which will inevitably result in increased audience revenue as well. Though some newsrooms refuse to invest in Hearken’s offerings without proof they will yield some measurable return, others seem eager to take the chance. The success of Hearken’s faith-based approach indicates that many in journalism innately believe the profession should improve its relationship with the audience.

What are the main takeaways for journalists, journalism educators and others who are interested in media metrics? 

Hearken’s effort to spread its interpretation of audience engagement is just one piece of an ongoing public contest to determine journalistic practice. There are countless conversations about audience engagement that occur annually at a variety of journalism practice and research conferences. These conversations tend to include editors, reporters, and publishers, but rarely include employees of companies like comScore and Nielsen who are in the business of understanding how audiences behave. What makes this omission confounding is the fact that these firms are having their own conversations about audience engagement. The fact that these conversations are taking place shows that the major players within the news media environment believe that how audiences engage with media is worthy of consideration. On the other hand, the fact that these stakeholders with disparate interpretations of audience engagement have yet to come together reflects just how convoluted the term has become. How the term is ultimately defined and measured will have consequences not just for how journalists produce the news, but also what they expect of public – as well as what the public expects of them.

Elia Powers, Ph.D., is an assistant professor of journalism and new media at Towson University. He writes regularly about news literacy, audience engagement and nonprofit journalism.

The post What Research on ‘Measurable Journalism’ Tells Us About Tech, Cultural Shifts in Digital Media appeared first on MediaShift.

A group of filmmakers in Canada took their project one step further — by adding an interactive game element to their project.

“Thank You For Playing,” produced by the National Film Board of Canada, is part documentary film and part web game. The game simulates how casino games that reward users with positive reinforcement can lead to gambling addiction.

In Canada, revenue from gambling is used for infrastructure, health care, social services — and even preventing and treating gambling addiction. More than 35 percent of those revenues come from individuals with a gambling problem, according to the film.

It’s a paradox that the film’s director, Andréa Cohen-Boulakia, said she found fascinating. The goal of “Thank You For Playing,” which was released in December, is to bring awareness to the public health issue of gambling addiction. The filmmakers initially intended to focus on how the casino industry targets elderly people who often have a lot of free time, money to spend and are no longer active in society.

But after spending months traveling throughout towns and provinces across Canada to do research and interview prospective characters, Cohen-Boulakia found a more compelling story: the risk of gambling addiction as a result of being in a vulnerable emotional state. She saw that, as with any vice, some individuals are more prone to addiction than others. For most gambling addicts, the addiction is the outcome — not the original source of an individual’s problems.

“People talk about this being an emotional disease,” she said. “When you’re in a place where you’re emotionally wounded, you’re much more vulnerable. That’s what this is about.”

The Interactive Game 

The film opens up with a digital version of the shell game: the viewer, now the player, is presented with three icons. The icons are quickly shuffled and the film’s narrator directs the player to watch closely in order to uncover the icon that’s in the shape of a heart.

As the game continues, there are more icons are they are shuffled faster as the game becomes more challenging.

“Now you think you’re in control,” the narrator says. “But the game is programmed to let you win just often enough to condition your behavior. What is happening is called positive reinforcement.”

The viewer can then choose to watch one of three stories, each about a different subject who is recovering from a gambling addiction. Those characters, including one subject whose three-year-old daughter had been diagnosed with leukemia, explain how they saw gambling as an escape from their personal problems. The chapters of the film are interspersed between more opportunities to play the shell game, which continues to increase in difficulty.

The slot machine-like game allows the viewer, who has now become the player, to witness firsthand the adrenaline rush and excitement that come with winning a game — as well as the desire to keep playing. That’s an especially important component to include when the three subjects of the film cannot touch a machine themselves.

Andréa Cohen-Boulakia

Merging the interactive game with the documentary film scenes proved challenging and took more than one try to get right. Cohen-Boulakia described the two pieces as different “stories.”

“We wanted it to be interesting and enriching,” she said. “How can both formats, really different formats, mix together in order to give the audience a better understanding of the issue?”

Both the game and the film had the same goal — to educate. The goal was to have the game not match the experience of the characters in the film, but to create a new parallel and connecting experience. That required a team effort between the programmers and the producers, and it’s the part of the whole project that took the longest to complete, Cohen-Boulakia said.

The Film’s Impact

Cohen-Boulakia said as an educational tool, she hopes the film will bring more awareness about addiction and lead to more open discussions about what leads someone to go down that path.

Ideally, the risk and reward aspect of playing the game can offer insight into how individual might feel while gambling. Knowing early on that someone is susceptible to addiction might prevent the kind of dramatic outcomes in which people later find themselves when they are struggling with addiction. Cohen-Boulakia had even heard stories of addicts committing suicide. 

She said schools can have stronger prevention programs and hopes even kids will watch the film and begin to think about the issue.

“I hope people will be able to look in and ask themselves if they are prone to be dependent or not,” she said. “I hope people will talk more about it.”

Bianca Fortis is the associate editor of MediaShift, a founding member of the Transborder Media storytelling collective and a social media consultant. Follow her on Twitter @biancafortis.

The post How Canadian Filmmakers Combined a Film + Game to Combat Gambling Addiction appeared first on MediaShift.

Less than a year after closing its daily print edition to go all-digital, Brazil’s Gazeta do Povo has made major strides in its efforts to track the impact of its journalism on a societal level.

With ICFJ Knight Fellow Pedro Burgos’ help, Gazeta compiled and released its first impact report earlier this year to show readers the results of its work with Impacto, the impact-tracking tool Burgos developed with support from Google News Lab before starting his fellowship.

By compiling reports like this, Gazeta is able to show subscribers that its work is producing positive change in society — and that its journalism is worth paying for. And it gives digital-first newsrooms a more accurate, meaningful way to measure success than counting clicks.

But what does “impact” look like? For Gazeta, impact often looks like BRL16.4 million being returned to public coffers after Gazeta’s investigations uncovered irregular contracts made by the Federal University of Paraná. Impact can be as small as a reader telling the news outlet that its reporting has made a difference in his or her life, as well.

As an ICFJ Knight Fellow, Burgos is developing Impacto so it can be adopted by more newsrooms in Latin America and beyond. He is also building a coalition of Impacto media partners with continuing support from Google News Lab; Gazeta is one of the founding coalition members.

IJNet spoke with Leonardo Mendes Jr., Gazeta’s editorial director, to find out more about how the newsroom transitioned to an impact-tracking mindset — and what other newsrooms can learn from their efforts:

Tell us a bit about why Gazeta first decided to partner with Pedro and begin using Impacto. What changes took place after you started using Impacto in the newsroom?

Gazeta do Povo changed its business model last year. We shut down our daily print edition and we put all our energy into the digital product and the subscription model. But it has always been clear to us that people will pay for content if they see a single value in it. And Impacto is a tool created to show the unique value of journalism. It is a tool that shows subscribers how their subscriptions are supporting journalism that changes the world.

The newsroom quickly adopted Impacto as a working tool. Whenever a reporter or editor sees that there was a repercussion from one of their stories, they tell colleagues: “That is impact” — and then someone registers the impact.

Was there anything that surprised you from using Impacto to keep track of Gazeta’s impact?

We’ve always been a local media organization. With Impacto, we began to perceive and register how our content feeds the debate on national issues such as politics, economy, education and ideology.

What advice would you give to fellow journalists/newsrooms who want to get better at tracking their impact?

The key is to engage the team permanently. The team must see it as fundamental to the strengthening of high-quality journalism. Whenever engagement was high, impact records were frequent and significant. When engagement is low, we do not take full advantage of the tool.

What do you hope people will learn by reading the Impact Report?

How journalism can impact society in different ways. For some people, journalism plays its role only if it improves the use of public resources or overthrows bad government. Of course, both these things are very important. But journalism can do more. Journalism can help to teach a class how to live with someone different. Journalism can make the world better by telling inspiring stories. Journalism can improve a reader’s day simply because it made him happier or more motivated.

Read more about Impacto

Impacto was developed by Pedro Burgos, a Brazilian journalist and developer based in Sao Paulo. As an ICFJ Knight Fellow, Burgos is continuing to develop tools that help newsrooms measure the impact of their journalism in society. Impacto, which has the support of Google News Lab in Brazil, is already being tested in five important media companies — Folha de S. Paulo, Gazeta do Povo, Veja, Nexo and Nova Escola — and will be expanding to other countries this year.

Sam Berkhead is IJNet’s manager.

The post How A Local Newsroom In Brazil Learned To Track Its Impact appeared first on MediaShift.

For local news organizations, tracking impact is a way to tie journalism’s value to revenue. Impact trackers can help show readers how news works and source powerful messaging for membership or subscription campaigns. Thirty-one percent of recent subscribers to local newspapers subscribed because they wanted to support local journalism, according to the Media Insight Project. Gannett, where Smith worked before joining WhereBy.Us, recently launched an internal impact tracker, and LION Publishers is now offering investigative reporting grants to local newsrooms and helping grantees use the impact of that reporting in fundraising.

WhereBy.Us has been at the forefront of using analytics to create news products for the local media market. This philosophy has led to a newsletter-first strategy, an emphasis on connecting people with the cities they live in, and a membership program with benefits that include merchandise, giveaways from local businesses and discounts for events. The company is currently using an institutional fellowship from the Donald W. Reynolds Journalism Institute to build a toolkit for using metrics in small, independent newsrooms.

I talked with Smith over email about her new role at WhereBy.Us, the metrics she tracks and why the company started tracking impact. Here is our lightly edited conversation.

Q&A

What’s a typical day look like for you as growth editor?

I’m not sure I’ve had a typical day since joining the WhereBy.Us team in December! Much of my work so far has been around building workflows and processes for our growing team. My role is set up so I’m like a consultant for our local brands — The New Tropic in Miami and The Evergrey in Seattle (and more coming soon!) I check in with the local teams daily to brainstorm how we can engage our current audiences and capture the attention of new people. I also lead project work related to growth — things like our content distribution plans and tracking, CRM-like people tracker for editorial use and organic and paid social media strategies.

What metrics do you track?

Since we’re a daily newsletter, subscribers and open rates are important. We use a net promoter score system to get regular feedback on how our newsletter subscribers feel about our daily product. For social media, we look at engagement metrics, so comments, shares and reactions. We also look at things like number of responses to our callouts and how many people attend our events. I care most about metrics that indicate people have found value in what we’ve created and want to participate in our community. And, of course, we keep tabs on revenue.

It’s a new position, so in the bigger picture what does the company hope to learn through your work?

How to grow effectively. I’m tracking our experiments, big and small, so that we can learn over time what works and what doesn’t. Test, measure, learn, test again.

What can impact look like for WhereBy.Us?

We organize how we think about impact in four buckets: informed awareness, informed conversations, informed action, external recognition. We look for and track when someone tells us we taught them something or changed their mind, shared or talked about our work, took an action, changed a behavior, things like that.

Is there a great example you can share?

Our first experiment with Facebook Groups. Our Seattle brand The Evergrey launched “Embrace the Grey,” a Facebook group designed to help their community (and themselves!) make the most of winter there. Members participate in daily challenges, share ideas, start conversations, and we ask if they’ve changed their minds about winter after trying new things or considering different perspectives. The group is coming together IRL in March to celebrate a holiday that we made up and members chose the name for — Greybreak. This is a great example of how we can bring a community of people together around a shared experience and help improve lives in one way.

The group is open to the public — check it out here.

What pros and cons did you talk about before deciding to start using an impact tracker?

A lot of these conversations happened before I joined the team. There are lots of pros — making sure we’re fulfilling our mission to engage the curious locals in our cities, learning what qualitative value our projects and events provide, showing clients more than quantitative metrics. I think one of the pain points on this for many organizations is the effort and time needed to keep the thing updated and useful. My smart colleague Anika Anand kept the input process for our impact tracker as simple as possible.

Anything else on your mind right now related to metrics and impact?

WhereBy.Us’ ecosystem. Is it healthy? Are people who subscribe, or have conversations on our social posts, or attend our events, or pay us for marketing work, delighted with what we’re offering? Are they participating in our work on different platforms? I’m thinking about ways to measure and check in with each mini-community we’ve created so we never rely too heavily on any one space.

Learn more

You can subscribe to WhereBy.Us’s newsletters at The Evergrey and The New Tropic.

Jason Alcorn (@jasonalcorn) is the Metrics Editor for MediaShift. In addition to his work with MediaShift, he works as a consultant with non-profits and newsrooms.

The post How WhereBy.Us Will Track Impact of Local Media appeared first on MediaShift.

Smart questions from Tony Marx of @nypl: How do we decide what information to trust as facts? How are we going to protect local news? How can we encourage people to get out of their filter bubbles? How can we figure out what to do about dark money? #infoneeds

— Amy L. Kovac-Ashley (@terabithia4) February 20, 2018

Over a day and a half, the future of algorithms and technology took center stage, with talks from tech pioneers and authorities Tim O’Reilly, Jimmy Wales and Amy Webb. But more intimate breakout sessions and panels offered a chance to see where philanthropic attention to media is being directed right now: discussions focused on local media collaboratives, national-local partnerships, community-focused journalism, community and ethnic media, and the science of storytelling among other topics.

Implicit in many of these: It’s a given that media has impact on the health of our democracy, and filling the growing gap in professional reporting is of utmost importance. It’s no longer a point that has to be proven each time it comes up, at least for this crowd. Rather, the gathered leaders in media, philanthropy and tech asked more nuanced questions about impact, metrics, news and how to pay for it all.

The time for talking in nice hotels has ended, it's time to DO stuff, says Tony Marx of Knight Commission on Trust, Media & Democracy. "The times require more of us, and our children require more of us." #infoneeds

— Media Impact Funders (@MediaFunders) February 20, 2018

What’s the impact of local non-profit news?

On Wednesday, a panel of non-profit news outlets and the community foundations that have supported them discussed the impact of news in their communities. At a fundamental level, what the Texas Tribune offers is the “maintenance of an educated public,” said John Thornton, the Tribune’s co-founder. The Tribune now boasts the largest statehouse news bureau in the U.S.

As the media gets hollowed out, the notion that someone in media has your back gets lost, Thornton said. “What’s filled that void is distrust.”

We’re taking a look at the ways foundations are investing in #localnews with an all-star panel of @lillyweinberg, @thorntonaustin of @TexasTribune, @LaMonteG of @LORfoundation, @GallowayVTD @vtdigger and @SandraSWClark @whyy. Tune in: https://t.co/D28BbmGexM #infoneeds

— Knight Foundation (@knightfdn) February 21, 2018

As in Texas, strong local coverage by a non-profit newsroom is making a difference in Vermont. VTDigger covers the state with impartiality, rigor and a 10-person newsroom. “Maybe not at the national level, but when we report, things change,” said Anne Galloway, the nonprofit newsroom’s founder. At the state and local level, you can be more pragmatic, she said, and there’s less political noise crowding out the journalism, even when covering the state legislature. “We give everybody a hard time,” she said.

Are attention metrics a destructive force?

The day before, tech publishing veteran Tim O’Reilly offered an explanation of algorithms in technology, as well as a warning. In response to a question about the incentives of commercial news, he said, “You put your finger on my biggest concern, we have not realized that we are living within an algorithmic society that is optimizing for the wrong thing.”

"You put your finger on my biggest concern, we have not realized that we are living within an algorithmic society that is optimizing for the wrong thing." — @timoreilly #infoneeds

— Jason Alcorn (@jasonalcorn) February 20, 2018

Algorithms, O’Reilly said, are designed with an objective function. Facebook’s News Feed algorithm is designed for engagement. Media, both real and propagandistic, optimizes for that News Feed algorithm in search of likes and shares. “The clicks are always going to be on the sensation,” he said, calling the feedback loop of attention and traffic a destructive force.

Can Facebook be fixed? O’Reilly is hopeful. In many respects, he said, Facebook’s current situation is analogous to Google in 2011. Then, Google faced its own “crisis of trust” and responded with a major update to its search algorithm to punish spammers and publishers of low-quality news and information. That effort worked, and whatever Facebook does not may also.

Still, “We have to find other economic models for the news business,” O’Reilly said. A model built on attention metrics can always be gamed. “You have to create value for people who care about that value. When people pay for the things they care about, you can have a better business.”

What’s the impact on communities?

In rural communities in Colorado and New Mexico, “Small Towns, Big Change” explored urgent questions like how to create robust economies, protect natural resources, and improve quality of life. Journalism itself wasn’t the aim, but rather a means to identify solutions for the LOR Foundation, who collaborated with Solutions Journalism and seven local news organizations. It came about after the foundation learned that most people in the regions it served got most of their news from Facebook instead of local or even national news, said LaMonte Guillory, communications director for the LOR Foundation.

.@LaMonteG shares insights from @soljourno's work in the Intermountain West asking communities "How well do your local news outlets cover the things you care about?" https://t.co/qkhWkByGfs #infoneeds

— Teresa Gorman (@gteresa) February 21, 2018

In Philadelphia, another solutions journalism project has had an impact both in the field of journalism and on the issues it has covered. The Reentry Project won a 2017 Philly News Award for best non-traditional news provider and led to 10 stories produced in collaboration with multiple news outlets. And its work led to changes at the city, in local business practices and in the reentry community’s perception of the media itself:

And impact on the issue #infoneeds @soljourno @ReentryUpdates @jeannyfr pic.twitter.com/37DFIzpfOq

— Lindsay Green-Barber (@WhimsicaLinds) February 21, 2018

What comes next for media funders?

Futurist Amy Webb is concerned about the future and urged funders to act. “You have the power to fund the future of news you want,” she said. With the rise of AI, fake audio and video, the alternative could be catastrophic.

the bad scenario per @amywebb at #infoneeds pic.twitter.com/kgL4N7ZRX3

— Media Impact Funders (@MediaFunders) February 21, 2018

The Knight Commission on Trust, Media and Democracy will be releasing its recommendations later this year.

Jason Alcorn (@jasonalcorn) is the Metrics Editor for MediaShift. In addition to his work with MediaShift, he works as a consultant with non-profits and newsrooms.

The post Knight Media Forum Focuses on Non-Profit News, Impact and Danger of Algorithms appeared first on MediaShift.

Click the image to read our entire series.

Investigative non-profit journalism is flourishing this year, likely because it is top of mind for so many people. Media credibility is in the spotlight, and those of us who dig deep into uncomfortable places and ask for our reader’s trust by being unbiased and fact-driven feel it shining especially brightly.

New York-based ProPublica, one of the country’s most high-profile non-profit newsrooms, opened a regional bureau in Chicago with a team of 12 reporters, editors and technologists. In Vermont, the non-profit VTDigger has become the country’s largest investigative reporting non-profit focused on local or state news.

And at inewsource in San Diego, we’re projected to reach $1.1 million in revenue this fiscal year — our most successful year yet.

inewsource’s small but versatile team of reporters covers a variety of topics but focuses on four: education, health, the environment and local government. Through our partnerships with the local PBS and NPR affiliate station, KPBS, and San Diego’s leading commercial news station, CBS8, our work reaches more than a million people a week through web, radio and TV. Our investigations have had more impact this year than ever before.

Here are a few examples of what that impact looks like:

• Our series on a long-ignored transparency law prompted the city of San Diego to enact a new law mandating disclosure of business interests behind billions of dollars in city purchases and contracts.
• Our data analysis on diabetes-related amputations in San Diego County and California uncovered a “shocking” increase that has mystified diabetes experts.
• An investigation into a local non-profit Christian college found administrators couldn’t account for more than $20 million in expenses. The college’s CFO was replaced shortly after publication.
• Our dogged pursuit of problems at a local school district, which serves one of the county’s poorest areas, forced the resignation of an interim superintendent and uncovered millions of dollars that were misspent or unaccounted for. It also prompted the state to begin what it calls an “extraordinary audit.”
• Our analysis of test score data posted by the California Department of Education found huge errors in the data. The state removed the faulty data and initiated a system to notify the public when bad data has been replaced.

Reporter Brandon Quester of inewsource on a reporting assignment. Used with permission.

Collaboration is the future

During a year of successful, high-profile collaborations across journalism, we’ve used partnerships with other local and national non-profits to have an even bigger impact. For example:

• We worked with KPBS reporters to investigate how one local refugee resettlement agency violated housing practices. The story prompted the agency’s executive director to step down and launched a federal inquiry.
• We collaborated again with KPBS in the first-ever mile-by-mile look at the U.S./Mexico border wall, as well as its effect on migration patterns and border patrol staffing levels over time.
• We partnered with the Investigative Reporting Workshop to highlight how San Diego schools are dealing with educating refugees.
• And we worked in tandem with the New York education journalism non-profit The Hechinger Report for a first-of-its-kind analysis of how race plays into community college completion rates.

What comes next?

In looking ahead to 2018, local investigative reporting non-profits are likely to have an even more important role in the media landscape, through more partnerships, increasing readership and an ever-more urgent need for trustworthy journalism. We will continue to produce stories that have immediate impact. As a non-profit investigative newsroom, we can only do this work with the support and generosity of people who care about credible, fact-driven journalism.

Through the end of the year News Match, the largest-ever grassroots campaign for non-profit news, is helping us and newsrooms like ours in communities across the country by matching donations by individuals to non-profit newsrooms. That’s how we will raise the funds necessary to pay for what’s to come in 2018, more local investigative reporting – journalism’s gold standard.

Brad Racino is senior reporter and assistant director at inewsource, an investigative journalism non-profit in San Diego, California.

The post The Growing Impact of Local Non-Profit Investigative Journalism in 2017 appeared first on MediaShift.

How does a documentary series inspire real social change?

A case study by the USC Annenberg Norman Lear Center’s Media Impact Project illustrates how a 2016 series first broadcast before the 2016 election was used to reinvigorate progressive groups in the election’s aftermath. For media makers intent on social impact, it’s a story with important lessons for how to plan engagement that makes a difference.

America Divided is an Epix documentary series that aired in the run-up to the 2016 U.S. presidential election and is now streaming on Amazon and Hulu. Its producers conducted an innovative engagement campaign to spur action against social injustice, and invited MIP to determine if their documentary series inspired real social change.

What we found was that documentarians can use innovative engagement campaigns to rally the public to action long after a program’s TV broadcast (if they were lucky enough to be broadcast!).​

America Divided producers took 10 steps to engage citizens following the series’ broadcast. Somewere tried-and-true, such as using celebrity prestige to attract attention, but others were more innovative, such as editing several shorter, single-issue versions of the series that could be screened before niche groups. This varied approach helped America Divided to meet its producers’ goals for impact by building coalitions around the project as well as re-energizing a demoralized liberal base post-election.

The MIP impact evaluation revealed that a key benefit was this re-invigoration of groups around the root causes of social injustice: the screenings and panel discussions gave activists a reason to assemble, to air their views and, more importantly, to talk about paths to solutions. Specific actions that grew from these assemblages include canvassing, starting social media pages, creating petitions, and signing up for planned marches and rallies, as well as organizing additional meetings and screenings.

What can media makers take away from this case study? We identified 15 key insights that producers of all kinds can use to create engagement around their own work.

15 Tips for Media Makers to Keep Social Issues in the Public Eye

1. Think of broadcast as just the first step in your engagement campaign.
2. Forge partnerships with groups who can screen and promote your work.
3. Seek connections between your issue and any arts, civic, philanthropy or cultural groups and their own activist work.
4. Focus screening campaigns around one rallying issue.
5. Partner with regional PR firms that know the local players and issues and can unite disparate groups.
6. Be open to re-purposing edits that allow for issue or region specific coalitions.
7. Exploit celebrity power to cross-promote online, draw live audiences and create a relevant “face” for the issue.
8. Use screenings as forums for people to assemble, discuss and create community.
9. Engage audiences in panel discussions following screenings to promote deeper dialogue and create plans for social action.
10. Support learning with fact-based resources for deeper study.
11. Keep the momentum going by continuing to supply content to your partner groups.
12. Reach out to different-minded audiences to promote civil discourse.
13. Make the information in your documentary searchable online
14. Make certain screening partners can easily access your materials.
15. Keep your website current so that it becomes a source for news on your issues, and maintain ongoing dialogue with your digital audience.

Download the full report

Laurie Trotta Valenti is a writer and media educator currently managing the Media Impact Project at USC Annenberg Norman Lear Center.

The post How Media Makers Can Plan Engagement That Inspires Real Change appeared first on MediaShift.

Social media bots (automated software designed to perform specific tasks, such as retweeting or liking specific content on Twitter) are a problem because of the part they play in spreading misinformation. What’s still lost in the commentary, however, is that simply proving bots’ existence, let alone purging them from social platforms, is extremely difficult. With these fake entities meshing deeply into the fabric of cyberspace, the biggest threat they pose is of turning it into a place devoid of human discourse.

Since the beginning of 2017, my team at Mentionmapp Analytics been tracking bots, sockpuppets (real people operating behind fake profiles, and usually promoting specific points of view) and the flow of misinformation using our network visualization application.

These are the key lessons we’ve learned so far:

• There’s no doubt that people with ill-intent are highly creative.
• Technology alone (e.g. trying to create “good” bots to confront “bad” bots) is unlikely to solve the problem.
• Bots’ personalities are evolving at a rapid pace. Creators are honing their craft daily to fool us and to fool the platform detection abilities.
• Bot network operators will adjust their tactics to stay in the game. There’s more to spreading misinformation that just high volume or distributing it like spam. Manipulating and inflating social metrics is an important way to game platform algorithms. For instance, some profiles exist simply to “like” certain tweets.
• And we are witnessing bots being deployed by interests throughout the full political spectrum.
• It’s also more than just a Russian issue. Bots are employed by other states, their proxies, and sometimes people simply trying to make a buck.

The major trends

Over the course of the past year, we have observed and documented the proliferation of bots in online conversations. From political dialogue, like the Dutch, British and German elections, to the anti-vaccination issue in the US, seeing firsthand the mechanisms at work to misinform and mislead has been troubling.

The major bot and sockpuppet patterns and trends we’ve observed include:

• Clusters of profiles retweeting or liking content within a tight time window of the original post.
• New profiles, with few followers and little account activity, receiving retweets in the hundreds (or thousands).
• Computer-generated profile display names and handles (with stolen profile photos, mismatched genders, and a variety of seemingly unlikely language used given the person’s stated location).
• Automated behaviors which allow profiles to generate responses to conversations at speeds and volumes that could only be executed by computers.

Overall the goals of social media bots are to distract, influence, or suppress everyday online conversations. In the case of sockpuppets, it’s the deliberate deception and misrepresentation of the digital self that’s cause for concern. Sockpuppets are the equivalent of paying protesters to show up to your rally: the optics suggest it’s a cause that the public cares about and supports, when it’s really a front to manipulate perceptions and opinions about an issue.

#Geertwilders and #FakeNews

The political events of late 2016 pushed my team to start thinking about digital communications in a different light and how we could apply our Twitter visualization business in a completely new way. We’ve transitioned from a niche Twitter marketing tool business into researching and identifying the wide variety of bots operating on Twitter, and understanding their behaviors.

We began our first case study in February 2017, tracking the hashtag #Geertwilders in the lead-up to the Dutch general election. We uncovered a group of 26 fake profiles acting as amplifiers in an orchestrated fashion. With near identical “fingerprints,” such as date joined, follower-to-following and number of tweets-to-like ratios, and retweeting one specific tweet connected to @abermans, a profile no longer on Twitter, it was our first clear picture of computational propaganda in action.

We’ve also seen bots use the hashtag #FakeNews to intimidate and harass journalists and activists. On August 28, 2017, five ProPublica journalists and the Open Society Foundation saw their accounts overwhelmed by a massive bot attack.

Seeing one tweet generate over 13,000 retweets and likes over the course of four hours was both a valuable learning experience for us and deeply disheartening. After generating more than 21,000 retweets and likes in barely eight hours, this incident removed any doubt that real online human discourse is threatened. Below is a small snapshot of what we saw, and the tweet that started the deluge.

The first tweet in the ProPublica #FakeNews bot attack (Image courtesy John Gray)

Data visualization of the #FakeNews bot attack against ProPublica (Image courtesy John Gray)

Is there a solution?

The online information ecosystem is messy for many reasons. Easy-to-access software automation tools mixed with open platforms of distribution make for a complex situation. 51 percent of those surveyed (including my team) for Pew Research Center’s study, The Future of Truth and Misinformation Online, are not optimistic about the situation improving over the next 10 years. Still, we aren’t discouraged from working to be part of a solution.

One part of the solution is to facilitate conversation around the issue. From more than one hundred investigations and a commitment to being part of an informed conversation, we wrote the ebook, “Ecosystem of Fake: Bots, Misinformation, and Distorted Realities” which looks at the trajectory of online information as it’s moved through the “post-truth” and “post-fact” eras.

Bots are like crabgrass: they are impossible to eliminate completely. The bot occupation of cyberspace is a significant problem and still an emerging conversation. With more awareness, discussion, and action around the problem, we can begin cleaning up the havoc created by software automation and those who sow misinformation in our public conversations.

John Gray is CEO & co-founder of Mentionmapp Analytics Inc. and a freelance writer. He is investigating Twitter interactions unfolding between real people, sockpuppets and Bots in consideration of how misinformation is impacting our socio-political discourse. John has a Bachelor of Applied Science (Communications) and a B.A. (English) both from Simon Fraser University.

The post How Bots Are Threatening Online Discourse appeared first on MediaShift.

Journalists don’t as a rule have a specific impact in mind when we begin our journalism. We list project goals, but other than bringing awareness to an issue or event we do not identify what we’d like to see happen next.

This is a story about the one time we did.

Rockland County, just northwest of Manhattan, is one of New York’s fastest growing counties. Neighborhoods that used to feel suburban and bucolic now choke with high-density sprawl, multi-family homes rise in backyards next door, traffic is a problem and religious schools pop up on residential streets. This is the lede of our story for the Journal News and lohud.com:

“A generation ago, there were few problems between Ramapo’s small ultra-religious Jewish communities and the gentiles and other Jews who made up the bulk of the town’s population. Things have changed. As the ultra-religious community has grown, Ramapo has become a flashpoint in a continuing conflict over what it means to live in the suburbs. … The conditions fueling that conflict are now threatening to spread beyond Ramapo’s borders. Surrounding communities have taken notice, and they are adopting measures aimed at heading off the strife that has become the norm for their municipal neighbor. No place, it seems, wants to become ‘the next Ramapo.’”

People in Rockland County, one of two counties we cover at The Journal News and lohud.com, argue over this. Things get ugly. Reader comments on the stories we write turn hostile and border on anti-Semitism. No matter what the topic, the conversation inevitably devolves into “us” vs. “them.”

Claudia Gollub speaks during a public meeting on blockbusting, in a photo that ran with long-form journalism on a town’s conflicts. (Photo: Peter Carr/The Journal News)

But here’s the real problem, and it’s not religion: Ramapo’s loose zoning, lax enforcement of fire and building codes and largely unchecked development puts everyone at risk.

We wanted to know: Do people understand what’s really happening in Ramapo? If we were to explain the issue — lay it out in long-form story, year by year and decision by decision — could we all (or most of us) finally agree? Could we change minds? If we could achieve consensus, would we stop arguing religion and start attacking the real problems?

So, after years of having the same tired, angry, circular discussion, we set out — in one project — to change the conversation.

We researched the story over several months. We talked to longtime residents, members of the Jewish community, government officials and the county’s Fire and Emergency Services coordinator. We wrote and rewrote, shot video and mapped the region.

“Ramapo nears breaking point” is a long-form piece of explanatory journalism that laid out all of the problems that began decades ago.

How we measured the story’s impact

At the end of our story, we included a short survey to measure the impact of our explanatory journalism — to find out if we successfully proved that the issues in Ramapo are a lack of zoning and safety code enforcement, not religion. (We considered asking for opinion at the beginning of the story to gauge change at the end, but felt that would inhibit readership and we could accomplish what we needed with one survey at the end.)

The survey was four questions, beginning with two easy Likert Scale queries:

1. How strongly do you agree that the issues in Ramapo are a lack of zoning and safety code enforcement, not religion?
2. How strongly do you agree that after having read this story, your opinion of the issues in Ramapo has changed?

Then we expected a steep drop-off in answers to the final two questions because the first takes time and the second raises privacy concerns:

1. How have Ramapo’s issues impacted you?
2. What is your email address?

We built the survey using PollDaddy because it prevents repeat responses and for the data it provides on the back end, including geolocation, a time/date stamp and referrer, and it assigns each respondent a unique PollDaddy ID. This helped us connect their answers from question 1 to question 2, and so on, and get a more complete picture of their response.

We embedded the survey at the bottom of the long-form story. For two days, that was the only place you could find the survey; we didn’t embed it elsewhere or share it on social media. After two days we added it to our editorial, “Ramapo’s shoddy governance is by design.”

Survey results

Surprisingly, nearly 900 people responded. Here’s what they said:

Do people understand what’s really happening in Ramapo? Yes. Most (67 percent) either agreed or strongly agreed with our explanation of the issues, with “strongly agree” ranking the highest (54 percent)

Could we change minds? Yes. Of those 67 percent, 14 percent said our reporting convinced them, while 35 percent said they agreed beforehand.

Finally, we believed we and the community could stop having the same conversation and instead move forward toward solutions.

Emails. Almost a third — 29 percent — gave us their email address for follow-up.

Sources and commentary. Even more surprising than the total number of respondents was the percentage of them — 38 percent — who also took the time to write about how Ramapo’s issues have impacted them. Hearing from homeowners, those who’ve moved away, educators, fire safety officials, people who have tried to move in but claim to have been denied housing gave us a list of future story ideas and sources.

A few wrote to thank us for our reporting, for exposing the issues and doing so in such an unbiased way. A couple said we should have written this years ago. One called it “conversation-framing analysis” and others called for more discussion on the topic. One response shows how we raised awareness and encouraged civic involvement:

“I live in a neighboring town and am concerned about Ramapo’s issues becoming issues here, too. So I will be paying more attention and doing what I can to ensure that we do not fall victim to the same lack of zoning and code enforcement that has plagued Ramapo.”

A narrow street and a crowd of homes in Kaser, a village in Ramapo, illustrate some of the issues with density explored through long-form journalism. (Photo: Seth Harrison/The Journal News)

What we learned

• People will answer a survey, even at the bottom of a long-form story, if they’re invested enough in the issue.
• Think about who you want to answer the survey and who you don’t. Burying our survey at the bottom of the long-form story and not promoting it weeded out casual readers that we didn’t want filling it out anyway.
• Close the feedback loop. Thank your readers and share the results of the survey. We didn’t do this at first, and our audience let us know that wasn’t OK.
• Be upfront about how you’ll use their specific information. Unless you’re clear you will use their comments in a follow-up story, contact people to ask permission before publishing their words. This demonstrates respect and encourages further engagement.
• Use all of the information you gather to learn more about the audience. We cross-checked our survey email list with our subscriber lists and the list of those who attended our project discussion in person to see who we’d reached. We learned that we have room to grow our subscriber base in Rockland when we otherwise would have thought we were talking to existing subscribers.

Anjanette Delgado is the digital director and head of audience for lohud.com and poughkeepsiejournal.com, part of the USA Today Network. Email: adelgado@gannett.com, Twitter: @anjdelgado. Special thanks to Lindsay Green-Barber of the Impact Architects, who helped with survey methodology.

The post Assessing The Impact Of Explanatory Journalism appeared first on MediaShift.

The research aims to quantify the effect of news media. Put in terms that are increasingly common when talking about journalism: What is the impact of news organizations?

The study by Harvard professor Gary King and collaborators found that a few, mostly small news outlets publishing simultaneously in a broad area of public policy concern increased the volume of conversation on social media by 19 percent the day after publication. Over a full week, the volume was increased 63 percent relative to the average day’s volume. The number of unique authors increased as well, and the composition of opinion changed in the direction of the published articles.

News outlets it appears, even in the face of social media algorithms, the echo chambers of political polarization and a barrage of digital information, have a profound and measurable effect on national discourse.

Independent outlets have outsized impact #indymediamatters #tmcmedia https://t.co/CVCNRCGyzY

— Jo Ellen Kaiser (@jgksf) November 9, 2017

How the experiment was conducted

On of the challenges with research on media impact is the lack of control researchers have over the publication of news itself. You can’t exactly randomize the news. The need to maintain editorial independence and the timeliness of reporting means there have been relatively few large-scale, controlled experiments attempting to experimentally measure the effect of news media.

The research team found a way around this by actually collaborating with news outlets, using the first three years of the five-year study to build trust. “Frankly, when we began talks with the researchers in 2012, what they required seemed impossible,” Media Consortium executive director Jo Ellen Green Kaiser wrote on MetricShift in 2016. However, the researchers began attending industry conferences and worked with the Media Consortium to test the experiment with a small group of participants.

In the end, the study recruited 48 news outlets who agreed, in small groups, to publish at the same time on a broad policy area agreed-to beforehand and typical of the the outlets regular coverage. The 11 policy areas were race, immigration, jobs, abortion, climate, food policy, water, education policy, refugees, domestic energy production and reproductive rights. The media outlets retained full control over what was published, as well as the option to opt-out at any time. (A list of outlets that participated fully in the experiment is available in an appendix to the report.)

For their part, the researchers picked a two-week publication window when they expected a slow news cycle and randomized the publication date to either the first week or the second week. They then were able to measure and compare changes in the volume and composition of conversation on Twitter between the publication week and the control week.

The experiment was run 35 times, and this group-publishing was found to have increased conversation about the general topic by 63 percent over a week, relative to the average day’s volume.

The organizations were mostly small publications, with a median 50,000 subscribers. Some members of the Media Consortium who participated were Truthout, In These Times, Bitch Media, The Progressive, Earth Island Journal, Feministing, Generation Progress, Ms. Magazine, Yes! magazine and Making Contact.

What do the findings mean?

For the Media Consortium, this research shows that “even small independent news outlets can have a dramatic effect on the content of national conversation,” Kaiser wrote in a blog post.

Actually, she says, independent media has some built in advantages when it comes to having a impact on the national discourse. Independent outlets boast loyal communities of readers and supporters who are eager to push national conversations on social media, a willingness to collaborate openly that amplifies the effect of news, and news coverage that can face down serious topics because there’s less imperative to chase pageviews.

Defining impact as influence on a national conversation is new, with trustworthy measurement having been a big hurdle to overcome.

There are some limitations to the study. Kathleen Hall Jamieson, director of the Annenberg Public Policy Center at the University of Pennsylvania, told the New York Times that without the researchers disclosing more information (names of individual stories and publishers were withheld for reasons of editorial integrity), there would be questions about its general applicability. Is the volume of conversation on Twitter a true proxy for the national conversation, for example, and how meaningful is the discourse there?

Nevertheless, this research may shift the discussion about media impact. “In fact, too often, philanthropic foundations refuse to support these outlets because they are ‘too small’ and ‘don’t have enough impact,’” Kaiser wrote.

After sending 153,861 tweets, I definitely *want* to believe that journalists at small outlets have "substantial power to affect national conversations about politics & policy" https://t.co/XpOLZTtmws Highlights need for diversity in media sources & amplifying trusted journalism. pic.twitter.com/bGCC01Lr4X

— Alex Howard (@digiphile) November 10, 2017

A more expansive definition of impact

We are in an age of digital media where reader-supported journalism and nonprofit news will increasingly provide a majority of public interest reporting. Against that backdrop, this research is a step toward a deeper understanding.

Until now, impact has typically been measured along a vector. In that formation, acts of journalism lead to responses from the public and ultimately a reaction in the form of individual or social change. Impact trackers like the ones used by Gannett, Chalkbeat, and the Center for Investigative Reporting measure vector-based impact quite well.

This research raises the prospect that whether or not we measure it every time, the act of publishing news is influential in still-surprising and robust ways. “Our results should remind us of the importance of the ongoing and interconnected national conversation Americans have around major issues of public policy,” write the study’s authors. “This conversation is a fundamental characteristic of modern large-scale government.”

If mostly small publishers working in small collaborations can influence conversation so significantly, what of non-profit news collaborations, local newspapers, and television news? What of collaborations between all three?

Correction: Mother Jones is a member of The Media Consortium but did not participate in the experiment. The article has been updated.

Jason Alcorn (@jasonalcorn) is the Metrics Editor for MediaShift and an independent consultant working with non-profits, newsrooms and philanthropy.

The post Harvard Experiment Finds Large Effects From Small News Outlets appeared first on MediaShift.