Dec 29th A thought experiment for the holidays

In which the empty self-aggrandizement of the University press release is outed and alternatives suggested.

This week, between Christmas and New Years, the University is closed. To set foot in the lab, a special form is needed, and signed by the authorities. I lodged one, but truth be told, I am fine with an enforced holiday, all the more because my son Nick is visiting. Among other things, it is a good time to catch up on back issues of Nature (you don’t think I’d be out rock-climbing with my son do you?).

In the 18/25 Dec issue (number 7521), an editorial alerted readers to a study just out in the British Medical Journal about poor science reporting in the news. This is an old lament and journalists are a well kicked scapegoat. But what is new here is to ask at what stage are errors introduced? The authors of the article (Sumner et al. 2014 Br. Med. J. 349 g7015) consider the University Press Release. This is the thing issued by the University Press Office that says in essence: “Hey World! Check out this week’s Science where our brilliant Professor Brainbaggins and team report their game-changing breakthrough of how to get sunbeams from cucumbers.”

They found about 400 University press releases where in addition they could get not only the original journal article, in which the discovery is reported for peer consumption, but also actual news stories in the press. They limited their search to print news, thus avoiding the maelstrom of TV and internet, and to medical research. Indeed, Sumner et al. were motivated by the possibility that reading mistaken reports about medicine would alter human behavior harmfully.

With the journal article, the press release, and subsequent news stories, they could look for the presence of exaggeration, a catch-all word that could just as well be called mistakes. Specifically, they could ask where the news was exaggerated, were the University press release similarly exaggerated? The answer is a resounding yes!

Here is a key figure from the study.

They broke exaggeration into three categories. The first is advice, as in “We recommend that children avoid eating yellow crayons” from a study comparing the toxicity of different crayon colors: if the journal article made no such recommendation, Sumner et al. count it as exaggeration. The second is claiming causality from correlation, an obvious pitfall. A study finds 70% of people who ate yellow crayons lost weight: claiming from this that eating yellow crayons causes weight loss is, to say the least, an exaggeration. The third is inferring human relevance from animals. This one is most exactly an exaggeration because animal studies are indeed relevant to humans, but by a little bit only. As the figure shows, in all of the categories, an exaggeration in the news was accurately predicted by exaggeration in the press release. As the authors themselves note, theirs is a correlational study, they cannot prove that the dodgy University press release is the culprit, but it sure is a plausible hypothesis.

Now, before moving on, let me say that the paper also points out that number of news stories a particular discovery garnered was not correlated with exaggeration in the press release, so pox on the idea that exaggeration is helpful to pique jaded reporters. And I’d be remiss if I did not point out that a major part of the authors’ work was turning qualitative concepts about exaggeration into numbers, as for example are plotted in the figure. For the purposes of this blog post, the details of this transformation don’t matter and I didn’t spend time parsing them out. If someone reading this knows of an undermining flaw, please comment.

With this background, I can ask: What is the point of a University press release? Presumably it is to alert the public to important things on campus. In the science side, those things would be mainly discoveries and research results. But the problem is conflating important with newsworthy. If our beloved Prof. Brainbaggins wins the Nobel prize, there is no need of University press release. What if our Prof. discovers that a kind of T-cell receptor binds avidly a particular antigen common on the surface of cancer cells? That sounds important, but is it news? News is someone using this discovery to cure cancer. But if it is one of those discoveries that helps us understand, in this case T-cell biology, but gets swept up into the great maw of knowledge, that is not news.

But in fact, the University will issue a squib only if Brainbaggins has gotten into Nature or Science or some other top drawer. If they publish in Protoplasma, forget about it. Thus, rather than news, the University press office is issuing a running tally of its scorecard for publishing high-impact papers. This is self-promotion. And as the British Medical Journal study shows, a promotion that is subject to dangerous distortions.

That we should not be surprised by this comes from considering sport. The fortunes of the many University of Massachusetts athletic teams are covered in fine detail the local newspaper, the Daily Hampshire Gazette. Not only who won and lost but also key plays and player development. No doubt, the University issues press releases on sport but I am confident that Gazette reporters pay them little heed and see the games for themselves. Would we have any use for a reporter who instead relied on the University press releases? That is what the science reporters are doing, failing to connect with the original article and its authors, abetted by the pap in the press release.

Clearly, University press offices need to clean up their acts. But I would like to propose taking a step further.

Here is an experiment. Run this experiment for half a decade or so and examine the results. It too will only be correlational but I predict manifest improvement in what the public knows about research on campus and its practioners, and indeed a rise for the general status of the University.

I challenge any University to scrap its press office and create effective avenues of communication. This will take care and money to succeed but as for the money, perhaps the same amount as spent on press release writers. Here are two examples. Organize a lecture series (or several), where academics address the public. The series needs to be properly advertised and transportation needs addressed. This sounds trivial but visitor parking on campus is usually expensive, inconvenient, and full. If a University is serious about showing its public what goes on inside the ivory then it cannot baulk at making parking free for 90 min around the hall where a public-directed lecture will take place. Organize a radio program or pod-cast where each week an academic is interviewed about their work. Making a good interview series will take skill, but would go far towards accomplishing the goal of showing off Prof. Brainbaggins and other great minds on campus and their fascinating endeavors.