Among many consequences of covid-19, the rise and fall of Citizen Science has been one of the more amusing ones to watch. It has fallen from grace significantly since its 2020 peak, when everyone was an expert on cloth versus surgical versus N95 masks and “did their on research” on which one was best for them. As with any progressive idea, it soon became adopted by the other end of the American political spectrum who “did their own research” on vaccines and genomic integration of mRNA. So it goes.
Another type of science also thrived during covid-19 but unlike the Citizen sort it is now stronger than ever. It began with daily updates on covid-19 incidence and mortality, usually from the same source, when every Twitter user became an expert on data visualization and every media outlet had a “data journalist” job posting. It now continues with a daily stream of (predominantly) opinions presented as hard facts, backed by pretty graphs above and a list of sources below. I speak of course of Journalist Science.
Before I go into why — spoiler alert — I don’t think Journalist Science is a net positive, a disclaimer: I like and respect many “data reporter”-type people, the Financial Times’s John Burn-Murdoch probably most of all. FT has a clear and known bias toward capitalism and markets, which makes it one the most legible sources of news around; it is in fact the only newspaper I subscribe to. And I’ve linked to Burn-Murdoch’s reports many times, even on this blog. His covid-19 charts, small multiples in particular, were the peak of data visualization and deserved to be included in Edward Tufte’s latest book. So when looking for examples of why Journalist Science is bad, I will use the Financial Times as the prime example: not because they are particularly bad but because they are one of the best newspapers around, and even they don’t get it right.
This article and the reaction to it is what got me to question the concept. It was about the diverging political paths of kids these days: girls becoming every more liberal, boys turning more and more conservative, in four “developed” countries (South Korea, US, Germany and the UK). On the day it came out, two of my friends who don’t know each other thought something was fishy, and both linked to an act of Citizen Science attempting to debunk it. The attempt failed, not through any fault of the citizen doing it but because the original sources were opaque, and the ways in which they were combined were unclear.
Before the pandemic, most data visualization exercises had a single source: a Gallup poll, a think tank’s projection, or just a CIA Factbook piece of miscellany. Covid-19 seems to have eased data journalists into looking at more than one source, combining them into the same graph, correcting for this or that anomaly that is bound to occur in any large data gathering exercise, all in order to perform a feat that is impossible to distinguish from actual science and in fact is proper science by any reasonable definition. This drift from painting pretty pictures to doing research proceeded over months and years, with each change explained in a footnote, and with the public familiar enough with the numbers that any particular graph did not need special introduction.
By the time covid became old news, the drift was complete: data journalists became data scientists in everything but the name, with an editor instead of a PI and retweets, likes and view counts instead of citations. As problematic as academic research is — and look no further than this very blog for a few thoughts on the rot — Journalist Science is even worse and let me count a few of the ways how:
- It does not provide its methods in sufficient detail to be reproducible;
- It is driven by editorial policy, making it even more prone to publication bias than Academic Science;
- Said editorial policy picks topics to which the readership will be receptive, leading to high degree of confirmation bias;
- Unlike academic publications and pre-prints, there is minimal to no cross-talk with other researchers, making any one publication a “dead-end” for anyone interested in the topic.
Two things absent from my list of problems are lack of peer review and lack of statistician input, the first because we don’t really need it, the second because we do but I see no evidence that journalists are any worse at statistics than doctors, molecular biologists, psychologists, or anyone else writing research papers who isn’t an actual statistician.
So how to fix this? A good start would be to publish the actual work of science you did, either as a preprint or as a full-blown academic publication — thought that would be overkill. This is the approach taken by Nassim Taleb: if you have something scientific to say, write a paper instead of a Twitter thread. BioRxiv and medRxiv are two well-respected depositories accepting papers of any length, but outlets like the FT produce enough research that they may as well have a preprint server of their own. This would ensure that at least some amount of rigor was given to the methodology, and would enable dialogue.
Less likely would be for newspapers to pre-register their studies, or at least name the Journalist Science articles that were considered but turned down. Least likely? Have the data journalists dedicate their considerable knowledge and talent to describe, enlighten and criticize the body of scientific work that’s already there, or alternatively dedicate themselves full to science and call themselves scientists. The uncomfortable middle ground that Journalist Science straddles is contributing greatly to the reputational decline of both of its components.