If you buy something using links in our stories, we may earn a commission. Learn more.
The Trump administration has made many stumbles in its response to the coronavirus pandemic, but one of the key failures was not having enough kits to test for the pathogen once it appeared in the United States. Instead of accepting kits from other countries—including the ones approved by the World Health Organization—the White House went its own way.
On March 17, Deborah Birx, the physician coordinating the administration’s scientific response to the Covid-19 outbreak in the United States, tried to explain the rejections. "It doesn't help to put out a test where 50 percent or 47 percent are false positives," Birx told reporters, suggesting that at least some overseas tests were deeply flawed. A few days later, FDA commissioner Stephen Hahn again mentioned the 47-percent error rate in an interview with National Public Radio, attributing it to “an abstract that was recently published in the literature.” He continued: “What that means is that if you had a positive test, it was pretty close to a flip of a coin as to whether it was real or not."
That sounds reasonable. After all, a test that is no better than a coin flip would do far more harm than good, burdening an already overwhelmed health care system with a tidal wave of well but worried people. Birx is a highly respected scientist whose résumé includes taking on the AIDS epidemic, and Hahn heads perhaps the nation’s most important health agency. But in this case, they appear to have relied on data that, for reasons that are still unclear, has been withdrawn from the scientific literature.
As it happens, both of Birx’s false-positive rates, the “50 percent or 47 percent,” came from a single study. That’s bad enough, if somewhat understandable given how quickly the pandemic has been spreading. Still, scientists are generally loath to base any decisions, including what they eat for breakfast, on a single, unreplicated finding. This time, they appear to have done so on a policy matter of the highest importance.
It’s far worse that the single study had already been retracted. As NPR has now reported, the paper was withdrawn a few days after its publication in a Chinese journal. In other words, the Trump administration may have blocked the importation of some Covid-19 tests at a critical moment in the evolution of the pandemic on the strength of a piece of research that is no longer in the literature.
This case is a particularly dismaying and consequential example of what happens when no one bothers to engage in scientific fact-checking. But it will not be the last time that something we thought we knew about the coronavirus because it was in a published paper will turn out to be wrong.
The last few weeks have seen a blizzard of publications about Covid-19—and, like the outbreak itself in this country, it’s just the beginning. Many of the papers have come from China, where researchers have had more time to study the infection than the rest of the world, but important papers have emerged from Italy, as well.
All of these articles have at least one thing in common: They were written, and published, in great haste. And while desperate times may dissolve norms, speed remains the enemy of rigorous science. (Thoughts and prayers for the poor armchair epidemiologists trying to follow along at home; everything they think they know changes by the hour.)
Consider: The research behind a normal paper takes months or even years to complete, and then months more—or longer—to be published. Journals subject manuscripts to peer review before deciding whether to accept them; once accepted, they must be edited and then ushered through what has typically been a lengthy production process. With the Covid-19 papers, however, the timeline has been radically condensed. At least one key paper—also touted by the Trump administration—was accepted just a day after it was submitted. (That paper is, as we write this, being fisked in multiple quarters.)
Chinese scholarship has been particularly problematic lately. Although the country is a research juggernaut, ethical controls are uneven, at best, and sometimes nonexistent. Chinese scholars have been paid cash bonuses to publish findings in prestigious journals, which almost certainly played a role in the country’s dubious distinction of having more papers retracted for fake peer review than any other. The government recently banned such bonuses as a way to help reduce the pressure to “publish or perish.” But the sudden surge of articles on Covid-19 coming out of the country could frustrate that effort.
In a sign of just how hard academia’s “publish or perish” habit is to break, incentives have put a brake on the publication of at least some findings. As The Wall Street Journal reported earlier this month, some of those who had access to data from early cases at a hospital in Wuhan “were reluctant to share data with others before publication in a prestigious medical journal.”
Meanwhile, many researchers have taken an alternate route to getting their findings in front of their colleagues and the world. They have been resorting to the use of so-called preprint servers to post preliminary drafts of their papers—rough cuts of research in progress.
Make no mistake: Preprint servers are an important development in scholarly publishing. They allow readers to get an early glimpse of potentially important findings and possibly to help shape the refinement of the research into what, ideally, will be a much more rigorous final product. However—like everything else in science—they are works in progress and should not be considered entirely reliable on their own as either scientific evidence or the basis of public policy.
Then again, no single peer-reviewed paper should be, either. The study of the test for Covid-19 had appeared not on a preprint server but in the Chinese Journal of Epidemiology, which, according to NPR, published the article on March 5 and retracted it shortly after. Unfortunately, the retraction did not appear in the US government’s PubMed database—the world’s largest repository of scientific papers about biomedical subjects—until this week, allowing Birx, Hahn, and others here to cite the article without knowing that it had been spiked.
If anything, the appearance of a retraction notice on PubMed within a few weeks of the paper’s being pulled was lightning speed. Many times, publishers never get around to alerting such databases to their retractions, leading many researchers to do exactly what Birx and her team did: Citing retracted papers in support of their work. (We have a database to prevent that!)
The truth is that much of the blitzkrieg of science that emerges in the coming days and weeks will turn out to be wrong, at least in part, and that’s not a bad thing. Science is not the exercise of getting it right the first time, every time. Knowledge comes incrementally, if at all. The problem, of course, is that we can’t always wait for that new knowledge to arrive. When it comes to findings, the Covid-19 train is an express, while the rigorous science coach is a local. Until that local arrives at its final destination, it may be wise to label all this research—preprints, peer-reviewed papers, and for goodness’ sake, pronouncements from Donald Trump— with a black-box warning: “There is some evidence for this now. It will likely turn out to be at least partially wrong.”
In other words, let’s all stay six studies away.
WIRED Opinion publishes articles by outside contributors representing a wide range of viewpoints. Read more opinions here. Submit an op-ed at opinion@wired.com.
- It's time to do the things you keep putting off. Here's how
- What isolation could do to your mind (and body)
- Bored? Check out our video guide to extreme indoor activities
- Blood from Covid-19 survivors may point the way to a cure
- How is the virus spread? (And other Covid-19 FAQs, answered)
- Read all of our coronavirus coverage here
