Understanding Coronavirus Research

Why the COVID-19 studies seem to contradict one another

research lab table with coronavirus blood samples in testing tubes and other medical equipment

Getty Images

By:

Meryl Davids Landau

En español

Published May 29, 2020

In science's battle against COVID-19, it's fair to say that headlines can be confusing. Everything from masks to heat to the drug remdesivir has been presented, depending on the week, in a vastly different light. It's easy to lose track of what's the next big hope in fighting the coronavirus and its spread and what's not worth the hype.

So you may be wondering how medical science can be so contradictory. But the reality is, for the most part, research is proceeding mostly as it should, which is similar to the way a football makes it to the end zone: inch by hard-won inch, with plenty of backsliding.

The difference with the novel coronavirus is that such early studies — ones that under more normal circumstances would be shared with only a tight circle of academics — now swiftly make news around the world. In essence, we're watching behind-the-scenes medical science unfold before our eyes.

Getting initial information out quickly has benefits for fighting a new disease that poses a widespread threat, but it also means people need to be especially discerning, says Anupam B. Jena, M.D., associate professor of health care policy at Harvard Medical School. “What you're reading in the news is work that is preliminary. Investigators are releasing their results so others can benefit, which is appropriate.” The problem, he says, is that no one has had time to completely vet the findings.

To experts, none of the seemingly contradictory back-and-forth that unfolds is surprising. “Ideally, as soon as researchers find something, it would always be true. But that just does not happen,” Jena says. Instead, knowledge about every disease is continually updated with newer research until, eventually, a more complete picture emerges.

"Medical research for COVID-19 is proceeding at such a rapid pace that the process of getting information, evaluating it, further studying it and readjusting what we know is happening more publicly than usual and on a much more compressed schedule,” explains Deborah Doroshow, M.D., an assistant professor of medicine at the Icahn School of Medicine at Mount Sinai.

Of course, the process of new research shifting what was previously understood in medicine isn't new. After insulin was discovered, experts thought it would treat a variety of diseases; then it became clear the drug was a game changer just for diabetes. And 50 years ago, scientists became convinced that many cancers were caused by viruses, only to learn that this is true just for cervical cancer and a few others, Doroshow says.

How preliminary drug findings play out

In the case of remdesivir, the first news report about the drug came from a study of hospitalized patients in China. The study was stopped before it could be completed, reportedly because cases of the disease were dropping in the country, so not enough seriously ill people could be enrolled. Still, preliminary data that was posted on the World Health Organization's website and picked up by the media showed that the drug was not effective.

The second, ongoing study is sponsored by the National Institutes of Health. Preliminary results from that trial of roughly 1,000 patients, published in the New England Journal of Medicine, show that remdesivir enables patients to leave the hospital several days earlier than those not on the drug.

Despite the dueling headlines generated by both reports, many steps remain in order to understand this medication (and many others being tested). For one thing, findings from the China study were not published in a journal or peer-reviewed. That process, in which experts assess the quality of research before allowing its release, is important, as is how studies are often further dissected after publication. “Doctors sit down around the table in ‘journal clubs’ and have vigorous discussions about whether the design of a study allows it to adequately answer the questions it claims to,” Doroshow explains. This often leads to additional research.

The incremental, often zigzag pace of science is playing out in many other areas of coronavirus news. For instance, estimated fatality rates for the disease change regularly. “This takes us back to grade-school fractions,” Doroshow says, because rates vary dramatically, depending on if you are calculating people who have died as a percentage of only those hospitalized with documented COVID-19 or counting everyone who is thought to have it (whether or not they're hospitalized). As more people in the community are tested, rates will keep changing, she says; in the meantime, experts make the best estimates they can.

Masks are another topic subject to refinement. Initially, experts focused on whether a mask could keep a person from contracting the disease and determined that common cloth masks do not. But then studies began to emerge about how masks keep droplets from an infected person from spreading as readily to others. Now the Centers for Disease Control and Prevention (CDC) recommends that people wear cloth face coverings to slow the virus's spread.

What to consider when you look at studies

When evaluating news about COVID-19 research, look closely at the number of people studied, recommends Leslie McClure, chair of epidemiology and biostatistics at Drexel University in Philadelphia. If the group is small (generally fewer than 100), the results may not be generalizable to others. Also ask yourself whether the benefits shown seem important, such as whether a treatment saves lives. And don't confuse correlation with causation. A correlation means that in a given study, two things seem to be linked; those results are a lot softer than outcomes that prove one thing definitively causes another.

It's meaningful to note, as well, where the study was published, which can range from a medical journal to a “preprint” site (which is not peer-reviewed) to a researcher's personal website. But right now, even well-regarded journals are publishing the kinds of preliminary studies about the coronavirus they wouldn't for other diseases, such as research with only a handful of subjects or those that didn't give a placebo to half the participants (making them the control group) in a study of a particular virus treatment, says Holly Fernandez Lynch, assistant professor of medical ethics at the University of Pennsylvania.

Ethicists are debating whether this is a good idea (Lynch thinks it isn't helpful in the long run to sacrifice quality for speed), but it means that even findings published in major journals are likely to be clarified or changed by later research, she says.

When this happens, it isn't a sign that science isn't working but that it is, Doroshow asserts. “The scientific process involves being humble enough to know that what we thought to be true can change over time. This doesn't mean we're doing a bad job. It means that by evaluating and reevaluating, we're doing a good job."

Benefits Recommended For You

See All