The BMJ warns that a “deluge of poor quality research [in COVID-19] is sabotaging an effective evidence-based response,” while Nature argues that more patients are needed for better trial results.
A little over 100 years since the so-called Spanish Flu that killed millions across the globe, a particularly cruel fate after the millions already lost in World War I, another once-in-a-lifetime pandemic is sweeping the globe.
The death rate is not nearly as high but the threat is no less daunting. And right now, we have little recourse than those in 1918: To try and keep away from each other.
But, we now have better tools and better science at our disposal than we did a century ago; the speed at which the life science world has understood and mapped the genome of SARS-CoV-2, the novel coronavirus virus causing COVID-19, was astounding.
We already have one drug, a failed Ebola antiviral from Gilead called remdesivir, now repurposed to help some COVID-19 patients get out of hospital faster, already allowed in the U.S. and Japan, with more than 100 vaccines in development and dozens of PCR and antibody tests also now on the market to find the virus or evidence of its time inside a host.
But, with the great need for speedy results and a near fanatic focus on one disease area comes some major issues, not least the role of evidence-based medicine. Making a new drug or diagnostic takes time: It is a process of trial and error, with safety and efficacy paramount. To assess these two things, you need lots of people, rigorous scientific method, and deep interrogation of the data.
Two new editorials from BMJ and Nature argue the speed at which science is moving is at risk of forgetting this. The BMJ was particularly scathing, with Paul Glasziou, professor of evidence-based medicine, Sharon Sanders, assistant professor, and Tammy Hoffmann, professor of clinical epidemiology, arguing that a “deluge of poor-quality research is sabotaging an effective evidence-based response.”
There are now more than 1,000 COVID-19 studies underway, according to ClinicalTrials.gov, but many of these are “too small and poorly designed to be helpful, merely adding to the COVID-19 noise,” the authors write.
They point out that there are 145 registered trials of the anti-malaria drug hydroxychloroquine, recently touted by U.S. President Donald Trump as a potentially beneficial drug for COVID-19 after some early anecdotal data from Europe. But the BMJ authors say that 32 have a planned sample size of fewer than 100 patients, while 10 have no control group, and 12 are comparative but non-randomized.
“Outcome measures vary widely, and only 50 seem to be multicenter. Strikingly, only one provides a protocol, and even limited registry details reveal unjustified outcome switching.” Data from these tests will do little to tell us how well this drug works in COVID-19.
The scientific journal Nature makes a similar point on trial sizes: In an editorial, they say that while some tests—such as the World Health Organization’s Solidarity trial of four potential COVID-19 therapies—are “large and collaborative,” it argues that too many others “are smaller, do not always include a control group and don’t test medicines on enough patients to provide statistically meaningful results.”
The trials for remdesivir particularly irk the journal, saying this shows up the “clinical chaos” that can ensue when trials are not well designed. There have been conflicting results, with some failing to show benefit (and hit by stalled enrollment), and others showing it could help some leave hospitals a few days earlier than those not taking the medicine, while another test said it worked as well with a five-day course of the drug as with a ten-day course, “but because the study lacked a control group it was impossible to conclude with any certainty whether the drug had worked,” the editorial said.
The former trial was fairly big and placebo-controlled, but Nature says that as the results were announced at a press conference and the full data have not yet been released, “[w]e do not know, for example, how often participants experienced unwanted side effects, or how well matched those in the control and treatment groups were in terms of age and other medical conditions.”
In the midst of a pandemic that also threatens to turn into a global economic depression, and more excess deaths expected outside of COVID-19 as a result of this downturn, and the lockdown, the need for speed is obvious, but should not curtail good science. We want antivirals, diagnostics and vaccines, but we need to know just as much that they work, and are safe.
Nature concludes that: “In the end, the search for a successful drug needs the power of scale and the learning that comes from collaboration. More trials must, moreover, include control groups and ensure transparency with data.”