When doctors want to help untangle confusing and sometimes contradictory findings in the scientific literature, they often turn to specially crafted summary studies. These are considered the gold standard for evidence. But one of the leading advocates for this practice is now raising alarm about them, because they are increasingly being tainted by commercial interests.
For many years, these studies — called meta-analyses and systematic reviews — seemed to solve a big problem. Doctors who had once relied on each other's expert opinions to select the best treatments gradually turned to careful scientific studies instead.
But the number of studies mushroomed and often came to different conclusions. So in the 1990s, doctors and medical advisory committees started relying on studies that combined results from many different research projects to streamline the search for answers.
These kinds of studies are "extremely important," says Dr. John Ioannidis, a professor of medicine health research and policy at Stanford University. He has conducted many of these types of studies over the course of his career. "They're trying to make some sense out of a very convoluted scientific and medical literature."
But Ioannidis says unfortunately things have gotten out of hand. First, "the problem is that there are just too many meta-analyses," Ioannidis says.
In a recent study on the subject, titled "The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-Analyses," he chose as an example studies involving antidepressant drugs. "There are 185 of them published in the literature within seven years, which means about 25 of them published every year for the very same drugs and the very same indication, major depression," Ioannidis said.
What's worse, they're increasingly being generated by scientists who have financial interests in the outcome, Ioannidis found.
"About 80 percent of them have been funded or have some other conflicts of interest with manufacturers of these drugs," he says. "If you look at what their conclusions are, those that have been authored by industry employees, practically all of them, with one exception, have claimed that there are no caveats about antidepressants."
You have to read deep into the studies to find warnings about potential suicide risks, for example — if downsides are mentioned at all.
That's potentially misleading to doctors who turn to this kind of analysis to get a quick take on what works and what doesn't. Ioannidis says the drug industry has started using meta-analysis for commercial purposes, rather than as a disinterested look at the evidence.
"They can get the results or at least the interpretation that fits their needs. So you have the most powerful and most prestigious design in current medical evidence, and it can be easily manipulated as an advertisement, as a marketing tool."
And that defeats one of the main purposes of these studies, which is to make an overflowing scientific literature more manageable.
Peter Kramer, a clinical professor emeritus at Brown University and author of Listening to Prozac, took a deep dive into meta-analyses when he was writing his latest book, Ordinarily Well. He found the situation even worse than Ioannidis suggests.
"In some ways my doubts are stronger than his," Kramer told Shots.
Some analyses he looked at were trying to parse very subtle differences — for example, comparing two very similar antidepressants. You could get any answer you want, depending on how you set up the study, Kramer says.
And the problems he found aren't simply commercial conflicts of interest. For example, he saw biases among academics who were wedded to the notion that placebos are just as good as actual drugs for depression.
"Even on that side of the equation, where there's no adverse sponsorship but just an allegiance to an idea, I thought it wasn't always the case that everybody's hands were on the table," Kramer said.
But financial conflicts are easier to identify. Five years ago, the National Academy's Institute of Medicine (now called the National Academy of Medicine) suggested dozens of standards that should apply to systematic reviews of the research literature.
The influence of funders is a concern, says Dr. Alfred Berg, a professor emeritus of family medicine at the University of Washington who chaired one of the committees.
"If there's obvious sponsorship from an organization that might have a conflict of interest, that's probably not a good idea," he says. It's reasonable to suggest that scientists with clear financial conflicts of interest should not be producing these studies. "Is it going to happen in my lifetime?" Berg says. "Probably not!"
Pfizer, one of the many drug companies that fund meta-analyses involving their own products, declined an interview request. But the company noted in an emailed statement that the drug company "shares data from its clinical trials with outside members of the research community for them to conduct their own independent meta-analyses."
Fortunately, scientific journals are starting to do a better job of making sure researchers disclose their financial interests. It's up to readers now to take heed.
Email Richard Harris at rharris@npr.org.
Copyright 2021 NPR. To see more, visit https://www.npr.org. 9(MDEwMTk5OTQ0MDEzNDkxMDYyMDQ2MjdiMw004))
