Science. It Works. #2

Science is often flawed. It’s time we embraced that. by Julia Belluz and Steven Hoffman

Stories like Stapel’s are what most people think of when they think about how science goes wrong: an unethical researcher methodically defrauding the public.

But outright fraud is just one potential derailment from truth. And it’s actually a relatively rare occurrence.

Recently, the conversation about science’s wrongness has gone mainstream. You can read, in publications like Vox, the New York Times or the Economist, about how the research process is far from perfect — from the inadequacies of peer review to the fact that many published results simply can’t be replicated. The crisis has gotten so bad that the editor of The Lancet medical journal Richard Horton recently lamented, “Much of the scientific literature, perhaps half, may simply be untrue.”

When people talk about flaws in science, they’re often focusing on medical and life sciences, as Horton is. But that might simply be because these fields are furthest along in auditing their own problems. Many of the structural problems in medical science could well apply to other fields, too.

That science can fail, however, shouldn’t come as a surprise to anyone. It’s a human construct, after all. And if we simply accepted that science often works imperfectly, we’d be better off. We’d stop considering science a collection of immutable facts. We’d stop assuming every single study has definitive answers that should be trumpeted in over-the-top headlines. Instead, we’d start to appreciate science for what it is: a long and grinding process carried out by fallible humans, involving false starts, dead ends, and, along the way, incorrect and unimportant studies that only grope at the truth, slowly and incrementally.

Acknowledging that fact is the first step toward making science work better for us all.

In an analysis of 300 clinical research papers about epilepsy — published in 1981, 1991, and 2001 — 71 percent were categorized as having no enduring value. Of those, 55.6 percent were classified as inherently unimportant and 38.8 percent as not new. All told, according to one estimate, about $200 billion — or the equivalent of 85 percent of global spending on research — is routinely wasted on flawed and redundant studies

Indeed, another review found that researchers at Amgen were unable to reproduce 89 percent of landmark cancer research findings for potential drug targets.

The researchers discovered that university press offices were a major source of overhype: more than one-third of press releases contained either exaggerated claims of causation (when the study itself only suggested correlation), unwarranted implications about animal studies for people, or unfounded health advice.

Worse, the scientists were usually present during the spinning process, the researchers wrote: “Most press releases issued by universities are drafted in dialogue between scientists and press officers and are not released without the approval of scientists and thus most of the responsibility for exaggeration must lie with the scientific authors.”

Right now, taxpayers fund a lot of the science that gets done, yet journals charge users ludicrous sums of money to view the finished product. American universities and government groups spend $10 billion each year to access science.

A failure to appreciate how science works, its faults and limitations, breeds mistrust. At a meeting at the National Academy of Sciences this month, health law professor and author Tim Caulfield pointed out that one of the things readers often use against his pro-science arguments is that “science is wrong” anyway, so why bother. In other words, people hear about research misconduct or fraud, see the contradictory studies out there, and conclude that they can’t trust science.

Instead, if people saw science as a human construction — the result of a tedious, incremental process that can be imperfect in its pursuit of truth — both science and the public understanding of science would be better off. We could learn to trust science for what it is and avoid misunderstandings around what it is not.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s