I Think You'll Find It's a Bit More Complicated Than That (46 page)

And so I give you my taxonomy of bad science, the things that make me the maddest. First, of course, we shall take on duff reporting: ill-informed, credulous journalists, taking their favourite loonies far too seriously, or misrepresenting good science, for the sake of a headline. They are the first against the wall.

Next we’ll move on to the quacks: the creationists, the new-age healers, the fad diets. They’re sad and they’re lonely. I know that. But still they must learn. Advertisers, with their wily ways, and their preposterous diagrams of molecules in little white coats: I’ll pull the trigger.

And the same goes for the quantum spin on government science. I’m watching you all.

And finally, let us not forget the strays, the good scientists who have passed to the dark side. Was it those shares in that drug company, or the lust for fame and glory? Bad scientists, your days are numbered.

If you are a purveyor of bad science, be afraid. If you are on the side of light and good, be vigilant; and for the love of Karl Popper, email me every last instance you find of this evil. Only by working joyously together can we free this beautiful, complex world from such a vile scourge.

What Eight Years of Writing the Bad Science Column Has Taught Me

Guardian
, 4 November 2011

I’ve got to go and
finish a book
: I’ll be back in six months, but in case it kills me, here’s
what I’ve learned
in eight years
of
writing this column
.

Alternative therapists
don’t kill many people, but they do make a
great teaching tool
for the basics of evidence-based medicine, because their efforts to distort science are so extreme. When they pervert the activities of people who should know better –
medicines regulators
, or
universities
– it throws sharp relief onto the role of
science and evidence in culture
. Characters from this community who wonder why people keep writing about them should look at
their libel cases
and their awesomely bad behaviour under fire. You are a
comedy factory
. Don’t go changing.

Next: the real story of
how the world works
is much weirder than anything a quack can make up.
The placebo effect
is maddening, the nocebo effect more so, but the research on how we make decisions, and are
misled by heuristics
and mental shortcuts, is the wildest of all. Knowing about these belief-hacks gives you
thrills, and power
.

Pharmaceutical companies
can behave dismally. Most important, they
still won’t publish all
the results of all the clinical trials conducted on humans. This is indefensible, and because
we tolerate it
, we don’t know the true effect sizes of the medicines that we give. This absurd situation mocks the whole of medicine: we need legislation to fix it, and popular movements to drive that. I’ll join yours.

Journalists
can mislead the public
about the answers of evidence-based medicine, which is bad. But they also mislead us on
the methods and techniques
. We live in a new era of doctors and patients – at our best – making decisions together. For that collaboration to work, everyone needs to understand how we know if something is good for us, or bad for us. The basics of evidence-based medicine, of trials, meta-analyses, cohort studies and the like should be taught in schools and waiting rooms. It’s interesting, but it’s also life and death: people care about it.

Politicians misuse evidence
,
and distort it
to shameful degrees
. But more than that, there are endless cases where we could do randomised
trials on policies
– old and new – to find out
if they achieve
the outcomes they’re aiming for. There is
no honourable excuse
for failing to use
the fairest tests
we can design.

Real scientists
can behave as badly as anyone else. Science isn’t about authority, or white coats, it’s about following a method. That method is built on core principles: precision and transparency; being clear about your methods; being honest about your results; and drawing a
clear line between the results
, on the one hand, and your judgement calls about how those results support a hypothesis. Anyone blurring these lines is iffy.

Conflict-of-interest stories – where someone has a vested interest in the results of their study – are important, because they tell you when there’s a risk that something’s wrong in a piece of science. But this is only motive: the gruesome, fascinating mechanism of a crime against science – the methodological flaws – that’s where the action is. People who don’t really understand science can only critique it in terms of motive. Let them have that; we’ll do the details.

Last,
nerds are more powerful
than we know. Changing mainstream media will be hard, but you can help create parallel options. More academics should blog, post videos, post audio, post lectures, offer articles, and more. You’ll enjoy it: I’ve had threats and blackmail, abuse, smears and formal complaints with forged documentation.

But it’s worth it, for one simple reason: pulling bad science apart is the
best teaching gimmick
I know
for explaining
how good
science works
. I’m not a policeman, and I’ve never set out to produce a long list of what’s right and what’s wrong. For me, things have to be interestingly wrong, and the methods are all that matter.

So keep the nonsense coming, I’ll see you next year for more, and if you miss me, I’ll be procrastinating at
badscience.net
, and
@bengoldacre
on Twitter.

I haven’t yet gone back to a weekly column. The things I write take a huge amount of time: not just for the research, but also from chasing down endless blind alleys to find that one shining gem of bad behaviour that can illustrate an interesting bit of science.

To reassure you, I haven’t been smoking dope on the sofa. Over the past three years I’ve written
Bad Pharma
, and pushed hard on the policy fallout that it triggered (which you can read about in the updated second edition). I’ve fallen back in love with seeing patients; dived headlong into teaching; had exciting, busy day jobs; done hundreds of talks; put fingers into pies; and fallen into lobbying and policy work on the changes I’ve advocated for (which you can read about in the last book and, especially, the next one). Lastly, I’ve become obsessed with the technical fun of shaping a big argument over a book rather than 700 words: there are two of these to come very soon. Also, we’ve had babies.

I am extremely optimistic about the growing role of science in society. There is still an endless stream of nonsense in mainstream media, politics, and medicine: but the last ten years have seen a spectacular flourishing of science, in live shows, and most importantly online. Rebuttals, fact-checks and evidence can be thrown onto Twitter or blogs within minutes of a dubious claim being made, and anyone can get access to good quality information, wherever they’re motivated.

If you are a purveyor of bad science: the mountain of bullshit is slowly shifting beneath your feet. You should be worried.

And if you are a nerd: keep talking, stay vigilant, and be proud.

This is our time.

Footnotes

Nullius in Verba

1
This column uses the example of some work by the
Guardian
’s health correspondent to illustrate the importance of transparency about research methods, as well as results. This is a growing issue, as raw data and the tools for analysis have become more accessible (excitingly) – and more widely used – outside of traditional academia. It is the only column by me that the
Guardian
has ever declined to publish.

How Myths Are Made

1
See
here
.

Publish or Be Damned

1
Where there are updates, occasionally, throughout the book, they are after the piece in italics.

Pink, Pink, Pink, Pink. Pink Moan

1
Unless you have an Athens login, you are not allowed to read what the researchers actually said, instead of what the media said they said. Because although they are publicly funded academics at the University of Newcastle, and although this work has been publicised in every major mainstream media outlet in Britain and the US, and although the journal is edited by academics you fund, and paid for by subscriptions from university libraries … the actual academic article is behind a paywall, with a payment model geared towards institutions, rather than interested individuals. Bad luck you. I guess you have to rely on journalists.

‘Hello Madam, Would You Like Your Children to Be Unemployed?’

1
See
here
.

Confound You!

1
In case it’s been puzzling you, epidemiologists use ‘odds’ (e.g. 366 ÷ 2300 for the top row of Table 1) rather than ‘proportions’ (which would be 366 ÷ 2666 for the top row of Table 1) because odds work more neatly when you use ‘logistic regression’, which is the more advanced technique mentioned above. If you’re interested to know more, I recommend coming to London School of Hygiene and Tropical Medicine to do our MSc in Epidemiology.

Bicycle Helmets and the Law

1
This is an editorial I wrote for the
British Medical Journal
with David Spiegelhalter about the complex, contradictory mess of evidence on the impact of bicycle helmets. Like most places where there’s controversy and disagreement, this is a great opportunity to walk through the benefits and shortcomings of different epidemiological techniques, from case-control studies to modelling. Epidemiology is my day job –
Bad Science
and
Bad Pharma
are both, effectively, epidemiology textbooks with bad guys – and since the techniques of epidemiology are at the core of most media stories and squabbles on health, it’s very weird that you don’t hear the word more often.

Medical Hypotheses Fails the Aids Test

1
See
here
.

2
See
here
.

Building Evidence into Education

1
Writing is just a hobby, alongside seeing patients, doing research, and – increasingly – putting these ideas into practice through campaigning and lobbying. In 2011 I co-authored a Cabinet Office White Paper explaining how randomised controlled trials can be used to improve government policy (this is the subject of my next book). And in 2012 I did an Independent External Review for the Department for Education, looking at what could be done to improve the use of evidence and data in schools. This was a dry internal report, but I was also asked to write something to explain what these changes might look like, aimed specifically at teachers, which is reproduced here. If you like it, and want to share it, there’s a PDF on the DfE website. I feel fairly hopeful, but these are long, slow cultural shifts. In June 2014 DfE advertised a public tender to assess progress towards the goals I set out in the internal report; so we shall see.

A Rock of Crack as Big as the Ritz

1
I chose titles on my blog spontaneously when I posted the pieces. ‘A Rock of Crack as Big as the Ritz’ is the title of a short story by Will Self.

Heroin on Prescription

1
There might be other explanations; I was only a medical student – BG, 2014.

The Noble and Ancient Tradition of Moron-Baiting

1
From the preface to the second edition: ‘The first edition of this book prompted many curious letters from irate readers. The most violent letters came from Reichians, furious because the book considered orgonomy alongside such (to them) outlandish cults as dianetics. Dianeticians, of course, felt the same about orgonomy. I heard from homoeopaths who were insulted to find themselves in company with such frauds as osteopathy and chiropractic, and one chiropractor in Kentucky “pitied” me because I had turned my spine on God’s greatest gift to suffering humanity. Several admirers of Dr. Bates favoured me with letters so badly typed that I suspect the writers were in urgent need of strong spectacles. Oddly enough, most of these correspondents objected to one chapter only, thinking all the others excellent.’

Why Don’t Journalists Mention the Data?

1
The thirty-seventh study was released in the fortnight between the previous article and this one.

Empathy’s Failures

1
In 2014 Rolf Harris was found guilty of several sexual offences against children and jailed for over five years. I’ve left this piece in, partly as a reminder that abusers don’t always have the word ‘monster’ tattooed on their forehead. That phrasing comes from PinkZapCat, when I asked on Twitter about deleting the column with Rolf in it. I think it’s extremely wise.