Bad Science

Ben Goldacre

Summary: An overview of the problems with the current state of science and its methods and procedures

Score: 80 / 100

health—half of all science stories in the media are medical—and (Friday, August 30, 2013, 02:32 PM, page 56-57)

Homeopathy pills are, after all, empty little sugar pills that seem to work, (Friday, August 30, 2013, 02:35 PM, page 65)

Nutritionists are alternative therapists but have somehow managed to brand themselves as men and women of science. (Friday, August 30, 2013, 02:37 PM, page 70-71)

There is only one internationally recognized method for identifying something as earwax: pick some up on the end of your finger, and touch it with your tongue. If your experiment had the same results as mine, both of them taste a lot like candle wax. (Friday, August 30, 2013, 03:55 PM, page 187-88)

Yom Kippur in Judaism, Ramadan in Islam, and all manner of other similar rituals in Christianity, Hinduism, the Baha’i faith, Buddhism, and Jainism are each about abstinence and purification (among other things). (Friday, August 30, 2013, 04:11 PM, page 240-42)

now that science is our dominant explanatory framework for the natural and moral world, for right or wrong, (Friday, August 30, 2013, 04:15 PM, page 257-58)

set of experiments from the March 2008 edition of the Journal of Cognitive Neuroscience, which elegantly demonstrated that people will buy into bogus explanations much more readily when they are dressed up with a few technical words from the world of neuroscience. (Friday, August 30, 2013, 04:32 PM, page 300-302)

children are predisposed to learn about the world from adults, and specifically from teachers; they are sponges for information, for ways of seeing, and authority figures who fill their heads with nonsense are sowing the ground, I would say, for a lifetime of exploitation. (Friday, August 30, 2013, 04:43 PM, page 338-40)

First, you want your expensive cream to hydrate your skin. They all do that, and Vaseline does the job very well; in fact, much of the important early cosmetics research was about preserving the moisturizing properties of Vaseline, while avoiding its greasiness, and this technical mountain was scaled several decades ago. (Friday, August 30, 2013, 05:23 PM, page 378-80)

alpha hydroxy acids, high levels of vitamin C, or molecular variations on the theme of vitamin A. These have genuinely been shown to make your skin seem more youthful, but they are only effective at such high concentrations, or high acidity levels, that the creams cause irritation, stinging, burning, and redness. (Friday, August 30, 2013, 05:26 PM, page 391-93)

In general, you don’t absorb things very well through your skin, because its purpose is to be relatively impermeable. When you sit in a bath of baked beans for charity, you do not get fat, nor do you start farting. (Friday, August 30, 2013, 06:58 PM, page 415-16)

The simple theme running through all these products is that you can hoodwink your body, when in reality there are finely tuned “homeostatic” mechanisms, huge, elaborate systems with feedback and measuring devices, constantly calibrating and recalibrating the amounts of various different chemical constituents being sent to different parts of your body. If anything, interfering with that system is likely to have the opposite of the simplistic effects claimed. (Friday, August 30, 2013, 07:03 PM, page 425-28)

The link between the magic ingredient and efficacy is made only in the customer’s mind, and reading through the manufacturer’s claims, you can see that they have been carefully reviewed by a small army of consultants to ensure that the label is highly suggestive, (Friday, August 30, 2013, 07:08 PM, page 443-45)

Just like the lottery, the cosmetics industry is playing on people’s dreams, and people are free to waste their money. (Friday, August 30, 2013, 07:09 PM, page 448-49)

I can very happily view fancy cosmetics—and other forms of quackery—as a special, self-administered, voluntary tax on people who don’t understand science properly. (Friday, August 30, 2013, 07:10 PM, page 449-50)

Homeopathy was devised by a German doctor named Samuel Hahnemann in the late eighteenth century. (Friday, August 30, 2013, 07:18 PM, page 486)

For this purpose Hahnemann had a saddle-maker construct a bespoke wooden striking board, covered in leather on one side and stuffed with horsehair. These ten firm strikes are still carried out in homeopathy pill factories today, sometimes by elaborate, specially constructed robots. (Friday, August 30, 2013, 07:24 PM, page 506-8)

The typical homeopathic dilution is 30C; this means that the original substance has been diluted by one drop in a hundred, thirty times over. (Friday, August 30, 2013, 07:29 PM, page 537-38)

We know that two sugar pills are a more effective treatment than one sugar pill, for example, and we know that saltwater injections are a more effective treatment for pain than sugar pills, not because saltwater injections have any biological action on the body, but because an injection feels like a more dramatic intervention. We know that the color of pills, their packaging, how much you pay for them, and even the beliefs of the people handing the pills over are all important factors. We know that placebo operations can be effective for knee pain and even for chest pain. The placebo effect works on animals and children. It is highly potent, and very sneaky, and you won’t know the half of it until you read the placebo chapter in this book. (Friday, August 30, 2013, 07:49 PM, page 603-8)

Voltaire said, “The art of medicine consists in amusing the patient while nature cures the disease.” (Friday, August 30, 2013, 07:50 PM, page 620)

claims for miracle cures should be treated with caution, because “miracles” occur routinely, in 1 percent of cases by their definition, and without any specific intervention. (Friday, August 30, 2013, 08:06 PM, page 661-62)

trials with inadequate blinding exaggerated the benefits of the treatments being studied by 17 percent. (Friday, August 30, 2013, 08:27 PM, page 740-41)

(The placebo control for acupuncture, in case you’re wondering, is sham acupuncture, with fake needles or needles in the “wrong” places, although an amusing complication is that sometimes one school of acupuncturists will claim that another school’s sham needle locations are actually their genuine ones.) (Friday, August 30, 2013, 08:30 PM, page 745-47)

people have studied the effect of randomization in huge reviews of large numbers of trials and found that the ones with dodgy methods of randomization overestimate treatment effects by 41 percent. (Friday, August 30, 2013, 08:42 PM, page 789-90)

there is an almost linear relationship between the methodological quality of a homeopathy trial and the result it gives. The worse the study—which is to say, the less it is a “fair test”—the more likely it is to find that homeopathy is better than placebo. (Friday, August 30, 2013, 08:48 PM, page 815-17)

Cochrane Collaboration, an international not-for-profit organization of academics that produces systematic summaries of the research literature on health care research, including meta-analyses. (Friday, August 30, 2013, 09:00 PM, page 850-51)

it’s a little-known fact that this very phrase has been effectively banned from the British Medical Journal for many years, on the ground that it adds nothing; you may say what research is missing, on whom, how, measuring what, and why you want to do it, but the hand-waving, superficially open-minded call for “more research” is meaningless and unhelpful. (Friday, August 30, 2013, 09:13 PM, page 883-86)

People do experience that homeopathy is positive for them, but the action is likely to be in the whole process of going to see a homeopath, of being listened to, having some kind of explanation for your symptoms, and all the other collateral benefits of old-fashioned, paternalistic, reassuring medicine. (Oh, and regression to the mean.) (Friday, August 30, 2013, 09:16 PM, page 888-90)

Henry Beecher, an American anesthetist, wrote about operating on a soldier with horrific injuries in a World War II field hospital, using salt water because the morphine was all gone, and to his astonishment the patient was fine. (Saturday, August 31, 2013, 08:26 AM, page 969-71)

the Declaration of Helsinki, the international ethics bible). (Saturday, August 31, 2013, 08:29 AM, page 986)

He found, spectacularly, that four sugar pills are better than two (Saturday, August 31, 2013, 08:33 AM, page 1003)

Sham ultrasound is beneficial for dental pain, placebo operations have been shown to be beneficial in knee pain (Saturday, August 31, 2013, 08:44 AM, page 1057)

“Electrical machines have great appeal to patients,” wrote Dr. Alan Johnson in The Lancet in 1994 about this trial, “and recently anything to do with the word LASER attached to it has caught the imagination.” He’s not wrong. I went to visit an alternative therapist once, and she did gem therapy on me, with a big shiny science machine that shone different-colored beams of light onto my chest. (Saturday, August 31, 2013, 08:49 AM, page 1078-81)

both what the doctor says and what the doctor believes have an effect on healing. (Saturday, August 31, 2013, 08:54 AM, page 1098)

alternative therapists don’t just give placebo treatments; they also give what we might call placebo explanations or placebo diagnoses: ungrounded, unevidenced, often fantastical assertions about the nature of the patient’s disease, involving magical properties, or energy, or supposed vitamin deficiencies, or “imbalances,” which the therapist claims uniquely to understand. (Saturday, August 31, 2013, 09:01 AM, page 1126-29)

there has been a huge amount of research into the value of a good therapeutic relationship, and the general finding is that doctors who adopt a warm, friendly, and reassuring manner are more effective than those who keep consultations formal and do not offer reassurance. (Saturday, August 31, 2013, 09:02 AM, page 1134-35)

His results suggest—albeit it in a very small sample—that a drug could be made to have the opposite effect from what you would predict from the pharmacology, simply by manipulating people’s expectations. (Saturday, August 31, 2013, 09:11 AM, page 1181-83)

the selfsame drug became less effective with time, as new drugs were brought in. (Saturday, August 31, 2013, 09:21 AM, page 1217-18)

it’s routine marketing practice for homeopaths to denigrate mainstream medicine. (Saturday, August 31, 2013, 09:29 AM, page 1263)

The philosopher professor Harry Frankfurt of Princeton University discusses this issue at length in his classic 1986 essay “On Bullshit.” Under his model, “bullshit” is a form of falsehood distinct from lying: the liar knows and cares about the truth but deliberately sets out to mislead; the (Saturday, August 31, 2013, 09:45 AM, page 1318-20)

There have been an estimated fifteen million medical academic articles published so far, and five thousand journals are published every month. (Saturday, August 31, 2013, 10:24 AM, page 1447-48)

In the nineteenth century, as the public health doctor Muir Gray has said, we made great advances through the provision of clean, clear water; in the twenty-first century we will make the same advances through clean, clear information. (Saturday, August 31, 2013, 10:44 AM, page 1474-76)

free radical theory of aging. (Saturday, August 31, 2013, 11:01 AM, page 1506)

Similarly, when you take a snapshot picture of the people who take antioxidant supplement pills, you will often find that they are healthier or live longer: but again (although nutritionists are keen to ignore this fact), these are simply surveys of people who have already chosen to take vitamin pills. These are people who are more likely to care about their health and are different from the everyday population—and perhaps from you—in lots of other ways, far beyond their vitamin pill consumption: they may take more exercise, have more social supports, smoke less, drink less, and so on. (Saturday, August 31, 2013, 11:09 AM, page 1532-36)

The people having the antioxidant tablets were 46 percent more likely to die from lung cancer, and 17 percent more likely to die of any cause,10 than the people taking placebo pills. (Saturday, August 31, 2013, 11:25 AM, page 1585-86)

describing the experiences of 230,000 people in total. This showed that overall, antioxidant vitamin pills do not reduce deaths, and in fact, they may increase your chance of dying. (Saturday, August 31, 2013, 11:29 AM, page 1598-99)

And calcium supplements once looked like a good idea for osteoporosis, but now it turns out that they probably increase the risk of heart attacks in older women, so we change our view. (Saturday, August 31, 2013, 11:31 AM, page 1607-9)

there is essentially no difference between the vitamin industry and the pharmaceutical and biotech industries (that is one message of this book, after all: the tricks of the trade are the same the world over). (Saturday, August 31, 2013, 11:46 AM, page 1618-19)

Eat lots of fruit and vegetables, and live your whole life in every way as well as you can: exercise regularly as part of your daily routine, avoid obesity, don’t drink too much, don’t smoke, and don’t get distracted from the real, basic, simple causes of ill health. (Saturday, August 31, 2013, 12:01 PM, page 1649-51)

The Women’s Health Initiative was another huge randomized controlled trial into dietary change, and it gave similarly gave negative results. They all tend to. (Sunday, September 01, 2013, 06:52 PM, page 1899-1900)

pawn (Sunday, September 01, 2013, 07:02 PM, page 1948)

This sounds low, but it seems the more common treatments tend to have a better evidence base. (Monday, September 02, 2013, 07:17 AM, page 2194)

bringing a drug to market costs around five hundred million dollars in total. (Monday, September 02, 2013, 07:26 AM, page 2254)

only 10 percent of the world’s health burden receives 90 percent of total biomedical research funding. (Monday, September 02, 2013, 07:29 AM, page 2273-74)

in 2003 a systematic review found thirty separate studies looking at whether funding in various groups of trials affected the findings. Overall, studies funded by a pharmaceutical company were found to be four times more likely to give results that were favorable to the company than were independent studies. (Monday, September 02, 2013, 08:29 AM, page 2368-70)

First, when you get a negative result, it feels as if it’s all been a bit of a waste of time. It’s easy to convince yourself that you found nothing when in fact you discovered a very useful piece of information: that the thing you were testing doesn’t work. (Monday, September 02, 2013, 08:37 AM, page 2387-89)

In 1995, only 1 percent of all articles published in alternative medicine journals gave a negative result. (Monday, September 02, 2013, 08:39 AM, page 2402-3)

this ingenious experiment shows is how bad we are at correctly identifying random sequences. (Tuesday, September 03, 2013, 07:21 AM, page 2575-76)

We see patterns where there is only random noise. We see causal relationships where there are none. (Tuesday, September 03, 2013, 06:08 PM, page 2604-5)

our tendency, in our unchecked intuitive reasoning style, to seek out information that confirms a hypothesis, (Thursday, September 05, 2013, 08:59 AM, page 2620-21)

attributional bias: we believe our successes are due to our own internal faculties, and our failures are due to external factors; whereas for others, we believe their successes are due to luck, and their failures to their own flaws. We can’t all be right. (Thursday, September 05, 2013, 11:15 PM, page 2714-16)

Lots of people have argued that we evolved to reason and do math with concrete numbers like these, and not with probabilities, so we find them more intuitive. Simple numbers are simple. (Thursday, September 05, 2013, 11:28 PM, page 2751-53)

standard cutoff point for statistical significance is a p-value of 0.05, which is just another way of saying, “If I did this experiment a hundred times, I’d expect a spurious positive result on five occasions, just by chance.” (Thursday, September 05, 2013, 11:43 PM, page 2850-51)

This breaks a cardinal rule of any research involving statistics: you cannot find your hypothesis in your results. Before you go to your data with your statistical tool, you have to have a specific hypothesis to test. If your hypothesis comes from analyzing the data, then there is no sense in analyzing the same data again to confirm it. (Friday, September 06, 2013, 12:05 AM, page 3015-17)

people aren’t stupid. Anybody can understand anything, as long as it is clearly explained, but, more than that, if they are sufficiently interested. (Monday, September 09, 2013, 06:28 PM, page 3686-87)

To academics, and scientists of all shades, I would say this: you cannot ever possibly prevent newspapers from printing nonsense, but you can add your own sense into the mix. E-mail the features desk, ring the health desk (you can find the switchboard number on the letters page of any newspaper), and offer them a piece on something interesting from your field. They’ll turn you down. Try again. (Tuesday, September 10, 2013, 09:05 AM, page 3731-34)

Start a blog. Not everyone will care, but some will, and they will find your work. Unmediated access to niche expertise is the future, and you know, science isn’t hard—academics around the world explain hugely complicated ideas to ignorant eighteen-year-olds every September—it just requires motivation. (Tuesday, September 10, 2013, 09:08 AM, page 3745-47)