(1) You can't say that
The 1998 Wakefield et al Lancet study2 includes just two easy to parse lists of the twelve individuals included in the study, all of whom had received an MMR vaccination. Eight of the twelve (66.67%) children were judged by their parents to have developed autistic behaviors shortly after receiving their MMR vaccinations. That IS IT, my friends. No matched control group. No replication in a larger, more representative sample -- or using alternative measures that we could expect would be more reliable. No common-sense checks that should bear out if the ridiculously large effect size was indeed representative (e.g., did autism diagnoses spike in 1988, the year MMR vaccines were introduced in the UK?).
In short, this should not have been published in the first place, particularly by a medical journal that should see the public health implications a mile away. Sensational claims need to be supported by sensational(ly rigorous, high-quality and not made-up) data. Like The Amazing Randi sez,
"Look, if I told you that I keep a goat in the backyard of my house in Florida, and if you happened to have a man nearby, you might ask him to look over my garden fence, when he'd say, 'That man keeps a goat'. But what would you do if I said, 'I keep a unicorn in my backyard'?"1
(2) Evil or ignorant?
Now we come to Andrew Wakefield, the first author of the study. Turns out he was paid what would today be $80K by lawyers planning to sue vaccine manufacturers to run the study. And his study's "self-enrolled" participants were the very same vaccine suers. Oh, and on top of all that, he also made up the data.
The psychologists among The Science Police wonder, is Wakefield someone who knew what he was doing, and why it was wrong -- but did it anyway? E.g., for fame, money, love, etc. Or is Wakefield a True Believer -- someone who just never thought things through deeply enough to see any one of the many serious, and important flaws in the study -- and perhaps was even motivated a true desire to help people. This would be a POLL in itself, if we weren't more interested in your opinion on another matter...
(3) Who just f***ed up?
This shiny nugget was published in 1998. It's March 2009 now, and the legal hearings aren't expected to conclude until April. The Science Police are here to ask why it is taking over a decade to settle this thing. We put it to you in the poll below.
Finally, The Science Police thanks Brian Deer, a journalist who seems to have developed an amazing sense of smell, for his help keeping the streets safe. And for better people than we, who have been covering this stuff early and giving the events a human face.
1Maddox, J., Randi, J., & Stewart, W.W. (1988). "High-dilution" experiments a delusion. Nature 334, 287 - 290.
Posts
Who f**ked up? That's a really interesting question.
ReplyDeleteWakefield clearly behaved disgracefully, but medial science - and society in general - should be able to deal with individual researchers who act that way. We didn't.
I think the fundamental lesson of the British MMR fiasco (and every other vaccine scare) is that the doctrine "Better safe than sorry" is no longer applicable. We have a tendency to be very sensitive to possible vaccine harms, and there's a great temptation to give any report of vaccine harms the benefit of the doubt. Better safe than sorry, right?
I think this was the attitude of the BMJ editors and I know it was the attitude of many in the media. Private Eye magazine, for example, kept saying "We're not saying MMR causes autism but we think people need to know it's an open question".
this sounded reasonable at the time but we know now that it had dire consequences for public health amongst other things. I think the lesson is that society as a whole f**ked up by obsessing about the "dangers of modern medicine", and forgetting about the dangers of infectious diseases and the dangers of the media.
Neuroskeptic, this is an intriguing hypothesis you raise, and The Science Police are inclined to agree: perhaps this was all done to a trick of framing.
ReplyDeleteRather than approaching the 1998 result by trying to estimate the number of people who would be harmed as a result of falsely calling into question the safety of the MMR vaccine -- the frame became, "how much damage will be caused if we falsely put our trust in this vaccine?"
For more on framing: http://en.wikipedia.org/wiki/Framing_(economics).
Thanks for your comment, Neuroskeptic. This was an angle we hadn't thought of before.
Who had the up of f*ckery? I thought this was going to be a difficult question to answer, but no, not really.
ReplyDeleteIt is my contention, and I have written about this randomly but at length, that the key problem is a background low level of good science and critical thinking education. I don't mean that everyone needs to be able to come out of school knowing how to analyse a full research paper, what the constituents of a vaccine are and why the LHC will never form a black hole.
I mean that people need to come out of school knowing what science is, how it is generally carried out, what "theory" means, what "balance of probability" means and how to look at a claim, raise an eyebrow and say "Interesting, but it smells fishy. Let me investigate..."
This may result in less chance of a dodgy paper being approved, researched, written and published by the scientific community or, failing that, less chance of a cautious or uncritical dissemination in the mass media or, failing that, less chance of a scare arising among the public.
Optimistic? Almost certainly. However, although there are numerous gatekeepers between a researcher and the public, the public are way too gullible, inbred psychological factors notwithstanding.
I come from the perspective of an Honours science education, followed by years working with the public, followed by being a researcher and bullshit filter on behalf of the public, followed by studies in journalism in the hopes of being a science journalist, so this may give me a different view of things compared to those who work within science and see the professional/public meeting-point from the other side.
It almost certainly leaves me unqualified to comment on Should Wakefield have been published at all?
However, from being a public educator I can tell you that a simple Baloney Detector Kit would have made most of my clients considerably better off, even before they came to me.
Additional comment:
ReplyDeleteI'd be intrigued to know what you think of my cobbled-together baloney detection kit:
http://sumdubito.blogspot.com/2008/05/science-101-how-we-know-that-what-we.html
Greetings Dubito
ReplyDeleteThe Science Police are in full agreement that -- along with instilling a sense of wonder in the natural world -- strengthening fishy smell detection would be a worthwhile emphasis of any basic elementary school education. And a CORNERSTONE of training in journalism. We are happy if our humble thoughts have reached someone who has been / will be involved in communication with the larger non-professional science world (which we can only dimly sense from afar).
"Extraordinary claims demand extraordinary proof." -- Dubito, is this idea based in a quote from somewhere? Or is it just the case that great minds thing alike? (c.f., "sensational claims need to be supported..." above)
Walking the beat,
The Science Police
TSP: That's right - although I'm not a big fan of the terminology of framing myself. But that's the idea. Almost everyone saw the issue as being "Does MMR cause autism?". To which the answer is "Almost certainly not", but you can never rule out some small risk.
ReplyDeleteWhat they should have asked is "Do the risks of MMR outweigh the benefits?", to which the answer is much more clear.
In a similar vein, people should not ask "Can antidepressants cause suicide?" but "Do antidepressants cause more suicides than they prevent?" - and so on.
In general, people tend to focus on the vaccines, or the drugs, and want to know whether they are "goodies" or "baddies", rather than focusing on the health outcomes and the best way to achieve them.
See also thalidomide, which despite being dangerous in pregnancy is very useful for a range of conditions. so long as you're not pregnant, it's a fine drug. Yet many people are surprised and even unsettled by that statement. They see thalidomide as a "bad drug" - when in fact like everything else, it's bad if you misuse it and useful if you use it right.
Neuroskeptic: Unfortunately, the mass public really isn't very good at dealing with shades of grey. That takes background knowledge, a fair bit of thought, and the leisure and energy to break out of the "Will it kill me?" survival-focused mindset that we all slip into when confronted with flash-card bits of information, like an orange stripe glimpsed through the trees.
ReplyDeleteWhich is a nuisance, and makes the job of science communicators an extremely cautious one.
TSP: "Extraordinary claims..." is a fairly common phrase, I didn't make it up :)