|
|
|
|
|
|
|
|
|||||
| Home Help Feedback Subscriptions Archive Search Table of Contents | |||||
First published online December 11, 2006
doi: 10.1242/10.1242/jcs.03245
Sticky Wicket |
|
Okay, kids, you can come back in; Uncle Mole is in a better mood now. If you're just joining us, you should know that I was in a very foul temper, because I'd only just found out that a prominent scientist, whose work I'd valued, had been exposed as a fake. His lovely work wasn't simply flawed, it was made up. My whole faith in science has been shaken, and I want to fix it. Instead, I've watched TV.
I've just seen one of my favorite episodes of the old Twilight Zone. In `It's a Good Life' we meet a mind-reading, omnipotent monster who is terrorizing a small town. The monster is a six-year-old body played by Billy Muni, who would later gain quasi-immortality as the youngest member of the Robinson family on Lost in Space (as in, `Danger, Will Robinson!'). But in this Twilight Zone story by Jerome Bixby, he is a small boy who can do anything, and when he is displeased, he can transform townspeople into horrors, or make them disappear altogether, by `sending them to the cornfield'. So I imagined doing this to our scientific fraud, and I felt better. Maybe this wasn't the point of the story, but it made me feel better.
The problem of scientific fraud isn't new, but it seems as though our efforts to eradicate it have not worked. We get tougher, but the fakers just get better at faking. Perhaps we need to further tighten security - remove your shoes and laptops prior to submission...
There are two views of this problem, and the one we take will dictate what we should do. The `tip of the iceberg' position says that the fraud that has been exposed represents only the tiniest bit of a problem that is rotting science from the inside. Some have advocated a zero-tolerance policy, and have taken it on themselves to act, vigilante fashion, to publicize any discrepancies they find in publications, demanding satisfaction. The standard operating procedure here seems to be to contact the journal and the community, via emails for example, intimating that every questionable figure is evidence of fakery - mistakes cannot be tolerated. I know of one investigator who is being hounded to explain two identical images in a paper, which he asserts is a post-proof printing error (and was immediately corrected) but he can't prove it. But the vigilantes contend that zero tolerance demands that everyone subject themselves to a `trust no-one' process in the hope that we'll weed out the worst offenders.
Don't get me wrong, I do think that there is a lot of fudging in the literature. My old Oxford English Dictionary defines fudge in this context as "to fit together or adjust in a clumsy, makeshift or dishonest manner". (The most romantic etiology traces this to one Captain Fudge, c1664, a.k.a. Lying Fudge, who was probably a real person, although he may have made himself up.) In science, fudging data can be elimination of compelling results that don't fit the hypothesis (which may be for perfectly valid reasons or not) or adjusting the results, say, when molecular weight markers seem off. It occurs because researchers are under tremendous pressure to publish on a timetable - the need to publish any work that has used up time and resources, however questionable the conclusions. Journals promote this problem by demanding additional results that are conditions for publication, usually on even shorter timetables. Fudging seems inevitable. I am not forgiving it; I'm only saying why I think it happens. But when it goes too far it becomes fakery, and it is unforgivable. The tip of the iceberg view is that much of what we see is not simply fudged; it is faked.
The alternative view, to which I subscribe, is that true fraud is exceedingly rare. Mistakes, misinter - pretations, and wishful thinking are more common - and problematic - but I think we can deal with them. But outright fabrication is rare enough to be news.
Can I prove this second view? No. But I can demonstrate that it is a useful and profitable position to take. And the demonstration points to a route to the solution, not only for fraud, but also for errors and other problems.
Unless you live in a cave and, for that matter, a cave without an internet connection, you know about eBay, the massively successful online auction system. Anyone can buy or sell anything on eBay (including, apparently, fabulously expensive grilled-cheese sandwiches) and can do so with a remarkable level of confidence. It is based on a seemingly naive, but ultimately profound, precept: most people are honest. This is backed up by a readily accessible rating system, where buyers and sellers provide feedback on their transactions, thereby exposing problems if and when they arise. The system is largely transparent: those who lie are quickly flamed, and anyone who gives inordinate numbers of negative comments is discredited. It is freewheeling, but for the most part it is wildly successful.
Once, when science was conducted by an elite, feedback occurred in the literature and at meetings. This still happens, but in a manner that is difficult to assess unless one is in the center of the action (again, one of the elite). High-impact journals have no interest in publishing work that refutes other work, regardless of the rigor of the refutation, and the group psychology among researchers translates this into `high impact = true, low impact = less true'. Even when we think that everyone knows that a particular finding is flawed, one only has to take a stroll into a related but different venue, such as the department upstairs, to find that others who might be peripheral to the field can evince surprise at our suspicions. We need a feedback system that everyone can access.
I propose that we take a cue from eBay. Link a system to PubMed, for example, by which we can identify a paper and offer feedback ("we repeated this finding, but couldn't reproduce that one" or "this result may be an artifact for the following reasons"). It must be transparent - commentators are registered and their identities known, and we can similarly access their other reviews. Vigilantes who only find fault will find their comments of less value than those from reviewers who are balanced in their views. And, of course, the authors will be able to respond to criticism if it is especially important. We will have a way to evaluate the experiences of the community, far beyond a paper's `impact', which is more likely to reflect the extent to which a finding is easy to mention. I think we'll gain confidence in the literature, we'll expose fudges, and we'll find very little fraud.
For this to work, however, we need a fundamental change in the community of scientists. We need to realize that making mistakes is common and that honesty requires that errors be owned up to. We need to reward, not punish, those individuals who can say that they got it wrong. It happens all the time, and we pretend it doesn't, and, as a consequence, the fudging goes on. We can make it stop, but only if we take away the pressure not to admit to it.
But what of the real monsters who are out there? The ones who simply make it up? Such allegations are serious and must be dealt with by informed investigation, as we do now, and evidence of genuine misconduct must come from close to home (fellow researchers with intimate knowledge of the lab and methods). We can deal with this, and only open such investigations when the evidence is overwhelming. But why do they do it, this outright fakery that is so antithetical to the entire process of scientific inquiry? I think I know, because I've just been watching the Twilight Zone.
The monster in `It's a Good Life' can do anything, knows everything and, because he is only a child, has no goals but his own desires. He understands only that whatever he wants to happen, happens. When a bright, young, and very ambitious scientist begins his or her career, one of two things occur early on. They can chance on a set of ideas that happen to be correct, and their experiments flow effortlessly towards a happy conclusion. And if this is an important conclusion, rewards come quickly. Or, alternatively, they can be wrong, and they learn at this formative stage that no matter how wonderful an idea may be, and how much they need it to be true, it can still be wrong. This is an extremely important lesson that our first, lucky researcher may not learn, unless, of course, we teach them. The successful student guesses again, and again may be right - more rewards. By the time they come up against something that they cherish that turns out to be mistaken, they may have already become our monster-their idea, their need to be right, exceeds all other goals. It doesn't happen all at once - but someone who is always right begins to believe that they are special, and does not realize that luck is a major factor in all of this. So they make it right. They have made the leap to quasi-omnipotence. I once met a monster like this, and it was truly frightening.
'It's a Good Life' was remade, years later, as a vignette in The Twilight Zone Movie. In the rewritten work, the ending was changed: a teacher takes on the task of educating the monster/child, who is desperate for guidance. And this, of course, is what we need to do with our most gifted, lucky young scientists. We have to teach them that ideas are frequently wrong, and this is fundamental to science. And we have to stop stressing that being right brings rewards, while being wrong brings despair. Let's stop giving awards for best poster, best thesis, best student. Science is a reward. We don't need more monsters. Let them be wrong sometimes.
Otherwise, I'll want to send you to the cornfield.
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati 
Twitter What's this?
Related articles in JCS:
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
