Why Replicate

An article on the pitfalls of ‘peer review’ was linked to here thanks to Adam Gallon.

The scientific fraudster who dazzled the world of physics

The article is quite interesting. There were several pertinent points which have AGW corrilaries.

Schön’s fraud was the largest ever exposed in physics; he ended up without a job, and was forced to leave America in disgrace. But the ease with which his fraudulent findings and grotesque errors were accepted by his peers raises troubling questions about the way in which scientists assess each other’s work, and whether there might be other such cases out there.

The ease of which the papers were accepted by the club is understandable. Once you have a reputation it’s easy to be accepted or rejected by your peers. McIntyre’s latest attempt at publication of the same math Santer used on the same datasets is a good example of the opposite effect. The name and opposite conclusion preventing publication of the exact same calculations.

On one occasion, after a trip to Constance, Schön showed colleagues in a neighbouring lab at Bell an astonishing graph. It appeared to show the output of an oscillating circuit. He claimed he had built the circuit in Constance using crystals from Bell Labs. The result were fantastic – but not impossible – and admiring colleagues helped Schön prepare a manuscript reporting the new claims.

They sent that manuscript, with more than a dozen others over the next year and a half, to the journal Science. Along with its British competitor Nature, Science was – and is – the most prestigious place to publish research. This was not only because of its circulation (about 130,000, compared with 3,700 for a more niche publication such as Physical Review Letters), but also because, like Nature, it has a well-organised media operation that can catapult the editor’s favourite papers into the pages of national newspapers.

This next paragraph is very important – my bold.

By the middle of 2001, more than a dozen laboratories around the world were trying to replicate Schön’s work on organic crystals, motivated by the prospect of building on the findings in Science and Nature. Yet without access to the details of his methods, which were strangely absent from the published reports, no one was successful

I don’t know how strange it is. In Climatology it’s become standard. After all the consensus doesn’t want to be questioned. Santer won’t release his data, Steig and Comiso have refused to release their code and methods, hell even the NOAA doesn’t release their data sources.

One of the most cherished beliefs of scientists is that their world is “self-correcting” – that bad or fraudulent results will be shown up by other experiments and that the truth will emerge. Yet this system relies, far more than is generally realised, on trust: it was natural for his peers to question the way Schön interpreted his data, but taboo to question his integrity.

Self correction only comes from replication. Without the data required, replication is a nearly impossible task. In the case of the physics world a lot of money was riding on replication, after all a molecule sized transistor can make you a mountain sized pile of money. In Climatology, there is no positive product, no new invention which helps people instead there is only a negative one – taxation and regulation. Therefore there are no labs attempting to disprove, only government funded work to increase the knowledge and funding only goes in one direction.

This is the mechanism which has created a consensus, those who disprove claims are not welcome to the discussion, they are not wanted no matter how reasonable their argument. Consider Steve McIntyre’s latest attempt which uses the same math and techniques as Santer yet is meeting resistance to publication for no reason other than it disproves the accuracy of computer models using the same math and data Santer used to prove the accuracy. The only change is to update the data series, a series Santer et al had to chop off in order to make its conclusions.

The question scientists should be asking is why was the data chopped in the first place and then if the extended data reverses the conclusion was there motive involved. Santer is promoting some highly political ideals in the video linked at CA HERE. In the consensus of the government funded environment, this is not openly questioned.

Schön was, in effect, doing science backwards: working out what his conclusions should be, and then using his computer to produce the appropriate graphs. The samples that littered his workspace were, effectively, props. The data he produced were not only faked, but recycled from previous fakeries (indeed, it was this duplication of favoured graphs that would prove his Achilles’ heel).

This is exactly what Mann does. He knows the conclusion and his papers (the two I’ve replicated) work very hard to reach those conclusions using blatantly flawed math. I fully believe that Mann did this with intent in his 08 paper and am highly suspicious (quietly) of the Antarctic temperature paper. The strength of my words are not lightly considered.

But Schön’s case also suggests a recipe for judging any science headline: any time a major breakthrough is reported without the researcher in question showing details of how they carried out the experiment, it’s time to start asking questions. If something seems too good to be true, it very possibly is.

This last paragraph is a weak end to an otherwise excellent article. The questions have been asked for centuries, it has been time for a while to ask the questions of climate science. So far I’ve read dozens of papers and worked on calculations for several. Of the three which caught my attention in particular two had massive flaws in their calculations leading to obviously erroneous conclusions ( the 08 Hockey Stick and the Antarctic). The third was a paper by John Christy on RSS vs UAH and I reached his same conclusion using GISS data to correct the satellite data before I read the paper.

The realization I came to some time ago, was that CA provides a huge service to the world community in exposing the flaws of climate science. SteveM can’t do it all by himself and it’s important for people to realize the magnitude and type of flaws are critical to the base assumptions of anthropogenic global warming. This has been a substantial part of my own drive to dig into some of these papers. It takes hours of time and a lot of study but replication can and must be done.