the Air Vent

Because the world needs another opinion

A Different Kind of Hide the Decline

Posted by Jeff Id on December 6, 2009

This is a serious one which has not been covered yet.

In the past year, there has been substantial discussion on CA, WUWT, tAV and other blogs about the filter types used to present temperature data. We evil deniers found that “scientists” have been stiffening the ends of their filters to make sure that the uptrends in graphs look as scary as possible. I personally have been censored from advocate global warming blogs for even suggesting that the practice was be going on. Now we have PROOF that it’s been colluded for.

I’m not sure if readers get the fact that scientists were not speaking openly by email but have been careful not to reveal their bias (especially in the more recent emails). Just not as careful as if talking to reporters. Like other smart scientists, Michael Mann was careful throughout the emials not to reveal his position on these things. After all, if guy’s like Mann knowingly create artificial hockeysticks for a living, you don’t expect them to write it down.

He went a bit too far in this email though.

Look at how Michael Mann writes this;

Michael E. Mann wrote:

Dear Colleagues,
Several you have inquired about the below claims by the notorious “Idso brothers” which
relates to the paper by Mann and Jones that appeared in GRL a couple weeks ago.
Of course, its the usual disinformation we’ve come to expect from these folks, but a few
details on why:
1) The supposed “Co2 fertilization” argument is a ruse. The only evidence that such an
effect might actually play some role in tree-growth trends has been found in high
elevation sites in western North America (consult Malcolm Hughes for more details). As
in Mann et al ’99 (GRL), any such effect, to the extent it might exist, has been removed
from the relevant series used in the latest (Mann and Jones) paper through the removal
of anomalous differences between low-elevation and high-elevation western North American
temperature trends during the post 1800 period, prior to use of the data in climate
2) We haven’t in the past extended the proxy reconstruction beyond 1980 because many of
the proxy data drop out. However, the repeated claim by the contrarians that post-1980
proxy data don’t show the warming evident in the instrumental record has finally
prompted me to go ahead and perform an additional analysis in which the
proxy-reconstruction is extended forward as recently as at all possible (to 1995, for
which 3 out of 8 of the NH records are available, and 1 of the 5 SH records are
available). The SH and GLB reconstructions are thus obviously tenuous at best, but they
do address, to the extent at all possible, the issue as to whether or not the proxy
reconstructions show the post-1980 warming–and they do.
See the attached plot which compares the NH (blue), SH (green), and GLB (red) series
through 1995. The late 20th century is the nominal maximum for all 3 series *without any
consideration of the information in the instrumental mean series*. This thus refutes
the 2nd criticism cited by the Idso brothers.
One note about the 40 year smoothing. As in the trends in the instrumental series shown
by Mann and Jones, a boundary constraint on the 40-year smooth has been used that
minimizes the 2nd derivative at the boundary–this trends to preserve the trend near the
end of the series and has been argued as the optimal constraint in the present of
nonstationary behavior near the end of a time series (Park, 1992; Ghil et al, 2002). I
favor the use of this constraint in the smoothing of records that exhibit a significant
trend as one approaches the end of the available data. This might be worth talking about
in the next IPCC when the subject of adopting uniform standards for smoothing data, etc.
are discussed…

In retrospect, Phil and I should have included this analysis in the GRL article, but its
always hard to know what specifics the contrarians are going to target in their attacks.
This analysis however, will be included in a review paper by Jones and Mann on “climate
in past millennia” that is presently being finalized for “Reviews of Geophysics”.
I hope that helps clarify any questions any of you might have had. Please feel free to
pass this information along to anyone who might benefit from it.
Now, back to fighting the “Shaviv and Veizer” propaganda along w/ Ben Santer and David
Parker out in Italy…

The bold is the critical part. Stiffening the 2nd derivative is math code for ‘hide the decline’ . He’s discussing it and trying to rationalize the moronic practice through science. This is one of the most biased statements in the entire email set in my opinion and is flat, slap you in the face evidence of the bias Mann continually imparts to science. This is the another form of Hide the decline. The original had to do with Phil Jones deliberately hiding data which didn’t agree using hidden and undisclosed mathematical methods.  This method, is similarly ‘black box’ and is based on the assumption that the upslope of the previous twenty years will match the next.

But Mann is smart. Consider though if it was such a minor point, why would he resort to discussing filters in the context of other papers? Why would he try to standardize the approach? Would the other scientists change their minds based on the goofy explanation? The explanation is for you the accidental reader and only partially for them. In plain English he’s saying, look we have an excuse we can fall back on — what do you think?

This is ugly ugly science, written with the intent to hide the decline and make things ‘ worse than we thought’. There are endless examples of this in these emails. Since PNAS is investigating Mann (yeah right) perhaps they ought to be looking for a pattern in behavior.

46 Responses to “A Different Kind of Hide the Decline”

  1. stevemcintyre said

    Jeff, while there is much to criticize in this and other emails, the Hide the Decline issue is a little different IMO. It’s an interesting bit of information about MBH99 tho.

  2. Jeff Id said

    I believe I’m aware of the technical details of the MXD data – thanks to you. People have interpreted it as hiding the decline in temperature data. This is an example of that.

  3. Jean S said

    Jeff, I don’t think that refers to “hiding the decline” but it is about Mann’s “minimum roughness criterion”, see e.g.

  4. carl said

    first derivative is velocity
    second derivative is acceleration

    just saying for clarity

    by reducing or dampening the acceleration you flatten the curve

  5. chopbox said

    Hi Jeff,
    I’m not sure I get your point here.
    It seems to me minimizing the second derivative is a pretty reasonable way to graft some (instrumental) data onto some other (proxy) data. It’s not like he’s suggesting a method that hides the first derivative (negative slope); he’s just trying to make the graft smooth (minimizing the second derivative). (This is not saying it’s ok to graft the one onto the other, but rather that once you’ve decided to do that, making the graft smooth seems pretty reasonable to me.)
    I don’t think he’s providing a secret excuse here. After all, he asks people to “feel free to pass this information along to anyone … ” Not very secretive, that.

    Also, I haven’t said this in a while, so let me say again: thanks for your blog. (And please thank your wife from me for granting you the time to do it.)

  6. mack520 said

    You don’t have to forge ahead overcoming all obstacles at the moment. Your loyal readership will go through this and keep you informed- this is why you have administrative assistants. For the underlings to profit the big shots have to stay on top of their game. Yer a big shot. Quit working so hard, follow the GS example to the extent you are able.

  7. Jeff Id said

    #3 When you minimize the second derrivitive as the hadcrut series reaches its endpoint do you not see a stiffened upslope i.e. every hockeystick temperature overlay ever presented?

    I don’t plan to win every discussion on this but as JeanS points out in his linked post, this is a reflection of temperature data both on the horizontal and vertical axis. What you get in a 40 year tend is a mirror of the previous 40 year trend – continued upslope. What am I missing here?

  8. timetochooseagain said

    4-Straighten would be more accurate. Plus, the second derivative is rate of change of rate of change, which can be of either sign-acceleration or deceleration. if you smooth out deceleration the result isn’t a flatter curve it’s a more dramatic one that keeps going and going and going.

  9. carl said

    @ jeff Id

    if you basically reduce acceleration of the curve to zero or close to it yes the curve will continue on in whatever direction it was going

    All that is going on is basically some fudging to attempt to smooth out the curve(s) to get rid of all the noise and there is lots and lots of noise versus the actual tiny little signal they are seeking which statistically is the heart of the problem. The data sets do not match and are full of noise. So they do lots of smoothing and averaging and from I have read of the data/programs/emails some cherry picking too. Problem here is the fudge factor and smoothing is huge compared to the actual values they are seeking and the time line is relatively short. I am not a statistician but it appears to me that the fudge factors add more error than the actual signal being sought.
    The old causation vs correlation issue is also at play here which is often ignored or omitted from AGW discussion…….if the medical/pharma researchers played these same kinds of games people would get killed by bad medicines all the time and when they have that indeed has happened. All other variables must be isolated and tested and when it comes to climate the science simply isn’t there nor can some of it even be tested sufficiently. Simply tracking temp and CO2 isn’t close to enough. Small temp changes ie 0.1C over many years is damn tiny.

  10. Jeff Id said

    #9 attempt to smooth out the curve(s) to get rid of all the noise

    Mann is talking about the temperature curves. The noise IS the decline and this IS one of the ‘cherry picks’ you describe.

    My opinion is that the noise issue has got a few people confused. The point of this is to straighten the endpoint of any curve which fits the bill. We’ve seen it regularly done across climate science and in the context of what we’ve just learned, it’s difficult to imagine why we would reach to say — he’s trying to hide noise that’s all. The “noise” is the “decline” and a 40 year smooth is a big thing. And this is NOT just a case of simple smoothing but rather a case of projection of a trend according to pre-determined conclusions.

  11. stan said


    I assume you are talking about examples like the Rahmstorf 2007 debacle. Would this be the most extreme example of this type smoothing?

  12. Jeff Id said

    #11, that’s very similar to how I see this. The fact that Rahmstorf used ungodly fancy math to make a triangular filter was one oft he most entertaining things in climate science this past year.

    – your little mind couldn’t possibly understand the sophistication of my eleven dimensional manifold filter. haha.

  13. Duke C. said

    Just thought some context would be useful…

    What Mann wrote was in response to an opinion emailed by Timo Hämeranta on Aug. 20, 2003. (ref. 1061625894.txt)

    Looks like Mann and Wigley spent the next day getting their stories straight. And on Aug. 23, Mann broadcast the discussion to a wide group of individuals. (Look at the Cc: field. 16 names. Reads like a who’s who of climate science.)

    If anyone thereafter had questions about Timo’s findings, they would all be on the same page.

    After that, “smoothing the 2nd derivative” was accepted science, Because Mann said so.

  14. Douglas Hoyt said

    Isn’t this the method that Soon in GRL criticized? See
    where it says:

    “I did not really come up with a reasonable recommendation except to say that I disagree with the approach of defining a trend,” Soon said. “To get even close to what they are trying to say, they have to pad in a lot of data, and that procedure ought not to be recommended. To get roughly what they get, you have to define a point in 2000 and fill in the data up to 2020. That is a totally arbitrary process … The quantitative information is highly uncertain, because it depends on the future,” Soon said. “I stop the red line at 1980. I don’t make any extrapolation. This is the most conservative.”

  15. stevemcintyre said

    Jeff, again this is an issue, but it is not the Hide the Decline thing. Jean S confirms. If you applied this method to the real Briffa MXD, it would go down from 1960-1990. I wish that you’d rename and amend this post so that people don’t get wrongfooted. Cheers, Steve

  16. Jeff Id said

    Ok, as Steve says this is NOT the hide the decline of the Phil Jones email. It is however a method being proposed to hide the decline in temperature records.

    #15, I doubt very much there is any way Mann would ever apply this method to extend a downslope in a proxy or temp.

  17. Paul Linsay said

    The real solution is to reject smoothing entirely. Unless one has a physical theory that predicts what the data points should be, any smoothing curve is merely going to “guide the eye”. There is no theory here. If it’s absolutely,totally, positively necessary to to have a curve, connect the dots with straight lines.

  18. Amber said

    I was going to say something similar to what has already been said along the lines of it being a storm in a teacup but Jeff has already calmed down in #16 so I’ll leave it at that, except to say that we must always be careful not to overread anything. Genuine science is methodical not impetuous.

  19. Ian said

    The paper that Mann is referring to, is that the one that Ed Cook described in the following terms: [1062592331.txt, 3 Sept 03]

    “I am afraid the Mike and Phil are too personally invested in things now (i.e. the 2003 GRL paper that is probably the worst paper Phil has ever been involved in – Bradley hates it as well)”

    Just wondering…as it would suggest that most of the Team were offside as well, perhaps in part for the concerns identified by Jeff, and only held in check by (fear?) of Mann.

  20. Jeff Id said

    #18, you again mistake the point of others. This was a suggestion of intentional hiding of the decline in temperature data. It was not as blatantly disingenuous as Phil Jones ‘hide the decline’ but you should spend a few hours grocking the link of Jean S. The way I interpret it is that Steve doesn’t want me to dilute the main issue the media was able to figure out.

    In the meantime this is being proposed as a method to stiffen the ends of graphs of surface temperature data. This would only be applied to upslopes. Do not presume to teach ‘genuine science’.

  21. Morgan said

    For those who want to dive in, here is a Mann paper…

    …that references both the Park and Ghil papers. Park’s is a chapter in Statistics in the Environmental and Earth Sciences. The Ghil paper is here:

  22. wattsupwiththat said

    I call BS on Michael Mann for this statement:

    The supposed “Co2 fertilization” argument is a ruse.

    Your statement is BS Manny.

    Here’s a peer reviewed paper:

    Greenhouse gas carbon dioxide ramps up aspen growth

    The rising level of atmospheric carbon dioxide may be fueling more than climate change. It could also be making some trees grow like crazy.

    That is the finding of a new study of natural stands of quaking aspen, one of North America’s most important and widespread deciduous trees. The study, by scientists from the University of Wisconsin-Madison and the University of Minnesota at Morris (UMM) and published today (Dec. 4) in the journal Global Change Biology, shows that elevated levels of atmospheric carbon dioxide during the past 50 years have boosted aspen growth rates by an astonishing 50 percent.

    Mann: “some role”?


  23. Ed Snack said

    Yes, the CO2 statements are worth noting. If one looks at the supposed CO2 adjustment in MBH99, the correlation between CO2 levels and CO2 adjustment must be negative. The Idso’s might not be in Mann’s cheering group, but they know a damn sight more than Mann does about CO2 fertilization, so his characterisation of them is praise indeed.

  24. Heide de Klein said

    So, if the “trend” is upwards, they exaggerate it. If the trend is downwards, they truncate it (or invert it).

  25. W. W. Wygart said

    What was the date/filename on this email?

    Comments have been made elsewhere in the past by persons much smarter than me, that based on the lack of citations of works in statistics outside of climate research in their papers, the statistical methodologies of certain people working within the AGW field seem to have become detached from reference to the larger field of statistical research, and has there fore may have become [my term] very self-referential. This might be a case of this of this phenomenon in action, where a cultural pattern has established itself within the field where a ‘convenient method'[fishy formula] is invented by one worker and then standardized as statistical practice without any reference to outside expertise in the statistical sciences to ‘keep them inside the guardrails’ [or the error bars] so to speak. Here we also seem to see one person interested in guiding that process.


  26. IanH said

    I agree with Jeff, this 40year smooth is not to lose the wiggles, it’s done to hide a decline at the end. I also can see that Steve & Jean do not wish to conflate this particular process with Briffa terminating a series to hide a decline. I think Jeff’s title (didn’t catch if it’s changed) is fine to introduce another form of trick, as it’s clear Mann is steering the team to adopt a 40year filter with a stiffened trend as they know it’s going to get worse as time passes and this will help future publications.

  27. Amber said

    #20 Jeff, you again read things that aren’t there to be read, a pattern of yours becoming increasingly obvious. Now let me say something to you bluntly so there can be no confusion about what I’m saying:

    *do not infer motive from the written word*

    I have said more than once that I agree with you and others that there are things in these stolen files that do not look good and that I fully support the investigation of the issues raised. However, it is absolutely wrong of you to say that someone did something “intentionally” — how the hell would you know? Have you any concept at all of due process or rules of evidence? Have you acknowledged to yourself the obvious fact that you’re missing substantial portions of evidence?

    Keep up the good work with the actual science, thank your very understanding missus, drop the role of chief prosecutor and stop being such a patronising bastard. No offence intended but some direct language is required here because your patronising is wearing a bit thin.

  28. Jeff Id said

    #27 Let me be equally blunt. You have not one clue what you’re talking about.

  29. Dr. Dweeb said


    Not sure where to make this comment.

    In my researches I discovered this paper.

    While it is clear this gentlemen quotes a lot of his own research as well as others, it is equally clear that he has something to say on a matter with which he is intimately involved (I am guessing he has had frostbite – lol). His thesis is that the ice core data has been falsly manipulated to show lower CO2 concentrations than were actually present in the deep and less deep past.

    This is the other half of the AGW hypothesis, apparently shot to pieces by inappropriate data manipulation – published in the peer review literature no less!

    Anyway, this is the first I have heard of serious problems with Ice core data, and the 83 year shift to splice proxy to instrumental data seems suspiciously familiar to me.

    Just something else to inject into the blogshere.


    ps: The “climate” here in Copenhagen is palpably nauseous and if I read one more article postulating as scientific fact that the polar bears are going extinct due to to AGW, I will add to the local environment, my personal technicolour yawn on the steps of Bella Center. The “weather” on the other hand is wet and dismal, as one would expect in this dismal country at this time of year.

    pps: “Climatgate” has received scant coverage here in “Hopenhagen”, and my detailed review of MSM shows that London correspondents have been paraphrasing various UK paper’s coverage and performing zero investigative activity. This is what I expect of the “free press” in one of the world’s most left wing “democracies”.

  30. Amber said

    #28 Hey Jeff, nice ad hominem attack, mate!
    Haven’t I seen you criticising Mann and friends for doing the same thing? Nice double standards.

    You and I agree on some things and disagree on others. Show some dignity.

  31. Jeff Id said

    I cannot show you the dignity you request because you are making false claims with no understanding of the words written. Read more carefully.

    For instance – look up the meaning of ad hominem.

  32. Mark T said

    Jeff is correct, Amber, you apparently do not know what an ad hominem is.

    As for inferring motive from the emails: Jeff doesn’t need to infer anything, they (the authors) clearly state their motive(s) in many instances along with the suspect methodologies.


  33. Kondealer said

    Jeff is right about CO2 fertilisation. The Idso’s are plant physiologists, they know what they are talking about. I’m a plant physiologist- I agree with the Idso’s 100% on this.
    Where Mann MAY be correct is in thinking that high elevation trees may differ in their response from low elevation trees. It is to do with the partial pressure of CO2 at different altitudes. High elevations have a lower PP CO2 than low elevations, so any increase in CO2 concentration should have a more marked on high altitude trees. But that is not what Mann is saying. and what is more it is the high elevation trees that are supposed to be “treemometers”.

    One question that has been bugging me. The team look for (the few) trees that are apparently “temperature sensitive” and select these (not a correct statistical prcedure). However their reconstructions go back over 1000 years. Now some trees will live that long, but not many.
    Two points;
    1) given the “divergence problem” there is no sound scientific basis to assume that these trees are “treemometers” all their lives.
    2) I assume that they also look for tree ring overlap matches to “splice” records for living and dead trees to extent the proxies back in time. Given that there is no physical evidence and they can’t look for a correlation with past temperatures (because that is precisely what they are trying to reconstruct) how can they assume that these dead trees were also “treemometers”?

  34. Layman Lurker said

    #33 Kondealer

    Would you consider doing a guest post on this topic? It is something which I have done a little reading on but would appreciate an article or essay on this matter from someone like you.

  35. Jeff Id said

    #34, I agree that it would be interesting. I’ve got no more expertise than the climatoknowledgests on tree growth.

  36. vsaluki said

    “The only evidence that such an
    effect might actually play some role in tree-growth trends has been found in high
    elevation sites in western North America (consult Malcolm Hughes for more details).”

    I see that Hughes is showing a CO2 fertilisation effect at high altitude. I don’t see anything in Hughes showing that this doesn’t happen at lower altitudes. Even if the effect is lessened at lower altitudes, it’s still there. Mann, in his usual dishonest way, wants to pretend that we can simply ignore the fact for low altitude trees.

    Concerning the effects of smoothing, if you can use a long up period to effect a shorter down period, then of course you are hiding the down period. I guess I don’t understand where Jeff is being critizised for pointing this out. It’s a different kind of hiding than splicing in instrument data – but it’s still hiding.

  37. vsaluki said

    “However, it is absolutely wrong of you to say that someone did something “intentionally” — how the hell would you know?”

    Well, we know that Mann intentionally ignored McIntyre’s corrections to his analysis methods. We know that he intentionally ignored the fact that he was using the Tiljander data upside down after it was pointed out to him. We know that he claimed to be making proxy data clearer to the public when he spliced instrument data on to proxy data. How does taking proxy data that went down and hiding it’s direction make anything clearer to the public? We know that Mann is involved in the RC propaganda campaign, where they cover up the truth by inventing rationalizations and then censoring any comments that would expose their actions. Here are Mann’s own words:

    “Anyway, I wanted you guys to know that you’re free to use RC. Rein any way that you think would be helpful. Gavin and I are going to be careful about what comments we screen through, and we’ll be careful to answer any questions that come up to any extent we can.”


    “On the other hand, you might want to visit the thread and post replies yourself. We can hold comments up in the queue and contact you about whether you think they should be screened through or not, and if so, any comments you’d like us to include.”

    Jeff and I have both had the experience of having our comments censored when it was obvious that RC had no possible response. This has happened hundreds of times. I have screen shots of the deleted comments. And during the Briffa Yamal debates, I had the second experience that Mann points out. My comments were held up for hours while others were streaming through. Obviously, RC was looking for Briffa or someone to answer them. And when they couldn’t find an answer, they finally deleted my comments. This makes it utterly clear that Mann is supporting a propaganda campaign on RC. Victory by censorship! And let’s not forget Mann’s comments about freezing out journals that published things that he didn’t like:

    “I think we have to stop considering ‘Climate Research’ as a legitimate peer-reviewed journal. Perhaps we should encourage our colleagues in the climate research community to no longer submit to, or cite papers in, this journal.”

    So, I don’t know where your reservations are, Amber. But personally, I have no doubt at all that Mann is intentionally seeking to perpetuate a fraud.

  38. DaveJR said

    Kondealer said: “However their reconstructions go back over 1000 years. Now some trees will live that long, but not many.”

    In the Briffa “Yamal” reconstruction, none of the trees stretched back that far! IIRC, 300 years was about the maximum!

  39. MikeH said

    Slightly off topic, I have followed the AGW debate for some years now, but not as closely as these last few weeks. And today, I spent some hours reading comments and threads on RC and other pro-AGW sites and also spent similar time reading sites that are skeptical. As a result, I have to congratulate the skeptics and condemn the AGW supporters. It is clear to me and I hope to the public at large, that the AGW supporters are acting more like religious zealots than scientists. On balance, I perceived their sites as containing a great deal more propaganda and vitriol than rational analysis and the reverse seems to be true of the skeptic sites. And I hope that the skeptics maintain that position because the only way for the science of global warming to return to being actual science is if as many scientists as possible return to it (and those that won’t are purged). Jeff – bravo!

  40. Bernie said

    If you really want a contrast in tones between the skeptics and the CAGWers, take a look at Andy Revkin’s recent blog
    There are nearly 600 comments – a non-random sample for sure of the two camps – and the overall pattern mirrors what you see at RC. The latter is not too surprising since many of the commemts come from the same individuals who comment (or try to comment) at RC.

  41. Kondealer said

    Jeff & Layman Lurker, I appreciate your faith in my abilities here.
    Yes I could put something together, but not for the next 10 days or so.
    It is the last week of my teaching semester and I’m pretty snowed under (excuse any unintentional pun here) with marking student assignments.

    If that timescale is OK then I’ll do it.

  42. Amber said

    #31 Jeff, when you try to win an argument by diminishing the person making the argument (which is what you have done) instead of debating the substance of the argument, that is ad hominem. Perhaps you should doublecheck your own dictionary.

    #37 Vsaluki, thank you for taking the mature approach, it’s a model example for us all. I’m not defending Mann (see for example #27), I’m defending due process, which is too often IMHO lacking in posts regarding the stolen files. If anyone has done anything wrong, let them be caught and punished. But not in a kangaroo court.

    But in responding more directly to your words, I might just add that you have had first-hand experiences and noone can take that away from you, and comments in the CRU files *support* those experiences, in your view (I assume). That is not the same thing as someone using the files as prima facie evidence.

  43. Jeff Id said

    #42 I didn’t try to win anything. If I tried to win it would take a reasoning with so many off the wall points – like the meaning of ad hom. Lucia does that quite well and patiently, however I have no interest in that.

    It is up to you to learn. I’ve spent the last 2 hours reading technical papers and calculating sea ice statistics. What have you done?

  44. Layman Lurker said

    #41 Kondealer

    Excellent. This will be a great topic for a post. Many thanks.

  45. Jeff Id said

    I’m a grumpy bastard. hehe.

    #41, It would be awesome any time you get a chance.

  46. Kondealer said

    #45, Jeff, just one question how can I post an article with graphs and stuff?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: