the Air Vent

Because the world needs another opinion

Another Change, Neil Found this at Climate Audit

Posted by Jeff Id on November 22, 2009

You have got to see this. I’ve looked at the comments in several code modules but he found the best one so far. Remember Phil Jones now famous denial of knowledge as to what he meant about ‘hiding the decline’ in this post.

Busted- Phil Jones Doesn’t Recall Divergence

I’ve just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) amd [sic] from1961 for Keith’s to hide the decline.

Check out this quote from the code – cudo’s to Steve Neil for digging it out.

function mkp2correlation,indts,depts,remts,t,filter=filter,refperiod=refperiod,$
datathresh=datathresh
;
; THIS WORKS WITH REMTS BEING A 2D ARRAY (nseries,ntime) OF MULTIPLE TIMESERIES
; WHOSE INFLUENCE IS TO BE REMOVED. UNFORTUNATELY THE IDL5.4 p_correlate
; FAILS WITH >1 SERIES TO HOLD CONSTANT, SO I HAVE TO REMOVE THEIR INFLUENCE
; FROM BOTH INDTS AND DEPTS USING MULTIPLE LINEAR REGRESSION AND THEN USE THE
; USUAL correlate FUNCTION ON THE RESIDUALS.
;

pro maps12,yrstart,doinfill=doinfill
;
; Plots 24 yearly maps of calibrated (PCR-infilled or not) MXD reconstructions
; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually
; plot past 1960 because these will be artificially adjusted to look closer to
; the real temperatures.

;

;
; Plots (1 at a time) yearly maps of calibrated (PCR-infilled or not) MXD
; reconstructions
; of growing season temperatures. Uses “corrected” MXD – but shouldn’t usually
; plot past 1960 because these will be artificially adjusted to look closer to
; the real temperatures.

Hide the decline and artificially adjusted to LOOK closer to real temperatures. There ain’t a lot of middle ground left in my opinion. Spin away Real Climate.

Can we get rid of this crazy series now?? Please???

Steve’s link is here:

these will be artificially adjusted

How much clearer can they make it. — Say thanks to Steve and write your congressmen. I wonder how Gavin can explain that.


33 Responses to “Another Change, Neil Found this at Climate Audit”

  1. Jeff Id said

    I had to copy Calvin’s code from the CA thread. hahaha.

    Calvin Ball permalink

    void function fubar(void); {

    if dataset == hockeystick then plot(dataset); else fudge(dataset);

    return; }

  2. John F. Pittman said

    From: Michael Mann Date: 27/10/2009, 16:54
    Perhaps we’ll do a simple update to the Yamal post, e.g. linking Keith/s new page–Gavin t? As to the issues of robustness, particularly w.r.t. inclusion of the Yamal series, we actually emphasized that (including the Osborn and Briffa ’06 sensitivity test) in our original post! As we all know, this isn’t about truth at all, its about plausibly deniable accusations.

    Add this and Steve’s work above together, what middle ground they had is now quicksand.

    I still think that Yamal is going to be even more important. It was used like “crack” to avoid the “divergence” problem. Its problems were known from what I have seen. So much for that “plausibly deniable accusations” whatever that means.

  3. Craigo said

    Jeff – someone else is having fun with the code in the “HARRY_READ_ME.txt”:

    http://www.tickerforum.org/cgi-ticker/akcs-www?post=118625&page=13

    Ok, one last bit to finish that last one off:

    Quote:
    ..knowing how long it takes to debug this suite – the experiment
    endeth here. The option (like all the anomdtb options) is totally
    undocumented so we’ll never know what we lost.

    22. Right, time to stop pussyfooting around the niceties of Tim’s labyrinthine software
    suites – let’s have a go at producing CRU TS 3.0! since failing to do that will be the
    definitive failure of the entire project..

    I eagerly await more reading to find the results of that.

    2009-11-21 02:01:15

    I didn’t feel up to reading the entire saga so feel free to snip if O/T

  4. stevemcintyre said

    reader “Neal” spotted this, not me.

  5. Jeff Id said

    #4 Updated, fantastic though.

  6. Kenneth Fritsch said

    The scientist is going to concentrate on what is unsual about the proxy, i.e. the proxy does not respond correctly to temperature after 1960. She would not think of plastering something into that time period to deflect attention. She would want to talk about and let you know what she has found – and attempt to explain it.

    The advocate and all his defenders, on the other hand, will want to sell their message and when they are questioned will in bewilderment say ” well of course we would not use “bad” proxy data when we have “good” instrumental data and how dare you think we intended to deceive.

  7. cbullitt said

    Seen this yet?

    http://cbullitt.wordpress.com/2009/11/22/the-harry_read_me-file/

  8. Slayer said

    This is the very email that all the Left wing blogs are using to defend the crooks. They insist the quote was taken out of context and that we non-scientists don’t know what we’re talking about.

  9. Jeff C. said

    Not sure if anyone has discussed this yet, but if you unzip the file “mbh98-osborn.zip” in the documents directory, there are a crapload of “censored” directories.

    Once it is unzipped follow this path:
    \mbh98-osborn\TREE\ITRDB\NOAMER

    Five directories with censored in the name such as
    “BACKTO_1200-CENSORED”

    Haven’t dug into these yet but I’m assuming these are the recon runs without the bristlecones. There is also a file called “pca-censored.f” which appears to be the code for reducing the datasets to the PCs. Nothing obvious as to what might be different from the file “pca-noamer.f” file. Date on everything in the zip file is 5/22/2000.

    This might be all the exact same stuff Steve Mc found years ago in the censored folder of their FTP site, but it is pretty to see. : )

  10. Jeff C. said

    Yep, sure enough, BACKTO_1000 – hockey stick, BACKTO_1000-CENSORED – no hockey stick. Nice handy little reference file, too. The following trees are missing from the censored directory:

    az510
    ca528
    ca529
    ca530
    ca534
    ca535
    co511
    co522
    co523
    co524
    co525
    co535
    co545
    co547
    nv510
    nv511
    nv512
    nv513
    nv514
    nv516

    This is obviously the same stuff Steve found long ago, but just how in the world did the whistleblower get this? With all the grief this caused the team, I can’t believe they just left this lying around on a server years later using the same “censored” directory names.

  11. stevemcintyre said

    This looks like the same material that I analysed. I’ll crosscheck in more detail some time this week.

  12. Stan said

    How much will one have to dumb this down so an investigastive journailst can understand it and use it?

  13. wattsupwiththat said

    The source files with the comments that are the topic of this thread are in this folder of the FOI2009.zip file

    /documents/osborn-tree6/mann/oldprog

    in the files
    maps12.pro
    maps15.pro
    maps24.pro

    These first two files are dated 1/18/2000, and the map24 file on 11/10/1999 so it fits timeline-wise with Dr. Jones email where he mentions “Mike’s Nature trick” which is dated 11/16/1999

  14. Nick Stokes said

    This conspiracy theory makes no sense. If the data has been “artificially adjusted to look closer to the real temperatures” post 1960, then why in the same sentence do they say not to plot past 1960? What’s the point of that cleverness if they tell you not to use it?

    As far as timeline goes (and file naming), this fits better with Briffa et al 2001, which was actually about trees, MXD etc. And there they emphasise that indeed they don’t use data past 1960:
    “The period after 1960 was not used to avoid bias in the regression coefficients that could be generated by an anomalous decline in tree density measurements over recent decades that is not forced by temperature”

    So what is this “artificial adjustment”? Research papers are peer-reviewed; code comments not so much.

  15. Jean Demesure said

    @jeff,
    NOAMER has been commented at a French blog http://skyfal.free.fr/?p=421&cp=1#comment-23601
    BTW, no sign of ClimateGate in the French MS media (apart from a translation of the Ministry-of-Truth AP).

  16. Amabo said

    Nick, the data HADN’T been “artificially adjusted to look closer to the real temperatures”. They didn’t plot past 1960 because they had to “artificially adjust it to look closer to the real temperatures” first. Duh.

  17. Richard Blogger said

    All very silly. The only email that is interesting is 1256353124.txt:

    I’m not thinking straight. It makes far more sense to have
    password-protection rather than IP-address protection. So, to access
    those pages

    Username: steve
    Password: tosser

    Have a good weekend!

    Mike

    I am assuming that the “steve” here is a reference to Steve MacIntyre. ROFL!

  18. curious said

    More interesting, IMO, is the admission of something many spectators have understood for a while:

    “I’m not thinking straight”

    coupled with a demonstration of an inability to recognise the value of valid and accountable work.

  19. Kondealer said

    Here’s another nice email that shows the doubts about proxies within “The Team” themselves.

    From: “Raymond S. Bradley”
    To: Frank Oldfield
    Subject: Re: the ghost of futures past
    Date: Mon, 10 Jul 2000 08:57:19 -0400
    Cc: alverson@xxxxxxxxx.xxx, jto@u.arizona.edu, k.briffa@xxxxxxxxx.xxx, mhughes@xxxxxxxxx.xxx,pedersen@xxxxxxxxx.xxx, whitlock@xxxxxxxxx.xxx, mann@xxxxxxxxx.xxx

    But there are real questions to be asked of the paleo
    reconstruction. First, I should point out that we calibrated versus 1902-1980, then “verified” the approach using an independent data set for 1854-1901. The results were good, giving me confidence that if we had a comparable proxy data set for post-1980 (we don’t!) our proxy-based
    reconstruction would capture that period well. Unfortunately, the proxy network we used has not been updated, and furthermore there are many/some/ tree ring sites where there has been a “decoupling” between the long-term relationship between climate and tree growth, so that things fall apart in
    recent decades….this makes it very difficult to demonstrate what I just claimed. We can only call on evidence from many other proxies for “unprecedented” states in recent years (e.g. glaciers, isotopes in tropical ice etc..). But there are (at least) two other problems — Keith Briffa
    points out that the very strong trend in the 20th century calibration period accounts for much of the success of our calibration and makes it unlikely that we would be able be able to reconstruct such an extraordinary
    period as the 1990s with much success (I may be mis-quoting him somewhat, but that is the general thrust of his criticism). Indeed, in the verification period, the biggest “miss” was an apparently very warm year in the late 19th century that we did not get right at all. This makes criticisms of the “antis” difficult to respond to (they have not yet risen
    to this level of sophistication, but they are “on the
    scent”). Furthermore, it may be that Mann et al simply don’t have the long-term trend right, due to underestimation of low frequency info. in the (very few) proxies that we used. We tried to demonstrate that this was not a problem of the tree ring data we used by re-running the reconstruction
    with & without tree rings, and indeed the two efforts were very similar — but we could only do this back to about 1700. Whether we have the 1000 year trend right is far less certain (& one reason why I hedge my bets on
    whether there were any periods in Medieval times that might have been “warm”, to the irritation of my co-authors!). So, possibly if you crank up the trend over 1000 years, you find that the envelope of uncertainty is comparable with at least some of the future scenarios, which of course begs
    the question as to what the likely forcing was 1000 years ago. (My money is firmly on an increase in solar irradiance, based on the 10-Be data..).
    Another issue is whether we have estimated the totality of uncertainty in the long-term data set used — maybe the envelope is really much larger,
    due to inherent characteristics of the proxy data themselves….again this would cause the past and future envelopes to overlap.

  20. Layman Lurker said

    #19 Kondealer

    John F. Pittman, did you catch this quote above? ;)

    We tried to demonstrate that this was not a problem of the tree ring data we used by re-running the reconstruction with & without tree rings, and indeed the two efforts were very similar — but we could only do this back to about 1700. Whether we have the 1000 year trend right is far less certain (& one reason why I hedge my bets on whether there were any periods in Medieval times that might have been “warm”, to the irritation of my co-authors!). So, possibly if you crank up the trend over 1000 years, you find that the envelope of uncertainty is comparable with at least some of the future scenarios, which of course begs the question as to what the likely forcing was 1000 years ago.

  21. Jeff C. said

    Following up on #9 and #10, the zip file ”mbh98-osborn.zip” in the documents directory has some other interesting stuff. The directory ”mbh98-osborn\TREE\COMPARE” appears to have multiple versions of various recons presumably to compare the differences. There is even a file named “resid-fudge.dat” that has quite a hockey stick. Probably just someone goofing around with the data but the filename makes one wonder.

  22. Jeff Id said

    Actually from Mann’s emails the resid-fudge could be a big one.

  23. Kondealer said

    And this gem. Trying to smear the Wegman report.

    From: “Michael E. Mann”
    To: Phil Jones
    Subject: Re: EGU
    Date: Mon, 15 Jan 2007 12:45:46 -0500
    Reply-to: mann@xxxxxxxxx.xxx
    Cc: raymond s bradley

    thanks Phil,
    not suggestion you not cite Wegman report, just suggesting you make sure the citation makes clear what the report is…
    mike
    p.s. where/when did Tom Crowley use it?

    Phil Jones wrote:

    Mike,
    Thanks.
    I agree Wegman isn’t a formal publication. This was the highest profile
    example I could come up to show abuse of the curve. if you know of any
    others then let me know.

    Phil,

    I would not reference Wegman report as if it is a publication, i.e. a legitimate
    piece of scientific literature. Its a piece of something else! It should be cited in
    such a way as to indicate it is not a formal publication, wasn’t peer-reviewed, i.e.
    could be references as a “criticism commissoned by Joe Barton (R, Exxon).

  24. Jeff C. said

    Looks like someone might have been indulging in a bit of wishful thinking with the “resid-fudge.dat” file. If you compare it to several others in the same directory (resid-lowf.dat, resid-lowf-1750.dat, and resid-best.dat) look what you find:

    year resid-lowf resid-fudge resid-lowf-1750 resid-best
    1952 0.199307 0.199307 0.199307 0.199307
    1953 0.203005 0.203005 0.203005 0.203005
    1954 0.206704 0.206704 0.206704 0.206704
    1955 0.211684 0.211684 0.211684 0.211684
    1956 0.216662 0.216662 0.216662 0.216662
    1957 0.222889 0.222889 0.222889 0.222889
    1958 0.229115 0.229115 0.229115 0.229115
    1959 0.236529 0.236529 0.236529 0.236529
    1960 0.243945 0.243945 0.243945 0.243945
    1961 0.25247 0.25247 0.25247 0.24
    1962 0.260995 0.260995 0.260995 0.24
    1963 0.270529 0.270529 0.270529 0.24
    1964 0.280065 0.280065 0.280065 0.24
    1965 0.290492 0.290492 0.290492 0.24
    1966 0.300919 0.300919 0.300919 0.24
    1967 0.312104 0.312104 0.312104 0.24
    1968 0.323288 0.323288 0.323288 0.24
    1969 0.335084 0.335084 0.335084 0.24
    1970 0.346879 0.346879 0.346879 0.24
    1971 0.359128 0.359128 0.359128 0.24
    1972 0.371377 0.371377 0.371377 0.24
    1973 0.383913 0.383913 0.383913 0.24
    1974 0.396452 0.396452 0.396452 0.24
    1975 0.409108 0.409108 0.409108 0.24
    1976 0.421766 0.521766 0.421766 0.24
    1977 0.434374 0.534374 0.434374 0.24
    1978 0.446982 0.546982 0.446982 0.24
    1979 0.459376 0.559376 0.459376 0.24
    1980 0.471771 0.771771 0.471771 0.24

    Note how the values are the same up until 1975. After that someone helpfully added 0.1 to the “resid-fudge” values from 1976 to 1979 and 0.3 to 1980. Again, no smoking gun but it does make me think this deserves more scrutiny.

  25. Layman Lurker said

    #24 Jeff C.

    Could it be someone playing around with numbers to quantify a gap between actual data and a level needed for a robustness claim or something?

  26. Charlie said

    Regarding #21,22,24 resid-fudge:

    Also note the comment by debreuil (03:30:22) and debreuil (04:25:44) : on wattsupwiththat CRU e-mail thread. It references a fudge array in
    (\FOIA\documents\osborn-tree6\briffa_sep98_d.pro)

    ; Apply a VERY ARTIFICAL correction for decline!!
    ;
    yrloc=[1400,findgen(19)*5.+1904]
    valadj=[0.,0.,0.,0.,0.,-0.1,-0.25,-0.3,0.,-0.1,0.3,0.8,1.2,1.7,2.5,2.6,2.6,$
    2.6,2.6,2.6]*0.75 ; fudge factor
    if n_elements(yrloc) ne n_elements(valadj) then message,’Oooops!’
    ;
    yearlyadj=interpol(valadj,yrloc,timey)

    The amount of “correction” applied doesn’t seem to match, but it is setup to apply to 5 year periods.

  27. Jeff C. said

    #25 Could be, 1980 is the last year of the proxies and MBH is exclusively thermometer records after that. Possibly smoothing of a transition discontinuity that negatively affects the stats?

  28. Layman Lurker said

    #27

    Yes, that would make sense. Perhaps this could be some of the intermediate work which lead to “Mike’s Nature Trick”: http://www.climateaudit.org/?p=7810#more-7810

  29. lucia said

    Uses “corrected” MXD – but shouldn’t usually
    ; plot past 1960 because these will be artificially adjusted to look closer to
    ; the real temperatures.
    Oh common… This must be edited in. Must. Must. Must. I mean… come on. Someone actually put this in this way as a comment?!!!

  30. mrpeteh said

    #29 Lucia,
    Yes, people write notes to themselves so they don’t forget in the future.

    First thing to remember: these people LONG ago forgot that they are public servants working on public funds and their work is not in any way private.

    They have assumed and presumed that nobody else will ever see the code.

  31. Jono W said

    Its great fun wrapping self up in a cape / deerstalker and a spyglass, my sleep pattern has gone out the window. BUT, we need ONE result out of all this… release of all Hadley data/code as a national/nternational priority and fast. As they say there is no better way to win an arguement than convincing your enemey. release the source data.

  32. John F. Pittman said

    #20 Layman Lurker

    Yes, I did. I just emailed JeffID. I have an update but findings have already made it less substantive than it could be. I hope that JeffID gets my email and waits until I add some more stuff. There has been great quotes popping up world wide that support those posts I did.

  33. gravamen said

    Wikipedia articles that should record this event in a neutral and balanced manner, with journalist-written sources:

    http://en.wikipedia.org/wiki/Climatic_Research_Unit_e-mail_hacking_incident

    http://en.wikipedia.org/wiki/Phil_Jones_(climatologist)

    http://en.wikipedia.org/wiki/Michael_E._Mann

    http://en.wikipedia.org/wiki/Hockey_stick_controversy

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
Follow

Get every new post delivered to your Inbox.

Join 142 other followers

%d bloggers like this: