the Air Vent

Because the world needs another opinion

Audit of an Audit of an Auditor

Posted by Jeff Id on September 29, 2009

Just a short post tonight I hope. Tom P, an apparent believer in the hockey stick methods posted an entertaining reply to Steve McIntyre’s recent discoveries on Yamal. He used R code to demonstrate a flaw in SteveM’s method. His post was on WUWT, brought to my attention by Charles the moderator and is copied here where he declares victory over Steve.

Steve McIntryre’s reconstructions above are based on adding an established dataset, the Schweingruber Yamal sample instead of the “12 trees used in the CRU archive”. Steve has given no justification for removing these 12 trees. In fact they probably predate Briffa’s CRU analysis, being in the original Russian dataset established by Hantemirov and Shiyatov in 2002.

One of Steve’s major complaint about the CRU dataset was that it used few recent trees, hence the need to add the Schweingruber series. It was therefore rather strange that towards the end of the reconstruction the 12 living trees were excluded only to be replaced by 9 trees with earlier end dates.

I asked Steve what the chronology would look like if these twelve trees were merged back in, but no plot was forthcoming. So I downloaded R, his favoured statistical package, and tweaked Steve’s published code to include the twelve trees back in myself. Below is the chronology I posted on ClimateAudit a few hours ago.

http://img80.yfrog.com/i/schweingruberandcrud.png/

tomp

My version of TomP's graph - Click to expand

The red line is the RCS chronology calculated from the CRU archive; black is the chronology calculated using the Schweingruber Yamal sample and the complete CRU archive. Both plots are smoothed with 21-year gaussian, as before. The y-axis is in dimensionless chronology units centered on 1.

It looks like the Yamal reconstruction published by Briffa is rather insensitive to the inclusion of the additional data. There is no broken hockeystick.

He did a fantastic job in reworking R code to create an improved hockey stick graph. To see his code the link is here.
.
I spent some time tonight looking at his results. Time planned for analyzing Antarctic sea ice. I found that essentially the only difference in the operating functions of the code is the following line.
.
Steve M —- tree=rbind(yamal[!temp,],russ035)
Tom P —– tree=rbind(yamal,russ035)
.

The !temp in Steve’s line removes 12 series of Yamal for the average while Tom’s version includes it. I’m all for inclusion of all data, but I am a firm believer that Briffa’s data is probably a cherry picked set of trees to match temp or something. Therefore by inclusioin of the sorted Briffa Yamal version, we have an automatic exclusion of data which would otherwise balance the huge trend. However, this is not the problem with Tom’s result. The problem lies in this plot, also created by TomP’s code.

tompcnt

Tom P's Yamal Reconstruction - Count per Year. Click to Expand

Here is the zoomed in version:

tompcntzoom

Above we can see that everything in TomP’s curve after 1990 is actually 100% Briffa Yamal data.

So the question becomes – What does the series look like if the Yamal data doesn’t create the ridiculous spike at the end the curve?

I truncated the black line at 1990 below.

tomptruncsh

The black line is truncated at the end of the Schweingruber data and it looks pretty similar to the graph presented in the green line by Steve McIntyre again below.

rcs_merged_rev[1]
Don’t be too hard on Tom P, he honestly did a great job and took the time to work with the R script which is more than most are willing to. Steve is a very careful worker though and it’s damn near impossible to catch him making mistakes. Trust a serious skeptic, it’s not easy to find mistakes in his work and some of us check him just as I spent over an hour checking Tom’s work. In my opinion Tom deserves congratulations for his efforts and checking, this way we all learn.

I’ve now been all the way through SteveM’s scripts from beginning to end and can’t find any problems with the script, maybe others can!


36 Responses to “Audit of an Audit of an Auditor”

  1. Antonio San said

    Great post indeed!

  2. rephelan said

    Jeff:

    I’ve been following TomP’s postings and have added him to my ignorant, obnoxious, deceitful troll list. I hope he is posting from a university server because it is so much more satisfying to see acadenic trolls burned for offenses they labored so hard to see ithers burned for.

  3. rephelan said

    ahh.. that was “others” not “ithers” as opposed to “druthers”… and as for the troll list… I know where TCO lives down to about 16 blocks.

  4. […] Jeff Id audited the auditor of an auditor and found that Steve’s work still holds up “robustly”. – Anthony Jeff Id writes on The Air Vent […]

  5. It was therefore rather strange that towards the end of the reconstruction the 12 living trees were excluded only to be replaced by 9 trees with earlier end dates.

    This statement by Tom P is totally false. I have no idea where Tom’s figure of 9 comes from and I’d appreciate an explanation. The russ035 data set had 34 cores, 17 of which ended in 1990 (the CRU archive only had 10 cores in 1990.) In the middle part of the 20th century, there were up to 15 more cores than the CRU 12.

    As I mentioned in my first post on this, the ID number sequence (missing ID numbers) convinces me that there are missing cores in the archive which require explanation every bit as much as missing invoice numbers.

  6. Jeff Id said

    Tom made several statements which are dubious at CA. I’ve just learned that he holds a technical PhD. This changes my perspective somewhat on his comments as he should know the difference.

    Time permitting, we venters are going to have some fun with the core numbering tomorrow.

  7. Hans Erren said

    Looks like it’s time for an IPCC funded expedition to find 20 trees to update the Yamal proxies since 1995…

  8. Jeff
    Thank you. This was important. I’m glad it went to WUWT. And I know how much work it implies.

  9. MrPete said

    Re: Hans,

    Is there a Starbucks near Yamal? :)

  10. MikeN said

    Can someone tell me how to get the charts to show properly? They are being cutoff, with the last chart I can only see 20 in 2000.

  11. MikeN said

    OT, but Jeff, you should include which hockey stick in your hockey stick posts.

  12. Kenneth Fritsch said

    I do not know where Tom P fits into climate science, but I find his somewhat twisted logic not unlike what I have seen in some climate scientists. Once one understands what Tom P presented in his graph it becomes rather apparent that, without a better explanation, he is being disingenuous and that what he presented in no way contradicts Steve M’s analysis or view of it as a sensitivity test.

    I find his postings do inform about a certain line of thinking that exists in at least some of climate science today. Just think if a climate scientist were able to publish the Tom P graphs in countering the Steve M sensitivity test and they were allowed to stand without further comment and explanation.

  13. Layman Lurker said

    I will post here instead of CA because I think traffic load is very heavy.

    I think Tom P.’s latest attempt to salvage his strawman argument fails both on logic and on method (according to Steve’s review of Tom’s filtering methods). The point of Steve’s excercise is not to create an alternative Yamal reconstruction. Even if it was then how can Tom argue that such a reconstruction should extend beyond 1990 with the very samples which have been shown to be a biased subset prior to 1990?

  14. Layman Lurker said

    I see that Bender made an almost identical point to mine: post #329 on the Yamal Divergence thread:

    I understand Tom P’s argument. After much back-pedalling, he is saying that there is no good reason for excluding Briffa’s sub-sample from a chronology intended to cover the area we are referring to as “Yamal”.
    .
    This, of course, has nothing to do with the question of whether Briffa’s sub-sample is representative of the trees in that broader area called “Yamal”. Which is the question Steve asks.
    .
    Tom P’s point, as I rephrase it here, is arguable, but not incorrect. Unfortunately, it is irrelevant to the independent point that Steve is making. Briffa’s sub-sample is not representative of other samples taken from “nearby”. QED.

  15. MikeN said

    >The point of Steve’s excercise is not to create an alternative Yamal reconstruction.

    Why not. I think people should just put together whichever datasets they can and produce dozens of reconstructions.

  16. Kenneth Fritsch said

    Why not. I think people should just put together whichever datasets they can and produce dozens of reconstructions.

    Because Steve M is doing a sensitivity test. That has been the brunt of the analysis work at CA and Air Vent and it can stand on its own without being peer reviewed.

  17. Kenneth Fritsch said

    I see that Bender made an almost identical point to mine: post #329 on the Yamal Divergence thread:

    Layman, this explanation was obvious after MrPete’s post on that thread and the fact that Tom P failed to show or note the Schweingruber series as a stand alone was very misleading. Showing/noting it would have explained the “differences” immediately – and the invalidity of the Tom P point.

  18. Jeff Id said

    It seems like Tom is backing off. His whole tone has changed in the last couple of posts.

  19. Jeff Id said

    That’s good because I lost patients with him when he refused to admit his errors. – He still hasn’t.

  20. Layman Lurker said

    #18

    The guy is acting like he has an agenda. Here is another strawman presented in this comment (copied from WUWT)regarding the incomplete numbering sequence of the Schweingruber series:

    “Excluding one subset of a core archive on the basis of incomplete label sequences but including another series with the broken label sequences might lead to accusations of cherry picking.”

    WTF. Again, this point is only relevant to the strawman premise. It has nothing to do with the point of Steve’s analysis.

    His real error is that he is missing the point in his comments. He is backing off because the more he pushes this the more people see this for what it is.

  21. Jeff Id said

    Lurk,

    I put this on the CA thread. I’ll go check WUWT next.

    Re: Tom P (#359),

    So since you now admit that your method is not valid for a sensitivity test — do you now retract this statement based on your incorrect sensitivity analysis at WUWT:

    It looks like the Yamal reconstruction published by Briffa is rather insensitive to the inclusion of the additional data. There is no broken hockeystick

  22. Kenneth Fritsch said

    Jeff ID, I do not see where an admission of error or worse in my view, being misleading, is important at this point. We all know now what Tom P did and how he thinks about selection criteria for proxies.

    I personally log all this information away for future reference when I see replies from known, honest to God climate scientists. It sometimes helps in understanding where these people are coming from. Tom P is no dummy, but in my view he is more interested in selling than in understanding.

  23. Jeff Id said

    Kenneth,

    It’s a about honesty and integrity at this point IMHO. You know from TCO that the one thing that drives me nuts is dishonesty for a motive. When TCO crossed the line from being a good critic is when I started moderating his endless (even drove your own near limitless patients to the edge) posts.

    Tom is about to get a whole lot of venting. I’ll work on sea ice or something to try and settle down.

  24. rephelan said

    Jeff:

    There are trolls and then there are trolls. TomP is a TCO style troll, talented, maybe even brilliant, but the point of every encounter is himself…. but not confident enough to come out from behind an anonym. Then there are Flanagan style trolls… who enjoy a kind of blogging S-M… and, of course, there are Don Baccus (better known, perhaps, as dhogaza) type trolls, who are simply vicious, poisonous ankle-biters who are best stomped on sight. The top of the heap, however, are Tamino-type trolls who have hard time earning a living in the real world except when their unique combination of talent and viciousness make them useful to Russian-emigre billionaires with a profoundly anti-American agenda, but who will otherwise not be named here….

  25. Jeff Id said

    I would love to know if Grant Tamino Foster was funded by a Russian billionaire.

  26. Carrick said

    Jeff ID:

    It’s a about honesty and integrity at this point IMHO.

    I wonder why some people have such a hard time admitting mistakes?

    Based on my understanding of what Tom has done, he essentially wrote a series of R-codes that came to the same conclusion one could arrive at without running a single line of code: Namely if you only include Briffa Yamal data after 1990, you will get identical results for that period to only including the Briffa Yamal data. Well of course you will, since that’s a tautology.

    TomP somehow missed this, and that is fine, but it appears dishonest to now argue a different point entirely (and one that need not involve any R scripts at all) and at the same time not withdrawal his original argument, namely that he proved using R code that Steve had made an error.

  27. Kenneth Fritsch said

    Carrick and Jeff ID, I agree with Carrick’s rendition of what Tom P did. I went through all of Steve M’s code that Tom P indicated he used for his graph and could not find any plots that show only the CRU series with the composite series. If Tom had shown the single Schweingruber series in any of these plots, the original error would have been obvious to Tom P and others.

    I have asked him for his code for his graph and have not been able to extract it from him. He has ignored my last request at CA. Jeff do you have the code he used that will produce the plot he showed. If those few snippets of changed code from Steve M’s original code are what Tom P used (I may want to run that) I think he may have mislabeled his plot and misunderstood what he had plotted. He kept insisting that he had not run his plots past 1990 when it was so obvious he did.

    Carrick, in my mind you have Tom P’s evolving criticisms correct and it now appears to me that he is being evasive to save face. Look at Briffa’s reply to Steve M’s comments and analysis. It sounds rather vague and evasive to me.

    Tom’s secondary point entirely misses the point of Steve M’s sensitivity analysis and instead looks at the combined series as a composite over the entire period, but then without Tom P putting weights on the parts of the composite after 1990 or clearly pointing this problem. Anyway, I agree that it is totally disingenuous of Tom P to change and confuse the issues. I just wanted to be sure that I understood all the issues involved.

  28. Jeff Id said

    In TomP’s original code is here:

    http://wattsupwiththat.com/2009/09/28/update-a-zoomed-look-at-the-broken-hockey-stick/

    Tom P (15:19:16) :

    The only material change I found was the single line shown in the post above.

    I think Steve removed some of my comments to Tom so I’m going to let Tom say whatever he wants. I’m tired of it anyway. When I first saw his work I emailed to Steve and Charles that I thought he was right, then I went through the entire thing until I found the problem – which turned out to be a pretty obvious mistake. But I went line by line through the whole thing.

    Had he at least admitted the obvious error, and retracted the obviously false claims, I wouldn’t have changed my opinion of him so quickly.

    Since Tom is not honest enough to admit error, and he continues to make one false point after another, he isn’t worth my time. When the goal of one side is confusion couched in reasoned dialog, the faulty argument put forth will will never resolve.

    There are whole cultures on Earth based on this kind of negotiation.

  29. Peter S said

    Jeff – your patience is acknowledged.

  30. Layman Lurker said

    Wow, the load at CA is unreal. Steve has a lot to deal with right now.

    I don’t think Steve is too worried about Tom P. anymore. I see that he has “moved on” rather than admit his strawman error (interesting that Briffa seemed to insinuate the same thing). Now, in true shotgun fashion, he implies that centennial scale signal distortion due to the core age differences between the CRU 12 and the Schweingruber series must bugger up Steve’s sensitivity comparison. He suggests that this injects noise into the (Schweingruber) signal.

    I am not really qualified to comment on this, but it seems to me that centennial scale signal distortion can hardly be categorized as noise during the period which Steve is comparing. Briffa makes no such complaint. If there is any LF distortion would it not be more likely due the differences between “RCS” and “Corridor” processing? Does it matter? Any comments Jeff?

  31. Jeff Id said

    #30, I actually read the TomP comment earlier. It almost prompted me to answer but I managed to resist the troll. I’ve noticed the pattern in most of his comments now. I interpreted his comment to say that it affected historic signal data and went away scratching my head because they are exactly the same.

    I’ve got very limited understanding of corridor standardization having read nothing but the two words themselves, however I guess it is similar to a regional standardization over large regions. Briffa’s series use the same RCS standardization across all samples. Sorry, I’m not much help.

    As far as injecting noise into this example, that is truly shotgun. Like a kid in a fight sticking his arms out and spinning in circles.

    Steve used Briffa’s method, so if Briffa’s method is sensitive to series length, that would be part of the point of a sensitivity test..no? Anyway the method doesn’t affect data in years it does not exist.

  32. Layman Lurker said

    Speak of the devil here is Tom’s latest to you at WUWT:

    Jeff Id (16:03:48) :

    Steve McIntyre has put up the chronology records for all the core data from live trees in the CRU and Schweingruber series:

    http://www.climateaudit.org/?p=7241

    While most of the CRU trees cover the entire plotted range back to 1800, and so are at least 190 years old, only 3 of the 18 Schweingruber are that old, and most are less than 100 years old.

    Hence the Schweingruber series does not contain the long-lived trees necessary to discern a centennial or multicentennial variations. The Schweingruber series is therefore of very limited utility for a valid comparison with the much longer-lived trees of the CRU archive. Steve McIntyre’s sensitivity test between them is comparing a signal (CRU) to noise (Schweingruber).

    Hence I am happy to withdraw my reconstruction based on the data from both series – the original Briffa plot stands unmodified. It is not the hockeystick of Briffa’s Yamal reconstruction that has collapsed, but rather the case against it.

  33. Jeff Id said

    Tom P (07:35:54) :

    This is a false interpretation of the standardizaton methods in tree rings. The net result of this reconstruction is a mean, nothing more. An average of the available data. There is no recentering, no RegEM no fancy PCA just an average of the ring widths as standardized by the same exponential function for every tree ring width.

    Pretending that you have killed Steve’s work again (a third time) and again by weak arm waiving means nothing.

  34. Kenneth Fritsch said

    I compared Tom P’s and Steve M’s code and the differences are exactly at the line:

    tree=rbind(yamal[!temp,],russ035) # Steve M
    tree=rbind(yamal,russ035) # Tom P

    and then the line:
    count.pick=countf(yamal[temp,]) # Steve M
    count.pick=countf(yamal) # Tom P

    and finally at line:
    legend(“topleft”,fill=1:2, legend=c(“Schweingruber Variation”,”CRU Archive”)) # Steve M
    legend(“topleft”,fill=1:2, legend=c(“Schweingruber Variation”,”CRU Archive”)) # Tom P

    What this changes in effect is Tom P putting the 12 CRU series back in the mix to get a composite whereas Steve M was removing them to look at Schweingruber and CRU separately. Steve had already combined them elsewhere but had cut off the series at 1990 for the composite. The scale makes it difficult for Tom P to see that his cut off is not at 1990. Had he been more forthcoming and direct in spelling out his code changes from Steve M’s the problem would have readily recogonized.

    However looking at the separate plots of CRU and Schweingruber would have revealed that what Tom P thought he had was not possible.

    So let me label that as a mistake followed by Tom P wanting to see something that was not there and then a pigheaded response by Tom P when the error was pointed out. However, we should not lose track of his secondary arguments and how much in line they are with Briffa’s reply in missing the point of what Steve M was doing and thus not even answering in any detail the question that that sensitivity testing presents: What is the proper why of selecting samples a priori?

    Other drive bys, like Lorax, at CA also show that same mistaken view as taken by Tom P and Briffa. I am not sure what it would take to show them the light.

  35. […] correctly but in fact Tom had mistakenly extended the Yamal data past the sensitivity data. My reply was here. I never did read an admission of error from him on that […]

  36. coping with bipolar disorder…

    […]Audit of an Audit of an Auditor « the Air Vent[…]…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
Follow

Get every new post delivered to your Inbox.

Join 148 other followers

%d bloggers like this: