Robust – Not so much

A reader sent me a link to a new paper by Berger, analyzing the coherence (covariance) of various popular proxy reconstructions (hockeysticks).  The paper can be accessed for free here. Similarly to global temperature metrics from the same data, such reconstructions are often declared to support each other confirming the accuracy of the methods.  Of course, it’s my opinion that they are bad data crammed together with bad statistical methods that are ‘designed’ to force a balance of noise in the calibration period which creates a correlation to temperature (the blade) out of basically — nothing.  The handles are basically noise and deamplified signal, which averages toward zero.

Burger’s analysis looked at the coherence (of the handles) of various reconstructions with each other, finding clusters of the most like of 10 reconstructions and at which significance thresholds their likeness represents. Burger writes:

By avoiding the (calibrating) instrumental period, and by using a fairly robust spectral
measure for low-frequency performance, the above coherence analysis has uncovered
several inconsistencies among the group of millennial reconstructions that figured
prominently in the latest IPCC report and elsewhere. An immediate lesson from this
10 is that simple visual inspection of smoothed time series, grouped and overlaid into a
single graph, can be very misleading. For example, the two reconstructions Ma99 and
Ma08L, which have previously been described to be in “striking agreement” (cf. Mann
et al., 2008), turned out to be the most incoherent of all in our analysis.

It’s an interesting paper, in several aspects, including the fact that despite claims by RC scientists that the reconstructions support each other, mutual comparisons of the reconstructions have not been carried out.  Yes the reconstructions all look like hockey sticks, but if they are temperature, we should have coherence.   Still, five groups of reconstructions were created from the 10 which which had a highly significant coherence >99%.  However, it’s not until we drop below 90% that the five groups begin to match each other.  An interesting result, considering that the papers often use the exact same source data.

Similarly, the 95% significance level (d=0.61) yields identical clusters, while under the 90% level (d=0.65) clusters {Br00, Es02, dA06} and {MJ03, Mo05} merge.

The graph above can be interpreted as the quality of match on the vertical scale having different reconstructions on the horizontal scale.  Where individual reconstructions are joined with a horizontal line is the quality of match between them, as indicated by the value d.   From this plot, our favorite whipping boy, Mann has the least similar of all the reconstructions not joining to the rest until d=0.77ish.

Table 2 is another way to look at the comparison:

There are a few notable quotes in the papers discussion section.

Using inconsistent reconstructions to approximate the temperature curve has one par25
ticular visual consequence. Whether overlaying them in one figure or forming an average,
the result tends to be a cancellation of larger amplitudes as inconsistency means
here to be indistinguishable from random covariations. Together with the mentioned
synchronization through the instrumental calibration period, such “synthesis” figures
automatically resemble a hockey-stick.

This is the point made in the hockey stick posts in the header bar above.  I don’t know Burger’s other work, but basically Mann08 IS exactly what he recommends against.  The noise in the pre-calibration period is essentially random as he has shown and forced syncrhonization in the calibration period — automatically creates a hockey stick.  In this case he’s just discussing a merging of the finished reconstructions which already have the temperature variance well matched since 1850.

The paper does have one minor issue though in the following statement.

The analysis is constrained to reconstructed data prior to 1850, to ensure that the estimated coherence is not 5 inflated by calibrating effects from the instrumental period.

Popular calibration methods other than decentered PCA which include CPS and various regression analyses, affect the reconstructions well prior to the calibration timeframe.  I’ve described regression style hockey stick math here as shock and recovery.  The frequency of the noise in the proxy determines the recovery rate and from my messing with M08, I would have suggested doing the analysis on pre-1650 instead of pre-1850, data, not that it would make that much difference.

It seems that Burger is far less skeptical than myself of proxies to represent temperature.  In his concluding recommendations he makes this point:

It was shown that the target area plays only a minor role. Furthermore, if type and
processing of proxies are sufficiently even, coherent reconstructions are produced. If
5 that is true in general, the main source of reconstruction inconsistency is the use of
mixed types of proxies (“multi-proxies”), and their role for temperature reconstruction
should be revised. One should systematically check whether “uni”-proxy reconstructions
tend to be more coherent than multi-proxy reconstructions, and if so, which types
of proxies actually create the inconsistencies.

By uni-proxy reconstruction Burger is referring to the types of proxy used.  While this is an interesting point, I wonder why he didn’t take the comparison analyisis one step further and look at the proxies themselves.  My belief is that they are highly, highly (two times) inconsistent and if there is a temperature signal in them, the statistical methods of paleoclimatology have no chance of recovering it.

Anyway, this statistical analysis is now usable as a reply to climate science which often makes the claim that independent studies came to the same conclusion.  You can point them to Burger 2010 and say, check again.. I’m actually surprised that there wasn’t better coherence between the reconstructions, especially considering they use much of the same data.

The paper can also be found at this site, which allows for interactive discussion that is currently open, it will be interesting to see if anyone replies.