# the Air Vent

## Because the world needs another opinion

• JeffId1 at gmail dot com

Everything you need to know about Climategate. Now available at Amazon.com

• ## Subscribe to Blog via Email

Join 187 other followers

## Robust?

Posted by Jeff Id on October 16, 2010

I continue to learn new details of paleoclimate.  The Ljungqvist proxies have been great fun so far.  I can’t plot them individually because some are confidentially revealed but I can average them.  One surprise I had was when I took all tree proxies and all non-tree proxies and created nearly identical reconstructions.  I mean these things are as noisy as any proxy series I’ve ever encountered but the average keeps coming out the same!

I ran a test today where I randomly selected series from the Ljungqvist series and created non correlation sorted composite plus scale reconstructions from them.  This non-sorted CPS scales proxies according to their entire series length and averages.  Basically it is a process which cannot create the ‘calibration period’ distortions encountered in Mannian regress-o-matic stuff.  If you are non-mathematical, think of the following plots as reasonable averages of subsets of the data.

My previous Ljungqvist replication is here:

In the past, we’ve seen other’s math create hockey sticks from random data with no signal.  Ljungqvist’s math and the following will create a statistically flat line from a truly random dataset whereas correlation sorted data or regression will create a hockeystick.  The following is reasonable math with Ljungqvist temperature proxies.

So in this post, I scaled all proxies to zero mean and 1 for SD for their entire timeframe. I then selected random numbers of proxies to see what the true range of reconstructions that could be generated from them.

Below is a plot of 5o reconstructions created from selection of approximately 9 random proxies for each one. Red line is the mean of all series in the following plots.  The red curve is a constant throughout this post.

The red line is the average, the grey mass is the 50 reconstructions plotted on top of each other.  Think of the gray as an uncertainty of perfectly known data rather than an uncertainty of possibility.  Expanding the graph to 500 series illustrates that the visible pattern is quite repeatable.

Note that we have an amazingly similar pattern.  A multi color 20 series version is below.

Fairly consistent pattern, wouldn’t you say?

Lets go more extreme.   5 series about 5 at a time.

Again, much of the general shape is repeated.  Certainly these five are more flat on average than might be expected, you have to look at individual series to see my point.

I then plotted no dendro and dendro proxies.

It appears to me that the signal in this data is fairly consistent throughout the series.  Looking at raw proxies, it is hard to see anything worthwhile but the averages always work.  It seems that any potential critique of Ljundqvist would have to come from the source data itself and the source data has a number of independent origins.

These proxies appear less noisy than the Mannian ones so I correlated the series with CRU and plotted the histogram of the correlations below.

Remarkably, only 8 of the 30 proxies had less than a 0.1 correlation to temp 22% whereas Mann08 the proxies had over 60% with less than 0.1 correlation to temp. I’m not sure what to make of all this, there isn’t much in Ljungqvist10 on proxy selection processes but they are obviously a more consistent set than Mann08.

1. ### timetochooseagainsaid

This looks similar to the kind of thing I’ve been looking for in proxy work. The pattern does seem to reliably emerge from the data and not the math as with some methods.

2. ### JTsaid

Is there anything in the data which obviously explains the radical step change apparent at about 1730 – 1750?

3. ### Søren Rosdahl Jensensaid

Many of the proxies in Ljunqqvist are already temperature series. That might be part of the reason why a large part of them correlate well to the instrumental temperature.
There is a list of the used proxies here:
ftp://ftp.ncdc.noaa.gov/pub/data/paleo/contributions_by_author/ljungqvist2010/ljungqvist2010.txt

Some the series can be found here, where it is possible to se if it is given as temps or z-scores:
ftp://ftp.ncdc.noaa.gov/pub/data/paleo/contributions_by_author/ljungqvist2009/ljungqvist2009recons.txt

Jeff, by the way, did you see the code fragment I posted in your first thread about Ljungqvist?
I would like to now if it is kind of correct and then I will go on and apply both approaches to the Ljungqvist data.

Also, it would be interesting to see Mann’s CPS algorithm applied to the Ljungqvist data.

4. ### Steve Fitzpatricksaid

#2

Maybe/probably the end of the Maunder minimum. Sunspots fell to near zero by 1640, and started to rise from zero around 1710. They ramped up to “normal” (20th century levels) by about 1790, dipped a bit in the early 1800’s, and then returned to relatively high from the mid 1800’s on. See: http://en.wikipedia.org/wiki/File:Sunspot_Numbers.png

If the Maunder minimum caused the dip/recovery in temperatures around the minimum (based on the above reconstructions) then this indicates a surface temperature response in the range of -0.2C for no sunspots.

Recent satellite data says the average radiative effect of a single sunspot is about 0.0021 watt per square meter. A change of ~55 spots coming out of the Maunder minimum would then represent ~0.116 watt change in solar intensity. Diagnosed climate sensitivity: 0.2C/0.116 watt = 1.72 degrees/watt, which is absurdly high (about 6.4C per doubling of CO2). Which all suggests that there may be more going on than just a change in total radiative flux; perhaps an influence of cosmic rays on clouds, or an influence of changes in solar spectrum with the solar cycle.

5. ### Steve Fitzpatricksaid

#2,
Or maybe the temperature dip near the Maunder minimum was just due to random variation.

What we really need is a time machine.

6. ### Steve Fitzpatricksaid

Jeff,

Doesn’t the CPS method reduce variance during the reconstruction period? Do the uncertainty limits you generate try to take this loss of variance into account?

7. ### Steve Kochsaid

CERN is doing some experiments on the effect of galactic cosmic rays on cloud formation on earth. Here’s a link to some pretty/interesting slides by the project manager explaining the connection:

Click to access Kirkby.CERN_.20091.pdf

8. ### TimGsaid

The end points bother me.

Does the smooth exagerrate the rise at the end?

9. ### Jeff Idsaid

#6 Steve,

CPS done over the whole set of data means that the data is simply being rescaled to a SD of 1. Since the calibration and historic amplitudes are taken into account, when applied this way it becomes averaging.

The Mannian CPS had two issues, 1 was that it was done in some cases only in the calibration period, but a more serious problem was the throwing away of uncorrelated data which just becomes a noise sorter.

10. ### Brian Hsaid

I’m particularly impressed by how deep the LIA was. But they survived it!

11. ### Kenneth Fritschsaid

I think what you have here is a well thought out sensitivity study. It does not mean that you have to accept any of the underlying proxies as valid indicators of temperature. Only that different proxies and different combinations of proxies and differnt methods lead to different results. That is to say that reconstructions overall without some better prior physical explanations are not robust.

12. ### Jeff Idsaid

Thanks Kenneth, I’m not going to be dishonest about anything here. Wrong unintentionally for sure, but dishonest – nope. This particular reconstruction was startlingly unchangeable. I sorted the proxies mann style – same result. I used the mann rejected proxies, same result. Nothing can change the curve substantially. If you were to pick the worst agreement proxies of the series mathematically and physically it wouldn’t change the result.

I did sort on annual basis though rather than by decade but this is what it is.

13. ### AusieDansaid

The following may well be a dumb comment:

But Jeff, from what you’ve just said (##12) the temperature seems to move up and down over the centuries.

It’s been rather hot recently.
At other times it has been much colder.
Perhaps (more tentatively now) it will be colder again in the future.
Timing is unsure.

The climate is a chaotic system – fully determined, non cyclical but periodic; a reverting to mean type of chaos (Hurst number less that 0.5).

Have I go that right?

14. ### Steve Fitzpatricksaid

Jeff,

Do you know how Ljungqvist selected his proxies? Could they have been pre-selected based on concordance with each other? That kind of pre-selection process would generate the robustness to changes in proxy subset that you have shown here. If Ljungqvist just grabbed a bunch of proxies and started working (no pre-screening) then his reconstruction probably is believable… within the calculated uncertainty range.

This reconstruction gains additional credibility because is shows a clear Roman warm period and MWP, both independently supported by non-climate historical data (harvest dates, etc.). Ljungqvist’s reconstruction is a very long way from MBH98.

15. ### Jeff Idsaid

Steve,

There was very little discussion on how pre-selection was done. I’m still skeptical that any of this is correct with respect to temperature. Cave speleo’s, trees and boreholes are very questionable proxies IMO. Also, there is great differences in smoothness between the proxies.

AussieDan,

I have a hard time accepting all of this as temperature but this looks a lot closer to science than other papers I’ve read. Basically it is difficult to for me to interpret because of unfamiliarity with much of the raw data but the data is in many cases completely independent and coming to similar results.

16. ### Jeff Idsaid

If one proxy is lacking any HF signal, scaling will make its lowf signal dominant over a series with a lot of HF stuff. That may be part of what is going on here and is the reason dendro with hf was separated from the others.

17. ### Kenneth Fritschsaid

Jeff, I would advise you and other readers here to be as skeptical of a result we might like as one we might not. My point is that different reconstructions/proxies give different results and when one looks at them like Carrick did with, I believe a 50m year moving average correlation. The correlations over the entire period could be quite good but all of the comparisons he made showed long periods with very poor correlations.

I would question the work that you have just done with regards to all the selections agreeing “too well”. Would not some of these proxies come from different parts of the globe and would we not expect them to give different responses? I would also suspect as Steve Fitzpatrick has that a pre-selection process could have been used.

If you did difference series between the reconstructions that you show above, it might be more instructive. Also look at the dendro versus non-dendro and the large variance difference. One might supect that the dendro has considerably more variance because of a much higher frequecy resolution of the dendro proxies over those of the non-dendro.

18. ### Kenneth Fritschsaid

On review of your graphs, Jeff, I see more clearly what you are displaying. The differences in the numerous reconstructions are what we want to look at and those spaghetti graps make that an impossible task.

You cannot show all the paired differences either as that number is even larger. Off the top of my head a table listing the paired correlations would be unimaginably large except for perhaps the 5 series. How about a table and differences series graphs for the 5 series?

19. ### Jeff Idsaid

Kenneth,

I don’t know the best way but it was surprising that the spaghetti didn’t have more spread.

20. ### Steve Fitzpatricksaid

Jeff,

I read over a long thread at Bart’s where you commented about Ljungqvist. The impression I got was that the reconstruction was based on OTHER reconstructions. But here you seem to be saying it is based on a group of individual proxy series. Can you clarify? Did I just misunderstand what you said at Bart’s blog?

21. ### Jeff Idsaid

Steve,

There is some of each. The link in #3 has the list.

Proxy location Latitude Longitude Proxy type* SampleRes. SeasonBias Period covered Reference
‡ 1. Lower Murray Lake, N. Canada 81°21’N 69°32’W V Annual Summer 1–1969 Cook et al. 2009
2. Devon Island, N.Canada 75°34’N 89°19’W Ice-core d18O 5 years Annual 1–1969 Fisher et al. 1983
‡ 3. Greenland stack 75°N–65°11’N 38°3’–43°82’E Ice-core d18O Annual Annual 200–1969 Andersen et al.2006
4. Taimyr peninsula,N. Siberia 73°00’N 105°E TRW Annual Summer 1–1989 Naurzbaev et al. 2002
5. Indigirka, NE. Siberia 70°32’N 148°9’E TRW Annual Summer 1–1989 Moberg et al. 2005
‡ 6. Big Round Lake, Baffin Island 69°52’N 68°50’W V Annual Summer 980–1999 Thomas and Briner2009
7. Lake Tsuolbmajavri, N.Fennoscand.68°41’N 22°03’E Lf Multi-dec/cent. Summer 1–1989 Korhola et al. 2000
‡ 8. Torneträsk, N. Fennoscandia 68°12’N 19°27°E MXD Annual Summer 510–1999 Grudd 2008
9. Polar Urals, NW, Siberia 66°52’N 65.38°E MXD Annual Summer 780–1989 Esper et al. 2002a
10. Donard Lake, Baffin Island 66°40’N 61°21’W V Annual Summer 800–1989 Moore et al. 2001
‡ 11. North Iceland Shelf 66°33’N 17°42’W Md 2–5 years Summer 1–1949 Sicre et al. 2008
‡ 12. Haukadalsvatn, W. Iceland 65°03°N 21°38°W Lf Decadal Spring/Sum.1–1999 Geirsdóttir et al.2009
‡ 13. Korallgrottan , C. Sweden 64°54’N 14°8’E Sp Multi-decadal Annual 1–1999 Sundqvist et al. 2010
‡ 14. Jämtland, C. Sweden 63°10’N 12°25’–13°35’E TRW Annual Summer 1–870, 910–1999 Linderholm and Gunnarson 2005
‡ 15. Farewell Lake, C. Alaska 62°33’N 153°38’W Lf Multi-dec/cent. Summer 1–1959 Hu et al. 2001
16. Gulf of Alaska 61°N 146°W MXD Annual Summer 730–1999 D’Arrigo et al. 2006
‡ 17. Iceberg Lake, Alaska 60°47’N 142°57’W Lf Annual Summer 450–1989 Loso 2009
18. Canadian Rockies 52.9°N 117.9°W MXD Annual Summer 950–1989 Luckman and Wilson 2005
‡ 19. Spannagel Cave, C. Alps 47°05’N 11°40’E Sp 1–10 years Annual 1–1929 Mangini et al. 2005
20. The Alps 47°00 7–13°E MXD Annual Summer 760–1999 Büntgen et al. 2006
‡ 21. Lake Silvaplana, Switzerland 46°27’N 9°48’E Lf Annual to decad.Summer 980–1999 Larocque et al. 2009
‡ 22. E. China 42°–27°N 110°–120°E D Decadal Annual 20–1999 Yang et al. 2002
23. E. China 40°–27°N 107°–120°E D Decadal Winter 20–1999 Ge et al. 2003
24. Shihua Cave, Beijing, China 39°54’N 116°23’E Sp Annual Summer 1–1979 Tan et al. 2003
25. Chesapeake Bay, E. USA 38°89’N 76°40’W Md Multi-decadal Spring 1–1989 Cronin et al. 2003
‡ 26. Lake Qinghai, Tibetan Plateau 37°N 100°E Lf Multi-decadal Annual 1–1939 Liu et al. 2006
‡ 27. NW. Karakorum 37–35°N 74–76°E TRW Annual Annual 620–1989 Esper et al. 2002b
‡ 28. Dulan, NE. Qinghai-Tibet.Plat.36°N 98°E TRW Annual Annual 1–1999 Zhang et al. 2003
29. Bermuda Rise, Sargasso Sea 33°41’N 74°26’W Md Multi-dec/cent. Annual 1–1969 Keigwin 1996
‡ 30. Yakushima Island, S. Japan 30°20’N 130°30’E Tree-ring d13C Decadal Annual 130–1949 Kitagawa and Matsumoto 1995

22. ### Steve Fitzpatricksaid

Jeff,

Thanks, I missed that. Looks like it would be a huge amount of work to read each of the papers and evaluate what the respective authors did. If these studies were each based on a single proxy series, then it is hard to see how calibration against the instrument record could cause variance loss. On the other hand, if each study pulled one (or a few) proxy out of a mix because that proxy matched the instrument record very well, then it would appear the the same issues with loss of variance could apply.

Have you looked at any of these published articles?

23. ### Jeff Idsaid

Steve,

Zactly!

I’ve started reading but at this point it is for my own edification. The paper has a chance to change my mind about our knowledge of history. It has an uphill battle at this point, but I’m going to stay open minded.

24. ### Steve Fitzpatricksaid

Jeff,

It’s good to be skeptical, of course. What made me most skeptical of MBH98 when I first saw the hockey stick was not the data treatment (I didn’t know enough then to appreciate it), but the obvious conflict with documentary data like records of planting and harvest dates, Viking farms in Greenland, wine grapes grown in central England, etc. The reason I think Ljungqvist is more credible is because the reconstruction (especially with the associated uncertainty) is not in terrible conflict with that kind of documentary evidence.

25. ### Kenneth Fritschsaid

Jeff, I am thinking that it would be better to do a random selection but without replacement. I would think that a 500 run of random selection of 9 -10 proxies in each run will have lots of overlap if you do it with replacement.

Why not show all 30 odd proxies in the same graph or at least a few graphs with the x axis ranges stack on top one another? Even where the proxies end and only the instrumental record is shown is difficult to determine with the typical spaghetti graphs.

I hate spaghetti graphs.

26. ### AMacsaid

This is a very informative analysis, Jeff. And great comments.

To second what others have already said — the place to start is with good math and good proxies. For the first time, you’ve shown that a paleo author’s mathematical approach is good, or more exactly, not disqualified by producing paleo temperature traces that are incredible, or clearly-bad. That’s a real accomplishment.

For good proxies, I’d second Steve’s cautions in #14 about pre-selection. Researcher X takes cores of ten trees at each at 3 promising sites (e.g. that are just below the modern treeline). When Dr. X gets back to the lab, he finds: Site A’s cores average into a jumble; Site B has a pleasingly small SEM and matches with the instrumental record; Site C also has a small SEM but the treering tracing over the recent past “makes no sense” with respect to the instrumental record.

I’ve assumed that Dr. X obliges himself to include all ten cores in the record for A, B, and C, rather than picking the “good” ones — which may or may not be standard procedure; I don’t know.

Which data will Dr. X submit to NOAA? Site B only, because his expert opinion allows him to qualify that one as the sole set that is informative as to paleotemperature? Or B and C, because for each site, the low SEMs show consistency of signal (whatever it is)? Or all three sites, because they all seemed “good” when Dr. X was hiking about the treeline, trunk-borer in hand?

For current pivotal clinical trials of investigational drugs and devices, there is only one acceptable answer: all three. Is that analogy relevant here? (I actually am unsure; it’s annoying that pro-AGW-Consensus advocates refuse to engage on this point.) In Bart’s comments, Eli Rabbett praised the merits of applying professional expertise to proxy selection:

there is an important role for expert judgment and it is not to be belittled. An elaborate version is the Delphi method, but, for example, the tree ring series for MBH 98 and 99 were selected by Hughes who had vast experience with them. He purposely selected series that he thought would have a strong temperature response based on his experience with dendrology.

The failure to recognize the value of experience is a sure sign of paranoid style, but very common in the everyone is Galileo school of science practiced by blog scientists…

I think it’s important to channel Feynman, and ask, “What if I’m wrong? What tests would indicate that this paleo reconstruction is or isn’t valid for the centuries prior to the start of the instrumental record?”

I offered some thoughts on this at Ron Broberg’s, though I don’t think they apply to Ljunqvist’s method. link.

27. ### AMacsaid

[off topic] I see that Ljunqvist included three varved lakebed sediments among his proxies; two are noted to have never been used prior to this study. The Tiljander data series are not among them. That’s good, since the four (three) Tiljander series are not proxies for temperature 1720-on, and only weakly so in earlier times.

I wonder how a Tiljander-plus-8-random-proxies graph would compare to one of the 9-random-proxies reconstructions shown in the third graph. My guess is that it would make pre-1800 temperatures look notably colder. [/off topic]

28. ### Søren Rosdahl Jensensaid

# 21 Jeff
To see if a proxy is given as temp or Z-score one need to look in the second link I gave, to the data for Ljungqvist 2009.
A number of the proxies in Ljungqvist 2010 are also in Ljungqvist 2009.

I have also tried to replicate Ljungqvist’s result. My result is not as close as yours. My reconstruction shows the period 1300-1700 somewhat warmer than Ljungqvist and your result.

Is it possible that you can post the cps code you are using? Or can you email it to me?

29. ### Kenneth Fritschsaid

AMac, I thought the Delphi method was used for predictions and forecasts (with expert opinion) and not for selecting indicators for measuring past variables. Also it depends, as I understand it, on a group consensus – and, of course, then the bigger question becomes who selects the individuals in the group.

Expert judgment can easily become expert bias and therefore we need a more basic level of a meta judgment that evaluates the potential bias in the expert judgment. But more straight forward is the application of simple and well thought out sensitivity tests that put to the test the question: Why did the expert(s) select that particular criteria or make the selections that they did? It is these sensitivity tests that are frequently not used or misused in climate science that have put much of the climate reconstruction results in doubt.
Unfortunately some climate scientists and defenders of the consensus immediately jump to the conclusion that sensitivity testers are putting their proxies/reconstructions up as the “correct” one(s) when in fact they are often merely showing that different selections lead to very different results which in the end may well mean that any reconstruction result is not robust.

I do not buy the use of expert opinion that you note in your example (and is avoided in medical/drug studies) or that to which E. Rabbett evidently is referencing. Nor do I buy it when we talk about the Medieval Warming period historical references fitting a given reconstruction better. Expert opinion should hold up well to proper sensitivity testing.

Jeff ID, is your proxy data used for your spaghetti graphs available for me to put in the form I prefer?

30. ### Jeff Idsaid

Kenneth,

Some of the data is not publicly released but I had already emailed it to you this morning. Please use appropriate caution.

31. ### Subsampled Confidence Intervals – Zeke « the Air Ventsaid

[…] when I still worked with numbers, I performed asimilar analysis on the Ljungqvist proxy data.  The method is just too simple and direct to argue […]