the Air Vent

Because the world needs another opinion

Mann 07 Pseudoproxies Part 2

Posted by Jeff Id on July 19, 2010

The Mann 07 paper uses synthetic temperature proxies to demonstrate that the methods don’t create artificial hockey sticks. I’ve maintained here, that they most certainly do create artificial hockey sticks and that calling the papers science is more than a slight stretch. What I’ve learned over the last few days is that Mann07 is a situation where CPS and RegEM don’t show signal variance loss in the historic portion of the hockey stick. The math of M07 is correct!!

Yup, you heard it here first, M07 is right and CPS and RegEM will not create signal distortions of any magnitude from these pseudoproxies.

An important table from the SI SupplementaryInfo shows the noise levels and type of noise used in each of Mann’s experiments.

There are a couple of important columns for this discussion, SNR or signal to noise ratio, percentage of noise, and the rho column which is the autoregressive component.   This basically can be thought of as how fast the noise component can change.  A rho value of zero means that the noise can change instantly from one point to the next without having any dependence on the last value.  Rho can vary from zero to one, a higher value means that the more recent point has some dependence on the previous value.  If you imagine a pan of water, you could measure its temp and then heat it with a stove burner but if you measure every second, the measurement you take at second #2 is going to be similar to second #1.  However if you measure every several hours hours, randomly turning the burner on and off,  the temps will vary wildly and independently of the previous measurement temp.  The dependence on the previous temp is autocorrelation.

The significance of white noise is that statistically nearly impossible to generate a trend from it. Whereas if we have autocorrelation (redness) in the noise the signal can walk creating the potential for locally induced trends after these standard paleo sorting methods are used.

You can see in the table above M07’s model r (first column – r) he used dataset A with a signal to noise of 0.4 which he refers to as 86% (does anyone know how this 86% number is computed?).  Anyway, I found the column averages of dataset A from experiment a and r above and calculated an r amplitude ratio of 0.4 matching well with the table. I then fit an arima (1,0,0) model to the data and found that the noise had an average autocorrelation of 0.31 confirming the 0.32 rho from the table above.  Mann wrote out the AR math himself rather than using stock functions so I wanted to check it .

I then generated 10,000 artifical noise proxies having rho of 0.32 and added in a signal, using CPS I tried to extract that signal to verify its distortion.  The important part of the next figure is not through the absolute value of the vertical scale, but by the distortion in the horizontal iso-temperature lines.  The deviations from linearity represents the magnitude of the distortion which is created by the CPS preferential proxy sorting method.

As you can see, the amount of amplification of the signal is absolutely minimal.  CPS and therefore RegEM can extract the ‘signal’ undistorted from this data, Mann is right.  I consider M07’s results completely verified from that aspect.   This plot looks one heck of a lot different than my previous CPS work which used a higher AR coefficient in the proxies and got the extreme distortions shown in the figure below.

Now Mann is of the opinion that demonstrations exactly like this last graph above, are disingenuous because they have excessive redness and therefore don’t accurately represent proxy data.  Perhaps though, it isn’t the other guy who’s got their thumb on the scale.

From M08 which was a CPS and RegEM hockey stick paper, 1208 series were used of which 484 preferred series were kept in creation of the hockey stick.  I ran an ARMA fit to each of the proxies in this paper.  Although several series wouldn’t converge due to very high redness, the vast majority did, allowing the creation of this histogram plot of the rho’s.

The red line is Mann’s 0.32 value whereas the average value of this histogram of rho’s was actually 0.44.  I found that by using his 484 series which passed the uniquely magical sorting of M08, this average didn’t change.  It’s almost certain that 0.32 is too low a value for annual proxy data but an average of 0.44 isn’t that different right?

I don’t think this is about the ‘average’ rho though.  I think its about extremes and the rho in the calibration period.  Now I wonder what this reconstruction will look like if we take out the high rho proxies, or use only high rho proxies.

In the meantime, you can see from the isotemperature line figure above that pulling any signal you want from this data will be extremely difficult. Since I can pull any signal we can imagine from the ‘actual’ proxy data of M08, as shown in the hockey stick cps part one post above and here, we know that M07 doesn’t resolve point of variance loss in any way whatsoever.

Stay tuned for the exciting pseudoproxy extravaganza, part 3.  We’ll look at some of the pseudoproxies in M07 and compare them to M08, also I’m interested in the redness of the infilled blade pasted on the end of each series. — now doesn’t that sound fun?


16 Responses to “Mann 07 Pseudoproxies Part 2”

  1. Steve Fitzpatrick said

    Jeff,

    Very interesting. What was the rho value for the synthetic data used in your previous CPS work? I read your earlier posts again, but I did not find it (of course, I may have missed it).

  2. Mary said

    This is where all the US taxpayer money is going:

    Fraud gets ex-Obama, Clinton fundraiser 12 years

    NEW YORK — A wealthy Manhattan investment banker and former top Democratic fundraiser was sentenced Thursday to 12 years in prison for defrauding banks of $292 million, some of which he donated to politicians including Barack Obama, Hillary Rodham Clinton and Al Gore.

    http://www.google.com/hostednews/ap/article/ALeqM5iC3CYATP4ke7JDlA0Hkb9nxXZ9tQD9GVM98O0

  3. Jeff Id said

    #1 I want to say it was about 0.8 but I don’t remember as I used the schweingruber proxies into a variable so the value was only visible when you prinited it.

  4. Mary said

    This is REALLY funny :) You can’t make this stuff up:

    http://www.urbandictionary.com/define.php?term=Al+Gore

  5. dearieme said

    In summary, are you suggesting that Mann made a mistake, in that he reported something that was true?

  6. Kenneth Fritsch said

    Jeff ID, I think you are getting to the essence of what is critical to the Mann interpretation of conditions that lead to the loss of variance in reconstructions versus what Zorita and von Storch claim.

    To answer your question about the percent signal noise, the derivation comes from the Mann et al. (2007) paper: Climate Field Reconstruction Methods under 3.3 Signal to Noise Characteristics on page7.

    Look at the original von Storch paper listed in the references of the Mann paper referenced above under Von Storch. Von Storch does find the reconstruction reduces variance and it depends on the noise level and the red noise particularly. Von Storch detrended the instrumental data in his first paper (why I do not know) and this made the variance loss even greater. In his comment reply to this problem found by Wahl he does the analyses without detrending and finds a reduction in variance not as large as in the detrended case by still large. He used an AR1 of 0.71 and found a very large reduction in the variance.
    The Mann paper is critical of von Storch for using the detrending approach, but does not bother to mention what von Storch obtained without detrending. Mann also vehemently disagrees with von Storch using the AR1 value of 0.71. Jeff, what you show would indicate that the proxies taken separately with the higher AR1 values are going to have huge reductions in reconstruction variance. Of course when this is simulated with a psuedo-proxy you will be looking at only one average AR1 value.

    Do we have any comments made publically by von Storch or Zorita on Mann’s 2007 paper?

    I have some problems with the Mann paper in that he, one, shows the pseudo-proxy instrumental temperatures tacked onto the end of the reconstruction whereas what one wants to see is the reconstruction data from start to finish, i.e. once one gets a relationship between proxy response and temperature using the instrumental data and proxy response in that time period that relationship needs to be applied to all the proxy response (do not hide the decline or hide or not show for any reason the entire reconstruction).

    A second problem is that his graphs make distinguishing the various different “experiments” from one another. I did a 400 % magnification on my computer of the graph SI Figure 5 and I find that the “experiments” with noise added, and particularly the blue colored one, do not match the amplitude of the high frequency spikes that is in the psuedo-proxy. There has to be a better way of displaying this information.

    And finally the Mann paper references a drift in the climate models used for psuedo-proxies by von Storch, and presumably the ones used by Mann that he claims have to be removed to make them more realistic. The statement is made that “any potential drift was removed”. Why not just say whether there was drift or not and it there was that it was removed thusly?

  7. Steve Fitzpatrick said

    Jeff #3. Thanks. My recollection is that the Schweingruber proxies contributed a big fraction of the Mann 08 reconstruction once the Tiljander lake sediment data was removed. Is that correct?

  8. Kenneth Fritsch said

    The von Storch paper in the reply to the Wahl comment is important and I am posting the link here to the abstract. To read the original paper in full and the comments you simply need to get an ID and PW and sign in.

    http://www.sciencemag.org/cgi/content/full/306/5696/679

    I also note that the Mann paper, like so many that he and his cadre author, finds that he has moved onto a new and better methodology and even points to weaknesses in past efforts, but, lo and behold, that when a test of robustness is applied to the past results they hold as reported. To me that type of approach is more advocacy motivated than science.

  9. eduardo zorita said

    You may be interested in reading this web blog and this paper

  10. Re: eduardo zorita (Jul 20 04:46),
    Following up on Eduardo’s comment, Christiansen’s paper is relevant. You can get a free (postscript) version here. Mann and Co have a comment saying why they think C et al implemented RegEM-TTLS incorrectly, which is why it performed poorly. Unfortunately, I haven’t found a free copy, only the abstract.

  11. Jeff Id said

    #9 Thank you very much for the link. I’ve spent three hours this morning reading and grasping the nuance – or trying. Your work basically has royalty status here because I’ve checked some of it and can’t find problems. It’s really too bad that you guys didn’t replicate the mistake Mann98 made in your 04 paper. It’s amazingly ironic that the mistake of others becomes the point at which your own work is criticized.

    ——-

    Nick, I can’t see the mannian comment either. The blog post is good and there is a paper linked by Christiansen in the thread.

    http://web.dmi.dk/fsweb/solar-terrestrial/staff/boc/loc_rec.pdf

    From which equation A5 and the nearby comments are nicely explanatory of the examinations here. I also enjoyed the explanations of variance correction methods in the discussion.

    On a more subtle point, C misses badly on the descriptions of result and ‘climate sensitivity’ but of course has some pressure. I’m an outsider, so I don’t know whether he believes this portion or is forced to write it to gain acceptance. My guess is that he is forced, but there is so much wrongthink on Earth, he may actually believe.

    It leaves me wondering if he grasps the possibility of non-perpendicular noise, but the guy is so smart, it seems like an unlikely explanation. After all, any non-temp signal is noise in a temp reconstruction. Then there are non-linearities and spurious correlations. I suppose it’s just more confusion for us non-climatologists but I don’t intend to call anyone out on it.

  12. eduardo zorita said

    Christiansen et al are not the only ones that find that RegEM also suffers from signal loss. Smerdon et al. also obtain similar results as Christiansen. I haven’t test those methods myself, but my impression is that they may be quite sensitive to the choice of some internal parameters, (e.g. the ridge regression parameter or of the order of the truncation in the Truncated Total Least squares version .

    http://www.ldeo.columbia.edu/~jsmerdon/papers/2010a_jclim_smerdonetal.pdf

  13. Jeff Id said

    #12, From the work done replicating and replying to the Steig Antarctic temp paper on the cover of nature which used RegEM, I can guess that you are right about the ridge or TTLS parameter.

    An interesting fact in the case of RegEM is that the temperature values are infilled rather than replaced. It essentially beomes a method for pasting proxies on the end of pure temp data. While Schneider’s methods don’t allow easy access to the B matrix, it can be obtained through several of the R replications of RegEM done here. We even have an emulation of the Ridge implementation now – which I’ve never tried myself.

    The B matrix is the linear weighting applied to the proxies, it is a series of vectors, 1 per timeperiod to combine the values of each year. If the same proxies exist in any two years the vector is the same, if one proxy or temp value drops out, the weighting vector is different and independent of the others. Dependency ends up being based on similarity of covariance or correlation. Some unique things can happen to the weightings including strong negative values or inversion of sign.

    B matrix from Steig et al.

    http://noconsensus.wordpress.com/2009/06/15/improved-weight-calculation/

    ——

    I’ve done some unusual things here with CPS to try and explain it to tech minded laypeople. I’ve varied redness, signal to noise, and used Mann08 proxies to demonstrate that any signal can be extracted. They are very much red enough to get whatever signal you want. I figured if Mann can criticize the redness of the pseudoproxy papers, why not use the actual data.

    http://noconsensus.wordpress.com/2009/06/20/hockey-stick-cps-revisited-part-1/

    and this is one plotting noise levels to look at signal loss which I found unique because of the gaussian bell shape of the loss plot.

    http://noconsensus.wordpress.com/2009/08/26/more-on-cps-signal-distortion/

    ——–

    This M07 post happened because I was going crazy with the fact that he claimed zero signal loss. I already know that’s not true, so I had to figure out what he did.

    Now that I know he used mostly white noise – according to the table – followed by very low rho AR1 noise, everything makes sense again.

    ——–

    Thanks for the paper. I’ll read it this morning.

  14. Jeff Id said

    And on the B matrix again, accessing it allows a method to see how well the proxies fit in the reconstruction. You can get the proxy fit in the calibration range rather than just temp data.

  15. [...] long and vociferously contested .  Neither of us had a good answer, but I had to know.   In Mann07 – Part II at the Air Vent, the mystery was solved.  The M07 paper uses model data as a ‘known’ [...]

  16. [...] I’ve long and vociferously contested .  Neither of us had a good answer, but I had to know.   In Mann07 – Part II at the Air Vent, the mystery was solved.  The M07 paper uses model data as a ‘known’ [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
Follow

Get every new post delivered to your Inbox.

Join 140 other followers

%d bloggers like this: