Forgive me the off topic but I’m now moderate on Climate Audit for some reason I do not understand. My answer to your last message is as follows:
Posted Dec 16, 2013 at 11:16 AM | Permalink
Your comment is awaiting moderation.
How can you be sure that the mess came from MXD?
What did you use? I advise you the Schweingruber data base and of course the raw data of Briffa et al. 2013.
Needless to dwell on the preprocessed data. Densities have no proven sensitivity to age but the RCS method is used in Briffa et al. 2013 (and in some other papers) against all logic. RCS methods are useless in the case of densities and have the great disadvantage of not allowing the necessary data normalization which is essential to avoid sampling bias.
I am curious enough to look at the data. It looks like Briffa got a much better match to temp but on skimming the paper there were several points where post-hoc selection seems to have occurred.
I have looked at Shweingruber several times here in the past, not sure if you are aware of that or not. Sorry for the bad memory.
I haven’t looked at RCS processing of MXD data only TRW, and have only used pre-processed MXD series. Does raw MXD also show the decay of TRW? I suppose I would have to plot it.
It seemed easy to bias series accidentally at the most recent endpoint using RCS so I was actually pleased that Briffa made some attempt to split the trees into a couple of groups, but any dart-throw split is pure mashmatics and may have unintended consequences. I learned from my work, not to trust the low frequency signal in the methods though, which correlates with Crag Loehle’s work on the subject of tree growth.
I do find the potential gains of a good proxy exciting, yet don’t believe in accuracy of any of the proxies used today.
I don’t know what you mean by stating that some result wasn’t what was wanted or “for accuracy: let the data speak”. Briffa’s splitting of the data, makes some sense in that trees in different situations experience different growth patterns, density and rates at different temperatures. Nonlinearity and non-stationary response is one of the main criticisms I and others make of the dendroclimate field.
This same process is often done by the scientist before regressing series together. The last step in the Briffa paper appeared to combine a bunch of series to “calibrate” to temperature data. This process is mathematically equivalent to the CPS correlation sorting method and can create a lot of that wiggle-match you are seeing. http://noconsensus.wordpress.com/2009/11/28/6858/ — explanation of why regression and CPS are the same thing
When I see a graph that matches temperature from MXD or TRW or Varves (Boreholes are their own story), I don’t see a science capable of predicting temperature trends, I see proxies affected by temperature in a highly non-linear way being mashed together with mathematics to give an inaccurately high correlation number. “Let the data speak” requires very careful math to avoid the pitfalls so common in the field and a careful eye to recognize when non-temperature noise has been mathematically pressed into a temperature pattern.
Are you saying that you believe Briffa has found good data?
I am moderately interested by the approach of Briffa and other climate experts of dendro, lot of tinkering sometimes very complicated hiding what actually contain data. Only RAW DATA from Briffa et al. 2013 are interesting and important.
“Perhaps what show this material is not what we wanted to discover ?”
Because the correlation of raw MXD with TLT is so exceptional that the climate community should rush to analyze this extraordinary fact. Why does hardly anyone interested? I guess the result is uncomfortable for many people.
“Are you saying that you believe Briffa has found good data?”
Yes, of course. But again, only RAW DATA are valueable (at least in the case before us).
“If so, do you base that on anything other than the close comparison to temperature?” :
glaciers melting, snow and reflections on the structure of stations data. Physiology ? no.
Some other considerations in the comments from these points:
The match to TLT is very good, but consider my point that high frequency “wiggle matching” can be achieved with enough sets of raw noise. Since tree growth data responds to temp, there is some kind of signal in the data. However, the long term MXD trend doesn’t match the thermometer data it experienced in your graph. It matches a lower trend more closely.
This is exactly the point I’m trying to make. The low frequency response is not necessarily the same as the high frequency curve when these math processes complete.
From my extensive experience in looking at paleo-correlations, I am very very skeptical that the match you show in your graph indicates good data. I strongly suspect large-scale correlation sorting or regression of the data at some step in the process. It would be very nice to find out I was wrong.
Until you spend some serious hours deconstructing Briffa’s paper, my recommendation is that you reconsider your opinion of the squiggles on the graph. They very well could have zero meaning at all.
First, high-frequency or inter-annual variations. In this graph (http://imageshack.us/a/img62/5610/ydd2.png), no possible manipulation because the values are the output of densimeter directly normalized. If the correlation is good, it can only be a good correlation between temperature and tree physiology. No place for a statistical trick here.
Secondly, low frequency or simply the trend. It is certainly more difficult, but, a priori, there is no special reason to expect a divergence. A phenomenon can obviously affect the trees in the long term but also all other means of measurement. That is why the comparison with a single reference is insufficient. This is also why the low frequency correlation with TLT is instructive. We are 2 against 1 in favor of TLT and MXD. In the case of the Alpine area, we are 4 against 1 (TLT, MXD, snow, glaciers against stations) and I do not even take into account the proven weakness of stations data. Who finds something to put in the other side ?
In my opinion, we are far beyond any reasonable doubt.
PS: Briffa et al. 2013 is not science, no need to dwell on it.
The data are comprised of more than one tree and typically more than one series, I don’t know the provenance of the Briffa curve you show so I cannot comment directly on its manufacture. If you sort that data by correlation, you create a high frequency match right from the noise in the data which is a big “trick” that has actually become standard in the paleoclimate field. If correlation is better in one tree than another, it doesn’t necessarily mean better temperature measurement was the cause, this is a false and statistically dangerous assumption. Often scientists pick series by hand for these papers based on correlation alone. This is scientifically and mathematically invalid.
Because high frequency amplitudes dominate the long term trend, sorting or regression cause the high frequency signal to match well in the calibration range, and historic signals are repressed.
“Secondly, low frequency or simply the trend. It is certainly more difficult, but, a priori, there is no special reason to expect a divergence.” – Actually, I don’t agree with this either. Even if there is a good short-term temperature signal, plants respond to all kinds of things other than temp such that divergence is the norm rather than the exception on most series I have examined.
“This is also why the low frequency correlation with TLT is instructive. ” Again, you should study the math carefully because the tree trend should correlate to ground data, If you were right that the data is good (which I doubt very much), you ,may be seeing the type of long term signal distortion I am trying to explain.
We aren’t even past the starting gate of reasonable doubt. I’m telling you, the key is in the details. If you can convince me that the details of the paper didn’t distort the signals, I will be very happy to change my mind but that kind of result would be a major breakthrough for dendrolimatology. I’ve looked deeply into too much of this kind of dendro work to expect anything revolutionary from existing Yamal region data.
Sorry for the style a little rough but I have difficulty to put civilized forms in English.
“Often scientists pick series by hand for these papers based on correlation alone.”
You’re right, it could theoretically be the case, I have no way to check. But think. In this case the excellent correlation relates to TLT !!! Briffa have selected samples for that someone can prove that he uses an incorrect methodology ??? This does not hold water.
Remember that we are talking about a better correlation with TLT and not with reference temperatures. Sorry but your argument is invalid.
“Even if there is a good short-term temperature signal, plants respond to all kinds of things other than temp such that divergence is the norm rather than the exception on most series I have examined.”
If you are using a single set of reference or several non-independent, you have no way to prove that the divergence comes from the proxy.
“…the tree trend should correlate to ground data…”
Data, what data? Tree should correlate the local temperature. What is the local temperature? Here is the whole question.
“If you can convince me that the details of the paper didn’t distort the signals,…”
My argument is the same one Steve McIntyre has been making for years, and it is not invalid. It has been published since 2008 several times also. VonStorch and Christiansen are two. You are falling for the closeness of a wiggle match between ground based temperature and TLT, and misunderstanding that to date, there are always problems in how these were created.
“If you are using a single set of reference or several non-independent, you have no way to prove that the divergence comes from the proxy.” — huh?… So you are saying that the magic tree “temperature record” suddenly shoots 7C in a different direction from a thermometer, and it is the thermometer we need to question not whether we are actually looking at a treemometer? From an engineering standpoint, that seems pretty unreasonable and I’m afraid I will have to disagree.
“Data, what data? ” — Ground temperature data not TLT data. TLT is data through an atmospheric thickness. The ground temp is what the tree experiences so it should correlate better to local CRU thermometer data. The fact that it correlates better to TLT is either anomalous local warming of the ground station or just plain luck – I will chose the latter as my best guess.
“What paper? ” Briffa 2013 where the data came from? Isn’t that where you found the curve?
I have personally no opportunity to verify if Briffa made any preselection. It is a fact that is my own, how can you say that this is not true? Can you help me?
“I achieved excellent correlation to a sine wave, temperature up and downslopes all from tree ring data including shweingruber MXD…”
You can do the same exercise including thermometers data directly, so it does not prove much on MXD.
“My argument is the same one Steve McIntyre has been making for years, and it is not invalid.”
I do not think Steve McIntyre argument for years specifically about data made public this year. I understand your argumentation, I just have shown you that it does not apply to MXD of Briffa et al. 2013. I believe Steve McIntyre is of the same opinion as me about it.
“So you are saying that the magic tree “temperature record” suddenly shoots 7C in a different direction from a thermometer, and it is the thermometer we need to question not whether we are actually looking at a treemometer?”
Not at all. On the one hand, this question appeared about low frequencies and secondly, I doubt very much that even at high frequencies there is such differences with MXD.
“TLT is data through an atmospheric thickness. The ground temp is what the tree experiences so it should correlate better to local CRU thermometer data.”
Are thermometers used by the CRU preferentially located in forests? No, of course. It is much more likely that the TLT anomalies better reflect long-term forest anomalies.
“Briffa 2013 where the data came from? Isn’t that where you found the curve?”
The curve? Not at all. The raw data ? Yes, they come from the appendices.
“I have personally no opportunity to verify if Briffa made any preselection” – The paper is online and free as is the SI. What would prevent you from checking which series were and were not used? I may do this work for you because you have my interest but time is very tight for me.
“You can do the same exercise including thermometers data directly, so it does not prove much on MXD.” — There is too much correlation thermometer data to do that. The noise (other factors) in tree and MXD data is very high, I estimated over 90% once. It ran at Climate Audit:
Unfortunately, I can’t agree with you TLT comment and don’t understand the frequency comment. It seems to me that you haven’t perused the discrepancies in the MXD series much. The data is actually quite noisy.
The last comment though makes sense and needs a reply because I still don’t know where you got your curve from. If it is from Briffa 2013 as your title said, then it is most certainly not raw. If it is from individual trees that you compiled into this curve that would be exciting, and I would like to know your method so I could replicate it! Verifiable temp proxies would be a fantastic development for someone like myself who has worked for so much time with it. How awesome would it be to truly know past climate for thousands of years!!
I think you need to look how the polar mxd series was constructed. I wouldn’t put it past Briffa to screen and select those series that matched CRU for a period that ended in 1995. Explains everything.
I seriously doubt MXD makes a better thermometer than instrumental thermometers.
In terms of lessons, I think you have a little quickly reversed the roles. It is possible that a serious problem of sampling affects Briffa et al. 2013. The origin is not post selection but the choice of the sampling sites. Steve McIntyre has mentioned this potential problem in the links I’ve provided and I tried to highlight it with several graphs. The interest of the method I used is precisely to be insensitive to such defects.
My confidence comes from having done a huge pile of work in this exact field with this sort of data. Much more than you seem to realize.
I’m still waiting for you to answer where exactly the data came from and whether you did any of the work yourself. This is critical in my understanding just what the squiggle you show actually is. It would be nice if you would tell me.
For my part, I downloaded the actual raw data this morning and spent a half hour planning an R routine to compile the data into timeseries. As you may be aware MXD data often comes in quite an unusual format.
I have no idea why you bring up glaciers. It wasn’t my comment but the “fidding” as you call it is 100% standard published practice in the field. They don’t hide it much at all – you just need to read the paper. Therefore, if you tell me the provinence of your squiggle – exactly – I can potentially see which thing was fidded by reading the paper.
I have no doubt that you have done a great and significant works on paleo. Simply, every case is different, Briffa is not Mann, MXD are not TRW or varves and Briffa 2013 is not Briffa 2001 and so on.
The file : raw/polar/polar.mxd.
I am for nothing in the core drilling, otherwise I built the graph on the basis that I have given to you:
– Calculation of the standard deviation for each series on the reference period 1951-1980 (eliminating incomplete series over this period).
– divide the entire series by the standard deviation obtained.
– Average of all series.
I talked about glaciers in reference to a graph of the Alpine area that I used in our broken discussion on Climate Audit.
“…the “fidding” as you call it is 100% standard published practice in the field. They don’t hide it much at all…”
Yes, but the idea that the good correlation between MXD and TLT is the intentional result of a selection by Briffa simply do not holds the road for a second.
Probably the pith offset concerns the calibration on temperature. This is arbitrary. My opinion is that a formal setting is not necessarily the best solution. Naturally, you should always keep in mind that a comparison of curves whatever the chosen solution is includes an element of subjectivity.
You are right. The conversation started with an insinuation that I should accept the data which is a very funny turn, yet he would not say which data I was looking at.
Interestingly, he/she seems to have knowledge that pith isn’t critical for MXD yet fails to answer the most basic questions like – where is the rest of your reconstruction series? I’ve often read Phi’s comments carefully at CA because they seem to have a technical foundation but I cannot follow the intent here.
I don’t even know if he handled dates right because Phi will not tell me the details. It took twenty comments to find out he hadn’t copied a graph from Briffa. Why did he title the graph Briffa 2013, if he did the reconstruction himself?
We do have a lot of questions and some are not related to the paper.
Phi, perhaps you would email me your code and results so I can understand?
it almost sounds to me as if he is graphing the data and then doing a “fit” without considering dates at all. Kinda like the old Hockeystick trick that selects for Hockeysticks or flipping Tiljander cause it “fits” that way. If that is really what he is doing it explains why he isn’t simply coming out and stating it.
He also still won’t give you a physical reason for why a tree that never leaves its local environment would match a Global Average Temp better than a local temp. Just reprising the Junk Paleo Science from the team.
For each series, the variance is calculated on the reference period and the entire series is reduced according to this value. Then all results are simply averaged. This is the easiest and most effective way to avoid sampling bias.
Well, I would say that I am also showing some patcience.
“…yet he would not say which data I was looking at.”
Oh well, the chart title refers to the name of the file and I gave you the source repeatedly.
“Interestingly, he/she seems to have knowledge that pith isn’t critical for MXD”
Pith offset is of interest if we consider the age of the rings. I’ve already explained why I did not take it into account. Do I have to repeat everything 50 times!
“where is the rest of your reconstruction series?”
I am primarily interested in the twentieth century and before Stevenson screens CRUTEM is too difficult to apprehend.
“I don’t even know if he handled dates right because Phi will not tell me the details.”
What do you miss? Densities are clearly linked to dates in polar.mxd.
“It took twenty comments to find out he hadn’t copied a graph from Briffa.”
“Why did he title the graph Briffa 2013, if he did the reconstruction himself?”
The title of the graph is polar.mxd the name of a raw data file. Briffa 2013 is in parenthesis to identify the source. What strange reviews!
“In the meantime, it would be nice to understand PHI’s explanation why pith offsets are independent of tree growth.”
I repeat it anyway: age does not have a proven influence on maximum density.
There is probably little influence but it is impossible to quantify. The sampling bias is much more important. I have already dealt with it all on cliamte Audit with graphs support.
“He also still won’t give you a physical reason for why a tree that never leaves its local environment would match a Global Average Temp better than a local temp.”
Who’s talking about global temperatures?
Your attitude is completely unwarranted and your ability to answer simple questions is atrocious.
For instance you could have written, “I took the raw data from x file and compiled it into the graph myself.”
How much effing clearer would that be! Instead you titled the graph Briffa 2013 and the data was not even Briffa’s –although he used it. These are both YOUR errors in communicating a very simple concept.
Now we find that you didn’t use the age offsets, which Briffa did use for MXD. Silly him I suppose.
At least now I can attempt to replicate your results. I should have time in the next few days. It will be pretty interesting if I can replicate your results, although it would still be nice to see your code for making the graph. There are plenty of series construction errors which could be made and I don’t have any values for comparison.
I have no doubt some wrongs but acknowledge that you do not put a lot of goodwill. Thus, the chart title is not “Briffa 2013″ as you say but “polar.mxd (Briffa 2013)”
“…and the data was not even Briffa’s –although he used it.”
What good play on words? These data are the substance of Briffa et al. 2013 and are published as part of Briffa et al. 2013 and not elsewhere.
“These are both YOUR errors in communicating a very simple concept.”
These are not errors and this is what I wrote in my first statement above:
“Needless to dwell on the preprocessed data. Densities have no proven sensitivity to age but the RCS method is used in Briffa et al. 2013 (and in some other papers) against all logic. RCS methods are useless in the case of densities and have the great disadvantage of not allowing the necessary data normalization which is essential to avoid sampling bias.”
And a little further down, I gave you a direct link to this post:
“Some clarification. It seems that all the interesting MXD material for the twentieth century is in the polar.mxd file. The series have been normalized in the period 1951-1980 before being averaged. The temperature anomalies are those of JJA. Some years are missing for HadCRUT, I could recover some by change in the reading program (http://imageshack.us/a/img21/1076/polar2.png). Still missing are 1905, 1907, 1961, 1962 and 1994. With these completed data, we see that the divergence appears earlier, around 1990.”
So yes, there are probably a lot of clumsiness in what I wrote, but not to the point of not understanding the substance.
“Now we find that you didn’t use the age offsets, which Briffa did use for MXD. Silly him I suppose.”
This remark is very strange. You do not understand that the pith offset has only a limited utility in the context of the RCS method and none in the one I use ?
The code would be of no use because you do not have the means to interpret it but I can give you the resulting values:
The link tells nothing about the provenance of your graph. The title says your graph is briffa 2013. That should say enough.
Briffa apparently found dating errors and corrected them by correlation, I believe these values landed in the offset files. I haven’t read his code yet or studied the paper enough to see how this was done but I certainly can’t tell just from your plot what was done. I have to ask details. I think if you explain what you did better Steve M may be interested to know, although he may already have worked with this data himself.
Of course the your code would have value, why can’t I read code? I always ask for code and usually publish it when I write here, it is standard practice.
Well, even if this graph was made by Briffa himself, by the pope, by the Queen of Atlantis or by a hydra in another galaxy, what would change? You do not even know if I do not match any of these eventualities.
I obviously did not look for dating errors. Anyway, in climatology, I almost never seen adjustments making anything different than worse data quality.
I must say that the reaction of Steve Mc surprises me. He obviously quite understand the significance of these data, so I wonder why he did not digs more the subject. It is still something other than muddy varves.
For the code, the treatment is so simple that it does not really seem to me a problem. I’ve also almost given his substance in the description. You could not interpret it anyway because the interpreter of the language used is not distributed.
The reason Steve didn’t react is probably because, like me, he didn’t realize you have only averaged the existing data. My guess is that he probably doesn’t know YOU had anything to do with the work. I really don’t know anything about what he thought though. The literal standard in the industry is to pre-sort the data and then regress to subdue any non-compliant data, which is WHY it matters WHO did the work. I wish you would listen a little better about that. A good high frequency match isn’t nearly the same as a good un-messed with high frequency match which I have NEVER seen. For your information, in paleoclimate, there is a very common risk that the RAW data is already pre-sorted leading to an excellent-looking yet still false quality of high frequency match.
The MXD data quality problem is so severe, that the original schweingruber data was simply chopped short and other data (usually temperature data) was pasted right on the end.
I have sent him an email explaining your claimed result.
The code is often a problem because there are plenty of pitfalls in processing the data. How did you handle the left justified loading of the files? Did you offset the years correctly? Did you do something unusual in your normalization? Heck you kept saying normalization without recognizing how many types of normalization exist, as though we could guess. There is a lot of detail there that is statistically interesting. When I make the series myself, I will provide the code right here for anyone to see.
This time, on the whole, I quite agree with your criticism. I also have a good experience of all errors that can be done and is done when programming. Replication is therefore welcome. Given the look of the result, I don’t think there are problem with dates.
About the small number of cores, here are two graphs that may interest you:
It is stable even very stable. In my opinion this makes manipulations on high frequencies impossible. It is quite different for bias due to different tree vigor. And this problem is solved by normalization.
See Phi, that is what is frustrating me about this conversation. You aren’t listing to me. Choosing a subset of data for its attributes is not “extraordinary”. It is 100% standard practice in paleoclimate to start with a lot of noisy data and only choose series which match temp or was a “preferred proxy” for other attributes. They just leave the rest out of the article entirely. I can’t remember a paleo-reconstruction paper offhand, which didn’t do some form of that. So please understand – 100 percent standard practice not 50%. They literally write it up in the papers that way, you should read more of them.
No, I am not insisting that is what has happened here, we don’t yet know, although the sparseness of the cores used is concerning. There is a very good possibility that this pre-sorting is exactly what you are looking at.
Patience, and caution. I’ve seen a lot of MXD series and none that I recall matched even high frequency temperature as well as what you show. You can understand that my suspicion of a pre-sorting step gives me little comfort when you show that the data is consistent. I haven’t seen much of that either.
I am interested though and we shall see what we find.
Briffa et al. 2013 is an awful pseudo-scientific study. The IPCC compatible character of the result is the mere fact of low frequencies bias. The wrong is so perfect that it’s necessarily intentional.
I just do not believe for a second to high frequencies manipulation, for obvious technical reasons and because low frequencies are so easy to fake.
I think it is called a Gaussian filter (?) This is at least what I remember trying to implement. It is over 5 years. There are some holes in CRUTEM and I could not tell you how the filter processes it (polyline simply ignores).
I think of something. You’ve never seen such high frequency correlation with MXD. In fact it is not very surprising. To my knowledge normalization is never used. This technique is difficult to implement for the long periods which are of interest to climatologists. I think we should be able to come up with something, but the tests I’ve done have proved unsuccessful. The problem is that without normalization, in addition to low frequencies bais, you always have a lot of noise.
Normalization to match standard deviation is how most iterative regressions in overdefined math problems start. Regressions on overdefined systems can have local minimums that provide a different set of weighting for the proxies than a different start point would. It gives a defined point where everyone can start from and each series is reasonably equally weighted on its influence on the error matrix.
Mann 08 started with normalzed data for instance.
By the way, there seems to be a good sized number of series in recent years in the polar mxd dataset so that eliminates that concern. I’m going to have to resurrect a couple of temperature downloaders I wrote to get the UAH and RSS series. I may post on just the MXD series alone first.
Yes. But do you have a reference to a specific regional MXD study using normalization?
I did not go very far in my attempt but it seems that these matrix or iterative techniques are not well suited for MXD (I don’t know why). And I would say that your surprise to find such correlations could be a proof of that.
LET Titre “UAH 65 E à 67.5 E Latitude 65 N à 67.5 N – juin à août”
LET TitresColonnes ?["Années", "Anomalie"]
Table [1979;-0.813, 1980;-2.057, 1981;1.493, 1982;-0.143, 1983;0.587, 1984;0.450, 1985;-0.797, 1986;-1.403, 1987;-0.047, 1988;0.400, 1989;1.007, 1990;1.227, 1991;0.160, 1992;-1.753, 1993;1.520, 1994;1.440, 1995;-0.283, 1996;-0.863, 1997;-2.827, 1998;1.073, 1999;-1.683, 2000;1.167, 2001;0.433, 2002;-1.327, 2003;1.237, 2004;0.277, 2005;-0.033, 2006;0.027, 2007;0.720, 2008;-0.323, 2009;-0.827, 2010;-0.903, 2011;-0.850, 2012;1.360]
!Première année dans le résultat :1979
!Dernière année dans le résultat :2012
!Années manquantes :
Normalization to match standard deviation is how most iterative regressions in overdefined math problems start. Regressions on overdefined systems can have local minimums that provide a different set of weighting for the proxies than a different start point would. It gives a defined point where everyone can start from and each series is reasonably equally weighted on its influence on the error matrix.
I agree it’s a standard practice in paleoclimatology… but, like much of what they do, I think it is not a good practice. The issue here is that we are dealing with a field that varies in both time and space. Standardization works best when the the quantity you’re trying to extract from the data can be assumed to be time independent (and the signal to noise ratio is constant across series).
In cases, like this, where you have polar amplification, ironically this method ends up weighting most heavily those with the poorest signal to noise ratios, and in any case, it’s easy to demonstrate that, unless the quantity of interest is constant over time, the method yields results than are not optimal in the OLS sense.
It would be really interesting to find series that looked so much like temperature though.
It couldn’t be MXD in general. Thermometers are designed to behave univariately with temperature. Tree growth (and hence density) is a complex response to many variables, with precipitation probably being a more important controlling variable than temperature.
I’m sure we only have the series that Briffa looked at, that matched up with the temperature record. He is not known for reporting adverse results. In fact, I can think of few less trustworthy “witnesses” than Briffa, sorry to say.
Of course we know that the Schweingruber data set is very heavily cooked. It is nowhere close to “raw measurements”, and I’ve refused to even consider it data in the past.
On precipitation, one of the things I have wondered about, given the withering under scrutiny of many paleoclimate methods, is how well some interesting drought reconstructions stand up to closer examination. I have seen a lot of work about that stuff and these days I file it under “interesting if true” because I am just so dubious of paleoclimate in general.
Well, high resolution late holocene paleo at any rate.
When I speak of “cooked data”, I don’t refer to deliberate falsification, but rather to the use of overly complicated and untested (and possibly untestable) algorithms to derive new “data” that are far removed from the original raw data set.
So, in this sense, my comment is not an accusation, it’s a fact, and easily demonstrable characterization of the Schweingruber data set.
When, after the data have been so heavily manipulated that I’m left with something that is difficult or impossible to fully replicate, I can’t really even call that product “data” anymore.
Candid answers to these two questions will identify flaws in the foundation of post-normal consensus models of climate change (AGW), the Standard Solar Model (SSM) of stars, the AGN (active galactic nuclei) model of galaxies, and the Big Bang Model (BBM) of the cosmos.
Below is the essence of a message just sent to the Congressional Space Science & Technology Committee:
Coincidence  exposed reality, and this was added as the last two pages of Chapter 2, A Journey to the Core of the Sun – Acceptance of Reality .
1. Einstein’s and Aston’s discoveries in 1905-1922 of “powers beyond the dreams of scientific fiction”  were selfishly used to kill, or threaten to kill, the inhabitants or leaders of other nations in 1945-1968.
2. Frightened world leaders apparently secretly agreed amongst themselves in 1971-1972 to:
_ a.) Save themselves,
_ b.) Sacrifice the integrity of science and
_ c.) Constitutional limits on governments, and
_ d.) Enslave their citizens by giving them street drugs, anti-depressants, psychotropic drugs, legalized marijuana, alcohol, reality TV, gladiator sports, video games, pornography, gambling casinos, etc., etc, ad infinitum !
With kind regards, - Oliver K. Manuel
Former NASA Principal
Investigator for Apollo
I do not know if you plan to share your results but in rereading the comments, I realized that I was not very clear about the calculus. I should certainly have given you the code but it would have taken us too far in the explanation of an exotic language. Normalization is done quite standard, it is centering, reduce. Apparently you could reproduce my results. If others want to dig deeper (it would be very interesting to have in particular several regional results using the Schweingruber data base), I formally specifies the method used.
V : all of the values in a series,
V’ : values for the years 1951 to 1980
m’ : average on V’
s’ : standard deviation on V’
Vr : resulting values of the series.
1. The series is eliminated if there are not 30 values in V’.
The reconstruction is simply the result of annual averages of all Vr.
I chose a 30-year period to obtain m’ and s’ significant enough and I started from 1951 to not remove too many series in the twentieth century.
I haven’t finished my analysis to the extent that it can be published. Certainly visually, my averages look the same as yours but the averages are not the whole story. I need to go to the original tree database and confirm which data was used and that will take time. The series shows very good correlation to temperature and that is highly unusual in the proxy world.
The effort you make to ensure the quality of data is meritorious. Yet I do not understand your reluctance. These data are
significant, significative and not favorable at all to the thesis supported by climatologists, exactly the opposite is true. Both the quality and the shape of these series brings almost IPCC to its knees.
I’ve seen a lot of this data. So far my opinion is that trees make bad thermometers. A set which makes a good thermometer would be interesting but if you have a lot of data which makes bad thermometers and one apparently good one, a scientist must ask themselves what are the implications for long term stationary? As it is, I’m not yet convinced that a bunch of other data was not thrown out to get the result. If the data was not thrown out, and this has good correlation, the balance of the type of data out there calls into question our ability to interpret this as temperature even with high frequency correlation.
The favorablity of the result for one argument or another is irrelevant. It is what it is and that is what we must live with.
I realize that the argument is given is that they’ve selected trees that have “temperature limited growth”, by selecting trees that are living in margins.
The trouble is, the margins shift over time in response to climate change, so what is temperature limited over some “calibration period”, probably isn’t for the entire period that you have core measurements for (this has been well documented already on ClimateAudit, which may explain McIntyre’s lack of interest in phi’s claims).
You could hypothetically end up with a period where you have temperature limited growth, in which case, you could get very good high frequency correlation between temperature and tree ring growth (IMO, in non-temperature-limited trees, this is destroyed by annual variability in soil moisture and precipitation, which is well known to be the dominant short-period controlling factor in tree growth). I say “hypothetically” because I don’t believe that this has been established to happen, by selecting trees that are in margins. I think a further correlational-based selection has to occur to get what is just an apparent “pure-temperature” proxy.
But what happens, when climate shifts, so the trees are no longer in the margin? You would end up with a loss of high-frequency coherence and possibly a divergence between long-period growth patterns and temperature, due to the nonlinearity in the temperature response.
Mann has suggested that in some climatological zones, that precipitation is actually the controlling factor, but that temperature and precipitation might remain long term highly correlated with each other. Hence use precipitation proxies then throw out high-frequency data. (It’s the basis behind the use of many of the proxies in his 2008 paper.)
Anyway, as far as I can tell, it is phi, alone in the world, claiming that MXD could be used to produce more reliable high frequency temperature probes than ordinary thermometers, which are very linear devises, actually designed to measure respond purely to temperature and are relatively carefully placed to measure pure temperature.
If somebody wants to worry about microclimate changes at met sites, a common (and IMO a somewhat overwrought) criticism of the surface stations, how about what happens in forests, when trees die due to disease or fire or culling and your “ideal tree” suddenly has a shift in its microclimate? That happens at high frequency, in a completely uncontrolled fashion (from the perspective of dendroclimatology), in real-world growing environments.
At least with met stations, you can estimate the magnitude of the effects and put limits on the impact of this on estimates of long-term trends (the estimates of the magnitude of the effects relative to the observed trends are what suggest that the criticisms are overwrought). How you could possibly do this with trees, even ideally geographically selected ones in long-term pristine temperature-limited zones, completely escapes me.
When you can’t even model the magnitude of confounding effects is when you seriously have to look at whether these samples could ever be primary standards for climate temperature measurements. (Again, even with the enthusiasm among dendrologists for the use of tree ring samples in climatology, I’ve never seen one of them ever make the claim for standard-labs quality outcomes from the use of the sample in temperature estimations.)
For my part, all the raw MXD data I could analyze showed, at the regional level, excellent high frequency correlation with temperature. And also an excellent low and high frequency correlation with the few credible proxies available.
I think it is a mistake to try to reconstruct the millennia temperatures without understanding what happens in the twentieth century.
The authors certainly have not selected the data to be so unfavorable to the view of the IPCC. This is what gives them credit. Again, one can possibly suspect a selection on the basis of sites (at worst based on individuals vigor), but the method I use is insensitive to these biases.
Happy new year Jeff. Looking forward to you digging into this. I have been interested in this ever since the post I did a couple of years ago.about Briffa’s 1960 truncation. That truncation would have been done after RCS processing though so it was an open question (to me anyway) whether the trend difference between composite MXD and temp was due to distortion from RCS (Briffa himself has mused about this in the climategate emails). Once the simple linear trend alignment was done the comparison between Briffa’s composite and temp was quite remarkable and showed no divergence at 1960 at all. To me the obvious next step after my post was to look at the raw data and then perhaps replicate the RCS process. It could be that a lack of adjustment for age related growth in Phi’s standardized MXD series might account for the difference with temp as the composite would be composed of younger and younger living trees as one moves to current time.
It is not measured directly and is closely dependent on climatic and environmental conditions.
2. Ring width (TRW).
It is closely linked to growth. And age for at least the first 100 rings.
3. The average density of tree ring.
To my knowledge, this measure is not used in climatology. I do not know of its dependence on climatic conditions, but its dependence on age is well known to those who work with wood.
4. The maximum ring density (MXD).
Closely related to temperature, no proven links with age.
The purpose of the RCS method is to take into account the age effect. This makes sense for the first 100 or 150 rings of each individuals in the case of TRW. As it is impossible to identify a dependency of MXD to age, using the RCS method in this case is just a joke.
Maximum densities as a function of age in polar.mxd :
The pith offset is not taken into account but it changes nothing. It is quite impossible to conclude anything of an eventual dependence on age. What is seen is essentially the result of the non-uniformity of the sampling. We also see the large interindividual variation (red curves are one standard deviation).
Phi — There is no proven linear relationship between latewood density and temperature.and there is quite a bit of evidence against it. You need to do more work with different data before you can make these unreasonable claims.
You MUST explain the provenance of the graphs you present. There is a lot of sorting in this field. Without it, your plots have literally zero value.
Really Phi, please listen and be patient. If you are not patient, read the articles behind your plots and find the raw data yourself. MXD is NOT a universally good thermometer. In fact, it has a terrible track record for linearity.
And since you asked where you assumed linearity, both short and long term linearity of a proxy is the key relationship you must assume for any kind of conclusion for or against the IPCC.
Linearity and stationarity with respect to temperature are THE key issues in proxy based reconstructions as the pro’s will tell you. Hide the decline is one of many examples of divergence from the temperature signal.
I spent several hours today reading Briffa’s paper. It is my favorite on of his at this point. Specifically, he avoids making the faux-errors of his and others past work where data is pre-selected for signal.
In support of my statements about standard practice, Briffa included this reference which is in reference to the “standard practice” that has been so thoroughly criticized by right thinking folks. From reading, some attempt was made to avoid the problems I was most concerned with.
“11. Summer temperature inferences
The development of the tree-ring chronologies, including the
selection and processing of the tree-ring measurements described
in the earlier sections, was carried out in isolation of any comparison
with the instrumental climate data. The aim here has been to
maximise the common signal within the tree-ring chronologies
and to minimise time-varying biases. This is distinct from maximizing
the climate or temperature signal, though of course the
expectation is that the common signal in this case will be a summer
temperature signal because the samples were collected from near
to the Arctic tree line.”
Also, Briffa 13 included some notes on MXD standardization which when you consider the curves presented in the article would account for the trend difference between your simple average and the ground data.
“Series of radial TRW or MXD measurements must be ‘standardised’
to remove long-timescale variations that represent systematic
patterns of growth associated with changing tree geometry
rather than the influence of any external environmental factors
(Fritts, 1976, p246e311).”
My own investigation will continue but I have spent my time this weekend already so it will take more time.
Monotony and stability are the key points, linearity is just a convenience.
If there was something to hope from pros, your blog may not exist and I would not waste my time discussing paleoclimatology.
Nobody has infused knowledge of what the signal of regional temperature is, any interpretation can only be derived from a critical comparison of the largest possible number of sources. What really means the comic episode of Hide the decline depends on such work.
No doubt Briffa et al. 2013 spread their good intentions. Unfortunately, I think I have amply demonstrated that the result is exceptionally bad, so bad that I do not believe in the sincerity of intentions.
As I already said and shown, the method used by Briffa et al. 2013 for MXD is inappropriate. The result does not differ much from a simple average while sampling bias are huge and not treated at all.
“Monotony and stability are the key points, linearity is just a convenience.”
That is completely ridiculous. You need to read more carefully. We MUST have linearity of response to temperature if we want to have any form of conclusion from math which assumes a linear response!
I literally cannot follow your hide the decline comment but what ‘hide the decline’ represented, besides a coverup of the problem, was that MXD data does not always linearly respond to temperature. In some cases, it is obviously either overwhelmed by other growth impacting factors or growth/temperature is so nonlinear it can have an opposite response! as Carrick showed with Dr. Lohele’s graph. Pithy comments to the contrary have no validity.
You seem to now be claiming that briffa’s methodology is faulty? Odd considering that you had not explained Briffa’s methodology prior to my discussing it.
Why is a simple MXD average better than a RCS corrected version? We know that an uncorrected age related signal would have a particularly biasing effect on recent years, have you demonstrated somewhere that the age-signal does not exist and therefore correction is not required?
This can become very complicated, but if monotony is observed, it is theoretically possible to use a proxy that does not have a linear response.
To my knowledge, attempts to explain the divergence are moving towards a stability problem.
The maximum densities can not be assimilated to growth. For the remainder, read the post of Layman Lurker, the whole bullshit about the divergence is contradicted by statistics.
Almost all my posts on B13 threads of CA were a review of the methods of the paper. Again I constantly reminded here that the method used by Briffa for MXD was stupid for because it is supposed to take into account the age effect which is not actually quantifiable. You really read what I write? Look up my charts?
It is useless for me to do anything other than ask you to reread all this thread.
Phi: “This can become very complicated, but if monotony is observed, it is theoretically possible to use a proxy that does not have a linear response.”
Of course it’s possible to use a proxy that does not have a linear response… you are using MXD after all.
However, it’s not just complicated to correct for the nonlinear response of the proxy (which is what I think you meant), it’s not even possible to correct for the nonlinearity, when you have a broad-band signal with unknown frequency content.
There simply isn’t a unique solution for this case.
Also, the high frequency content suffers the most from lack of uniqueness/distortion. So the very features you are excited about with MXD are the ones most affected by the lack of a linear response.
Of course there’s no reason to assume a monotonic response, since MXD is known to not have a monotonic relationship with temperature.
Divergence between MXD and instrument temperature outside of some range where they mysteriously agree well in the high frequency (where, as I pointed out, they shouldn’t) just points to the fact that culling has been done to produce this great correspondence of high frequency responses over some “calibration period”.
You’ve also claimed more about what you’ve shown than you’ve really shown. You don’t know that culling hasn’t been done, AFAIK Briffa doesn’t say, and the fact that there is good agreement over some interval with instrument temperature and divergence outside of it, certainly both points to culling and undermines the validity of MXD as an alternative measure of temperature.
Divergence is fatal for proxies that are trying to reproduce long period reconstructions. This is why Briffa’s consistent efforts to hide the divergence or otherwise paper it over are viewed by many of us as extremely dishonest.
“This can become very complicated, but if monotony is observed, it is theoretically possible to use a proxy that does not have a linear response.”
Of course it can but I did not say it couldn’t. What I said was – “from math which assumes a linear response!” — in this case we used averaging.
As to re-reading the thread, your partially stated thoughts makes interpretation of your meanings very, very difficult. For instance you wrote – “ITo my knowledge, attempts to explain the divergence are moving towards a stability problem.” It sounds like you are talking about stationarity in which case we are in agreement, but I’m not really sure.
I’m curious why Briffa’s method is stupid or impossible. I don’t think it is actually. He lines up the data by tree age not year, and runs a non-linear fit through the bunch of them to correct for age effects. It looks to me to be fairly reasonable and it also seems 100% necessary if you are looking to trends at the recent end of the curve. An average can only be a first order approximation and will demonstrate trend problems at the most recent end of the curve.
Sorry, I’m well aware that I commit errors of vocabulary. Yes, I think we agree on the stationarity.
For the method used by Briffa, I had already linked this graph:
Briffa relies on a similar analysis. Unfortunately, it is quite impossible to draw from that any indication of dependence on age. The shape is essentially the result of the non-uniformity of the sampling. See Jim Bouldin interesting considerations on this subject but about TRW. Compare also with Figure 3a of B13 for TRW. And even in the case of TRW, we see that the RCS method is worse than the normalization as sampling bias are so great. See here for MXD (http://imageshack.us/a/img22/2053/44qm.png) the difference between red and blue curves point out a sampling bias.
You seem well versed in much of this. If sampling bias is so great that we can’t fit an age curve, how can you be so certain that the trend at the most recent end of the data isn’t mucked up by a simple average?
“Given the correlation with TLT, I would not talk about mucked.”
That is not a reasonable position PHI. Everything we see in the data shows that there are age related effects. These do exist and will cause trend problems. Everything the tree experiences is ground related. There is no physical way for the trees to sense high level atmosphere so your expectation that it will is a little wild.
It is also unreasonable to take correlation as a guarantee of long term trend accuracy. The whole concept that trees are good enough thermometers that thermometer data should be contradicted is just silly. This is the best series I have looked at and I have looked at quite a lot of them. MXD thermometers are full of problems so you cannot take one set with good correlation (bad correlation groups KHAD I believe was rejected for Briffa’s paper for example) and claim long term stationarity either. MXD isn’t that perfect and a bit of correlation changes nothing.
“Everything we see in the data shows that there are age related effects.”
Really ? But I see nothing ! Can you help me ?
“These do exist and will cause trend problems.”
Assertion without proof.
“Everything the tree experiences is ground related.”
Yes, of course. The temperature which affects the trees can be quite well approached by the concept of regional ground temperature. Its evolution is fairly well rendered by TLT. Station data, it’s just a joke. Most of these data show abrupt jumps that can exceed 1 °C. The correlation between thermometers is actually awful.
“It is also unreasonable to take correlation as a guarantee of long term trend accuracy.”
Indeed, do I pretended that ?
“The whole concept that trees are good enough thermometers that thermometer data should be contradicted is just silly.”
Only the comparison of multiple proxies of temperature (stations data are only a proxy for regional temperature) can tell us anything on this subject. It is silly to pretend otherwise.
“This is the best series I have looked at and I have looked at quite a lot of them.”
Maybe, but you told me not using the raw data and you certainly do not use the normalization method that I advocate. It is essential to obtain a reliable result.
I confess to be quite tired by this discussion going in circles. You simply refuse to admit evidence because it does not fit your preconceived ideas.
Take your time and try to understand what you see on this graph:
“Anyway, as far as I can tell, it is phi, alone in the world, claiming that MXD could be used to produce more reliable high frequency temperature probes than ordinary thermometers, which are very linear devises, actually designed to measure respond purely to temperature and are relatively carefully placed to measure pure temperature.”
[Carrick, January 3, 2014 at 9:33 am]
But what do Giss, CRU, NOAA and other national agency ?
They apparently do not know that thermometers are very linear devises, actually designed to respond purely to temperature and are relatively carefully placed.
Still just an observation. What one seeks to obtain is ultimately an estimate of the evolution of regional temperatures. The analysis of data from stations and the study of energy balance of urbanized and rural surfaces show unambiguously that thermometers of stations are not at all suitable for this objetive.