DeWitt, Getting the answer to that one right is a bit of a chore, I’m afraid, so I don’t have any ready answers.
The problem is that climate noise is highly correlated, made more so by the algorithms used by BEST (in spite of which they still have more high frequency noise, go figure).
The nominal confidence levels are very high in both cases, but that is uncorrected for correlated noise.
However, I don’t think anybody has tried analyzing this particular data set (land only with their weighting) to the degree where we can reliably place confidence intervals for them. Sorry that I don’t have more answers right now.
I’m not answering because I don’t have one. It seems reasonable to use weighted least squares but to what advantage? It brings up the arguments about whether to fit a line or not. You know, the discussions where the statistician thinks they have the answer on what to use (line or curve) but the scientist is simply left asking why.
It seems reasonable to use weighted least squares but to what advantage?
It’s weighting the data points according to their uncertainty instead of assuming they have equal uncertainty, when they do not.
Part of the problem with climate science, IMO, is the ignoring of uncertainty in the measurement process and for global mean temperature, the manner in which you sum individual stations to “measure” global mean temperature or global mean land temperature is part of the measurement process.
“It’s weighting the data points according to their uncertainty instead of assuming they have equal uncertainty, when they do not”
I read up on the process this weekend. It does make sense that a weighted least squares would be appropriate for data with such large time based variations. You had a long discussion with Nick and Tamino on it at his blog. I wonder if this might be that day you are interested enough to write something up.
I have always thought that determining valid estimates of CIs for temperature series and trends of those series is more complicated than those producing them are discussing.
I am currently looking in more detail at calculating breakpoints for the stations in the USHCN TOB and Adjusted maximum and minimum temperature series and determining exactly how the Menne corrections affect these series. I think there is a lot of “good stuff” in these details that are overlooked in published papers where authors are more intent to simply show a novel method of adjusting data.
I was very surprised (and perhaps this is because I am a layperson in these matters) how easy it was to find the breakpoints that could be attributed to climate change and how geographically localized those changes are. Further it appears that correcting station related breakpoints can affect (obscure) real climate related breakpoints – but this conjecture needs more analysis. Once you embark on these analyses it appears to me that things can get complicated real fast. and you see things that beg for further analysis and details.
One thing I haven’t been able to get anybody to comment on is the “right way’ to do the least-squares fit.
BEST and CRUTEM3 both list uncertainties in addition to their times and temperatures.
Here’s a comparison of weighted versus unweighted:
BEST unweighted: 0.279
BEST weighted: 0.344
Here’s what the fits look like.
What are the confidence limits on the two trends? Are the weighted and unweighted trends significantly different?
DeWitt, Getting the answer to that one right is a bit of a chore, I’m afraid, so I don’t have any ready answers.
The problem is that climate noise is highly correlated, made more so by the algorithms used by BEST (in spite of which they still have more high frequency noise, go figure).
The nominal confidence levels are very high in both cases, but that is uncorrected for correlated noise.
However, I don’t think anybody has tried analyzing this particular data set (land only with their weighting) to the degree where we can reliably place confidence intervals for them. Sorry that I don’t have more answers right now.
Carrick,
I’m not answering because I don’t have one. It seems reasonable to use weighted least squares but to what advantage? It brings up the arguments about whether to fit a line or not. You know, the discussions where the statistician thinks they have the answer on what to use (line or curve) but the scientist is simply left asking why.
Jeff ID:
It’s weighting the data points according to their uncertainty instead of assuming they have equal uncertainty, when they do not.
Part of the problem with climate science, IMO, is the ignoring of uncertainty in the measurement process and for global mean temperature, the manner in which you sum individual stations to “measure” global mean temperature or global mean land temperature is part of the measurement process.
“It’s weighting the data points according to their uncertainty instead of assuming they have equal uncertainty, when they do not”
I read up on the process this weekend. It does make sense that a weighted least squares would be appropriate for data with such large time based variations. You had a long discussion with Nick and Tamino on it at his blog. I wonder if this might be that day you are interested enough to write something up.
As long as it’s “someday” I can always say “yes”. 😉
Seriously, I’ll see if I can write something up.
I have always thought that determining valid estimates of CIs for temperature series and trends of those series is more complicated than those producing them are discussing.
I am currently looking in more detail at calculating breakpoints for the stations in the USHCN TOB and Adjusted maximum and minimum temperature series and determining exactly how the Menne corrections affect these series. I think there is a lot of “good stuff” in these details that are overlooked in published papers where authors are more intent to simply show a novel method of adjusting data.
I was very surprised (and perhaps this is because I am a layperson in these matters) how easy it was to find the breakpoints that could be attributed to climate change and how geographically localized those changes are. Further it appears that correcting station related breakpoints can affect (obscure) real climate related breakpoints – but this conjecture needs more analysis. Once you embark on these analyses it appears to me that things can get complicated real fast. and you see things that beg for further analysis and details.