# the Air Vent

## Because the world needs another opinion

• JeffId1 at gmail dot com

Everything you need to know about Climategate. Now available at Amazon.com

 hunter on Scientific Warming Another Ian on Scientific Warming Frank on A good discussion Paul_K on A good discussion Frank on A good discussion Frank on A good discussion Jeff Id on A good discussion hunter on A good discussion omanuel on Scientific Warming Jeff Id on A good discussion hunter on A good discussion Jeff Id on A good discussion Paul_K on A good discussion Frank on A good discussion Jeff Id on A good discussion
• ## Follow Blog via Email

Join 148 other followers

## Area Weighted Antarctic – Offset Reconstructions

Posted by Jeff Id on August 31, 2009

Ok, I know you guys are tired of area weighted reconstructions but for those who say publish, the detail of this is important. What I’ve been attempting to do is verify that the anomaly based area weighted reconstruction is of good quality. As we discussed before, even two thermometers of the same exact measurements value can have different anomalies when the timeframe is different. As an example, assume we have an noiseless linear upslope in temp for 20 years. Thermometer 1 measures for all 20 years and thermometer 2 measures for only 10 but both record the exact same number. When anomalized the mean of each record is zero, the thermometer 2 anomaly will have a lower value than thermometer 1 and the average will have a sudden step when the second thermometer is introduced.

These steps can be corrected by looking at the beginning of the new record and making sure it is offset to match the longer term record. This method implicitly makes the assumption that both records are the same even though we’re not sure what the heck thermometer two would have measured had it had existed for the same time as thermometer 1. Confounding the issue is the fact that we’re looking at tenths or hundredths of a degree C in noisy data that can vary by 10C anomaly per month.

To calculate reasonable offsets for shorter length surface stations, an algorithm was created that starts from the earliest 1957 ground station records and works its way forward. When a new station is introduced, it finds the absolute closest already offset station and computes an offset for the new station in the hopes that we can remove the step. Unfortunately, the noise level of the data makes the whole process less simple than we might imagine. First, the no offset anomaly data looks like this.

Figure 1, no offset temperature data

The trend for this data is 0.052 Deg C/Decade. This or a version of this, is the most important super simple version reconstruction as it is verifying the validity of the complex algorithms both in spatial distribution and magnitude. The regression modes used do not have the ability to offset station data to create trends. The question then becomes, how much of a problem is that, because we want to make sure we’re doing good work and not just adding on to an improved hockey stick paper or something.

After creating the algorithm for calculating the step in the data, the mean of the overlapping data of both stations was used to calculate the offset. I found that final trend was sensitive to the amount of overlap use to compute the offset. By overlap, I mean the number of months of data points available at both stations.

If we use all of the overlap data to compute the offset, the result is almost the exact same thing as the no – offset data. This makes sense as the ground stations which have long histories control the trend and make sure the short record stations have mean values corresponding to the long station trends. Consider what happens when thermometer 2 were centered (mean) perfectly on the graph of thermometer 1. Even if 2 had a steep trend in comparison, the net trend wouldn’t change much. On checking the amount of overlap I found that most stations had very long records of overlapping data with the next closest station, therefore the full record length is not ideal for determining a true trend as differences would be suppressed.

Figure 2 is the full record overlapped data. The trend for this was – 0.057 C/Decade which is very similar to the 0.052 for the anomaly only data. This is a good confirmation of the reasonableness of the method. You can see there is very little difference between Figure 2 and 1.

.

Figure 2, Maximum Possible Overlap

So the question becomes, what is a reasonable level of overlap, too much gives no difference, too little and we get ridiculous results. It seemed impossible from poking in different numbers and getting different trends over and over. However, after a few dozen attempts, I decided to plot the trend vs months of overlap in reconstructions. Each reconstruction takes 30 seconds or so so when I ran about 100 of them it was a clear case of computer abuse but the graph is interesting.

Figure 3, Antarctic Continental Trend vs Overlapping Months

So this graph that the number of overlapping months used to compute the offset has an effect on trend. Fortunately the trend starts from as low as zero and even negative overall trends and launches as more months are added to a plateau of around 0.7C/Decade. After enough months are added it drops back down to the no-offset average of around 0.5C/Decade. The plateau saves our butt, it appears that the true trend according to the thermometer data is actually about 0.065 Degrees C/Decade and is reasonably independant of months average for a fairly wide range. The result, however, is exciting in that it makes very little difference if these offsets are accounted for or not. Figure 4 is a plot of the Antarctic area weighted trend with a 60 month maximum overlap as chosen according to the Figure 3 graph.

Figure 4, Antarctic Continental Trend with Confidence Limits

The temperature slope distribution is shown in Figure 5.

Figure 5, Antarctic temperature trends with station offsets calculated from 5 year data overlap

The trend from this version is 0.067 +/- 0.0848 C/Decade.

So what does it mean Jeff, right?

Well, the change in trend from reasonable accounting of station offsets results in a warming trend of very slightly higher than the far simpler no-offset anomaly averaging used previously. Considering that Hu had the continental confidence interval at +/- 0.10 C/Decade the difference of 0.015 C/Decade is hardly worth mentioning. However, the various reconstructions take the basic form of Ax=b. The x value is a multiplier matrix which converts the surface stations into the missing data. This matrix is complex in time for RegEM which actually helps fit temp data to principal components in RegEM (an unnatural calculation as noted by RomanM a long time ago). You’ll notice there is no method in the equation for compensating for offsets in anomaly (i.e. Ax +c = b). Therefore, if I’m understanding everything correctly, different forms of expectation maximization and regression, when correctly applied, should converge to a value similar to the no-offset solution trend of 0.052 and contain spatial distributions more similar to Figure 1 than Figure 5.

For reference, Figure 6 is the Regularized least squares version of the reconstruction. There have been many but this one does the best job visually of localizing individual stations. I need to do a post explaining why it’s superior to RegEM for this reconstruction but that will be for a later project.

Figure 6

1. ### Ryan Osaid

Damn if that doesn’t look good. :D

2. ### Jenssaid

Let us not forget why the Steig paper made such a splash in the media. Antarctica has been a stone in the shoe for the AGW proponents. It does not behave as it should. The sea ice is not melting and the temperature is not increasing. That annoying behavior is still obvious in your figure 4 (and in Steig’s figures): The slight warming takes place before 1970, when the Arctic and the globe on average were cooling, and there is no warming in the period from 1970 to 2000 when the rest of the globe was getting hot.

3. ### Terrysaid

Jens I agree. I still cant for the life of me understand why the entire focus has to be on the entire record when there is clearly not a linear trend. I fully understand Jeff and Ryan’s point of replicating the published data, but for my mind Steig et al would have been better to look at the actual behaviour of the anomaly rather than fitting a linear trend where there appears to be a fair bit of faith required to assume that it it linear.

4. ### Carricksaid

Can you post the raw data for Figure 4? If you can’t upload it to your site, I can provide you a url where you can host the file (or you can email it to me and I will post it with your permission and provide the url here).

Basically I’m interested in looking at the fit 1960-1980 and 1980-current.

Thank you and nice work.

5. ### Mark Tsaid

I still cant for the life of me understand why the entire focus has to be on the entire record when there is clearly not a linear trend.

Because a linear trend is a) the assumption and b) the only way they can get the results they want. Any higher order function doesn’t show continual warming. Even the Nature paper made that clear, but did we see any of that in the press release? Nope… Instead we get cherry-picked endpoints and “Oh my gosh! It really is warming!” Just put Antarctica right next to the MWP in the “things AGW theory cannot possibly explain so we willed them away with magic statistics” folder.

Mark

6. ### Jeff Idsaid

Carrick,

I can get to it after work for you.

7. ### Kenneth Fritschsaid

Instead we get cherry-picked endpoints and “Oh my gosh! It really is warming!”

Mark, I think it is a little more subtle than a straight cherry pick here because the use of 1957 start date can always be rationalized as the start of the manned weather records in the Antarctica. In fact a retort from the defenders of Steig et al. might well call our sensitivity testing to start dates as cherry picking.

My point here, as it was in the case of the choice of a start date for trends in TC activities in the NATL of 1970 being in a cyclical valley and the present time near a peak, that regardless of the apparent legitimacy of the start date selection, we still need to look beyond that choice and so should the authors using those start dates. The NATL start date is always rationalized as being the start of the period of more accurate TC activity records.

Ok, I know you guys are tired of area weighted reconstructions but for those who say publish, the detail of this is important.

Never do I tire of posts like this one – in fact I find it the very essence of your blog advantage. Jeff ID, have you considered polling your audience as to their preferences?

I know you need your blog for venting. I do mine by yelling at the TV – be it a political or economic commenter or one of my favorite sports teams.

Also, I would like to see the CIs for trends when you report them.

Finally, it might be informative to do a breakpoint analysis of the time series from the various reconstructions.

8. ### Jeff Idsaid

Terry,

It’s not linear, however a linear fit is basically the lowest order filter which can be applied to the data. If there is a warming signal under the assumed and alleged weather noise, the linear trend is a good way to see it. If the linear trend doesn’t show warming in 50 years, the models have a bit of work to do. I’m no modelologist but it seems warming of the west antarctic in particular is of concern for agreement with models.

9. ### Layman Lurkersaid

#7 Kenneth Fritsch

Good point Kenneth. Even if the 50 year trend shows warming, I don’t believe there is sufficient data to rule out low frequency ocsilations – patterns with long term persistance (ala Koutsoyanis).

10. ### Jeff Idsaid

Kenneth,

We’re kind of narrowing in on a final reconstruction method. Nic found another detail which needed work and that’s thrown a bit of a wrench into things. When everyone is happy with a final result I’ll do a post with various trends, and a more complete look at what’s really happening in the Antarctic. This post was one of my favorites because of Fig 3. Everything looks reasonable, and I have a great deal of confidence that we’ve got a good target trend set to check our work.

As far as polling, I do it every day. WordPress provides stats in the background on every post. My last area weighted post which took hours only got a few hundred views compared to the ten points post which took an hour and had a thousand in a short timeframe. Still I need to be a little selfish and do the math that I enjoy.

You should be proud that I haven’t even mentioned the horrendous evil health care bill. :) Since the science posts take so much time and blogs are basically radio transmissions it’s difficult to output enough to keep people interested. So, as a suggestion, hehe, I’m happy to accept a science related guest post on any weather topic from people who have something to add. It doesn’t need to be earth shattering either, some stats some weird paper people have read.

I’ve felt that several of your comments on CA could have been written as head posts here with little more work for people to discuss and consider. There would be a lot less time for venting about whatever is bugging me.

Caught in a recruiting trap now eh?! Blogland rules!

I’ve tried to get Ryan, Jeff C and Nic to also consider it for the same reasons. TonyB did that amazing historic post which was slow starting but has had some of the best reading and comments. Some other blog linked to it last week and sent a couple hundred people over.

11. ### Ryan Osaid

Yah, I think once I finally un-screw my code I’ll have something to post.

BTW, the Peninsula is somewhat of a problem for the models. If you examine the ModelE results in Steig’s paper, you’ll see that ModelE predicts more warming for the continent than the Peninsula when forcings are included. However, ModelE does okay when forcings are excluded. So something’s not quite right with ModelE.

12. ### Anomaly Aversion « the Air Ventsaid

[…] and self appointed skeptic of month for November 09, I calculated the continental trend using area weighted offset anomalies.  This method increased the trend from 0.05 (simple average) to 0.06.  Consider that Ryan and Nic […]

13. ### Comparison and Critique of Anomaly Combination Methods « the Air Ventsaid

[…] Area Weighted Antarctic Reconstructions […]