the Air Vent

Because the world needs another opinion

Stats Ain’t Everything

Posted by Jeff Id on September 19, 2009

Ok, here is an obvously lousy reconstruction of Antarctic temps. Besides the trend being0.10 C/Decade the distribution of trend is astoundingly bad. The difference is, it has very nice verification stats.

After watching each pixel converge using a variety of positive matrix factorizations, I realized that one of the main problems is the satellite calibration to ground data. There are obvious and large offsets in the satellite data when compared to ground stations. There are a lot of ways to calibrate the data. Ryan had a sophisticated step detection using statistical tests followed by calibration of steps outside the predetermined limit. Here I took the difference between satellite and ground data at 62 locations and filtered it by 12 month filter and subtracted that difference from the entire AVHRR dataset.

The filtered difference curve looks like this.

sat minus ground

Calibration Curve

After making this adjustment which removes the big steps in the satellite record fixing the exaggerated long term trend, the regression was visibly more accurate. I expected not only better verification stats but an improved long term trend match to the area weighted version due to better location of surface station information.

Figure 2 shows the 50 year trend calculated using this method.

PMF calibrated satellite 1957-2007

Figure 2 Trends 1957-2007

The trend was 0.106 C/Decade but obviously the trend map isn’t very accurate. The unique thing about it was that the high trend we usually see in the peninsula is spread around all the edges of the continent. Temperatures at the edges are substantially less variable than those in the center. Since the circumpolar current is continuous around Antarctica we might expect the edges of the continent to be more similar in temperature and therefore more likely to be associated with continental edges.

Well the crazy thing about this crap reconstruction is the verification plot. Figure 3 is r^2 match from ground to satellite data.

Figure 3

Figure 3

The mean r^2 is 0.57 which is pretty good.

The 1982 -2007 portion of the reconstruction is basically a normal regression. Values pre-1982 are an extrapolation. It therefore makes sense to look at the post 1982 trends in comparison to AVHRR data.

PMF calibrated satellite 1982-2007

Figure 4

Compare to the recalibrated satellite AVHRR data trends.

satellite 1982-2007

Figure 5

The differences in trend aren’t terribly surprising because the noise in the data is pretty high. Anyway, it took several days to make the pretty red picture in #3 even though it doesn’t seem to have much meaning. I’ve got a couple more tricks to try before I give up on this method.

5 Responses to “Stats Ain’t Everything”

  1. MikeN said

    How are you deciding on what methods to use?
    The results after you use a method shouldn’t inform things.

  2. Jeff Id said

    Ryan and Nic have already got a good method, I was exploring some other techniques which I thought might work well. They haven’t been applied anywhere before. From the Area weighted recon I know what a good reconstruction should approximate and can tell this one didn’t work well. The verification stats would be excellent though. Far better than Steig from what I can tell, it wasn’t worth finishing though — cause its crap.

  3. MikeN said

    How do you know what a good reconstruction should approximate?

  4. Jeff Id said

    #3 We only have the actual temperature stations as a check. I figure those are what we should go by.

  5. Layman Lurker said

    Off topic: I just watched a documentary (BBC I think) on You Tube called “The Lost World of Lake Vostok”. This is one of the coolest science documentaries I have seen in a long time. Check it out

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: