Well the Red Wings won, they got their crap together and played at least twice as good as they have for the past 5 months. It’s good to see guys who want to win.
In the meantime I ran a little RegEM 3PC reconstruction based on the raw satellite data but this time it’s without the peninsula.
The first concern is to create the mask, to decide what a reasonable mask is. Anthony Watt’s linked to this plot (Figure 1) of temperatures in the Antarctic. Before I show the plot, I have to remark that those of us who work in anomaly for too long really lose sight of the temperature gradient in the anomaly.
Anyway, I think the plot of ‘acutal’ temperature of the antarctic is more telling than the anomaly in this case. This is what Anthony Watts left as a comment on my area based antarctic reconstruction.
There has been some discussion that the Aleutian peninsula might be better served if newly classified as EM (Maritime Polar) This would separate relatively mild marine locations such as Ushuaia, Argentina and the outer Aleutian Islands like Unalaska The climate of Unalaska from the colder, continental climates. The mean annual temperature for Unalaska is about 38 °F (3.4 °C), being about 30°F (−1.1°C) in January and about 52°F (11.1°C) in August. With about 250 rainy days a year,
Contrast that to interior Alaska temperatures which are not moderated by the presence of the sea. Fairbanks for example has an mean annual temperature for Fairbanks is 26.9°F (-2.8°C) and with 106 rainy days.
Using the Unalaska to Fairbanks comparison, the Antarctic peninsula would be a candidate for this new “Maritime Polar” (EM) classification IMHO.
In support of that, here is a seasonal temperature map submitted to Wikipedia by Stoat’s William Connelly
You can see the tip of the peninsula has temperatures more similar to the ocean than the continent. In addition it has been suspected by many here and at other blogs that the warming trend from the peninsula was smeared across the continent creating the net warming of the antarctic.
Since we can do a reasonable recreation of the Steig et al. antarctic reconstruction, I used the same code and modified it to mask the satellite data at the tip of the peninsula. Figure 2 is the mask and surface stations used.
The black area of the peninsula wasn’t used. Blue dots are the 24 non peninsula surface stations. The 3 pc’s returned from RegEM looked a bit more normal. (Edited thx to Nic L), there are also two southern (off the bottom of map) surface stations included in this reconstruction).
The Antarctic trend after 3 pc RegEM Figure 5.
For comparison to the complete Antarctic using the same math including the peninsula stations to reconstruct Steig’s results gave 0.108 C/Decade. (Figure 6)
The temperature trend changed by 0.034 C/Decade.
The temperature distribution across the antarctic is shown in Figure 7.
It’s good to see some cooling related to the south pole station. I really don’t like RegEM because of the smearing of station data and the handling of inverse correlations. So it’s challenging to push myself to make more plots like this. I did run a trend graph from 1967 onward as I’ve done in some of the more straightforward reconstructions.
So we see a similar pattern to the other RegEM’s where the post 1967 trend is still positive while all surface station data is basically flat. Jeff C referred to this as temporal smearing. Keep in mind that the post 1982 values of any RegEM method so far consist entirely of the 3PC breakdown of Comiso AVHRR TIR satellite data while pre 1982 consist of the mashed up surface station data.
In the end the trend from the peninsula dropped 68% from the no peninsula reconstruction to a more reasonable value (which I still don’t believe has much to do with reality). I’ll redo the spatial weighted reconstructions next with no peninsula so we have something to compare it too.
Also, I’ve decided to retrend (recalibrate) the satellite data and surface station data grid point by grid point so all stations and sat gridcells match long term trend and rerun RegEM. In a reasonable reconstruction method, the trend should come out to whatever value I set all the data to. The alternate reconstructions I made would prove that out. Does anyone want to guess what the RegEM reconstruction trend will be?
Just to be mean maybe I’ll set all the trends sligthly negative, that has a chance of making a bit of drama.
the Air Vent 
So it is clear now that rank 3 RegEM does not recognize that the 20 pre-1982 peninsula stations originated from less than 5% of the continent. Had RegEM not been constrained from processing to higher order reconstructions, it might not have been a “problem free” reconstruction but at least the spatial distribution of warming would have been more regional and the overall trend closer to reality.
“spatial distribution of warming would have been more regional”
I believe it would only appear more regional because the high amplitude high frequency signal used for correlation would be localized. However long term trends would still be inverted and smeared.
If I’m wrong then linearly re-trending all the data to the same slope and applying RegEM will produce the same result as my area weighted reconstructions.
Great work on deconstructing this paper, and on your blog in general. A small gripe, though, about some of your typos, as in the above “loose site” comment. I know spell checking doesn’t catch grammar or non sequiturs but I’d appreciate it if you would proof read your posts a little more carefully. It pains me to imagine the sniggering over at RC when they “lose sight” of your logical points because careless commentary. Or maybe I’m being “sligthly” negative because I have nothing to “compare it too.”
Agreed. IMO it would still “random distribution of trend” as you coined before. Just a higher order version of the same correlation and smearing issues. But higher order processing necessarily confines the trends regionally. If peninsula warming is a regional phenomenon with little connection to the continent, then any attempt to reconstruct cannot constrain spatial distribution of trend to broad based, low order patterns without it causing smearing.
It seems you are now moving toward sensitivity “if/then scenario” type analysis to verify exactly what the method defects are.
“I believe it would only appear more regional because the high amplitude high frequency signal used for correlation would be localized. However long term trends would still be inverted and smeared.”
You’re conflating two different issues here. And note that we have been VERY critical of trend matching in the past that lacked high freq correlation.
#5 Actually I’m attempting to point out the conflation problem created by the assumptions of RegEM correlation on spatially correlated high noise data. Noise being local monthly weather fluctuations.
Excellent work, Jeff. Could I just ask you to clarify a couple of things?
You say just above Figure 4 “The 3 pc’s inserted into RegEM looked a bit more normal”. Aren’t the 3 PCs inserted into RegEM those from the satellite data, which only start in 1982. As Figure 4 shows PCs from 1957 on, are these in fact the 3 (satellite data) PCs from the RegEM reconstruction output? And although these PCs will be the same as the input PCs from 1982 on, I think that the original reconstruction PCs only looked abnormal in the 1957-1981 period?
Also, you say in relation to Figure 2 (actually titled Figure 3, I think) “Blue dots are the 24 non peninsula and non-island surface stations”. I count 21 blue dots – I can see that Scott Base and McMurdo would only show up as one dot but that only makes 22 (which is the number of non- peninsular, non-island surface stations that I used when looking at this issue). Can you clarify, please?
From a visual inspection, the 3 PCs in Figure 4, post 1981, look very like those from the entire satellite data without the peninsular masked. And I note that the reduction in the 1957-2006 average trend that you get as a result of both masking the pensinsular satellite data and excluding the peninsular and island surface stations, at 0.034 C/decade, is similar to the 0.038 C/decade reduction I obtained when using the 1982-2006 PCs from the Steig’s reconstruction but with the 15 peninsular and 5 island surface stations excluded from the RegEM data set (see my post #41 on your “Steig AVHRR Reconstruction from Satellite Data” thread). As I was using the same 3 satellite PCs from Steig’s reconstruction in both the full and the ex peninsular and island surface stations cases, it seems to me that the key factor may be excluding the peninsular and island surface stations rather than masking the peninsular satellite data. That interpretation appears to be supported by the fact that the 3 satellite PCs that I obtained in the ex peninsular and island surface stations case look very similar to those that you show in this post for the 1957-1981 period (as well as for the 1982-2006 period, as stated above). What do you think?
On a separate matter, I was concerned to learn that Roman M (I think) appears to have found differences between results using the Matlab Regem (TTLS) function and Steve M’s R version thereof (which is what I have been using). Hopefully Roman and/or Steve will get to the bottom of this. In the meantime, can you confirm which version of RegEM you used for your reconstruction, please?
To follow up on #7, did you get my email with the infilling done with R instead of Matlab?
Also, just to let you know what I’m working on at the moment . . . presently I’m attempting to duplicate Steig’s reconstruction statistics. I’ve got a script that will do the necessary Monte Carlo simulations, but the problem is that the real “noise” in the time series for the Antarctic is not AR(1). More to follow on how I determined that. Anyway, by using an AR(1) process for their Monte Carlo simulation, Steig et al. end up overstating their verification statistics.
Once we are able to duplicate their verification statistics, we can directly compare how alternate methods result in better/poorer verification.
#8 I’m going to have a chance tomorrow to work on your recon. I won’t forget.
Regarding the ARMA, there are some methods recommended in texts as to the best methods for determing ARMA coefficients. I think they are basically try until residuals are minimized but I don’t remember. I’m not surprised about overstating verification.
Did you determine the coefficients by the R process? Tamino did his own method which was an interesting couple of posts and was worth reading for me. I found his method resulted in simulations with too high a standard deviation. Which just happened to conveniently help his conclusion.
#7 good questions, give me a short time to figure all the answers out.
#7, I edited some of the text per your comment.
First it should stat RegEM output PC’s looked more normal. The second issue is that the 24 stations included two that are off the southern edge of the map. I’m not certain how Steig et al. handled these points but I think they are part of the original 42 and I needed to do the same reconstruction to compare the results.
Perhaps you could shed some light on these. I believe they are Campbell and Macquarie. Good catch though.
I agree that elimination of the surface stations is the key. I actually tried it both ways, the sat data elimination didn’t have that much effect from memory the trend was like 0.95 or something.
I’ve been using the matlab version of RegEM.
Ryan,I posted some correlations of the residuals at CA with the monthly and annual versions of the Steig et al. TIR and AWS reconstructions and I would agree that you see higher order significant correlations. Hu M and Bender made some comments on handling corrections for the standard errors in these cases. Of course, if you are modeling them with ARMA and doing Monte Carlo simulations you should be able answer these questions without a formulaic approach. I am looking forward to seeing your analysis.
#11 Actually, it was your posts that even made me look at it. AR(1) is clearly not right. AR(2) or AR(3) may work.
#12 He will do that to you.
#10, Campbell and Macquarie were indeed included in Steig’s reconstruction, but as they are islands well away from the antarctic continent, with latitudes of 52S and 54.5S, there seems little justification for his inclusion of them. Hence my removing these two island as well as the three near the peninsula, in addition to those on the peninsula, when attempting to produce a reconstruction that represents temperatures on the main land mass (as opposed to trying to reproduce Steig’s reconstruction). Temperature trends on both the peninsula and the islands are likely to be affected by different factors than those affecting the main antarctic land mass.
#14, I couldn’t agree more. I haven’t plotted them individually but I wonder why they were included.
My guess is that the Campbell and Macquarie have a high trend.
#14 In my opinion, Grytviken also should not be included.
#14 Also, I forgot Signy Island and Orcadas. Both are off the tip of the Peninsula and in the middle of the ocean. They have no business being in an analysis of the continent.
#15 I don’t know if they have a high trend or not, but their correlation with the continent is suspect. Including them in the 42-station RegEM weights ocean stations (Campbell, Macquarie, Grytviken, Signy, and Orcadas) as 12% of the total stations used to form the covariance matrix and increases the chance of spurious correlations.
Earlier I had run Steve’s RegEM and done a reconstruction without them. The overall trend ended up being about .108/decade (if I remember properly).
#18, I agree. But I still bet on high trend b/c there isn’t any reason why they should be included.
#19 I wouldn’t bet against you. 😉
#15, #16, #17 Using a simple measure of trend (C/decade)in anomalies I get -0.011 for Campbell (data for 1961-1999 only), +0.051 for Macquarie , +0.113 for Grytviken, +0.215 for Orcadas and +0.125 for Signy (data for 1957-1995 only).
Speaking of trends, I have noticed that if one calculates the antarctic average trend for Steig’s reconstruction for the sub-periods 1957-1981 (being 0.241) and 1982-2006 (being 0.260), they are both slightly over double the trend for the whole period (0.118). This oddity appears to be due entirely to PC1, which is much higher towards the end of 1981 than it is soon after the beginning of 1982.
#21, We’re getting close to something here, the key for the island stations is the pre-1982 trend since the post 82 data is only used for correlation for which the small trend has little use.
Calculate the pre 1982 trend for the islands, or wait 10 minutes and I’ll put it up :).
#22 For 1957-1981 I get -0.011 for Campbell (data for 1961-1981), +0.213 for Macquarie , +0.147 for Grytviken, -0.013 for Orcadas and -0.006 for Signy (data for 1957-1995 only).
#23, I’ve made mistakes before, I’ll check Campbell.
Who’s anomaly data are you using?
#22 A number of the peninsula stations have high slopes for 1957-1981: that presumably ties in with their influencing the trend of the whole reconstruction so much. Eg Deception +1.313 (1957-1967), FaradayV +0.595, Marambio +0.896 (1971-1981). Interestingly, if the 15 pensinsula and 5 island stations are excluded from the reconstruction the shape of PC3 changes, with the pre-1982 part no longer having much smaller fluctuations than the 1982 on part. This change seems to be linked primarily to the peninsula, rather than to the island, stations.
#23, I got the same thing as my next post. My version of Campbell is an upslope???
#24 Sorry, I meant to write +0.155 for Campbell to Dec 1981; -0.011 was the full period trend.
I am using READER data that I downloaded from the BAS website. According to the BAS, there have been no changes to that data since before Steig would have downloaded it.