the Air Vent

Because the world needs another opinion

Comiso’s Data

Posted by Jeff Id on March 28, 2009

Sometimes there are surprises in reconstructing papers. I can’t claim to have spent the time like this before but as I do now I’m learning. Our friend the Laymen asked for a plot of the new data as compared to surface station data. It’s rather impressive.  The data in this post is from Steig O9 raw data compared with the closest gridcell to the surface station..

Comiso is an Antarctic Jedi. I don’t know how he did it but the data he produced is of fantastic quality. Check out these plots. There’s no facetiousness here, this is just plain impressive. I have to know how he did it.









Ok, I’m tired. How did he turn NSIDC data which looks like this


Into this


It’s really quite good.

9 Responses to “Comiso’s Data”

  1. Layman Lurker said

    So how did Dr. Comiso do it? That is a good question. A good start would be to select a set of individual grids – it could be the surface station grids – and look at the difference plots of pre and post AVHRR cloud masking. This, I think, would identify where the masking and infilling occured, and what the infilled values were. Then you could experiment with methods for replicating infilled values like RegEm on raw AVHRR data for example.

    When Jeff C. tried to run the original data into RegEm he said it crashed, even when data was reduced to monthly values. Could this be due to the temporal gaps (in 1994 and 2001 I think)? FWIW maybe you could try plugging in some surface or AWS data into these gaps. Just a thought.

  2. Jeff Id said

    I’m certain Jeff infilled the gaps using mean values or by removal. I will be running RegEM soon with 3 and up PC’s from the data. There’s no reason to run the whole dataset as Jeff did.

    There is a bit of foundation work to do before I can get it to run and I’m feeling a little lazy tonight BC I really worked hard yesterday on the Steig PC analysis and when I saw JeffC’s plot I put another two hours in.

    The great thing about today was that we get to continue forward. The AVHRR raw data looks to be in excellent condition from whatever method Comiso used. Their paper is extremely vague on sat data processing but when I see all these stations matching so well it gives me some confidence that this might be the single best dataset for the 1982 on Antarctic trend in existence.

    It looks good and I still want code and data!

  3. Jeff Id said

    BTW the Steig data has no gaps.

  4. Jeff C. said

    Some are much better matched than others, but they all match quite well. Amundsen-Scott looks almost identical, Marimbio is off by a bit.

    I think it has to be either one of two things. It could be that his cloud masking really is much better than that used in the NSIDC data and the daily threshold of +/- 10 deg C makes a big difference. Following up on Layman’s point, do you omit those days when calculating a monthly mean or attempt some sort of infilling? If you infill, hat method do you use for infilling?

    The other method would be some sort of surface calibration. I had experimented with using Amundsen-Scott to offset the entire grid, but that caused poor agreement with the other stations. I suppose you could use some sort of regional calibration with the offset transitioning between zones.

    Whatever he did, it is impressive. Jeff – how hard is it to run all of the stations? It might be interesting to see if there is a regional pattern to the level of agreement. Let me know if I can help.

  5. Jeff C. said

    #1 and #2 – I was trying to use RegEM to fill the missing months in the NSIDC data. No luck at 5509 series or even using only a quarter of them.

  6. Mike Davis said

    If it looks to good to be true! You guys have been doing a good job!

  7. Layman Lurker said

    “There is a bit of foundation work to do before I can get it to run and I’m feeling a little lazy tonight BC I really worked hard yesterday on the Steig PC analysis and when I saw JeffC’s plot I put another two hours in.”

    You have just given me another reason to put off learning R! 😉

  8. Jeff Id said

    #7 Oh come on, just imagine the entertainment of having 1,000 people tell you you screwed up. How fun is that!

  9. Layman Lurker said


    “BTW the Steig data has no gaps.”

    Yes but if he infilled for cloud masking then he should have used the NSIDC – pre processed data. The thing that I find intriuging about this is that eigenvectors for this data did not exhibit the autocorrelation artifacts, and ocean pixels notwithstanding, may have potential for realistic values for infilling.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: