Measuring the Clouds
Posted by Jeff Id on March 31, 2009
Guest post by Jeff C
Jeff C has located a possible problem with cloud contamination of the surface dataset. This is an ongoing investigation but there are some apparent differences in result from the most recent NOAA 16 satellite as compared to the rest. This is an important issue as the methods section of Steig’s Antarctic paper cites clouds as the single greatest source of error.
Cloud masking is probably the largest source of error in the retrieval of TIR data from raw satellite spectral information. Wehave updated the data throughout 2006, using an enhanced cloud-masking technique to give better fidelity with existing occupied and automatic weather station data. We make use of the cloud masking in ref. 8 but impose an additional restriction that requires that daily anomalies be within a threshold of 610 uC of climatology, a conservative technique that will tend to damp extreme values and, hence, minimize trends29. Values that fall outside the threshold are removed.
As you might have guessed, there is not much information about the ‘enhanced’ masking technique given so the question becomes, what about the data.
In the previous post HERE , I plotted cloud cover percentages over time using the “cldtype” (cloudtype) parameter of the U Wisconsin AVHRR dataset. Cloud type is a bit of a misnomer, as the data is actually something known as “cloud fraction”. Cloud fraction is a value from 0 to 1 that expresses the frequency of cloud cover. It is very similar to the cloud mask, but not identical. Here are the cloud types that would be displayed if we were looking at the cloud cover for a single day/time:
0 – clear
1 – partly clear
2 – warm water cloud
3 – super cooled water cloud
4 – opaque ice cloud
5 – non overlapped cirrus cloud
6 – overlapped cirrus cloud
However, we are looking at monthly means, not single day/time data. Since cloud cover changes from day to day, the monthly means can’t list a single cloud type. Instead it lists the cloud fraction; the percentage of time the cloud cover was any of #1 through #6 (i.e. not clear sky). The cloudtype algorithm is not contained in the released version of CASPR (Cloud and Surface Parameter Retrieval algorithm version 4.03) and is not described in any of the documentation supplied in the released version. Cloudtype is included in the latest unreleased version CASPR (version 5.0), which was used in generating the UWisc dataset I have been reviewing.
Since the cldtype parameter is new, poorly documented, and giving funny results, I decided to generate new plots from the “cmask” (cloud mask) dataset from the UWisc server. These are different files from those containing the temperature and the cloud type information; I just recently noticed them on the server and downloaded them last night. Below is the same plot as in the previous post, but with cloud mask (black) and cloud type (red) overlayed.
The traces are quite different, and seemingly out of phase with each other. The large discontinuities have also shrunk with the exception of the step at the 2000 to 2001 transition. Clearly there is something wrong with the “cldtype” data. After checking with some knowledgeable sources, it appears that the cloud type data doesn’t have a correction applied for the diurnal drift that occurs over the life of the spacecraft. This may be by design, or it could be a bug in the cloudtype algorithm of CASPR version 5.0. It doesn’t seem like the data is very useful without the correction applied.
Here is the same cloudmask plot (the set that does appear to have the correction applied) with the spacecraft transition from NOAA-14 to NOAA-16 shown. It has a large discontinuity that did not disappear when using the cloudmask data instead of cloud type data.
I was gratified to see that the cloudmask data is much cleaner than cloudtype as now we may be able to actually make something of it. First off, here is a cloud-cover map from 1982 to 2001 for Antarctica. This covers the era up to the large discontinuity in the plot above (the transition from NOAA-14 to NOAA-16).
Note there is no blue on the plot as cloud cover doesn’t drop much below 50%. If you plot cloud cover during a shorter period of this same time frame, you get virtually the exact same result, the pattern is very consistent. Here is 1990 to 2000.
Do these patterns look at all familiar? They certainly did to me. The Comiso data (also known as the Steig “raw” data) has a virtually identical temperature anomaly trend pattern over the same period. Here is the Comiso trend plot accompanied by a similar 1982-1999 trend plot from Steig’s SI.
Note how the areas of heavy clouds correspond very close to Comiso’s areas of warming. I’m not sure why this would be, but it seems that it is probably more than coincidental.
There is also another oddity. The cloud pattern over NOAA-16 (2001 to 2004 in this data set) has a distinctly different cloud pattern than that seen previously. This is not a huge surprise as the plot over time shows a noticeably different signature during this period. Clearly there is a major difference in the cloud detection/masking of data from the NOAA-16 satellite.
Interestingly, Dr. Comiso’s trend plot over this period also shows a dramatic change. It is not as close a match to the cloud patterns as those seen previously, but it is close enough that it makes me wonder if the Comiso dataset is cloud contaminated.
The abrupt change from 2001 on is also troubling as Comiso’s dataset has a net continental warming trend over this period; the UWisc dataset has pronounced cooling.