Data is dangerous as many a paleoclimatology paper can demonstrate. Big data is statistically more dangerous, and the satellite sea ice data record is a multi-gigabyte set. Some years ago, your not-so-terribly-humble correspondent made a small error which was published here, and at WUWT. I downloaded the global satellite data for sea ice and did not correctly apply the missing “hole” in the data for the entire time series. The result was an inaccurate trend and associated conclusions. It was of course, embarrassing, and I did apologize quickly and fixed the data set. Any scientist would. The reason for the problem was that the documentation at the NSIDC wasn’t really up to snuff and I was new to the data. They very efficiently and quickly corrected the metadata record. Grant Foster, who posts as Tamino on the inappropriately named “open mind” blog, took a couple hours of his day to write a teasing post on the error.
Well today, on perusing the NSIDC site, I received a little redeeming gift. It turns out that even the NSIDC which is comprised of the literal best experts on the sea ice data, is capable of the exact same mistake I made on my first ever compilation of the massive dataset:
Correction in Application of SMMR Pole Hole for Daily Data and its Consequences
In March 2015, while editing the data processing code to apply the SSMIS pole hole mask, it was found that the Sea Ice Index processing code had been applying the SMMR pole hole mask over the entire Sea Ice Index time series when computing the daily extent numbers instead of using the SSM/I pole hole mask beginning in July 1987 as the monthly data processing does. Because it is assumed that the entire region under the pole hole mask is ice-covered and because the SMMR pole hole mask is relatively large, some sections of open water, regions with less than 15 percent ice concentration, were being unknowingly included as sea ice in the daily Arctic-wide ice extent number.
At the time of this mask correction, 132 incursions of open water into the SMMR pole hole mask had taken place in recent years. The first occurred in 1987. However, until 2007, all of these incursions had areas of less than .01 million km2 which is below the precision of these data. Between 1987 and 2013, the size of the open water in the SMMR pole hole mask ranged in area from 1800 km2 to 10,000 km2. Figure 19 shows the worst-case scenario, 22 September 2009, where approximately 10,000 km2 crept into the SMMR pole hole area (red area in Figure 19).
The version 1.2 release of the Sea Ice Index corrects this processing oversight. All concentration data up to the latitude of the appropriate pole hole are used in the calculation of Arctic-wide extent. As a result of this change, there are slight decreases in the corrected daily extent number for some days in the record. These small changes do not affect the monthly anomalies and trends, however, which are reported to two significant figures.
Again, they professionally and scientifically announced and corrected the record. Kudos again NSIDC.