I’m getting better with R now and am able to perform much more flexible calculations (It’s starting to get fun). After noticing the discontinuity in the satellite data difference graphs from my previous posts I plotted the trend before and after the discontinuity I found the raw data on either side of this discontinuity is in excellent agreement between RSS and UAH. I was interested in making a correction to this discontinuity and didn’t want to just pick one or the other. leading to the following post which uses rescaled GISS ground data to re-trend the area of the discontinuity.
Since the Chinese censorship made it nearly impossible to post, I had a bunch of time to consider a variety of possibilities for this post and really had a lot of time to confirm the math and try different methods. After a lot of consideration, I think the result is fairly substantial in its implications and is one of my better works here.
For those who are really laymen, do not pay attention to the graph offsets, absolute value is not the point of anomaly data. Only the trend is important.
——————————————————–
UAH and RSS Lower Troposphere Temperature Anomaly Trend Homogenization Using GISS Ground Measurement Data.
Over the years the reasons for number of discrepancies between UAH and RSS have been identified and corrected. A plot and linear fit of the trends of the latest versions from the NOAA reveals that there is a statistically significant difference between the trends of the two metrics.
By analyzing the stationary weather variation level of the difference of unfiltered data between GISS and Satellite measurements a 95% two sigma slope variance of +/- 0.002C/year. Figure 1 depicts the two metrics with least squares linear trend overlay’s. The difference in slope of the anomaly plots per century RSS-UAH is 0.003 deg C/Year. This places the trend difference of the two satellite metrics well outside of the instrumental noise variation level.
It has been noted that by subtraction of the UAH from the RSS metric a visible step in the data at about 1992.5 becomes apparent (black circle). The green line in Figure 1 is a plot of the UAH – RSS data filtered and offset -0.5C for better visibility of the step.
Figure 1.
The apparent slope of the difference between the metrics is visibly flat both before and after the discontinuity possibly indicating a strong degree of trend correlation in the unfiltered raw data. Figure 2 demonstrates the high degree of accuracy of the match of RSS to UAH by splitting the data, removing the sudden offset at 1992.5 and graphing the trends. The Red and Black lines are the UAH and RSS trend and data respectively. The data from 1991 – 1994 was removed for trend analysis. Although the trend between the early data has some minor divergence, another removal of the earliest portion of the satellite record confirms a high degree of trend accuracy prior to the 1992 step (not plotted). The Blue and Green lines represent the later years of satellite trend, blue and green trend lines match well and exhibit slopes within a very small fraction of each other.
Table 1 depicts the early and late slope information as well as the full raw data trend lines
UAH Total — 0.0127 C/yr
RSS Total — 0.0157 C/yr
UAH Early — 0.008 C/yr
RSS Early — 0.01 C/yr
UAH Late — 0.0123 C/yr
RSS Late — 0.0129 C/yr
From the table above, the RSS early and UAH early differ by 0.002 C/year placing it within the instrumental noise level for 30 year trend and is probably quite good for the 12 years (1979-1991) of the early data in this trend. In consideration of the past difficulties in merging the various satellite time series, data, it is reasonable to assume that the difference is a result of inaccuracies in processing in satellite metrics at this point. It is the attempt of this article to use GISS ground measured data to correct the trend of both metrics in this region in order to determine an improved long term trend agreement and accuracy of the satellite data sets.
The method used for correction was to match GISS amplitude and trend individually to each of the lower troposphere satellite measurements to achieve best slope fit. Then the GISS trend in the area of the data in question was applied to correct the slope of both satellite metrics in the area of the discontinuity. To create the best trend fit the GISS data was truncated and linearly de-trended for the 30 year period corresponding to the full length of satellite data. The GISS data was then multiplied by 1.23 as determined by SD analysis on a previous post which confirmed the result as suggested by John Christy of UAH – i.e. Satellite lower troposphere is expected to have a 1.3 times multiplier for tropics and 1.2 for the rest. The linear full length trend for each of the RSS and UAH was added onto the trendless GISS data to create an overlaid version of the ground data. The results of the overlay are shown in the Figures 3 and 4.
The gray rectangle indicates the width of the region to which GISS trend was linearly matched to the UAH and RSS data individually. Data outside this rectangle was not used in the trend match.
Figure 3.
Figure 4
The black lines in Figures 3 and 4 represent the rescaled GISS data. The green segment is the two years of data from RSS and UAH which is being trend corrected due to the inconsistency. The GISS trend over this period was found to be more negative than the satellite data from both metrics. The resulting corrections are plotted in Figure 5 below.
Figure 5
After addition of the corrections in Figure 5 derrived from the GISS ground trend the UAH and RSS are in excellent agreement. The offset of the two trends inside the circle has been completely removed.
Figure 6
The slope temperature trend for the corrected RSS trend in the above plot is 0.00682 C/yr and the UAH is 0.00675 C/yr with a difference of 0.00007 C/yr which is now well within the 95% confidence limits of the instrumental measurement error of +/- 0.002 C/yr. It is also well within the one sigma measurement error indicating that this single discontinuity is now the major point of disagreement between the two metrics.
Of course with unscaled GISS data proclaiming a 30 year trend of 0.018 C/yr and satellite records of lower troposphere expecting a 1.23 times multiplier from GISS data such slow temperature rises should be concerning. The actual satellite data should demonstrate a 0.022 C/year slope but in fact is only about 0.007 in this analysis.
————
Conclusions:
After correction of the single discontinuity in the difference satellite data the two time series come into excellent agreement. The unexpected strong negative slope of GISS ground temperature data in the area of discontinuity had a significant impact on overall satellite anomaly trend. After correction of this single area of contention, both satellite records fall well within the expected 95% certainty of “typical” measurement noise level. The net slope of the two corrected trends is about 0.007 degrees C/year with a surprisingly small difference between the corrected RSS slope and UAH slopes are only 0.00007 C/yr after correction of this single area of disagreement. Since this slope is dependent on the length of the GISS data slope match ,numerous fits were done of varying time widths, all revealed the same slope correction to the satellite data within about 0.001 degree indicating independence of the result from the time window for the GISS slope correction. The GISS data is strongly more negative in slope than the satellite data at this point.
——-
Editorial,
I was shocked with this result I fully expected the GISS correction to satellite data to agree with the long term trend of one or the other satellite series. This correction is based on what seems to me to be quite reasonable math and I wouldn’t have been surprised to see an increase in trend rather than a decrease. The whole initial premise of the calculation was to use a third source to determine which metric was correct UAH or RSS. The satellite data in this range is suspect simply due to the fact that the RSS and UAH don’t match well at 1992 (the point of correction). The rest of the satellite time series from RSS and UAH are in agreement within the margin of expected measurement error for other areas of their time series (see Table 1). This means there is a difficulty for those who work with the raw instrument data in the region corrected here. After hours of reading I was unable to find a reasonable explanation for this particular discontinuity.
This analysis demonstrated to me that it is possible that even the actual satellite trends may be lower over the 30 year timeframe than are regularly reported. The slope of the RSS and UAH data before the discontinuity and after the discontinuity are below the long term trend (Table 1). Since GISS trend is so heavily corrected, the fact that it is much higher than Satellites does not invalidate my result.
Personally, (it is a blog after all) this has been quite an eye opener for me. I consider myself a skeptic on global warming. To me the theory of AGW is sound but the magnitude is in queston. Politicians and polyscienticians regularly exaggerate the outcome of global warming which only increases suspicion of the open minded. After the discovery of the rubbish in Mann08 it really pushed me to look harder. My thinking directs me so much that I actually found myself hoping for a stronger trend in this data but it just isn’t there. No matter what (reasonable thing) I did I couldn’t make the correction increase the slope. I did try…hard.
The results from GISS data were quite independant of the window the deterending was applied to. I used from the full 30 years of GISS, down to about 5 with the same results. I also corrected varying widths near the discontinuity. The resulting 30 year satellite trends only changed within about 0.001 deg C no matter what I did.
I now believe the true satellite trend is most likely slightly less than the UAH and RSS late values from Table 1. The noise in the GISS data could have magnified the effect of my correction, however the independence of the correction from the width of the slope match or width of the trend corrected data, left me with the impression that this slope reduction is really what happened. These slopes 0.129 C/yr RSS and 0.123 C/yr UAH are substantially lower than the 0.157C/yr and 0.128 C/yr reported by RSS and UAH for long term trend.
Future analysis of the statistical variation of short term slopes between GISS and satellite records seems to be a good next step to determine the accuracy of my corrected long term slopes.
the Air Vent 
You might want to try the same analysis using Hadley surface temperature data as well. The anomaly is in column 2, the other columns are different 95% confidence intervals with column 11 and 12 being the combined confidence intervals from all effects.
Welcome back Jeff.
Jeff, I wonder if it could be correction factors that are disgreed on (perhaps the same correction factor) for orbital drift or replacement of a channel/satellite? Such that as the years progress, the difference drifts (increasing). I see this problem in instruments, and it is mostly a “drift”. As easy one, it is that two instruments are drifting in opposite directions. If you have such, combined with a disagreement of a correction factor, you can confound this phenomena. One drift is instrumental. One is computational. Sometimes you have both confounding the initial analysis. To help detemine this, I look at the third metric and see if one of the two has a more pronounced (magnitude) problem than the other. Such that if I find a small instrumental problem, I continue to look for a computational on one or both. The third metric can help determine the magnitude that one is looking for. Sometimes it is only one that has the problem.
John,
I thought that might be the case here where one instrument or the other had a problem. Since the trend between the two satellite measurements outside of the corrected area is so good a match I thought GISS could be used to correct the area of difference.
The satellites data actually don’t drift apart at all they are quite nearly perfect outside of the step. By rescaling GISS to match and replacing only a small piece of the data I really expected one or the other satellite to match (Figure 2)
I will take DeWitt Payne’s advice and use HadCRUT to check the analysis again. The thing I keep thinking about is that the trends prior to 1991 and after 1994 are matched well and both are less than the full series trends. I think my analysis, while surprising, may be correct in that the true trend is less than the reported trend I have doubts that it is as low as my corrected graphs.
I wrote Dr. Christy to ask him to review and let me know what he though of the cause of the step. Hopefully he has some time for us.
Further to DeWitt’s suggestion, Radiosonde / Satellite comparisons have been used as one way to validate satellite trend data. Data adjustments for inhomogeneity correct artificial cooling bias but some suggest further adjustments are needed. It could be another possible source of discontinuity correction and it could also serve as a check on your corrected RSS / UAH slopes.
Examining and comparing the discontinuity to Hadley, Radiosonde as well as GISS might verify the stronger negative slope you saw with the GISS comparisons in your post.
Wouldn’t that be a twist, a demonstration that both satellite metrics need a negative trend adjustment.
Jeff,
Are you sure the 1.2 multiple applies to the lower troposphere? I thought applied to comparisons with the mid-troposphere.
Raven,
I followed how to correct this trend from a CA post by Christy. There are a number of papers on this as well.
J Christy:
December 19th, 2008 at 11:27 pm
Ratio factor:
Climate models display a fairly robust ratio of the troposphere anomalies/trends represented by the surface variation vs. the LT profile of MSU temperatures of 1:1.2 globally, and 1:1.3 in the tropics (Steve: it is 1.3 for the tropics and 1.2 for the globe). These have been called amplification factors. Models indicate that anomalies and trends of the surface become larger by these factors in the LT profile (See CCSP report for example or Douglass et al. 2007 for model results). Observations have not agreed with these model-calculated ratios for long-term trends, but do agree on monthly scale anomalies. (see Christy et al. 2007 on Tropical Tropospheric Temps) and Douglass et al. 2007.
This was ‘accidentally’ confirmed by a SD analysis of sat data as compared to GISS in my previous post
https://noconsensus.wordpress.com/2009/01/03/giss-temp-slope-is-exaggerated/
Welcome back. I was over there in the latter stages of the presidential campaign, and I really missed the insights from some of my favorite blogs.
Keep up the good work. Follow the numbers, let the implications lay as you find them. I think that is in keeping with the “scientific method”.
DavID
Jeff, this seems like a sensible approach to the splicing problem, but I caution that it’s not an issue that I’ve studied.
While I understand why they want to use radiosonde data to effect the connection, there seem to be even worse homogeneity problems with radiosonde than ground data. So it seems like a good usage of relatively continuous ground data in a period where UHI changes or such isn’t going to be relevant.
Steve,
Thanks for the comment. That was my thinking on the ground data as well. From the paper that Dr Christy sent, he used 2 and 3yr section averages on either side of the discontinuity of differenced metrics (i.e. UAH-Sonde) to determine the accuracy of UAH slope was more correct than RSS. I learned more about the changes in balloon data from that paper than I expected. I am amazed by the number of steps in that data as well.
I think Dr. Christy’s method could improve the accuracy of my work here but I haven’t had time to work it all through in my head. I want to combine that with a tighter window around the actual step to change as little of the data as possible.