Overconfidence Error in BEST

UPDATE:  The post started as a general review of BEST several days ago including some critiques by others.   As I finished, an error in the CI calculation became apparent to me.  If you are familiar with the work, jump to the section titled Jackknife for the description of that problem.

—-

Finally, a technical post.  As most here know, the Berkley Earth Surface Temperature (BEST) has released its early results. The media has worked tirelessly to misrepresent the results to the public. I even heard Chicago progressive radio refer to the authors of BEST as denialists who decided they were going to overturn the results of other surface temperature analyses and found that they couldn’t, proving again that global warming is going to doom us all and that skeptics are fools. Really, that’s what mainstream Chicagoan progressives are being told about this. The purpose of the project was actually to create an open and transparent global surface temperature that people can read and critique at will. The authors made it clear in their mission statement which begins with:

Our aim is to resolve current criticism of the former temperature analyses, and to prepare an open record that will allow rapid response to further criticism or suggestions.

A difficult proposition considering the quality and number of the temperature records involved. I have read all four papers including a multiple read of the methods paper to understand some of the more sophisticated points presented. I’ve also read critiques by several bloggers/sceintists some of which I agree with and others which I believe are mistaken, but in the end I don’t believe any of the critiques could possibly have any appreciable effect on the trend results without a surfacestations style analysis of the raw data. The only places I have concern are in the brush-over given to the UHI effect and the obviously over-tight confidence intervals. Even if the CI’s were widened to more correct levels, it wouldn’t change the result and the UHI effect isn’t going to reverse any trend so despite some statistical critique, I believe the result is very close to the actual global surface temperature average minus some unknown amount of warming by UHI.  Still, I do believe that I have identified a specific error in the confidence interval calculation which must be corrected and is discussed below.

Is it the Message or the Presentation

I haven’t read more than the title of this article.  No time for blogging.  I have read and re-read the Berkley temperature study and have a bit different of an opinion than some of my fellow ‘skeptic’ bloggers.  Hopefully I’ll have time for that in the next day or two.

In the meantime, this article’s title  leads to one of my favorite chicken little points of the AGW movement:

New Research Casts Doubt on Doomsday Water Shortage Predictions

By measuring the isotopes in river water, scientists have determined that mountain glaciers contribute less than thought to downstream water supplies

I wonder if the greens wonder why the 1970’s greens used to call our planet a blue marble?    Green/blue, what’s the difference?

Well, seeing that the earth is about 70% covered with ocean water and that rain  storms are constant, how is it that warmer Earth might somehow ‘dry’ the atmosphere and cause less fresh water to fall from the sky?

Just how stupid are we non-scientists?

Then I suppose if we’ll sit with our Abercrombe cloths, facebook and IPhones in the streets and shelter in mass produced tents and blankets to protest big companies and capitalism, the scientists may just have us intellectually beaten.

Less water from the sky though?  So we’re going to thirst to death while drowning from higher oceans.  That is a tough message to sell.  I’m thinking if they just “communicate” better, it may work out.

Why WWII Got Us Out Of The Great Depression, and Why Present Government Spending Would Not Do The Same

A guest post by:

Leonard Weinstein, ScD

August 16, 2011

The Great Depression, starting about 1929 and running through the 1930’s, had several root causes, including major government actions and the bubble of buying stock on inadequate margin. I will not get into details on the causes, as the issue of the present write-up is how we got out of the Depression.

It is clear that President Roosevelt’s actions to spend large amounts of money on public projects in the 30’s did cause short-term job increases, and many of the projects had a valuable long-term benefit, especially infrastructure development. However, these gains did not end the Great Depression, and may have made it more severe and last longer than otherwise.

Many think WWII caused the end, and this is in fact almost certainly true. However, how the war ended the Depression is badly misunderstood. Some think that just the fact of massive government spending, and temporary employment supporting the war effort was the mechanism. That is wrong.

There were three major factors that ended the Depression:

1. The destruction during the war of most of the industrial capacity of most of the major industrial powers (including in Europe and Japan), but not including the United States, left us with a near monopoly on production of major items. In fact, many of the factories greatly built up their capacity for the war, and the increased capacity was used to advantage after the war. This advantage lasted several decades, and gave us a long head start on establishing markets.
2. A pent up needs for automobiles, appliances, and many other items developed due to the manufacturing plants converting to manufacturing supplies for the war. After the war, the conversion back allowed huge amounts of sales of these items. This lasted long enough to establish many businesses solidly.
3. The GI bill allowed huge numbers of military personal to buy homes, and even more important, go to collage. The large increase in well-educated people resulting had a major effect on the level of technology that could be developed. The middle class grew to a much larger percent of the population, and consumer buying increased greatly.

Nature’s Sneak Peek into AR5

Nature magazine has given us an article by a Jane Qiu interviewing glaciologist Qin Dahe which provides a preview of the upcoming AR5 report.  The article is a piece of triple A rated, (alarmist, advocate,  activist)  – of course – but it confirms our suspicion that the reasonable critiques of the alarmism are having no appreciable impact on the “science” or the cloak under which it is performed.  Whle Qiu, spoke generally, he said it doesn’t mater if Himalayan glaciers melt in 20 or 200 years.  I wonder if the ‘actual’ projection of 350 years matters to him as a glaciologist.  The projection that far out is insane anyway but that is another matter.  This paragraph contained a unique quote which I wonder how true it will turn out to be. My bold in the reply to the question below.

What can we expect from the fifth assessment report in 2013?

There will be more data on changes in glacial volume as well as other components of the cryosphere, such as permafrost and snow cover. There is encouraging progress in the IPCC climate-modelling studies, with smaller gaps between observed and modelled global average temperature and carbon-dioxide concentration. We will be tackling questions such as whether climate change is accelerating, whether the Greenland ice sheet is stable, what roles clouds and aerosols have in climate forcing, and whether there will be more extreme climate events, such as droughts, floods, typhoons and hurricanes.

I wonder, did they change the models, the data or the interpretation.   Perhaps all three but it doesn’t sound like McKitrick, McIntyre, Herman will be front and center in the modeling section.

More Bad Science Courtesy of Government

Fox News has an article highlighting an attempt by Republicans to point out the mountains of known-bad science being paid for and misrepresented by people in government.

Several Republican lawmakers are challenging the Obama administration’s science czar over what they claim are repeat incidents of “scientific misconduct” among agencies, questioning whether officials who deal with everything from endangered speciesto nuclear waste are using “sound science.”

The recent Yucca mountain nuclear waste disposal cancellation, is one more example of fake hurdles put in place to prevent the development of  real energy for real industry.   It is dictatorial politics at its worst.  Why do we have a ‘department of energy’ if its sole purpose is promoting the propaganda of the current commander in cheif?

In their letter, the lawmakers focused most on concerns about the 2009 decision to pull the plug on the controversial Yucca Mountain nuclear waste site in Nevada. The project years in the making faced heavy opposition in Nevada.

Energy Secretary Steven Chu in 2009 said the project was simply not a “workable option.” In early 2010, the department withdrew its license application for the site, and moved instead to impanel a commission to look at alternative sites. A department filing at the time noted that scientific knowledge on nuclear waste had “advanced dramatically” in the 20 years since the project started.

But the Government Accountability Office said in an April report the DOE did not cite “technical or safety issues” in its decision.

“Amid uncertainty over whether it had the authority to terminate the Yucca Mountain repository program, DOE terminated the program without formally assessing the risks stemming from the shutdown, including the possibility that it might have to resume the repository effort,” the report said.

A June report from Republicans on the House Science, Space and Technology Committee also said the panel could not find a “single document” to support claims that Yucca Mountain is unsafe for nuclear waste.

The government has strayed far from its original roots.   The left in particular, no longer appears to have any interest in helping people live better lives.  It is as though every single policy coming out is intentionally designed to be bad for industry, bad for our future and bad for our society.  Why people want to live like Cubans is beyond me, but we’re working hard to get there.

Hiding Future Declines

This is kinda huge — Jeff

———-

BREAKING: An IPCC backchannel ‘cloud’ was apparently established to hide IPCC deliberations from FOIA.

 

CEI has learned of a UN plan recently put in place to hide official  correspondence on non-governmental accounts, which correspondence a federal inspector general has already confirmed are subject to FOIA. This ‘cloud’ serves as a dead-drop of sorts for discussions by U.S. government employees over the next report being produced by the scandal-plagued IPCC, which is funded with millions of U.S. taxpayer dollars.

By Christopher Horner, CEI.org for WUWT

Although this is seedy and unlawful at any time, it also goes in the ‘bad timing’ file. Or it’s good timing, depending on one’s perspective.

Just as a brand new book further exposes the UN’s Intergovernmental Panel on Climate Change (IPCC)(which scam I dissected here, and in more disturbing detail here), and on the heels of the weekend surprise of a 2005 memo showing President Obama’s cooling/warming/population zealot of a ‘science czar’ John Holdren is the kind of guy Mitt Romney turns to to develop his ‘environmental’ policies, we’ve exposed the Obama administration and IPCC have cooperated to subvert U.S. transparency laws, run domestically out of Holdren’s White House office.

——

CEI reminds OSTP that this practice was described as “creat[ing] non-governmental accounts for official business”, “using the nongovernmental accounts specifically to avoid creating a record of the communications”, in a recent analogous situation involving lobbyist Jack Abramoff. CEI expects similar congressional and media outrage at this similar practice to evade the applicable record-keeping laws.

This effort has apparently been conducted with participation — thereby direct assistance and enabling — by the Obama White House which, shortly after taking office, seized for Holdren’s office the lead role on IPCC work from the Department of Commerce. The plan to secretly create a FOIA-free zone was then implemented.

See the rest at WUWT

Cycles In Temperature Trends

Recently, a paper by Craig Loehle and Nicola Scafetta has come to my attention.   The simplicity of the math makes it attainable for the technical reader who doesn’t want to spend a month learning climate jargon.  This isn’t a Mannian stew.  The paper received a strong critique at WUWT, much of which was undeserved in my opinion.   What the authors did was recognize the widely discussed 60ish year signal in climate data and project it onto temperature records to determine the percent of recent trends are due to human contribution.   The problem is that measured temperatures are 150 years long so our recently invented thermometers (think about that sometimes folks) have only recorded two of the suspected 60 year cycles and those barely visible cycles are further masked by a  warming trend.   So the question becomes, are the 60 year climate cycles regular or are they not?  We don’t have proof of anything but if they are real, there is some additional realistic evidence of lower climate sensitivity to CO2.  A smaller amplitude high frequency component was also used in the work.

The abstract is a good place to start, and may have caused much of the problem in interpretation.

Abstract: The climate change attribution problem is addressed using empirical decomposition. Cycles in solar motion and
activity of 60 and 20 years were used to develop an empirical model of Earth temperature variations. The model was fit to
the Hadley global temperature data up to 1950 (time period before anthropogenic emissions became the dominant forcing
mechanism), and then extrapolated from 1951 to 2010. The residuals showed an approximate linear upward trend of about
0.66°C/century from 1942 to 2010. Herein we assume that this residual upward warming has been mostly induced by
anthropogenic emissions, urbanization and land use change. The warming observed before 1942 is relatively small and is
assumed to have been mostly naturally induced. The resulting full natural plus anthropogenic model fits the entire 160
year record very well. Residual analysis does not provide any evidence for a substantial cooling effect due to sulfate
aerosols from 1940 to 1970. The cooling observed during that period may be due to a natural 60-year cycle, which is
visible in the global temperature since 1850 and has been observed also in numerous multisecular climatic records. New
solar activity proxy models are developed that suggest a mechanism for both the 60-year climate cycle and a portion of
the long-term warming trend. Our results suggest that because current models underestimate the strength of natural
multidecadal cycles in the temperature records, the anthropogenic contribution to climate change since 1850 should be
less than half of that previously claimed by the IPCC. About 60% of the warming observed from 1970 to 2000 was very
likely caused by the above natural 60-year climatic cycle during its warming phase. A 21st Century forecast suggests that
climate may remain approximately steady until 2030-2040, and may at most warm 0.5-1.0°C by 2100 at the estimated
0.66°C/century anthropogenic warming rate, which is about 3.5 times smaller than the average 2.3°C/century
anthropogenic warming rate projected by the IPCC up to the first decades of the 21st century. However, additional
multisecular natural cycles may cool the climate further.

So if we have a global temperature record which looks like this:

The top pane is hadley temperature with an overlayed model, the bottom is the residual from a full length model fit showing a cyclic plus trend method.  Since there is long term signal still visible (kind of a U shape in the bottom data), the authors determined the model is absolutely misspecified.   The thick black line over the top temperature pane doesn’t seem entirely unreasonable though.  It is quite possible that there is a cyclic nature to the signal, but without more information, little can be concluded with certainty.

This is the comment Leif Svalgard left at WUWT:

Leif Svalgaard says:

July 25, 2011 at 3:14 pm

The paper is cyclomania at its worst

A comment which I believe is way over the top.   Perhaps the conclusions are too strongly worded but the concept behind the paper is not bad, it actually could be perfectly correct.  Willis made more complete arguments with other fits having different timescales.  Nicola Scafetta replied with this comment which will sum up my own understanding.

Dear Willis Eschenbach,

I am sorry but you continue to say no-sense.

1) you cannot disprove our research by simply using a model slightly different from our, for example using a 62-year cycle instead of a 60-year cycle, and claim we are wrong because you get similar results! Why don’t you use a 100-year cycle instead of a 62-year cycle?

2) the existence of major quasi 60-year and quasi 20-year cycles are easily detected by power spectrum analysis, see figure 3, 10 & 11 in N. Scafetta, “Empirical evidence for a celestial origin of the climate oscillations and its implications”. Journal of Atmospheric and Solar-Terrestrial Physics 72, 951–970 (2010), doi:10.1016/j.jastp.2010.04.015

The bold is mine.  The paper by Loehle and Scafetta isn’t disproven by the mere mention that different cycles can be fit to the same data.   If the cyclomania is even close to a real process – say some cycles are 40 and some are 70 years, the fits address a substantial fraction of the climate warming we’ve seen.   Solar cycles experience some of the same imperfect variance as Leif will attest to himself.   Where the critics went really wrong was to fail to recognize the proxy data shown in the same paper, having a longer timeframe and similar cycles.

See, the 60 year cycle may not be only real since 1850.  It may not be a perfect cycle, but like sunspots, ice ages, and many other climate phenomena, there is potentially a generally repeated nature to the variance.  All that is required is for the cycle to exist and Loehle and Scafetta have a reasonable way to estimate the percentage of warming caused by CO2.

The last paragraph of the conclusion may be where they have gotten themselves in trouble.

It is also worth noting that by starting in a cycle trough in
1900 and ending on a cycle peak in 2000, graphical
depictions of warming in IPCC and other reports give an
exaggerated depiction of the rate of warming, a point noted
by Karlén [75] and Soon et al. [76]. By further failing to
mention solar effects and warm bias in the instrumental
record, it is implied that almost all the warming of about
0.8°C over this period is human caused, when really no more
than 0.4°C is likely to be. Reporting such a lower number
would probably cause much less alarm than what was
recently claimed by Rockström et al. [77]. Moreover,
because the 60-year natural cycle will be in its cooling phase
for the next 20 years, global temperatures will probably not
increase for the next few decades in spite of the important
role of human emissions (see Fig. 5A), as predicted by
multiple studies (reviewed in Loehle [69]; Scafetta [18]).

While the methods seem fine to me, they are far from definitive.   The paper did achieve an excellent fit to the data based on reasonable assumptions of observed natural cycles, like predicting the next sunspots, the total variation in the natural cycles is basically an unknown.   I would have prefered a less definitive conclusion on the order of, “if the previously observed cycles are an accurate component of recently observed temperatures and continue for the near future then we can make the following conclusions:”   Although similar statements are made in the paper, without more of that, a zealous reader can easily go too far critiquing the work.   The proxies are a longer record and do seem to support the cycles.

I don’t think that anything in the work is unreasonable though and the critiques of it in the WUWT thread were too severe IMO.  It is yet another paper which lends support to the ever more likely fact that climate models are running too hot and the potential that AGW is a far less severe effect than the IPCC likes to state.

Divide and Cool

So the other day I was reading around the internet and came across this document:  ebs_372_en[1] 

It is a ‘study’ commissioned by the EU has this amazing bit of political-speak straight out of a apocalyptic sci-fi novel:

This survey has been requested by the Directorate-General Climate action and coordinated by Directorate-General for Communication (DG COMM “Research and
Speechwriting” Unit).
http://ec.europa.eu/public_opinion/index_en.htm
This document does not represent the point of view of the European Commission.
The interpretations and opinions contained in it are solely those of the authors.

I’m not sure how you combine ‘Research and Speechwriting’ into a department, but I can make a few guesses as to why.  The study introduction’s first sentence makes no bones about the “authors opinions”.

Climate change is one of the greatest challenges of the modern age.

Anyway, I continued to read.  The very first graph stopped me in my tracks.   Jaw in lap, I stared for several minutes contemplating the meaning of what these surveyors were claiming.

Several things stood out in this poll.  First, since the poll is reported to one percent, which one hundred people could you ask these questions to and get no ‘others’ as answers.   The average group of 100 people wouldn’t give a single ‘different’ answer than this list.   The result is so goofy, that I bet 1 in 100 would say lack of faith is the biggest issue, but as you know, I’m often wrong.  The top issue according to the poll is poverty, hunger and lack of water with climate change in second place with 1 in 5 Europeans worried about that more than the economy, war etc…    I thought Europe was the enlightened group and America is comprised of gun toting dumb-hick morons.  Aren’t there riots going on for jobs in Europe?  Yet the people are worried more about climate change than their own livelihood?

Free Thinking

WUWT has highlighted an article from the Independent where James Hansen is quoted as saying the skeptics are winning.   My own opinion is that nobody who wants less government interference in energy creation is winning.   I have to laugh that people don’t have enough energy even today.  It is insane, but because of the ignorance of the public’s approach to the issues stupid government policies are the norm rather than the exception.

Climate sceptics are winning the argument with the public over global warming, the world’s most celebrated climate scientist, James Hansen of NASA, said in London yesterday.

Now Hansen is widely celebrated, he’s a triple A rated climatologist  activist, advocate, alarmist.  He has no fear of retaliation for his overpaid government job or the quid-pro-quo payments/awards from pro-government, pro-socialism organizations.  The false claims were flying everywhere:

It is happening even though climate science itself is becoming ever clearer in showing that the earth is in increasing danger from rising temperatures, said Dr Hansen,

That statement is flatly false.  The reason the skeptic argument (which is really a lukewarmer style argument) is “winning”, is that the DATA and the SCIENCE are supporting a more moderate warming in the future and with vastly reduced consequences in relation to some of the absolutely insane IPCC predictions.  Of course the winning comment is simply the recognition by the public that perhaps the IPCC is really a political organization just as any thinking person already knows.

It is amazing that a scientist puts crap like this out there with a straight face:

Part of the problem, he said, was that the climate sceptic lobby employed communications professionals, whereas “scientists are just barely competent at communicating with the public and don’t have the wherewithal to do it.”

Hmm… I’ve read nothing from a paid climate skeptic lobby and continuously read pro-climate information which can only be described as lies from paid for advertisements. I’d say that he’s a pot calling the kettle black, yet the kettle in this case was a glass. The climate alarmist campaigns are often government money or untaxed 501C’s paid to advertisers to support ever more controlling government.  Propaganda about why their message doesn’t work is continuously put in high profile news articles.  Really, I don’t know where the skeptic movement comes from except that the message from main stream climate science in their media organizations is heavily leftist, often verifiably inaccurate, and takes a completely negative approach of limitation, reduction, rather than expansion or development.

Who really wants less for themselves or their kids. Why not discuss expansion of real solutions like nuclear power or more efforts for research into real energy if they are so concerned?  My answer is simple, the politicians have different intent than the heavily leftist climate science movement can understand.  In my opinion it is the Achilles heal of leftism.  The progressive individual thinks the politician can just do X and if that doesn’t work do X + Y etc…  In practice, the politicians actions (even with the best intent) average out to whatever gives them more power/money.  Cap and trade was a perfect example. Scientists even hailed it as a “market” solution – selling an item with no value at a price set by the politicians who can alter the price of the commodity on a whim.   Sounds like a free market solution to me.

I have to wonder with the incredible ignorance of people who support the power mongers, will sanity ever return to Earth?

Polling the Educated

Some poll results were recently released by the EU.  They posted the questions on the form with the details.  From the totality of questions, this is intended to be a scientific poll on climate change.   Take the question seriously and write your thoughts.   We’ll do a comparison on Monday or Tuesday of the results.

A scientific poll of tAV readers who are widely educated on the mysteries of AGW:

 

 

AMSR-E Dead…..

The AMSR-E satellite just lost its ability to spin stabilize.   I’m not an expert on its funcitons but it had a high RPM rate previously which suggests a need for axial alignment.   This means that solar panels need to rotate in the opposite direction with similar speed and torque compensators must operate to prevent friction from changing the total spin rate.  So the thing had to stop spinning.   What the subsystems are comprised of is also beyond my knowledge but it doesn’t matter.   What does matter is that a number of VERY critical global doom timeseries are likely to have unfortunate steps introduced into them.   Some scientists will lose months or years of work in re-aligning the next satellite with the previous accurately and there will always be questions.   The first huge opportunity for atmospheric temperature consistency has been lost because we weren’t wealthy enough (or smart enough) to have the second instrument in space before the first failed.  This first one was apparently scheduled to fail 3 years ago.  Sounds like typical government to me.

If you are going to chuck trillions down the left side of the toilet, you would think measuring the parameters of doom might take a few hundred million dollar preference.   Think of the few hundred millon as a penny from a hundred dollars.

Nope, don’t know why we wouldn’t spend that bit. No, this is not a critique of the scientists, it is a critique of the government so willing to both steal and spend the trillions.

Really though, time series trends which are so sensitive to steps is very bad news.   Yes there are other data sources, but this particular satellite was a huge forward step in capability.  Hopefully, its value won’t be lost by this.  Some functions remain, although I expect that their quality will be compromised.

Roy Spencer’s blog is the place to address questions.  Corrections or additions to my limited knowledge on this are appreciated.  For instance: What is the scope of the data actually lost?

Premonitions

I’ve had a premonition. Using sophisticated computer modeling and regression analysis, I’ve used past IPCC conclusions as a temperature proxy and projected trends of the IPCC conclusions on the density of CO2 in the atmosphere. By applying an inertial consistency algorithm (based on past data autocorrelation), I’ve been able to project AR5 results a year ahead of schedule.

By happy coincidence, Donna Laframboise, from the well named nofrakkingconsensus blog, has done some research of her own to this effect. She has compiled the names and histories of the upcoming IPCC report AR5 on climate change and the doom they hope to provide in response to it. The links had to take her hours, so check it out.

We will be waiting on the edge of our seats to see what will comprise the summary for “policy makers” by such an august panel of minds. From my projections, were the IPCC reports published in movie format, we could make a college drinking game out of how many ways can they write “worse than we could have imagined”.

Thanks to Lubos again where I found the link.

I do need to point out for those who think I’m a conspiracy nut that both the WWF and the IPCC/UN have released documents expressing their role in the coming (very leftist) global government. I can’t wait until we sign up for that happy bit of paperwork.

Origin unknown but I like the crazy wizard with the crystal ball.