CRU Releases (Some) Data
Posted by Jeff Id on December 23, 2009
My next partially completed Siberia post will have to wait.
This is the beginning of the result of climategate. We owe a big debt to those who released the emails, and an even bigger debt to Steve McIntyre for his never ending efforts to force the revelation of some of this information. Now we can begin to understand the premier temperature dataset in climatology. Does this mean it’s done — nope. This is only the beginning, CA has a post describing the release of more metadata (data about data) and actual records for CRU than ever before.
However without seeing the data, the victory is partially hollow. All the criticisms developed over time still hold unfortunately. It appears that CRU hasn’t learned a damned thing from the experience yet either. Why the grumpiness in the face of what can only be described as progress (climate progress :D ), check out these quotes from the CRU page.
The database consists of the “value added” product that has been quality controlled and adjusted to account for identified non-climatic influences. It is the station subset of this value-added product that we have released. Adjustments were only applied to a subset of the stations so in many cases the data provided are the underlying data minus any obviously erroneous values removed by quality control. The Met Office do not hold information as to adjustments that were applied and so cannot advise as to which stations are underlying data only and which contain adjustments.
Underlying data are held by the National Meterological Services and other data providers and such data have in many cases been released for research purposes under specific licences that govern their usage and distribution.
Remember this pretty quote from the emails.
The two MMs have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone. Does your similar act in the US force you to respond to enquiries within 20 days? – our does ! The UK works on precedents, so the first request will test it. We also have a data protection act, which I will hide behind. Tom Wigley has sent me a worried email when he heard about it – thought people could ask him for his model code. He has retired officially from UEA so he can hide behind that. IPR should be relevant here, but I can see me getting into an argument with someone at UEA who’ll say we must adhere
to it !
I wonder how Real Climate will spin that as out of context.
This is what they say about data storage.
The data set of temperatures, which are provided as a gridded product back to 1850 was largely compiled in the 1980s when it was technically difficult and expensive to keep multiple copies of the database.
For IT infrastructure of the time this was an exceedingly large database and multiple copies could not be kept at a reasonable cost. There is no question that anything untoward or unacceptable in terms of best practices at the time occurred.
Since before 1990, the entire thing could have fit on a harddrive quite easily. I wonder what happened to the raw data implemented in 1990? I do suspect that the inital problem was partially data storage but what about the years since? And what about the future? And if Dr. Jones doesn’t mind, we’ll be the ones to determine if something untoward happened. I’m certainly not in the mood to have the towardness assigned by these people at this point.
This next point was very dissapointing.
. How can you be sure that the global temperature record is accurate? The methodology is peer reviewed. There are three independent sets of global temperature that all clearly show the rise in global temperatures over the last 150 years.
Peer review in AGW has just been exposed. I’ve not seen one single independent confirmation, sat/sonde data doesn’t agree very closely. Other datasets use the same information, so independent reads as a false claim. The real surprise in this section was the change in focus.
Furthermore, the strong scientific evidence that climate is changing as a result of human influence is also based on the growing evidence that other aspects of the climate system are changing; these include the atmosphere getting moister , global rainfall patterns changing, reductions in snow cover, glacier volume, and Arctic sea ice, increases in sea level and changes in global scale circulation patterns. There are also numerous changes in phenological records which point towards a general warming and support the veracity of the instrumental record.
Just what the f..k does that have to do with the accuracy of a measurement. I don’t swear often here, but come on!! Think about what these jackasses just wrote!! Have they learned nothing?!?!?!! How does sea level rise prove a temperature record within tenths of a degree!!! @@%!^$^@. If a special ed student made the statement you’d give them an F. How dumb do they think we are??? Wow, just wow.
Ok, – five minutes later, I’m calm again.
We have always provided the gridded HadCRUT product freely and without restriction for research usage. The data set is available through the Met Office, CRU and BADC and has been used widely in research papers and the media.
New question- why didn’t you bother to answer your own question … #$$%!?
The Met Office is not in a position to release that portion of the underpinning land station data for which we have yet to gain permission from the ultimate rights holders.
This rings hollow when you won’t release a list of the stations we need to get rights for? Maybe they did now, but at this point— who knows.
Ok then I skip down through endless – self questions – and find this beauty.
We take our responsibilities under the Freedom of Information Act very seriously and have, in all cases, handled and responded to requests in accordance with its obligations under the legislation.
It’s impossible to respond without adjective.
When the FOI requests began here, the FOI person said we had to abide by the requests. It took a couple of half hour sessions – one at a screen, to convince them otherwise showing them what CA was all about. Once they became aware of the types of people we were dealing with, everyone at UEA (in the registry and in the Environmental Sciences school- the head of school and a few others) became very supportive. I’ve got to know the FOI person quite well and the Chief Librarian – who deals with appeals.
The VC is also aware of what is going on – at least for one of the requests, but probably doesn’t know the number we’re dealing with. We are in double figures. One issue is that these requests aren’t that widely known within the School. So
I don’t know who else at UEA may be getting them. CRU is moving up the ladder of requests at UEA though – we’re way behind computing though. We’re away of requests going to others in the UK – MOHC, Reading, DEFRA and Imperial College. So spelling out all the detail to the LLNL management should be the first thing you do. I hope that Dave is being supportive at PCMDI.
[snip rest of irrelevent stuff to this topic]
What’s to say. Well let’s put it this way: There were a bunch of like minded pseudo-intellectuals, who’ve never made a payroll, never worried about ‘progressing’ outside of a sheltered system, that decided CA readers have a type. And that particular ‘type’ did not deserve protection under the law.
Ok, I’ve left several points alone as a careful reader knows, but the last point may be the best.
The code is a computer programme that is held in two files, using the scripting language “PERL”.
The first, called “station_gridder.perl”, takes the data files and creates a gridded data set. The second, called “make_global_average_ts_ascii.perl”, calculates the time series of global average temperatures from the gridded files that were created by the first program.
You will need PERL installed on your computer. This is available from www.perl.org.
I’ve not reveiwed the code but it seems that we now, after years of McIntyre and others work, can finally run software with replicates the CRU data. Although, I’m not sure how that works without all the actual data?
I’m just an engineer though.