the Air Vent

Because the world needs another opinion

Climategate Continued

Posted by Jeff Id on December 16, 2011

Fox news has an article on one aspect of climategate featuring some famous bloggers.

UPDATE:  The search warrant issued for confiscation of Tallbloke’s computer’s is available at CA.


139 Responses to “Climategate Continued”

  1. M. Simon said

    I remember when one of those famous bloggers had a bright idea at the other blogger’s blog.

  2. More fulmination about the “hiding” of the stations data. The irony is that it was released five months ago, and AFAICT no sceptics have bothered to look at it. Easier to go on bashing DoE etc.

  3. Bruce said

    Nick, where is the raw data? I’ve download the CRU release and it doesn’t look like raw data to me.

  4. Great quotes from them who must not be named. Replication replication replication. And if someone gets upset when you try to replicate their work…who is it that is not understanding the process?
    Nick–please point me to the raw station data(not gridded) and the adjusted data (not gridded) and the code for doing adjustments. I can’t seem to find it. I believe you are suffering from future present tense: since CRU promised they would someday release it, you think the data has already been released.

  5. Craig,
    I wrote about it here. The station data is here.

  6. phi said

    What is funny is that in general CRU does not even have the raw data and does not seem to care (it is understandable that some do ostriches, when comparison is available, we see that half of the secular warming comes from adjustments).

  7. Sorry Nick, not good enough. This is NOT raw but adjusted data and it is only MONTHLY not daily which means any missing values have been filled in some how.

  8. Craig,
    They never had, and never claimed to have, daily data. What they have posted is the data as supplied by the sources, who are listed in detail.

    This is how it goes. Firstly, a hullabaloo over thestation data. We must have it! It belongs to us!. Then on release, no interest, except, why don’t you have this? or this?

    It’s just harassment.

  9. Niels A Nielsen said

    Don’t be unreasonable, Craig. Poor Phil Jones has explained that he didn’t keep the raw data. Remember, the adjusted data is a “value added” product.

    Jon stewart is a punk: “V-a-l-u-e a-d-d-e-d d-a-t-a??? what’s that? Numbers fortified with art? truth plus? – now with lemon? doesn’t look good.”

  10. windansea said

    7.Craig Loehle said
    December 16, 2011 at 5:24 pm
    Sorry Nick, not good enough. This is NOT raw but adjusted data and it is only MONTHLY not daily which means any missing values have been filled in some how.

    Nick’s blind trust is amusing. CRU temp data acceptance is similar to getting a yearly account statement from your bank where all individual deposits/withdrawals have been “adjusted” and then averaged by month.

  11. Niels A Nielsen said

    Phil Jones, CRU: “..we have never had sufficient resources to keep track of the exact source of each individual monthly value. Since the 1980s, we have merged the data we have received into existing series or begun new ones, so it is impossible to say if all stations within a particular country or if all of an individual record should be freely available. Data storage availability in the 1980s meant that we were not able to keep the multiple sources for some sites, only the station series after adjustment for homogeneity issues. We, therefore, do not hold the original raw data but only the value-added (i.e. quality controlled and homogenized) data.”

  12. Wind,
    I don’t know if you know what you are talking about here. Daily data is huge. Until recently, it rarely left the place where it was asembled, with good reason.

    CRU is a small UK university department. They are not the source of the world’s data. They obtain and use what they can, just like the rest of us. If you want daily data, go to the sources.

  13. phi said

    The issue is that the data used by CRU are adjusted by suppliers and not a little, about 0.5 ° C warming added to the twentieth century. Strangely, all national agencies adjust series whereas, rightly, Hansen has shown that such adjustments distort trends (Hansen et al. 2001).

    That said, it’s still interesting because these adjustments are a lower bound for stations perturbations (UHI and other). In fact, by relying on proxies, these perturbations could reach a total of about 1 ° C in the twentieth century.

  14. Matthew W said

    “The original Climategate emails — correspondence stolen from servers at a research facility in the U.K. ”

    Still claiming that they were stolen !!

  15. Earle Williams said

    Matthew W,

    Of course, when facts are lacking you must infill with hyperbole.

  16. Mark F said

    When facts are lacking (obscured, hidden, misrepresented etc.) …

  17. Carrick said

    Nick:

    The irony is that it was released five months ago, and AFAICT no sceptics have bothered to look at it. Easier to go on bashing DoE etc

    Some people have been looking at it, so does that mean that if you looked at it, that means you’re not a skeptic?’

    What a stupid argument.

  18. Bruce said

    Nick, you know perfectly well almost every file says the data source is “Jones” or “Jones+Anders”, and way too much of it is -99.0 … for the 2000s. And the word extrapolated appears way too often.

  19. Bruce,
    Here is the source listing from the page I cited. It explains the code used for each station:
    10 – from earlier CRU work (from World Weather Record volumes, published by the Smithsonian Institution and NOAA/WMO from the 1920s onwards)
    30 – GHCN and for Australia (from Bureau of Meteorology)
    31 – GHCN
    32 – ALOCLIM (Austria)
    33 – NACD/NORDKLIM (Scandinavia and NW Europe)
    34 – NACD/NORDKLIM (Scandinavia)
    35 – IMPROVE project
    36 – CHTD (Canada)
    37 – GHCN (principally fUSSR but also some other regions)

  20. curious said

    Nick – my recollection is the same as Niels at comment 11. Do you disagree with that? Specifically:

    “We, therefore, do not hold the original raw data but only the value-added (i.e. quality controlled and homogenized) data.”

    btw – thanks for your response of a few days ago. I’ll respond on the thread Jeff gave your reply.

  21. #20 Curious
    Well, firstly the agitation, continuing with the Fox item, is about the data they have, and are supposed to be withholding. My point there was that they have released what they have, and nothing seems to have resulted, which raises the question of the good faith of the FOI demands.

    As to your quotation, as I recall that related to a query about the data used for one of his early papers. You’d need more context to match it to the current release. I checked some of the data which was from GHCN stations, and it seemed to be exactly the same as the GHCN version. Now currently that is as supplied by the Met offices, and I believe the historic data was also mostly preserved as found, with maybe some quality check.

    It’s often hard here to pin down what is being argued about. It seems to jump seamlessly from their reluctance to yield the data they have to complaints that it is not the data people think they should have.

    But really it comes back to unreasonable expectations of CRU. They aren’t data sources. If you don’t like what they have, do what they did – go to the sources.

  22. Brian H said

    One wonders how CRU et al got the data into their computer files to analyse and adjust, and then managed to lose it all.

    Computer storage, right up into the terabyte range, is dirt cheap.

    As for the Fox News article, rather ‘fair and balanced’, I’d say, in glaring contrast to any and all other MSM write-ups.

  23. #22 Brian, at the time this data is supposed to have been lost or discarded, I was saving up for my first 20Mb disk. And writing CFD programs at work with a 5 Mb memory allocation, total.

  24. Kenneth Fritsch said

    Nick Stokes, when you say the following I think you have pretty much nailed it. Given the history of CRU, and for that matter, GISS, I think my expectations have been fairly low since I have become interested in what they are doing. I thought that CRU admitted to losing/misplacing the original data that they had in hand at one time. I think that a repository of temperature data has been required for references in climate science papers. The initial efforts to come up with reasonably comprehensive ones were carried out almost like a hobby with not a lot expense or effort put into the process.

    I am not arguing here about the validity and accuracy of the data but only noting that these processes have not gotten the effort one would think would be required given the important studies such as reconstructions and climate modeling that depend on good instrumental data and uncertainty limits. In fact some of the published efforts of data handling showed sloppiness and a hesitancy to keep the data sets updated with all the current information available. My reading of the handling of the temperature data was that rather late in the game interested parties made efforts to determine the uncertainty of the temperature record by including most of the sources of uncertainty. I am not at all sure that that effort is complete.

    I think that with other groups looking into the temperature records that perhaps the closed fistedness that, for whatever reasons, CRU chose to apply are a thing of the past. Maybe the CRU data set will be a thing of the past.

    “But really it comes back to unreasonable expectations of CRU. They aren’t data sources. If you don’t like what they have, do what they did – go to the sources.”

  25. Kenneth, that’s a fair comment. The first efforts to gather temperatures could have used better data preservation etc.

    The point is, they were the first. Much criticism 25 years later, but at the time no-one else was doing anything at all.

    I can agree that a better repository is required. But this need is being heaped on CRU not because they have responsibility (or funding) to do that, but simply because they have produced, with limited resources, one of the best indices so far.

  26. EJ said

    Mr. Jeff, did you apply to be a reviewer?

  27. EJ said

    Who is the reviewer of uncertainties? Or do uncertainties don’t matter?

  28. Bruce said

    #18 Nick you know perfectly well the data for Canada is total crap. Is that what the C in CHTD stands for? Crap?

  29. George said

    Once you know the stations, the daily data are often available elsewhere on the ‘net. What Nick is saying is consistent with problems noted elsewhere. For example the time McIntyre noticed odd “hot spots” in Finland and Russia and it turned out some stations had re-reported the previous monthly average for the current monthly average. This is very obvious during spring and fall at high latitudes when the difference in temperatures between months is great. It might not be so noticeable in June/July or Jan/Feb.

    So that the stations report a monthly average is consistent with what has transpired in the past when there were problems. There are often problems in the daily data, though, particularly with things using METAR where a -15 will turn into a +15 because someone used a “-” char before the temperature on the METAR form and not “M” (for minus).

    The important thing is to know the station IDs. The data can be collected elsewhere. Yes, you will have to “work” for it.

  30. page488 said

    Ok – I wish someone would explain to me, in logical and reasonable terms, why original data can’t be kept, preserved, etc.

  31. Kenneth Fritsch said

    My point was, of course, that while many climate scientists have had to know for many years that accurate and transparent temperature records were required for their work, they were mostly content to use whatever was handed to them without asking a lot of questions or doing independent auditing or making a determined effort to estimate the uncertainty of the data. Temperature data collection and processing is not a very glamorous or probably personally recording work for a scientist and thus they are probably not going to do a very good job. I think that the entire climate science comunity has to share the blame in these matters – unless I missed one of them in past times voicing concern about the temperature record.

    Also a point that might get lost in these discussions is the matter of local temperature records that are required for reconstruction studies and might some day, when the resolution improves, be required for climate modeling. Currently the effort seems more orientated towards the temperature record getting the global temperature right and maybe large regions of the globe. The BEST effort appeared to make a blackbox out the temperature records in order to obtain global and regional averages. Other interested parties seem to be more interested in showing how few “good” stations are required to obtain a good spatial coverage of the globe or large regions of the globe and less interest in the local climates and what that might mean in the overall scheme of things.

  32. #30
    Easy, it is kept. By the originators, the met offices. Like PJ, you can get a copy, which you can keep or not as you wish. The data is still there.

  33. George said

    For example, see http://wattsupwiththat.com/2010/04/22/dial-m-for-mangled-wikipedia-and-environment-canada-caught-with-temperature-data-errors/

    There’s also a posting concerning this and GISS

    http://wattsupwiththat.com/2010/04/17/giss-metar-dial-m-for-missing-minus-signs-its-worse-than-we-thought/

  34. page488 said

    #32 I don’t buy that

  35. Niels A Nielsen said

    Nick: “But this need is being heaped on CRU not because they have responsibility (or funding) to do that, but simply because they have produced, with limited resources, one of the best indices so far.”
    For whatever good or not so good reason CRU did not keep record of which raw data went into their indices. Jones has now admitted as much. How could you know then if it is “one of the best indices”? You can’t of course and it seems similarly self-evident that this huge embarrassment is the main reason for Jones’ stonewalling of requests for data.

  36. #35 Niels,
    No, PJ isn’t saying that they don’t have a record of which data went into the indices. He’s saying that they haven’t kept the actual numbers. The data still exists, and can be got from the originators. Most is now in GHCN anyway and has been public for at least 15 years.

  37. George said

    “How could you know then if it is “one of the best indices”?”

    That’s my major problem with HadCRUT is they don’t KNOW which stations went into it according to Jones. How can they keep it properly “adjusted” over time if they don’t even know which stations were used during which time periods? I think that is the source of a lot of the skepticism of that database.

    I believe there should be an institution somewhere on the planet that does NOT engage in climate modeling that should keep a very high quality database that other institutions that are doing different sorts of modeling could use as a reference database. So whey the University of Hooterville produces their model that projects one possible future and Pixley College produces theirs, at least you know they started from the same place. As it stands now, not only are the models different, so are the adjustments, and the actual raw data collection. If two institutions vary in their conclusions you have no clue where the source of that divergence may be. They might CLAIM it is on their selection of forcings and feedbacks but if they have different sets of input data that are massaged differently when input, who the heck knows?

    I think there is a case to be made for a respected institution someplace creating and maintaining a standard climate research reference database. Or that might be overkill if we are at the end of a 30 year period if natural climate variability, climate begins to cool, CO2 had nothing to do with it, and nobody really cares to hear from a “climate scientist” again in their entire life. At that point the priority is for such a thing is going to drop pretty close to zero and the “green” cash generated by “climate change” will stop pouring in to UEA and other institutions. Tyndall will dry up and blow away, and that will pretty much be the end of the IPCC and the UNFCCC.

  38. #37
    “they don’t KNOW which stations went into it according to Jones”

    Do you have a quote for that?

  39. steven mosher said

    The purpose of steve mcintyre’s request for data was to compare raw data with adjusted data.

    The point was to illustrate that CRU added no value.

    Had Jones actually been willing to share raw data it might have been easier to recover. Some people who asked for it were more meticulous record keepers.

    as for storage. I still have 1/4 streamers from the early 80s. boxes of my old data.

    That said, one hopes that somebody goes about the task of getting orignal data.

    the answers wont change, but a complete accounting is a good thing

  40. George said

    “B) identification … of the stations used in the gridded network which was used as a comparandum in this study” I have been in conversation with Dr. Jones and have been advised that, in fact, we are unable to answer (B) as we do not have a copy of the station data as we had it in 1990. The station database has evolved since that time and CRU was not able to keep versions of it as stations were added, amended and deleted. This was a consequence of a lack of data storage comparable to what we have at our disposal currently.

    6. We’ve altered the version that you have anyway. We’re also in the process of doing more of this.
    7. You’d need to waste your time combining the two parts of the data and removing the stations that don’t get used.

    [implies to me that just because a station is in the 'list' doesn't mean its data actually got used]

    from 1111.txt

    ———–

    Gents,
    My head is beginning to spin here but I read this as meaning that he wants the raw station data; we don’t know which data belongs to which station, correct? Our letter stated:

    “We can, however, send a list of all stations used, but without sources. This would include locations, names and lengths of record, although the latter are no guide as to the completeness of the series.”

    from 1184.txt

    ————
    Dear Mr. Palmer:
    Thank you for your reply (attached below). However, I fear that it is totally unresponsive. I had asked for a list of the sites actually used. While it may (or may not) be true that “it appears that the raw station data can be obtained from [GHCN]“, this is meaningless without an actual list of the sites that Dr. Jones and his team used. The debate about changes in the climate is quite important. Dr. Jones’ work is one of the most frequently cited statistics in the field. Dr. Jones has refused to provide a list of the sites used for his work, and as such, it cannot be replicated. Replication is central to science. I find Dr. Jones attitude quite difficult to understand, and I find your refusal to provide the data requested quite baffling. You are making the rather curious claim that because the data “appears” to be out on the web somewhere, there is no need for Dr. Jones to reveal which stations were actually used. The claim is even more baffling since you say that the original data used by CRU is available at the GHCN web site, and then follow that with the statement that some of the GHCN data originally came from CRU. Which is the case? Did CRU get the data from GHCN, or did GHCN get the data from CRU?

    from 3965.txt

    ————

    I can then go through this list and extract the stations that have come from certain countries, with whom we have gotten data directly and signed agreements. This isn’t just from the countries (but does include a few scientists).

    from 4131.txt

    ————-

    And I really love this one:

    1. UEA has denied access to the data to McIntyre (and at least two others in the past) – in 2007. One of the three appealed and that appeal was rejected. We would look stupid if you released the data now.

    It certainly looked to me like they weren’t sure WHICH stations were used under which circumstances. They might have had an aggregate list of stations with available data, but it doesn’t look to me like they could ever confirm exactly which data were used when.

  41. StuartR said

    I don’t understand why every item of raw temperature data measurent isn’t being kept in single or central databases somewhere. I think laymen are quite justified to feel a basic discomfort with the data adjustments because people today are quite capable of understanding the meaning and life span of digital data, especially if you consider the example of digital photography. We know the original can be colour balance, adjusted, lightened, resolution reduced, etc, before being used in whatever presentation – newspaper or article. People even know that photos can be artificially stitched together. But at the same time people are aware that unless the original verified time stamped photo – that represents the light values that fell upon each CCD pixel when it was taken – can be shown to exists somewhere, they really are not required to 100% believe the story any derived photo claims to tell.

    So without this temperature database and also knowing that the scientist don’t even seem to care that there is an original temperature photo kept somehwere, and want us to buy the rather weak excuses that this would be too egregious a task for any one institution to manage, and we should just trust their best judgement for all time on this snapshot, I think we lose trust. It seems that the lack of push for this simple concept is because of some possibly overt but more likely subconcious desire to be quite content about the plausible deniability and confirmation bias that can be inserted along the winding trail and intervening steps – between someone trudging out to read a number (at whatever interval) from a thermometer in the snow, and then that number getting smoothed and adjusted along with a load of other numbers.

  42. George said

    People even know that photos can be artificially stitched together.

    It’s even worse than that. If I can scan photographic film with a pixel resolution higher than the silver grain resolution on the film emulsion, I can “create” a photographic negative of a scene that doesn’t even exist in real life. That is commonly done (or was in the 1980’s) for such things as advertising photos and royal British wedding photos to correct even the slightest imperfection.

    There should be a standard “source of truth” that is common to all of the models, all of the databases, etc. Fine if you want to use a different adjustment method, but the raw data should be the same.

    If group A and group B arrive at different conclusions, to what extent did a difference of input data have a role in that difference?

    Also, if I am doing BOTH climate modeling AND keeping a database of temperatures (and their “adjustments”) there is a certain temptation to make the database validate the models. This is particularly so when an institution has a direct (and very large) financial interest in that validation. For example: CRU showing warming, Tyndall Centre making a lot of revenue from consultation on implementation of policy resulting from that “warming”. They are have a direct financial interest because the “hotter” it gets, the more money they make.

  43. George:
    I agree with your suggestions about common accepted data, and yes, someone should be funded to do it. GHCN goes a long way towards what you want.

    But on your quotes:
    B) identification …
    just says they don’t have the station data, not that they don’t know which stations were used.
    6. We’ve altered the version …
    not specific
    7. You’d need to waste your time combining the two parts of the data and removing the stations that don’t get used.
    just a mechanics issue. Yes, some stations don’t get used, for whatever reason. And it’s troublesome to sort them out.

    Gents,
    My head is beginning to spin
    This is just Palmer trying to figure it out. He’s not there yet.

    from 1184.txt

    ————
    Dear Mr. Palmer:
    Thank you for your reply …
    This is just a complaint that he didn’t get the list.

    from 3965.txt
    This is just about agreements
    ————

    ————-

    And I really love this one:

    1. UEA has denied access to the data to McIntyre (and at least two others in the past) – in 2007. One of the three appealed and that appeal was rejected. We would look stupid if you released the data now.

    It certainly looked to me like they weren’t sure WHICH stations were used under which circumstances. They might have had an aggregate list of stations with available data, but it doesn’t look to me like they could ever confirm exactly which data were used when.

    I disagree. None of your quotes prove that, and I think it’s unlikely. PJ lived with this stuff for years. I think he would have endless station lists, maybe disorganised, and probably has them in his head as well. He would have innumerable (lineprinter) tables of results with station names.

  44. George said

    Nick, from my reading of other data, and it might have been the CG1 emails, particularly Harry’s stuff, I remember there were many stations whose data were not used for one reason or another. Maybe the series were incomplete, sometimes they were in the wrong format, sometimes there was inconsistent naming of the same station, there were all sorts of problems. He was having a real bugger of a time trying to get the data into the database. I think most of my recollection of this is from the emails sent by Harris in the climategate1 mails. It certainly made the database look like crap. I’ll go back and have a look after I’ve had some sleep.

  45. George said

    When you see things like Jones saying to Hansen that he is “rooting” for temperatures to increase:

    4184 Jones:

    [to Hansen] Keep up the good work! [...] Even though it’s been a mild winter in
    the UK, much of the rest of the world seems coolish – expected though given the
    La Nina. Roll on the next El Nino!

    It has to make any normal person wonder just exactly how objective they can really be. When you throw into that mix the rhetoric about “The Cause”:

    By the way, when is Tom C going to formally publish his roughly 1500 year
    reconstruction??? It would help the cause to be able to refer to that
    reconstruction as confirming Mann and Jones, etc.

    3940 Mann:

    They will (see below) allow us to provide some discussion of the synthetic
    example, referring to the J. Cimate paper (which should be finally accepted
    upon submission of the revised final draft), so that should help the cause a
    bit.

    0810 Mann:

    I gave up on Judith Curry a while ago. I don’t know what she think’s she’s
    doing, but its not helping the cause

    Don’t try to give me any bullshit about these people just being objective scientists here because it is abundantly clear that they are not.
    They have been completely discredited as far as I am concerned and I don’t mean at a scientific level, I mean at a more fundamental human level. I wouldn’t trust one of these people on any subject. They are very obviously slimy. “The Cause”, indeed.

  46. Brian H said

    #23;
    figured you’d go there. I go right back to the days of punch card data and program decks, and well remember the awe at seeing a 4′ high Bernoilli drive tower capable of holding 10 MB! But the point is that any data computer analysed was digitally stored, and at the very worst was/should have been on tape drives. “Lost in my messy office” (PJ) was a flat-out lie.

    And, as is repeatedly pointed out to you, but breezily ignored every time, the “original data” files you keep linking to are monthly massaged summaries, etc. As for the raw data being ‘in the hands of the originators’, that’s a fob-off of massively pathetic proportions.

  47. Craig Loehle said

    In the 1970s huge files were kept on 9-track tapes–England didn’t know about that? Nick seems pretty sure that Phil only received monthly values from stations around the world. How do you know that Nick? Where does it say that? Let’s figure out how big a file would be for 1 station: 365 days x 2 for max min temps times 20 bytes per record x 100 yrs. rounds to 1.6Mb or about the same as a single pic from my digital camera. And DOE paid these guys to assemble this data in the late 1980s. Are you claiming countries in the late 1980s couldn’t and didn’t send such a small file somehow to phil? Really.

  48. Craig Loehle said

    Let us grant Nick’s point that assembling the CRU data was an informal academic exercise at a time that it was not a policy-critical item. Ok, so they were sloppy as many academics are. But that does not inspire confidence, does it? It has been clear for at least 20 years that this data is important, and tons of grants (tens of millions a least) have been given for research USING this data but no one thought it important to get some funds to improve it? I guess database clean up isn’t exciting to big thinkers like Hansen and Jones but the rest of us care about it a little more. As it has turned out, there are hilarious errors in the data, like stations “located” in the ocean, errors in GISS in ascribing urban/rural, entire regions like Canada having recent data missing. It is not up to me to fix these–how about GISS and Hadley?

  49. cce said

    It’s interesting to compare adjusted and “raw” data series, and I suppose it’s possible to construct useful experiments with that “missing” data. It was stupid for CRU to stonewall the release of it. However, the the primary purpose of some (not all) is not to do any real science, but instead show difference plots of specific stations and then carry on about the nefarious intentions behind those adjustments.

    Of course, actual analysis says otherwise. e.g.:

    http://stevemosher.wordpress.com/2011/08/01/cru-data-in-rghcnv3/

    http://moyhu.blogspot.com/2011/08/first-templs-crutem3-reconstructions.html

    Question for Nick or mosher. Have you guys found stations in CRU for areas that are underrepresented in the others? That is, is there any “important” CRU-only stations?

  50. kim said

    George @ #45

    “They have been completely discredited….at a fundamental human level’. Yes, and they will need forgiveness ultimately, but where is the repentance, first?
    ================

  51. Kenneth Fritsch said

    “No, PJ isn’t saying that they don’t have a record of which data went into the indices. He’s saying that they haven’t kept the actual numbers. The data still exists, and can be got from the originators. Most is now in GHCN anyway and has been public for at least 15 years.”

    Nick, please consider the implication of your apparent attempt to defend CRU here. CRU lost the original records and you imply that finding the originals again is no big deal. That being the case, is not CRU’s reputation as the keeper of critical temperature data put in even more jeopardy? CRU has publically stated that they could work backwards to obtain the original data but thought that was a wasted effort, but you are implying that restoring the originals is even easier than that.

    The important point here is that any scientific, or even simply a data gathering body, worth its salt is going to be very careful about maintaining the original data when it provides that data in adjusted form. Any future changes in how that original data might be better adjusted requires the original data. USHCN/GHCN provides a good example of that with their Versions 1,2 and 3.

    I do not see on the face of it anything sinister in what CRU has done, but rather that the maintenance and potential improvements of their temperature data set was not well thought through and more likely they were merely going through the motions. The hesitancy to be forthcoming about the CRU temperature data was probably more a matter of embarrassment when CRU went to look and found substantial (original) data missing. CRU might not even have known about the missing data until it was requested. If that is the case then I can hardly see why the requests would be considered harassment.

    By the way when someone does make a request in this nature and the reply by the body requested or its defenders comes down to “if you want the data so badly you should put together your own data set”, I find that to be a very emotional and childlike reaction. Requests for data and transparency are not necessarily to obtain the data but rather to determine if the data continues to exist and in what form.

  52. phi said

    Cce,

    “It’s interesting to compare adjusted and “raw” data series…”

    Indeed, it is very interesting but I do not see such comparisons in your links.

  53. 47, 48
    “Let’s figure out how big a file would be for 1 station:”
    Well, no need to figure – I have the GHCN daily data on my computer. Mosh tracked it down. It’s 3.91Gb (794 Mb zip). Doesn’t sound much now, but it would take a year to download at 1200 Baud.

    In my only professional foray in climate science, I analysed part of the then recently digitized BoM Australian data (temp and rainfall) in 1980. Mostly daily – a little 3-hr. The only way they could transfer it to me was by putting it on a very expensive 20 inch “floppy” (PDP11) and sending by courier. Fortunately I was in the same city.

    “As it has turned out, there are hilarious errors in the data, like stations “located” in the ocean, errors in GISS in ascribing urban/rural, entire regions like Canada having recent data missing. It is not up to me to fix these–how about GISS and Hadley?”
    No, it’s not up to them either. They are data analysts, not suppliers. Actually, the Canada story is an interesting one in terms of how the legalism prompted by “auditing” just obstructs. EC actually submitted the CLIMAT forms to WMO, and they came to NOAA, which assembles the GHCN database. But they were not filling out the normals page – local long-term averages. It’s a trivial calc, but NOAA now won’t take the responsibility – they send to back to the source for amendment.

    I don’t know why EC didn’t fix it promptly, but I suspect it is more of the same. There’s plenty of data to calculate a long-term average, but something does have to be done about missing values. Scientifically trivial, but who wants the hassle that people make? Not EC, I suspect.

    And so it goes. GISS now won’t modify the data at all, even for obvious things. They send requests to NOAA, then it goes back to the provider.

  54. #51
    “is not CRU’s reputation as the keeper of critical temperature data put in even more jeopardy?”
    No. Again CRU is not the keeper of critical temperature data. It analyzes data. It does not originate it.

  55. #49 CCE,
    No, I don’t think there is any region where CRU has markedly greater station coverage than GISS. I made a KMZ file which shows all four station sets (GHCN, BEST, GSOD and CRU) in Google Earth in different colors, described here. Nothing stands out.

  56. Kenneth Fritsch said

    ““is not CRU’s reputation as the keeper of critical temperature data put in even more jeopardy?”
    No. Again CRU is not the keeper of critical temperature data. It analyzes data. It does not originate it.”

    Nick, quibbling and spinning what you hear like this does not become a man of your intellectual stature. Is not it rather obvious that I am not intimating that CRU makes the measurements? I believe we are all aware that CRU does not originate the data. It is clear that CRU collects the data, adjusts the data and keeps it in a data set. If you were making a small joke here I would suppose you could point out that a “keeper of data” would more aptly describe a group that actually keeps data and does not lose it.

    I would not expect someone of your POV to admit it, but I am sure that involved persons had higher expectations of CRU as a keeper of the data than was revealed more recently about the process. That CRU made mistakes is not the major issue but rather a transparent view of what they were doing is.

    By the way your comment does not reply to my main issue of a responsible data keeper and adjuster not making a seemingly simple effort to reacquire the original data and the importance of that original data in defending and improving the adjustment processes.

  57. Niels A Nielsen said

    Kenneth: “I would not expect someone of your POV to admit it..” I’m not convinced Nick has a point of view as the term conventionally understood. His endless “quibbling and spinning” is not intended to argue a point of view.

  58. This is not quibbling or spinning. In response to Kenneth’s
    ““is not CRU’s reputation as the keeper of critical temperature data put in even more jeopardy?”
    I say that that is nonsense. CRU seeks no such reputation and has no such responsibility. It is a university research and teaching department.

    And if you think it has, spell it out. Of what critical temperature data are they the keeper? Who assigned them that responsibility? Who funds them for it?

  59. Jeff Id said

    Nick,

    Can you provide all of the data and code from your recent temperature work?

    My guess is that the answer is yes.

  60. j ferguson said

    Nick,
    US Department of Energy? Maybe CRU is the wrong appellation, but i’d come to believe that although they might not have an official sanction to collect temperatures, they do and this is why they are supported by US funding (no doubt in part). From what you suggest, i suspect that the analysis is the reason for the support and the collection only incidental to it.

    so many of us seem confused on this issue.

  61. Jeff,
    Code I try to keep posted online, though I’ve been a bit slack lately. Temperature data, not systematically. I download GHCN etc as needed, and ditch old versions when they pile up.

    And there’s absolutely no chance I could find data from the 1980’s. If it still exists, the format is probably unreadable.

  62. AMac said

    Nick Stokes #40

    > I say that that is nonsense. CRU seeks no [reputation as the keeper of critical temperature data] and has no such responsibility…. And if you think it has, spell it out. Of what critical temperature data are they the keeper? Who assigned them that responsibility? Who funds them for it?

    I agree with Nick Stokes.

    His insight largely exonerates CRU, as far as many of the most serious charges leveled against it, and its director.

    There is however another facet. That relates to the position of the climate concerned community as a whole.

    I keep coming back to the example of the setup for clinical trials of drugs and devices. Which nearly everyone outside this insular group grasps. But which seems to cause no end of puzzlement to the leaders and foot soldiers of the c.c.c. itself.

    Clinical trials are used to make public policy decisions of great import. People regain their health, or die. Careers and fortunes are made, or lost.

    If global warming climate change research is a fascinating and sometimes-useful academic specialty, then I think things are pretty much okay. This field’s standards seem to be quite a few notches below those of more staid physical-science disciplines. But perhaps it’s on a par with trendy social sciences; maybe better-off than some.

    The problem arises with the claims of the more-excitable plurality or majority of the people in this field. That the need for public action is urgent; that billions or trillions must be re-allocated, that the crisis demands that lives and economies be upended.

    .

    It’s pretty clear, really. This time is different. Only bad people could disagree.*

    (* That last sentence is “per Michael Tobis, Michael Mann, et al.” rather than “per Nick Stokes,” though Nick’s comment inspired this rant.)

  63. John #60,
    PJ did collect copies of a lot of temperature data in the 80’s from the various mets, WWR etc. This was largely before he got DoE funding. And some derived database obviously continued to enable production and maintenance of CRUTEM and its predecessors.

    But even as a collection, it became not unique in the 90’s when GHCN started up. That was part funded through DoE, through CDIAC. It was made widely available, and was then maintained by NOAA.

  64. Amac,
    Yes, drug companies face onerous responsibilities for trialling and releasing drugs. But organic chemists, who discover the underlying science, do not. They operate as normal scientists, and do indeed discover things. The drug trialling is not an agent of discovery.

    For the people who want scientists to operate as engineers, auditors etc, I suggest, no, get engineers to do the job. Or folks expert in Sarbanes-Oxley or whatever.

    And of course it won’t work. The pressure is on scientists because they have the methods that do let you find out about the real world. And if you want science to work, there’s no use demanding that they all turn into engineers.

  65. AMac said

    Nick Stokes #64 —

    Are you talking about what the public-policy implications of drug & device work should be? Or about what the public-policy implications of global warming climate change work should be?

    In the most general terms, we are (I am, anyway) discussing the concept of “good practices” (Wikipedia), which are required in clinical trails and effectively absent from global warming climate change work.

    We may be in agreement.

  66. Amac,
    Science is science. It tells you about how the world is. You can’t change that – you can only find out about it, as best you can. That’s what scientists do.

    Public-policy implications are worked out in response by people charged to do that sort of thing. You can impose on them whatever conditions you think appropriate.

    Scientists are citizens, and have and express views. You should respect those as you do of other informed citizens. But they do not seek to control policy, and there is no mechanism for them to do so. It is up to the policy makers to get and evaluate scientific advice.

  67. AMac said

    Nick Stokes #66 —

    > But [global warming climate change scientists] do not seek to control policy, and there is no mechanism for them to do so. It is up to the policy makers to get and evaluate scientific advice.

    Yes. In the best of all possible worlds. (With apologies to Pangloss.)

  68. Brian H said

    Nick Stokes said
    December 18, 2011 at 9:29 am | Reply w/ Link


    Scientists are citizens, and have and express views. You should respect those as you do of other informed citizens. But they do not seek to control policy, and there is no mechanism for them to do so. It is up to the policy makers to get and evaluate scientific advice.

    Utterly disingenuous, as shown by the strenuous gatekeeping efforts of The Team. They very much wanted to shape and control policy, and succeeded.

    Ability to select the Garbage In is the ability to shape the Garbage Out.

  69. AMac said

    Y’know, I just looked at what’s near the top of my Google Reader page, and came across this short post on the genetics of human skin color by Razib Khan, based on his reworking of the assumptions and math of some forty-year-old work by geneticist L. Cavalli-Sforza and an associate.

    This is the caliber of work that an (admittedly excellent) science blogger bangs out in genetics, based on work done before much of anything was known about the details of human inheritance, by an (admittedly excellent) geneticist.

    If you were the CEO of a skin-care company and cared about this subject, would you bet your employer’s future on Khan being right? Probably not, considering the age of the study and its small sample size and all the rest. If you were prudent.

    Then I look back at what acclaimed global warming climate change science looks like. Think of policy based on that caliber of work. And shudder.

  70. Arthur Dent said

    As a scientist with many years experience working in the pharmaceutical industry I have to say Mr Stokes is talking absolute bullsh*t. The people who conduct clinical trials are also scientists NOT policy makers and they work to GCP because that’s the way all good scientists should work, especially when the outcome of your studies has potentially serious implications for patients. The policy decision, whether to allow the drug into the market is taken based on the high quality science delivered by the clinical trials.

    I agree with Amac #62 those people working in climate science who want us to take decisions costing society immense amounts or money need to work to a much higher degree of QC than is currently evident.

  71. Craig Loehle said

    It is reasonable to accept a lower standard in science when someone is doing exploratory work. The pioneer in an area has no data with which to even formulate a hypothesis, nor funding often. So he does exploratory work and then people see that this may be interesting to look into. Science should not stop there when the issue is critical, and excuse sloppy record keeping by CRU or upside down Tiljander (or any Tiljander since it can’t be calibrated).

  72. Kenneth Fritsch said

    “This is not quibbling or spinning. In response to Kenneth’s
    ““is not CRU’s reputation as the keeper of critical temperature data put in even more jeopardy?”
    I say that that is nonsense. CRU seeks no such reputation and has no such responsibility. It is a university research and teaching department.

    And if you think it has, spell it out. Of what critical temperature data are they the keeper? Who assigned them that responsibility? Who funds them for it?”

    http://www.cru.uea.ac.uk/cru/about/

    “The Climatic Research Unit is widely recognised as one of the world’s leading institutions concerned with the study of natural and anthropogenic climate change.
    Consisting of a staff of around thirty research scientists and students, the Unit has developed a number of the data sets widely used in climate research, including the global temperature record used to monitor the state of the climate system, as well as statistical software packages and climate models.

    The aim of the Climatic Research Unit is to improve scientific understanding in three areas:
    • past climate history and its impact on humanity;
    • the course and causes of climate change during the present century;
    • prospects for the future.
    The Unit undertakes both pure and applied research, sponsored almost entirely by external contracts and grant from academic funding councils, government departments, intergovernmental agencies, charitable foundations, non-governmental organisations, commerce and industry.

    Alongside its research activities, the Unit has an educational role through its contribution to formal teaching with the School of Environmental Sciences (most notably, the MSc in Climate Change) and various forms of in-service training including postgraduate education. It is regarded as an authoritative source of information on both the science and policy aspects of climate change by the media and maintains a high public profile.”

    Nick, you may want to quibble and I understand that, given your POV, but my replies are aimed at other readers of this thread. There is no doubt that CRU has promoted its temperature data set, amongst other work, as a means of obtaining research funds from outside sources. The CRU temperature data set has been widely referenced in climate science papers and is the basis of a lot of conclusions presented in those papers. The final sentence from the CRU posting should be read again:

    “It (CRU) is regarded as an authoritative source of information on both the science and policy aspects of climate change by the media and maintains a high public profile.”

    There is no doubt that CRU was/is considered an authoritative keeper of critical temperature data by other climate scientists and by CRU itself. Perhaps the defenders can imply that much of that consideration was only for marketing purposes and that CRU actually felt no responsibilities – which could be evidenced by their failures to maintain the original data. That view would, of course, make a charade of the science referencing the data set.

    The CRU compiles the land component of the record and the Hadley Centre provides the marine component. The merged record is used by the Intergovernmental Panel on Climate Change in all its publications.

    Again, I think no one really has an excuse for CRU being so nonchalant about misplacing some of their original data and not attempting to reacquire it.

    So who should I believe: Nick Stokes, CRU or my lying eyes?

  73. cce said

    Phi,

    Those posts by Nick and Mosher compare GHCN “raw” to the CRU (“value added”) data. If there was nefarious intentions afoot, it would be obvious in the results. Comparing raw and adjusted data from *specific* locations is interesting, and CRU should have released everything they had, and they shouldn’t have lost the “original” data. But it’s still just a sideshow. CRU holds contempt for skeptics that’s for sure, but they weren’t goosing the data to achieve a desired result.

  74. Jeff Id said

    Cce,

    “CRU holds contempt for skeptics that’s for sure, but they weren’t goosing the data to achieve a desired result.”

    Correction:

    They weren’t goosing the thermometer data to achieve a desired result. Paleoclimate data is another story.

  75. david said

    “2440 Jones:

    I’ve been told that IPCC is above national FOI Acts. One way to cover yourself and all those working in AR5 would be to delete all emails at the end of the process

    1577 Jones:

    [FOI, temperature data]
    Any work we have done in the past is done on the back of the research grants we get – and has to be well hidden. I’ve discussed this with the main funder (US Dept of Energy) in the past and they are happy about not releasing the original
    station data.

    “delete all e-mails” “has to be hidden” “why should I release the data to you when you just want to show something wrong with it” The hockey team abandoned the scientific method long ago, there is no excuse.

  76. david said

    63.Nick Stokes said
    e mail 1557 is particularly cogent to Nick’s comment..December 18, 2011 at 9:02 am
    John #60,
    PJ did collect copies of a lot of temperature data in the 80′s from the various mets, WWR etc. This was largely before he got DoE funding. And some derived database obviously continued to enable production and maintenance of CRUTEM and its predecessors.”

    A clear effort was made many times to obstruct anyone engaging in the scientific method of replicating the Global data sets. Nick please supply the missing context to e-mail 1557 as well as 2440. Thanks in advance.

  77. david said

    It is curious that the DoE was founded in the 1970 with the goal of helping the US become independent of Middle East oil. This is an easily achievable goal, yet the DoE, with 18,000 direct employees, and about 100,000 contract emplyees, Phil Jones being one of those, has only achieved the opposite goal, and accoding to Phil Jones, who in email 2440 is clearly under the impression that the IPCC is above FOI laws, the DoE also wished to keep the original data free from transparency. Full stop Nick, there is no excuse.

  78. phi said

    Cce,

    I would not call it nefarious intentions. That said, I’m not sure that Nick and Mosher actually use raw series, I mean series of temperatures as read on thermometers. I can only talk about what I saw and this concerns only limited areas, including the Alpine region. In this case, the adjustment of discontinuities causes a warming of 0.5 ° C in the twentieth century. These adjustments are not made by CRU who get series already adjusted. The values for the United States are of the same order of magnitude.

  79. Kenneth Fritsch said

    “But it’s still just a sideshow. CRU holds contempt for skeptics that’s for sure, but they weren’t goosing the data to achieve a desired result.”

    I do not think any thinking skeptic sees anything nefarious about CRU, but rather some may hold as I do that they did not do a good job of maintaining and improving the critical temperature set. I think that CRU and the climate scientists using the data were satisfied that it provided the right answers. We have many sources of temperature data over the past approximate 3 to 4 decades that all agree that we had warming over that period. My point in criticism of CRU is that I think they became satisfied and lazy and did not continue to sufficiently push the envelop in determining the uncertainty in the measurements, a more complete accounting local temperature and temperature variations that might reveal the degree of localization of climate and any statistically significant difference between temperature sets. I hear too much chatter in the generalized defense that all data sets provide the same picture and not concentrating on where the sets might differ – and they do.

  80. #78
    “That said, I’m not sure that Nick and Mosher actually use raw series, I mean series of temperatures as read on thermometers.”
    Pretty much. Nowadays “reading a thermometer” doesn’t describe it very well. But the GHCN data that we use comes directly from the CLIMAT forms submitted by the met organisations every month.

  81. #77
    “email 2440 is clearly under the impression that the IPCC is above FOI laws, the DoE also wished to keep the original data free from transparency.”
    OK. What FOI laws do you think the IPCC is subject to? Whose laws?
    And what “original data” are you talking about?

  82. Kenneth #77
    “the Unit has developed a number of the data sets widely used in climate research”

    If you look at that in the original, it has a link. And that takes you to a page, starting with 5×5 gridded data. Computed output, including their famous indices. This is not “critical temperature data” of which they are the “keeper”. It can be computed again any time. For CRUTEM I listed their sources above.

  83. AMac said

    Nick Stokes,

    You’re Racehorseing again. Reference. If you want to have a serious conversation, you’ll engage Kenneth Fritsch’s points, supra. Otherwise, just more climate silliness.

  84. j ferguson said

    Nick,
    suppose instead of what he did write, Phil had written something like “Why should i give you our raw data? We went to a lot of trouble to get it, but it’s incidental to our efforts and those you CAN have.”

    I’m beginning to think that the original temperature reports are something like field notes in my business where dimensions of existing construction as measured in the field are recorded. The dimensions are transferred to drawings and those become the basis of any further work – not the field notes. On the other had, field notes are ALWAYS preserved against the possibility of an error in data transfer. But I wouldn’t give anyone the field notes unless forced to.

    But maybe this isn’t quite analogous. I’m thinking that Steve M. wanted the raw data so that he could better analyze the effects of the homogenization – something he couldn’t reliably do unless he had the exact same things which Phil and company had started with.

    Jeff, sorry to take up this space, but I’m a slow learner and there was this opportunity…

  85. #83
    Well, Amac, I can make points too, and have. I asked before this simple qn – of what critical temperature data is CRU the “keeper”. Specifics? I haven’t heard any.

  86. diogenes said

    general question – if you as governmental agency provided funds to CRU to maintain a climate database, what standards of data retention would you reasonably expect? In your view, has CRU fulfilled those requirements?

    As an outsider, i would expect them to retain the basic measurement data. i would expect explanatory detail behind any adjustments or alterations made to that base data. And I would expect them to retain all the data they use – whether provided to them or generated by them.

    Are those reasonable requirements and were they implicit in the agreements drawn up by CRU?

  87. j ferguson said

    Diogenes,
    Retaining is one thing and publishing or otherwise making available is another. Isn’t your point more toward the publishing?

  88. tallbloke said

    #85 Nick

    Blimey Nick, even the CRU don’t know the answer to that. How do you expect Amac to know?

  89. TB #88,
    TB, that reminds me of the thread at your site
    IPCC declares itself immune to FOI requests.
    Evidence? Where’s that “declaration”?

    Oh, they kept it secret, of course!

  90. Re: j ferguson (Dec 18 19:20)

    JF, I agree with your point about not giving away the original material. I saw some of this at BoM back in 1980. Handwritten logs, typed tables etc. You need trucks to take it away anyway.

    But CRU was not withholding that material. They never had any (nor claimed to have).

  91. david said

    81.Nick Stokes said
    December 18, 2011 at 6:22 pm
    #77
    “email 2440 is clearly under the impression that the IPCC is above FOI laws, the DoE also wished to keep the original data free from transparency.”
    OK. What FOI laws do you think the IPCC is subject to? Whose laws?
    And what “original data” are you talking about?

    Nick I did not write the email, PJ did.
    2440 Jones:

    “I’ve been told that IPCC is above national FOI Acts. One way to cover yourself and all those working in AR5 would be to delete all emails at the end of the process.”

    The point has been made at least a 1/2 dozen times, each public funded participant is subject to their respective nations FOI laws. The IPCC has no authority to deny FOI requests. (Not hard to grasp)

    As to what orginal data?, well ask Phil, he wrote 1577 Jones:

    [FOI, temperature data]
    Any work we have done in the past is done on the back of the research grants we get – and has to be well hidden. I’ve discussed this with the main funder (US Dept of Energy) in the past and they are happy about not releasing the original
    station data.

    So, by any work I suppose P.J., the serial deleter of e-mails, by use of the phrase “Any work we have done in the past” means all the work he has done in the past, the word any indicting inclusion of all and exclusion of none.
    .

  92. Kenneth Fritsch said

    Nick Stokes, I suppose we could all say that it is fun to see how far Nick Stokes can reach to defend the indefensible, but it does get tiresome at some point.

    “Scientists are citizens, and have and express views. You should respect those as you do of other informed citizens. But they do not seek to control policy, and there is no mechanism for them to do so. It is up to the policy makers to get and evaluate scientific advice.”

    You are either naive or being disengenuous. How many times have we heard from both scientists, advocates and policy makers that there is a science consensus and it is that consensus that drives there arguments for government mitigation. Mann talks about the “cause”. Policy makers and the IPCC surely indicate to you that the policy should be driven by the science. With your POV you are perhaps not ready to admit that the science that the IPCC publishes is rather onesided and the scientists on the favored side could have immense influence on policy.

    “If you look at that in the original, it has a link. And that takes you to a page, starting with 5×5 gridded data. Computed output, including their famous indices. This is not “critical temperature data” of which they are the “keeper”. It can be computed again any time. For CRUTEM I listed their sources above.”

    You are making no sense here. That a group collects and adjusts data and refers to it on the web site as a major part of their mission and that data is in turn used for reference and analysis in a published papers would certainly qualify it as critical data and CRU as the keeper of that data since they maintain the data set. The fact that it can be computed anytime or that you can provide a general list of sources has nothing to do with the issue we are discussing.

    You vaguely refer the data as something easily reconstructed from the original. If you know this for a fact you should be able to provide more details than the hand waving you have been doing. Please provide the specifics in a comprehensible outline.

  93. Anonymous said

    David,
    “The IPCC has no authority to deny FOI requests. (Not hard to grasp)”

    The IPCC does not have to deal with FOI requests. You have not shown that any FOI law is applicable to them.

    As to what other organisations do with requests regarding material they hold, that is a matter for them subject to local law. It does not make the IPCC subject to FOI.

    And you said “the DoE also wished to keep the original data free from transparency”
    That was the original data I was asking about. What did you have in mind?

    As to the Jones quote that you gave, there’s clearly context there; he doesn’t mean that all his work has to be well hidden. He’s been publishing for years. It says that some material has to be well hidden because of something to do with the associated research grants. It’s not clear what, but there could well have been something in the agreements about release of the data.

  94. Anonymous said

    Kenneth #92,
    There are three things here:
    1. How the real world is actually going to behave
    2. What we can figure about how the real world is going to behave
    3. What should be done about it.

    Policy makers have to decide 3) and need to do their best to align it with 1. That’s their responsibility. Their best way to do it is likely to talk to the people who have been trying to figure it out on a full-time professional basis. Scientists. And yes, those have a consensus, as they do about most areas of science. Nothing wrong with that.

    But policy-makers can listen to whoever they like – in fact. Rep Rohrbacher has been doing so. Scientists can and should present their views as clearly and forcefully as possible. If their views are persuasive, then indeed they will become the “favored side”. And yes, they aren’t your views.

    “The fact that it can be computed anytime or that you can provide a general list of sources has nothing to do with the issue we are discussing.”
    Of course it does. How can you be the “keeper” of data that can be computed any time. Which particular printout do you have to keep?

    “If you know this for a fact you should be able to provide more details than the hand waving you have been doing.”
    I do it myself. Compute indices. I could easily, as a byproduct, put out things like 5×5 gridded output.

    Even the task of putting the GHCN-style data set from SYNOP or from CLIMAT forms is within reach nowadays. Blogger Ron Broberg put together GSOD basically from SYNOP reports.

  95. Opps, different browser, #94 was Nick Stokes.

  96. j ferguson said

    Nick,
    Isn’t the issue “their data?” Sure there is station data out there, but without seeing the specific dataset Jones used, there is no way to know whether his input was compiled accurately, the missing records realized, discoverable errors dealt with and other errors innocently or even carelessly introduced. It seems highly unlikely that the Jones dataset could be identical to one assembled from the other public sources.

    If this is what we are discussing here, why is it unreasonable to expect Jones to make his specific “raw” dataset available? I agree that despite Nick’s suggestion that they were not running a raw data service, their publication of what they suggest to be scientific analysis implies availability of the specific data on which the work is based. Maybe not everything they have, but certainly everything that is input into their published work.

  97. #96
    “If this is what we are discussing here,”
    No, I don’t think it is. The charge is that they failed as “keeper of crucial temperature data”. That implies that the world no longer had the data. And that’s not true.

    You’re talking about wanting to check Jones work in his 1986 paper, which was the original issue of this “lost data” story. That’s different. It would be good if that could be done. But in 1986 it just wasn’t always expected that data should be preserved indefinitely in the form used in case someone wanted to check for typos twenty years on.

  98. j ferguson said

    #97 Nick,
    I get it. sorry for the distraction.

  99. A. C. Osborn said

    Nick Stokes will condone any behavour as long it is for AGW and not against it.

  100. Kenneth Fritsch said

    “I do it myself. Compute indices. I could easily, as a byproduct, put out things like 5×5 gridded output.”

    Nick Stokes, I asked for an outline of the details in how one would go about using the data CRU receives (original data) from it sources and putting that data into a finished form that CRU maintains as the keeper of the data set. Your reply is again vague and unsatisfying. You keep referring to , I think, the finished product and not the process that makes CRU the keeper of the data set from start to finish..

  101. Carrick said

    Nick Stokes:

    But in 1986 it just wasn’t always expected that data should be preserved indefinitely in the form used in case someone wanted to check for typos twenty years on.

    I still have data and emails going back to 1984. It was well understood that you keep original data files, log books, etc.

    My own major professor made copies of everything he did and kept the duplicates at his house (in the even of a fire). My mother, hardly a renown scientist, made copies of her data and kept it in a bank vault.

    On what basis do you make the claim “But in 1986 it just wasn’t always expected that data should be preserved indefinitely”. Who told you this, where did you read this or will you admit to just pulling this out of the air?

  102. Carrick said

    In any case the idea is risible that there was ever a period that people would have considered it appropriate to destroy original data for a product that is continuing to be updated.

    The data books for Millikan’s oil drop experiment are still available. Roland Eotvös reprinted his entire data set in his early works on geophysical prospecting. And some how Phil Jones wouldn’t have known what was appropriate?

    Seriously Nick, I don’t know what your motivation is in saying some of the things you say, other than to undermine Jone’s own supporters. Those of us who are in experimental science know better, and have known better, than to do what he did.

    The fact you obviously find his behavior acceptable just undermines you and the movement you claim to represent.

  103. Kenneth Fritsch said

    Nick Stokes, here is some history of CRU linked and excerpted at the bottom of this post and specifically the part related to the issue we are discussing directly below:

    “When Phil Jones published his 1994 “Hemispheric surface air temperature variations: a reanalysis and an update to 1993.” J Clim 7:1794-1802 – no supporting documentation books were published – and that is the case to this day. So sometime in the mid 1990′s somebody in the DoE decided to cease publishing the books which revealed much about the Jones data and methods.”

    It is those methods that are important to CRU being a keeper of a critical data set and it those methods that you continue to imply are something different – I think. By using an example of regridding the finished product of that data set, I think you are not addressing the issue that I pose here about the original data and methods for processing it.. Do you have access to the missing methods and can you resurrect the original data and methods used to put it into final form?

    http://www.warwickhughes.com/blog/?p=1198

    “I see that Fox News has latched on to the Climategate2 email(s) revealing Phil Jones saying the DoE were happy with him not releasing station data. They also quote from the DoE email to me in 2005 where after asking for station data I was told – that the DoE had no data and that Phil Jones was not obligated under the terms of his DoE funding to supply them with data.
    This follows on from the Competitive Enterprise Institute (CEI) late November FOI action in the US to get the DoE to reveal their correspondence with Phil Jones from 2007. We await with interest the result of the CEI efforts. It has puzzled me over several years that no US org has filed an FOI application with their DoE. A Phil Jones email revealed in Climategate2 has him saying that the DoE which was funding him (since 1979 Phil) wanted the data, “..to be well hidden.” Phil goes on to say – “I’ve discussed this with the main funder (US Dept of Energy) in the past and they are happy about not releasing the original station data.”
    It is important to know that this was not always the case. Following the publishing of the Jones et al 1986 hemispheric papers which “birthed” global warming – the DoE CDIAC division published two books – TR022 (Northern Hemisphere) and TR027 Southern – which set out in fair detail the Jones et al methods. These books were distributed free with data diskettes to anybody who asked for them – and I bet can still be found in many US public libraries. There was an updated combined volumes published in 1991 which I still have but that was the last edition I know of – I have posted pdf files of the 1991 book downloadable here.
    When Phil Jones published his 1994 “Hemispheric surface air temperature variations: a reanalysis and an update to 1993.” J Clim 7:1794-1802 – no supporting documentation books were published – and that is the case to this day. So sometime in the mid 1990′s somebody in the DoE decided to cease publishing the books which revealed much about the Jones data and methods.”

  104. Carrick said

    Just to put a couple of Nick’s many prevarications to rest on the topic of CRU and data retention.

    His argument that CRU didn’t need a copy because they weren’t the original data source is also false. Actually, because the source they got their data from could modify their data over time (after providing it to CRU) from the point of view of replicability, this just makes it that much more crucial that CRU retains the versions of the data that were given to them.

    In terms of CRU being a “small organization”, well they were certainly a well-funded one, and it wouldn’t have cost much to have made a second copy of their data somewhere else on UEA.

    Are there any more excuses I’ve missed?

  105. Carrick said

    I’m Jones destroying his original data is McIntyre’s fault somehow or the abuse of FOIA or something else that happened in the future.

  106. Carrick,
    Your analogies are inappropriate. We are not talking about experimantal data that Jones may have created. We are talking about copies of other peoples’ data that he got from sources and used for calculations.

    Why do I bother pointing out these things? As a counter to the incessant Jones-bashing that goes on here. He isn’t perfect, but the accusations are ill-informed, and brought on by his success. Even with data lost, what he had put together was the best in the world at the time. It is better to achieve with imperfection than to achieve nothing at all.

  107. Kenneth Fritsch said

    Nick a reminder from above: Do you have access to the missing methods and/or can you resurrect the original data and methods used to put it into final form?

    Here is what I have from the CRU station data using the CRU code and source of data and my station count of that data:

    A. 10 – from earlier CRU work (from World Weather Record volumes, published by the Smithsonian Institution and NOAA/WMO from the 1920s onwards) My count=2846
    B. 30 – GHCN and for Australia (from Bureau of Meteorology) My count=1555
    C. 31 – GHCN My count=96
    D. 32 – ALOCLIM (Austria) My count=16
    E. 33 – NACD/NORDKLIM (Scandinavia and NW Europe) My count=21
    F. 34 – NACD/NORDKLIM (Scandinavia) My count=64
    G. 35 – IMPROVE project My count=7
    H. 36 – CHTD (Canada) My count =209
    I. 37 – GHCN (principally fUSSR but also some other regions) My count=158

    I assume that the CRU work (2846 stations) is what was lost or at least some of it was lost and/or cannot be tracked to the original data. I am not sure what the two GHCN designations mean for codes 30 and 31. Anyway I can compare the station data for GHCN to that of CRU and I started looking at that data.

    Should CRU finished data in the current data set agree with current adjusted GHCN data and should the time periods of data available from these 2 data sets coincide?

  108. Kenneth #107,
    No, I don’t have any interest in resurrecting “the original data and methods used to put it into final form”. The reason is this:

    1. Jones, back in the ’80’s, got together a whole lot of data from met offices and wrote a program that computed an index.

    2. As he points out somewhere, a lot of that data has in any case been updated by the Met Offices. Mostly correction of errors, some extra station data has been found.

    3. I have zero interest in resurrecting an index that preserves the original data errors. If such a calc were to be done with the old program, everyone would expect it to be done with the updated data.

    4. I also have no interest, now, in whether the old program is error-free, any more than I’d go looking for bugs in Windows 3.1. If it had errors, the time to find out was in the decade or so following publication. The program has come a long way since then. But there is no indication that results in his old paper show computing errors when compared with current.

  109. Kenneth Fritsch said

    Thanks, Nick for confirmining my inclination that a lot of what you do and say at these blogs is in more in line with an advocacy position and your speciality of defending the consensus climate scientists than it is in a true scientific curiosity. I will continue my analysis of the CRU data and comparing it with GHCN.

    Thus far I have found in a percusory audit that CRU uses the GHCN data for that part of their data set in the adjusted form from Version 2 (Version 3 is the current one). My data comes from the KNMI site. The dates do not coincide at this point, but I am going to look at the GHCN data set directly and not throygh KNMI for a final analysis.

    Interesting that using the adjusted GHCN temperature data implies, at least for that data, that CRU is not adding value.

  110. Carrick said

    Nick:

    We are talking about copies of other peoples’ data that he got from sources and used for calculations.

    No, this is what I was talking about. The whole point of replicability is showing how your algorithm takes the raw data and processes it, regardless of “which” raw data it is. If you can’t control the raw data, it’s doubly critical that you preserve the raw data used in your processing. For example, if you later find you’ve made a mistake in your processing, if you don’t have the original raw data

    My former major professor, the one who duplicates all of his notebooks and keeps one at home? He’s a phenomenologist, he doesn’t collect his own data. Well almost ever (sometimes he does collect his own data. One time he

    Why do I bother pointing out these things?

    No comment. ;-)

    As a counter to the incessant Jones-bashing that goes on here

    Well I don’t out of my way to bash Jones, but just because some people attack Phil Jones, doesn’t let him off the hook when he screws up. That’s kind of a non sequitur.

  111. Carrick said

    Someone walked in why I was typing sorry…..Forgot to complete my text Here’s the full sentence:

    No, this is what I was talking about. The whole point of replicability is showing how your algorithm takes the raw data and processes it, regardless of “which” raw data it is. If you can’t control the raw data, it’s doubly critical that you preserve the raw data used in your processing. For example, if you later find you’ve made a mistake in your processing, if you don’t have the original raw data, then what do you do?

    My former major professor, the one who duplicates all of his notebooks and keeps one at home? He’s a phenomenologist, he doesn’t collect his own data. Well almost ever (sometimes he does collect his own data. One time he literally lost his pants in an experiment he was working on ;-)

    In addition to acting as an advocate for Phil Jones, I really think Nick’s lack of experimental science grounding is affecting his judgement here.

  112. willard said

    > [M]y inclination that a lot of what you do and say at these blogs is in more in line with an advocacy position and your speciality of defending the consensus climate scientists than it is in a true scientific curiosity.

    An inclination motivated by true scientific curiosity, no doubt.

  113. willard said

    > That’s kind of a non sequitur.

    I thought Nick made an observation. Where’s Nick’s inference?

  114. “I really think Nick’s lack of experimental science grounding”
    You’d be surprised. I even have papers in J. Sound and Vibration :) But PJ isn’t an experimental scientist, and he wasn’t doing experiments.

    “One time he literally lost his pants in an experiment he was working on”
    I checked my log books. Never happened to me.

  115. Carrick said

    NIck:

    You’d be surprised. I even have papers in J. Sound and Vibration

    Hopefully you archive your data. And retain your log books, which are comprised of pages that are bound into the book spine.

    But PJ isn’t an experimental scientist, and he wasn’t doing experiments

    See comment on phenomenologists (which is what PJ is practicing when doing the temperature reconstruction). The requirement still exists for replicability both for external audit and for retracing one’s own footsteps. It’s a very hazardous thing to do to remove your “starting conditions” on an analysis. Plus it eliminates the ability to allow others to “follow in your footsteps”, which isn’t a very bright thing to do.

  116. Carrick said

    Willard:

    I thought Nick made an observation. Where’s Nick’s inference?

    Check the conversation again. You don’t have the nuances right, and there’s no point in responding until you do.

  117. Carrick said

    I’ve got a hole in one of my pants legs from an experiment (chemical, mundane, spilled battery acid on my pants leg). It is kind of fashionable these days to have these sorts of holes in your jeans, so I wear them to work on occasion (usually on days I get down and dirty.)

    The incident happened near this place.

  118. willard said

    Carrick,

    This is not about nuance. You claim:

    > Well I don’t (go) out of my way to bash Jones, but just because some people attack Phil Jones, doesn’t let him off the hook when he screws up.

    Show me where Nick is saying you’re doing this. Show me that this non sequitur you introduced is relevant to Nick’s observation regarding “the incessant Jones-bashing that goes on here.”

    If you can’t, you’ll have to admit strawmaning Nick here.

    This is not even subtle.

  119. Kenneth Fritsch said

    “An inclination motivated by true scientific curiosity, no doubt.”

    I think not, but rather by attempting to have a discussion about the thread topics with Nick Stokes and him changing the subject or misinterpreting what I have asked. I put a question to him about the CRU temperature data set concerning whether he had a method he could convey to obtain the original data and produce the CRU finished product. His answer was:” No, I don’t have any interest in resurrecting “the original data and methods used to put it into final form”. That reply avoids a direct reply to whether it would be possible the resurrect the original data and the process used to put it into final form. He could have replied that either he thinks that it is not possible or that it would not be a simple process but if someone wanted to attempt it he had some ideas on how to do it. Or as I pointed to his early contentions (probably applied to the final form but Nick was never clear on that) that if it were easy to resurrect the original data and provide the methodology for transforming it to final form, then CRU would have to be held negligent for not making what would be considered a simple effort to make them whole again.

    The discussion about CRU being a keeper of critical data got all hazy and vague by Nick only wanting evidently to talk about the finished data which he noted anyone can re-grid. That little detour in the discussion avoids talking about a keeper of critical data dealing with the issues of maintaining the original data for future references and providing the methods of transforming the original data into final form.

    What my early analysis of the CRU data sourced from GHCN is that that data is taken in final form from GHCN, i.e. CRU provides no value added by using the unadjusted data and doing their own adjustments. I need to and will do more analysis on this, but to date it seems that the data originating from CRU and adjusted by CRU and the GHCN data, used as is, makes up a large percentage of the total stations in the CRU data set.

    Think about this situation for a minute in regards to the hesitancy of CRU to reveal information about its data and methods for adjustment. CRU has a widely recognized and referenced data temperature data set and CRU surely uses that recognition as a means of showing their stature in the field to potential clients providing funds for CRU projects. Now if it were revealed that the data original with CRU was lost/misplaced/no longer available and the major other portion of the data set come from copies of GHCN finished data (no value added by CRU) there might well be concerns at CRU that their reputation would be tarnished and the flow of funding for projects adversely affected.

  120. diogenes said

    agreed Kenneth at 119, as my contribution upthread was meant to elucidate…and instantly diverted down strange side channels

  121. Carrick said

    Willard, you’re still carrying on a conversation with yourself. Really.

    Regarding this:

    Show me where Nick is saying you’re doing this

    Actually said he did and whether he did or not has nothing to do with the conversation he and I were having that you so kindly chimed in on.

    This is not even subtle.

    You’ve got that right. You’re lost as a goose.

    PS: I have plenty more rope to sell, if you’d like it.

  122. Carrick said

    * Actually [never] said he did ….

  123. Kenneth Fritsch said

    On further analysis of the CRU and GHCN match up I have found that the CRU stations that I looked at so far match up exactly in later years with GHCN V3 but not in the earlier ones. Both CRU and GHCN V3 match well for period of time that each has data. The GHCN V2, that I obtain from KNMI, does not contain the later years in the CRU series but that data that is available matches CRU perfectly. I now have to download the GHCN V2 data directly from GHCN to see how it matches with CRU over the entire station time period. At this time it would appear, if the KNMI data is valid and timely, that CRU is mixing and matching GHCN V2 and V3. That would be difficult to believe so I’ll dig deeper. GHCN was maintaining and updating GHCN V2 until the middle of this year. I would suspect that CRU and KNMI would have to be switching to the GHCN V3 in order to stay up to date.

    Anyone out there have any thoughts on this?

  124. Kenneth Fritsch said

    The station data in GHCN V2, direct from GHCN, agrees with the KNMI data. It would appear that CRU intermixes the data from the GHCN V2 and V3. Anyone have any ideas why they would do that and not simply use all the latest version from GHCN, i.e. Version3?

  125. diogenes said

    Kenneth

    are you suggesting that there is no clear and obvious link (hendiadys) between the base data and the published outputs?: In my book, the publisher ought to maintain that link…but Nick Stokes and J fFerguson disagree….

    And I am shocked that you might even think that the figures are rigged.

  126. Kenneth,
    I think you’ll find the main problem with GHCN V2 is that it has duplicates, which users have to resolve. Of course, not all stations have them.

    I resolve duplicates by simple averaging, and find very little difference between GHCN V2 and V3 in terms of computed indices.

  127. willard said

    Carrick,

    The non sequitur is relevant insofar you believe Nick holds it.
    Even if was meant sine dicendo.
    So your response to Nick’s observation amounts to deflect his observation.

    In a conversation, people usually agree about observations.
    Sometimes, they even distinguish observations from inferences.
    If you want to have a private conversation with Nick, you can reach him by email

  128. Kenneth Fritsch said

    126.Nick Stokes said
    December 20, 2011 at 9:01 pm

    “Kenneth,
    I think you’ll find the main problem with GHCN V2 is that it has duplicates, which users have to resolve. Of course, not all stations have them.

    I resolve duplicates by simple averaging, and find very little difference between GHCN V2 and V3 in terms of computed indices.”

    What I was looking at was not affected by duplicates. The CRU station records that I looked at were not from all GHCN V2 or GHCN V3 but a mixture of both versions. This was evident since the records matched exactly between CRU and either GHCN V2 or V3. My question was why would CRU mix the GHCN versions in their data set. Other than CRU simply not bothering to make a clean break from V2 to V3, I see no good reason. GHCN V2 and V3 are different and V3 has more station records.

    I plan to write CRU with my query and KNMI about when they will be converting to GHCN V3.

    Diogenes, I do not think that CRU’s problems with its temperature data set have any nefarious origins but rather what I see is sloppy work and neglect. I have noted the same problem in other climate science work and work that does not entirely emanate from the consensus scientists.

  129. Carrick said

    Willard here’s a suggestion.

    Look up the phrase “non sequitur” in Merriam-Webster, read definition #2. Then interpret my comment accordingly:

    Well I don’t out of my way to bash Jones, but just because some people attack Phil Jones, doesn’t let him off the hook when he screws up. That’s kind of a non sequitur.

    You’ll find, surprisingly, it fits. Whether people are picking on PJ doesn’t justify letting him off the hook when he screws up. Bringing up the fact that some people pick on PJ is “kind of a non-sequitur” here. In general, Nick explaining his motives for prevaricating doesn’t dismiss the fact he’s prevaricating.

    Hint: try reading the sentence without loading it with your own meaning. This isn’t interpretive dance, you don’t get to pick which meaning you like.

    This isn’t the first time you’ve butted into a conversation, assumed you knew more about English language than a poor old unedjumicated American like me, and blew the interpretive part of the language process.

    Simple pointer for the future: If you find your interpretation of somebody’s prose isn’t working, especially if somebody’s told you that you got it wrong, look to see if there is an alternative interpretation that makes more sense before launching into your half-baked rebuttals.

    If you want to have a private conversation with Nick, you can reach him by email

    Uh thanks.

    I’m sure that’s news to everybody here.

  130. willard said

    Carrick,

    Thank you for mansplaining “non sequitur”.

    Nick believes he witnesses an “incessant Jones-bashing”.

    In response, you rejected that you were bashing him, as if it suffices to refute what he claims.

    And then you toned down his “bashing” to “attack Jones”, an innocuous minimization.

    And then, not without humor, you introduced an inference he never made, returning to PJ’s “screw up”.

    Please mansplain how you consider that a response that moves forward a conversation.

    As if this was one.

    Answering a simple question can be more fortright than mansplanations:

    Do you disagree with Nick’s observation that there is an incessant Jones-bashing on this blog?

    Please, do say you disagree if you believe so.

    Your command of the English language should not prevent you from doing so.

    Thank you for your hint and your pointer. I always try to improve as a person.

    And the English language is so subtle. So thank you for your patience.

  131. Kenneth Fritsch said

    “Do you disagree with Nick’s observation that there is an incessant Jones-bashing on this blog?”

    Whatever Jones-bashing is done, as well as Jones-defending, can be easily ignored and we can concentrate on the more important matters such as the evidence and background of CRU’s handling of a critical and often referenced data set. Frustrating is all the detours and spinning we take in that pursuit.

  132. Carrick said

    Willard, how much extraneous meaning can you squeeze out of one causal remark?

    In response, you rejected that you were bashing him, as if it suffices to refute what he claims.

    I merely said I didn’t bash Phil Jones. I don’t. You’re generating inference where none is there or implied.

    And then you toned down his “bashing” to “attack Jones”, an innocuous minimization.

    No clue what the significance of this comment is. It’s meaning in the context of the sentence seemed plain enough to me.

    And then, not without humor, you introduced an inference he never made, returning to PJ’s “screw up”.

    I thought it was pretty obvious. Nick brought up the “incessant Jones bashing” as a motivation for his defending PJ. But his motivation for defending PJ doesn’t justify not accepting when PJ screws up. PJ screwed up, Nick should admit that and move on, without incessantly complaining about the Jones bashing. ;-P

    Do you disagree with Nick’s observation that there is an incessant Jones-bashing on this blog?

    Actually I don’t particularly agree with it, at least on this thread. If Nick wants to point to another thread he has in mind where “incessant Jones-bashing has occurred, we can reopen this.

    Incessant is “continuing without pause or interruption” and bashing is “a harsh, gratuitous, prejudicial attack on a person, group or subject.” That’s simply not happened on this thread.

    Nick has transformed interspersed critical commentary of PJ that has appeared on this thread into “incessant Jones-bashing”. I don’t think it’s an accurate or fair characterization of the conversation, but somehow I don’t see you going on the warpath on that account as you did with “non sequitur”. Speaks to your motives in challenging the one comment you’ve spent so much time on, and none on the only actual issue at hand, which is what is Jone’s responsibility for archiving the “raw data” used in his analysis.

    Last word to you if you want it.

  133. Kenneth Fritsch said

    By the way, I received an email reply from Geert Jan van Oldenborgh of KNMI within 24 hours of sending mine. I think very highly of the KNMI effort and the tools it affords for manipulating the extensive data it contains. I also think that Geert provides most of the manpower to maintain that data set.

    I have not received a reply from the CRU people to date.

  134. willard said

    Carrick,

    When I read how you defined “incessant” and “bashing”, I doubted for a second if it was really you who just answered me.

    You’re supposed to be a dissenting scientist whom I have all the reasons in the world to respect, and now you’re playing semantic games.

    This was a simple question.

    And you’re playing semantics, and minimizes the scope of Nick’s judgement, which makes no sense whatsoever.

    Simple due diligence: the word “bashing” occurs in the second comment of the thread:

    > More fulmination about the “hiding” of the stations data. The irony is that it was released five months ago, and AFAICT no sceptics have bothered to look at it. Easier to go on bashing DoE etc.

    Talk about prevarication… and having a conversation!

    Yes, let’s continue to talk about what we were talking about: CRU’s handling of the data.

    Because, well, that’s what curious scientists do, and certainly not because:

    > [I]f it were revealed that the data original with CRU was [let's abreviate to "mishandled"] there might well be concerns at CRU that their reputation would be tarnished and the flow of funding for projects adversely affected.

    Scientific curiosity. Of course.

    Merry Chrismas to all of you!

  135. kim said

    Carrick makes sense because he always does, and Willard doesn’t make sense because he never does, but honestly that’s the only way I could distinguish this conversation.
    ===========

  136. willard said

    Since Kim insists, here’s Nick Stokes:

    > Kenneth, that’s a fair comment.

    http://noconsensus.wordpress.com/2011/12/16/climategate-continued/#comment-63300

    Reading Kenneth’s comment should be enough not to particularly agree with Carrick’s implication that Nick never admits that Phil “screwed up”.

    That Nick should “move on” does not imply that others should not, as Nick Stokes hints here. That would be a non sequitur.

    Why does anyone move?

    Scientific curiosity, no doubt.

  137. kim said

    Nick Stokes? One of the least curious, yet most curious people I know.
    ==========

  138. Kenneth Fritsch said

    133.Kenneth Fritsch said
    December 22, 2011 at 6:47 pm

    “By the way, I received an email reply from Geert Jan van Oldenborgh of KNMI within 24 hours of sending mine. I think very highly of the KNMI effort and the tools it affords for manipulating the extensive data it contains. I also think that Geert provides most of the manpower to maintain that data set.

    I have not received a reply from the CRU people to date.”

    I have received an email from Phil Jones today (Jan 3, 2012) in reply to my email concerning the mixing of adjusted mean monthly temperatures from GHCN V2 and V3 in the CRU data set.

    I need to read it more carefully to determine what it explains. In the email Jones states that, “CRU series may appear similar to GHCNv2 and GHCNv3 because we are using the same original sources (i.e. the Met Services of the world, NMSs)”. That statement would imply to me that CRU (and GHCN) adjust the original data and that CRU has value added by way of their own adjustments. The sampling that I made of the CRU data set, where they use GHCN data, showed that the data were essentially identical with a mix of GHCN V2 and V3 for the same CRU station. I’ll need to look further, but my current impression is that CRU uses the GHCN adjusted data directly and does not do their own adjustments – unless their adjustment and GHCN’s are so similar that they produce identical results. I know that GHCN data is adjusted by GHCN because they have published their methods and the original data. I do not have the same information for CRU and I do not believe that it is published for the public to view.

    Jones did explain that my email was received when the CRU was closed for the holidays and this was their first day back.

  139. Kenneth Fritsch said

    Just to avoid any peeing contest about what I stated above about CRU adjusting data, I want to be clear that, while most of the data in the CRU temperature set is from their own sources or from GHCN, they do apparently adjust the part that is exclusively theirs even though the original data is not evidently available.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
Follow

Get every new post delivered to your Inbox.

Join 140 other followers

%d bloggers like this: