the Air Vent

Because the world needs another opinion

Chugging Along

Posted by Jeff Id on July 13, 2011

Steve Mosher has continued his work on GHCN station global temperature reconstruction package in R.  The most recent version is documented in his blog here.  He has developed an elegant set of commands for computing global temperatures in R and is working on release of an R library package for that purpose.   Recently he incorporated a method from Berkley which Tamino posted.  Steve has decided to work entirely in the global temperature datasets on his blog and originally I wasn’t sure if he would keep going or what it would turn into.   I check out what has been happening from time to time and got tired of having to google him to find it so I placed the links on the right.  He appears to be doggedly developing a very complete package of commands which will process the raw data by a variety of reasonable (and documented) methods.

10 Responses to “Chugging Along”

  1. Brian H said

    Average birdfeathers. Add x megajoules to a cubic mile of air over the Sahara. Note temperature change. Add x megajoules to a cubic mile of air over the Amazon. Note temperature change. Add x megajoules to a cubic mile of air over Antarctica. Note temperature change. Add x megajoules to a cubic mile of air over the South Pacific. Note temperature change. Average the changes to within 0.001K. This is ‘y’.

    Now, tell someone competent in atmospheric physics that you added 4x megajoules to 4 separate cubic miles of air and raised the average temperature by y degrees. (Give him only the y figure.) Ask him to derive x, to within 0.001mj or so.

    Should be good for a laugh!

  2. Thanks, Jeff.

    I am grateful to you, Steve and others for “doggedly” pursuing reliable information on Earth’s constantly changing climate.

    Perhaps my vision is distorted by eternal optimism, but I see encouraging signs that NASA employees, editors of and some politicians are beginning to see the connection between:

    a.) The fable of CO2-induced global warming,
    b.) Disintegration of our space program, and
    c.) Disintegration of the world economy.

    That is not a coincidence!

    See: “Dark fireworks on the sun,” by NASA’s own
    Dr. Tony Phiilips [ 12 July 2011]

    With kind regards,

  3. Steven Mosher said

    Thanks Jeff,

    Looks like Roman is plugging away at getting some code ready, if it’s as clean as tamino’s it will go in in a day.

    That will leave

    1. The actual Berkeley method with weighting. they are currently working in Matlab (yuuuk) BUT I saw their R programmer
    at a local R event, so maybe she is boning up to do the matlab to R conversion. die matlab, die
    2. Giss… YUUK doing Giss in R will be ugly city. I dont see why they just dont adopt CCC. They should at least use
    tamino station combine instead of the reference station method. RSM needs to die.
    3. Nick stokes: Nicks is a bit more of a challenge since he’s wrapped a bunch of stuff and options and job control things
    in. He’s built more of an application in R ( I tried this early on– ) His code would require a lot of work to turn into
    an Api.

    If people are wondering what I’m doing its actually building a bunch of functions that are designed to work on spatio/temporal

    In class terms.. consider the class of objects we call time series:

    1. Data : vector of data
    2: time: : vector of time
    3. frequency etc

    To that class we add the following elements to make a super class

    1: data units (C) this is for temperature
    2: time vector is regular ( all time from start to finish are present
    3: series name: every time series has a unique Id ( the station Id)
    4. Time series position: these are located in space
    5. metadata: still TBD how to handle this since #3 is a DB key

    Then on that class we define methods. Most of these methods will just be clones of basic R methods on the underlying
    classes, so window for example will just work, aggregate just works, basic math operators just work. Then we add
    methods, so Roman’s approach is just a method on that object.

    5 is the biggest hurdle design wise.. right now there are two different objects.. and they need to communicate or pass messages
    about changes in each other. kinda sorta

  4. Mark T said

    Why the disparagement for MATLAB?

    Just curious.


  5. Steven Mosher said

    cause it’s not open. I imagine what I will have to do is get octave ( is that it?0 and then run their matlab in that enviroment, trust that it gives the same answers, then use that version to test and verify the r implementation.

    as far as a language goes itlooks pretty reasonable, but I’m not in a position to fork over money so that I can do volunteer work

  6. Jeff Id said

    #5 Matlab is nicer than R by far IMO, but it is $$ and therefore not as useful as open source for the kind of openness that science deserves. I’m thrilled to see high quality work being published on line for this purpose. It would have been a very important tool 3 years ago, and my guess is that with some verification these Steve M R functions may find use in publications down the road.

  7. Gary said

    Mosh, that item #5 seems to me to be crucial. Need to know those station moves and changes. Yeah, in the long run it averages out, but for nearby station comparisons for UHI effects and such it’s important. Good work you’re doing. Any possibility of somebody writing a web-accessible front-end on the R engine so we who don’t have the time to get up to speed with R can work with the data?

  8. Kenneth Fritsch said

    Are Roman and Nick both providing algorithms to calculate regional and global mean temperatures using the spatially and temporally incomplete data that your code will provide?

    Is anybody looking at the uncertainties derived from spatially and temporally incomplete data?

  9. Steven Mosher said


    I will look into a web front end. there is a server version of R. Ideally, if this is done properly one would just drag and drop a series of boxes into something that looked like a filter graph ( err get the idea) and then the data/graph pops out. integrating R and web is a ongoing thing, usually guys have java front ends..

    the other thing I would look into doing was running stuff from a web page and then populating a GOOGLE fusion table.

    I’ve spent a ton of time integrating Google earth and other things… Now perfecting all that into a USEABLE fool proff system is another thing. That takes patience and testing and shit I think I wrote the main code for this at least 10 times.. 10 different ways at it.

    WRT#5. ya, I have a bunch of work to do there. how I handle metadata is going to be critical. I have NOT found a system/package that I am happy with. So for now the metadata sits in a R data.frame.. kinda ugly

  10. Steven Mosher said

    WRT 8.

    Roman is beavering away as we speak.

    I took a look at the code jeff Posted and while it was clear, it was not clear enough for me to clearly explain or integrate.
    So, I did a couple things: I changed some functions to output /input the kind of object Roman likes ( a mts) and I asked him to have a go at making the code a bit more accessible. I do have to understand the stuff i integrate to a certain extent. While he does that I will also start my own rewrite of the thermal hammer.. again just to get the feel
    of whats going on.

    Nicks code: I havent asked Nick yet. First I need to finish going thru his code. he has restructered it from when I first ran it and i’ve got no clear sense of it anymore ( my fault not his) He does some very clever things, but also
    does some annoying things ( uses ; at the end of lines)

    Mcintyre also has some things that will go in.

    Lastly people need to use the damn -> for assignments in R. Arrggg.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: