Open Mind

Dropouts

February 15, 2010 · 58 Comments

We’ve already critiqued the analytical skills of Joe D’Aleo and of course the incomparable Anthony Watts….. Incomparable.

They’ve joined forces to create a document which pretends to ask the question, “SURFACE TEMPERATURE RECORDS: POLICY DRIVEN DECEPTION?” but is really just a bunch of false claims intended to state outright that the surface temperature record is not just mistaken, it’s fraudulent.


There are too many misstatements in this work to deal with all of them in a single lifetime, but we can at least expose some of the most egregious. Let’s start with this, which is highlighted in extra-large font on page 6, and is one of their primary thesis points:


Around 1990, NOAA began weeding out more than three-quarters of the climate measuring stations around the world. They may have been working under the auspices of the World Meteorological Organization (WMO). It can be shown that they systematically and purposefully, country by country, removed higher-latitude, higher-altitude and rural locations, all of which had a tendency to be cooler.

It’s impressive how many things they can get wrong in so few words.

First of all, they’re referring to the reduction in reporting stations included in the GHCN (global historical climate network) data on which some global temperature estimates are partly based. But they just made up the claim that “NOAA began weeding out … systematically and purposefully …” The fact is that NOAA — a U.S. government agency — has no control whatever over which station data various nations contribute to the GHCN. If they have a complaint about a reduction in stations from, say, Canada, it’s blatantly dishonest to blame it on NOAA, they should take it up with Canada.

As for “They may have been working under the auspices of the World Meteorological Organization (WMO),” don’t they know? Didn’t they bother to find out??? Did D’Aleo and Watts actually decide to accuse NOAA of deliberately biasing the temperature record, without even bothering to find out who is doing what?

You could argue all day over what part of their claim is most dishonest. But it seems to me that the clear winner for “most stupid” is the very idea that removing locations “which had a tendency to be cooler” will somehow introduce a false warming into the global temperature anomaly calculation. Yet that’s what they claim. Are they counting on the fact that readers are too lazy or stupid to find out the truth? Or do they really not get it themselves?

They support their idea with this graph on pg. 11 from Ross McKitrick:

Note that the average temperature shows a clear step change in 1990. It gets warmer, because indeed the stations which no longer reported were more often cold stations than warm stations. But: cold doesn’t mean there’s a cooling trend nor does warm mean there’s a warming trend. This average doesn’t show a statistically significant trend either before or after 1990 — it only shows a step change. Of course it can’t be taken as a global temperature indicator, first because it’s not an area-weighted average (and the global distribution of stations is highly irregular) and second because this isn’t an average of temperature anomaly but of raw temperature. Talk about bias!!!

They do mention that


While a straight average is not meaningful for global temperature calculation (since areas with more stations would have higher weighting), it illustrates that the disappearance of so many stations may have introduced an upward temperature bias.

Of course it’s not meaningful for global temperature calculation. However, it does not illustrate that station dropout may have introduced an upward temperature bias.

They hasten to note that


The number of stations that dropped out tended to be disproportionally rural –

which they support by showing this graph:

They’re just plain wrong. While there were more rural stations which dropped out than suburban or urban, that’s only because their are more rural stations, period. The fractions of different types which are included tells the opposite story:

In truth, the post-1990 dropout saw a small increase in the proportion of stations which are rural, with a marked decrease in the proportion of urban stations. The real “disproportionate” dropout was of urban stations, not rural. They can’t even get that right.

If we look at the GISS global temperature data from 1950 to the present

There’s no sign of a step changed at 1990, which is exactly what their thesis implies we should see. Their thesis is wrong.

A disproportionate dropout of cold stations in favor of warm stations does not mean a disproporionate droupout of cooling stations in favor of warming stations. One of the benefits of computing grid averages using raw temperature rather than anomalies is that it computes an offset for each station which tells us whether or not it’s colder or hotter than the reference point, and how much. Then of course we anomalize the grid average so that the anomalies have mean value zero during the baseline period. We’ve already computed grid averages for grids just east of the prime meridian. Let’s take the two grids in this region which have the largest number of stations (by far) and compare the grid average anomalies using just the cold stations, to those using just the warm stations.

The two most populated grids we’ve looked at so far are latitude 40-50N longitude 0-15E, and latitude 50-60N longitude 0-15E. For each grid, let’s compute a temperature history using just the colder half of the stations, and compare it to the temperature history computed using just the hotter half of the stations. For the grid with the most data, we get this:

The jagged line is annual averages, the smooth line is a lowess smooth. Clearly it’s the cold stations which show more global warming over the last century+, while the hotter stations show less.

For the grid with the 2nd-most data, we get this:

Again: it’s the cold stations which show more global warming over the last century+, while the hotter stations show less.

This belies the claim that favoring hot stations over cold ones will inflate the global average temperature anomaly to create false warming. There’s absolutely no reason that using cold or hot stations would bias the trend, unless one or the other tends to have a greater or lesser trend. And the available evidence is that it’s the cold stations which will show more warming — so biasing the sample in favor of hot rather than cold stations will, if anything, tend to underestimate the global warming trend.

D’Aleo and Watts are dead wrong about NOAA undertaking any effort to control which stations are included in the GHCN and which are not. They’re dead wrong about NOAA even being able to do so. They’re dead wrong about “The number of stations that dropped out tended to be disproportionally rural.” And they are dead wrong about the idea that favoring hot rather than cold stations will introduce a warm bias into the global temperature anomaly record.

So I have to wonder, once again: are they counting on the fact that readers are too lazy or stupid to find out the truth? Or do they really not get it themselves?

Categories: Global Warming
Tagged:

58 responses so far ↓

  • Mike Bantom // February 15, 2010 at 6:43 pm | Reply

    Nice dissection Tamino. I still can’t believe how ignorant Watts is and the people who he has do guest posts on his blog are.

    That joint post of Copeland and Watts that you shreaded where they demonstrated an almost perfect correlation between the temperature series and the solar cycle by comparing the series AFTER they had SORTED them was hilarious in a scary sort of way. Yet an incredible number of people take them seriously.

  • Miguelito // February 15, 2010 at 7:08 pm | Reply

    Not to mention that the temperatures recorded by surface stations over the past several decades are consistent with satellite measurements, indicating that there’s no bias in the surface selections used.

  • mike // February 15, 2010 at 7:10 pm | Reply

    I used to think Watts just had an agenda. I’m beginning to think he’s truly stupid

  • Derecho64 // February 15, 2010 at 7:24 pm | Reply

    I just skimmed the “report” – more pictures from Watts, more nonsense from McIntyre, more hacked emails…

    The whole thing is a joke. They bounce between C and F, and usually use absolute temps in their graphs.

    They acknowledge:

    “Dr. Roger Pielke Sr., Dr. David Evans,
    Dr. Timothy Ball, Ken Gregory, Dr. Richard Keen, the New Zealand Coalition, Dr. Craig Idso, Dr. Robert Balling, Dr. Phil Klotzbach, Peter Leavitt, Willis Eschenbach, Basil Copeland, the Carbon Sense Coalition and Alan Cheetham.”

    A veritable “Who’s Who” of denialists.

    The whole thing is a mess.

  • Ben // February 15, 2010 at 7:25 pm | Reply

    I can’t imagine that these two could consistently get things so wrong by accident, so in spite of the adage that “one should never assume malice when stupidity will suffice” I have to conclude that their actions are indeed malicious.

  • Derecho64 // February 15, 2010 at 7:42 pm | Reply

    Watts is stupid, and does have an agenda. The two are not incompatible.

  • Chad // February 15, 2010 at 8:04 pm | Reply

    Nice post Tamino. I’ve read through the report briefly and it just doesn’t fail to amaze how so many of their damning graphs would disappear if they had used temperature anomaly like they’re supposed to!

    They brought up the issue of missing stations in Russia:

    This created 0.64C greater warming than was exhibited by using 100% of the raw data. Given the huge area Russia represents, 11.5% of global land surface area, this significantly affected global land temperatures.

    I’ve estimated the effect of Russian data on global land/ocean temperatures to be 0.00-0.01 °C/decade. Not infinitesimal but not really significant either.

  • Zeke Hausfather // February 15, 2010 at 8:31 pm | Reply

    Tamino,

    Any chance we can get a spatially-gridded anomaly for stations with and without continuous records post-1990?

    I made a first pass at doing the analysis awhile back (http://www.yaleclimatemediaforum.org/2010/01/kusi-noaa-nasa/) by simply averaging the anomalies for a common base period (1960-1970) for all stations in the continuous and discontinuous groups, but doing spatial weighting proved a bit beyond my ken. Given the preponderance of stations in particular locations (e.g. U.S. and Western Europe), it seems like it would be rather important.

    [Response: Shhh..... it's a secret. No, I guess it's not.]

  • Ray Ladbury // February 15, 2010 at 8:50 pm | Reply

    Wow! Light-bending stupidity at it’s most pure. Stupidity so powerful, no tinfoil hat can protect against it!

    It really appears that Watt’s & D’Aleo really don’t understand the difference between temperature and anomaly. Given this, I am not sure that there is enough frontal cortex there that they are capable of subterfuge

    Does the the DSM-V have a listing for hyper-stupidity disorder?

  • Derecho64 // February 15, 2010 at 8:51 pm | Reply

    Watts and D’Aleo are making claims that I think are defamatory and libelous. If I worked at NASA or NOAA, I’d forward on their “report” to my legal department and see if there are any statements that could be actionable.

  • Deech56 // February 15, 2010 at 9:02 pm | Reply

    I have to wonder – is there more stupid out there or is the stupid just showing up more and more in the mainstream media? The “skeptics” claim that the heretofore AGW-fawning media finally gets it.

    How convenient that this is taking place while the Senate considers a new energy bill. A coincidence, I’m sure. Yeah, right.

  • Didactylos // February 15, 2010 at 10:11 pm | Reply

    Tamino, you say there is “no sign of a step change”, but unusually for you, you don’t provide any analysis, and just “eyeballing” the graph, it is susceptible to the possibility that natural variation can hide a step change. (Slide the second half down 0.2 degrees, and *locally* it wouldn’t be noticeable.)

    I think there is *just* enough data before and after 1990 to show that there is, in fact, no step change.

    Now I’m suffering deja vu. Let me think….. ah yes. The GHCN discontinuity test is designed to identify exactly this sort of step change.

    I’m not asking for you to actually take the time to do this – the rest of the KiloWattErrors make it all moot.

  • Nick Barnes // February 15, 2010 at 10:21 pm | Reply

    are they counting on the fact that readers are too lazy or stupid to find out the truth?
    I think you mean “Are they counting on their readers being too lazy or stupid to find out the truth?”

    [Response: Quite so.]

    Otherwise you are – maybe inadvertently – casting aspersions on the readers.

  • KenM // February 15, 2010 at 11:00 pm | Reply

    Wow this is really bad.
    I think the ‘disproportionate’ claim can be looked at this way: “Of 6000 stations dropped, 4000 were rural” (I’m using made-up numbers). If there were three types of stations, and 2/3rds of the ones dropped came from one type of station, I think one can make the claim that they represented a disproportionate number of those dropped. That falls under the stupid category I think.

  • Ray Ladbury // February 15, 2010 at 11:09 pm | Reply

    Didactylos,
    Keep in mind that as soon as you introduce the possibility of a step change, you are introducing a more complicated model. As such, to assess this you’d need to use something like AIC based on maximum likelihood fits assuming your two models and, for lack of a better noise model, a normal distribution of noise about the mean trend.

    • Didactylos // February 16, 2010 at 1:40 am | Reply

      Ray, you make the mistake of thinking I have a clue about statistics. While I’m not at blog-idiot level, it certainly isn’t my specialist subject.

      That said, the GHCN method seems appropriate.

      To look for such a change point, a simple linear regression was fitted to the part of the difference series before the year being tested and another after the year being tested. This test is repeated for all years of the time series (with a minimum of 5 yr in each section), and the year with the lowest residual sum of the squares was considered the year with a potential discontinuity. A residual sum of the squares from a single regression through the entire time series was also calculated. The significance of the two-phase fit was tested with a likelihood ratio statistic using the two residual sum of the squares and the difference in the means of the difference series before and after the discontinuity was evaluated using Student’s t-test.

      • Ray Ladbury // February 16, 2010 at 10:56 am

        Didactylos,
        The problem with the likelihood ratio and Student’s t-test is that they do not consider the fact that a more complicated model will always give a better fit to the data while in many cases decreasing predictive power. My recommendation would be to consult Burnham and Anderson on use of AIC. But qualitatively, the goodness of fit (e.g likelihood) has to improve exponentially in the number of parameters (3 in the case of a linear fit–slope, intercept and width of the noise about the trend, and double that for the two-phase fit).

  • WAG // February 16, 2010 at 12:21 am | Reply

    Here’s what’s interesting – and scary – about this: it reveals how skepticism over global warming results not just from bad analysis, but from an inability to understand what good analysis looks like.

    When I was a student at Harvard, we often scoffed at the school’s explanation for its lack of practical courses: that “we teach you how to think.” Now I realize how important “knowing how to think” really is. It’s not enough to know how to crunch numbers as Watts and company do; you need to understand how to choose the numbers, why you crunch them the way you do, and what conclusions you can draw from the final numbers.

  • Mal Adapted // February 16, 2010 at 1:00 am | Reply

    Here’s what’s interesting – and scary – about this: it reveals how skepticism over global warming results not just from bad analysis, but from an inability to understand what good analysis looks like.

    Again we see the Dunning-Kruger effect at work. Incompetent individuals over-estimate their own level of skill, and fail to recognize genuine skill in others.

  • David Gould // February 16, 2010 at 1:06 am | Reply

    The confusion between the temperature of a site and how fast it is warming is stunning, really. No critical thinking skills; no logic; no mathematics.

    • Ray Ladbury // February 16, 2010 at 1:33 am | Reply

      David Gould says of Watts and D’Aleo: “No critical thinking skills; no logic; no mathematics.”

      If only they’d use their powers for good!

  • Eamon // February 16, 2010 at 1:20 am | Reply

    They’re not even accurate in their claim that sites have been ‘weeded out’. The Yale Forum on Climate Change & The Media has this:

    “During that spike in station counts in the 1970s, those stations were not actively reporting to some central repository. Rather, those records were collected years and decades later through painstaking work by researchers. It is quite likely that, a decade or two from now, the number of stations available for the 1990s and 2000s will exceed the 6,000-station peak reached in the 1970s.”

    I recommend the article, which regards an earlier claim by Smith and D’Aleo on the same subject. It can be found here:

    http://www.yaleclimatemediaforum.org/2010/01/kusi-noaa-nasa/

    It’s interesting to note that the max station count from Pererson and Vose in the article stands at 6000, so McKitrick’s graphic may be suspect – having twice the number of stations.

  • WAG // February 16, 2010 at 1:28 am | Reply

    To clarify, the thing I thought demonstrated “an inability to understand what good analysis looks like” was Watts’ publication, not Tamino’s analysis.

  • carrot eater // February 16, 2010 at 1:34 am | Reply

    The worst part is, whenever you tell them that they’re making a monumentally stupid mistake by averaging together absolute temperatures, they seem to get feisty. I don’t think Watts understands the issue at all, but EM Smith is just confused by what GISS does. He does not appear to realise that using offsets means that absolute temperatures aren’t being averaged.

    The guys at GISS made their own plot.
    http://data.giss.nasa.gov/gistemp/graphs/IntegArea.pdf
    You can see that removing the cold, cold SH high latitudes makes little difference. Removing the cold, cold NH high latitudes cools down the record a bit, because those areas have been warming more quickly than the rest of the earth.

  • Rattus Norvegicus // February 16, 2010 at 3:14 am | Reply

    I think you are not giving E.M. Smith his due time wearing the dunce cap, since I believe that he was the original source of this canard.

    Personally I tossed this once I examined a claim made by D’Aleo and Smith on their KUSI TV show. In it they claimed that temps in California were based on 3 (or was it 6, or maybe it was 4) stations. To bolster their argument they showed a chart which purported to demonstrate how evil homogenization adjustments were. It was the Davis Experimental farm, just west of Sacremento, CA.

    Taking this as a hint, I used the GISS database to display stations near Davis. Much to my shock and surprise, withing 100 miles of Davis I found around a dozen stations reporting in 2009. Hmm, this didn’t seem to fit with their assertions.

    The obvious conclusion was that they were either fools or liars and thus not attention should be paid to them.

    • carrot eater // February 16, 2010 at 3:14 pm | Reply

      The initial graph was McKitrick’s back around 2002, though EM Smith is the driving force behind picking it up and making an absolute hash of absolutes vs anomalies, and warm vs warming. So yes, Smith deserves the main dunce cap.

      Oddly, they introduce McKitrick’s plot with “While a straight average is not meaningful for global temperature calculation”, and then proceed to show many more such meaningless graphs made by EM Smith. Bizarre.

      But the ultimate inspiration appears to be Willmott, GRL 18: 2249-2251 (1991). This paper also shows a station count history superimposed on a global average absolute temperature. At least Willmott used spatial averaging, which is apparently beyond EM Smith’s capabilities.

      But I’m surprised Willmott made it through peer review. The analysis is just badly done. By 1991, people knew to use anomalies, not absolute temperatures. Willmott also claims that “Most studies of global temperature change do not explicitly address the problems of uneven spatial sampling”, when Hansen had done so years earlier in 1987, and done a much more competent job of it.

  • Rattus Norvegicus // February 16, 2010 at 3:15 am | Reply

    absent the other typos, that should read “no further attention should be paid to them”

  • jyyh // February 16, 2010 at 3:55 am | Reply

    Mathematics is dangerous to people who can only think in absolutes, Anonymous

    • Kevin McKinney // February 16, 2010 at 12:55 pm | Reply

      Exactly. The WUWT post about snow cover referred to above (I think it was on this thread) is a great example. The foofara about snowstorms “falsifying” warming is based upon the assumption–itself false–that there is a straightforward linear relationship between snowfall and cold temperature.

      The comments of Kevin Trenberth and others essentially said that this assumption is false: there are aspects of warming (such as the increase in absolute humidity) which can lead rather to increased precipitation–including snow.

      But the poster read this as simply inverting the assumption–in other words, read it as asserting that not only CAN warming increase snow, but that it always necessarily DOES. Another “absolute”–or quasi-absolute, if you want to be picky. This created yet another strawman argument, in a long line of same.

      Unfortunately, this is dangerous not only to the confused, but to the person affected, but to the quality of the whole discussion.

  • Kevin McKinney // February 16, 2010 at 12:56 pm | Reply

    Sorry–I meant to strike the words “but to the person affected” in the previous post.

  • Barton Paul Levenson // February 16, 2010 at 8:24 pm | Reply

    If NASA and NOAA are government agencies, they can’t sue for libel. Only individuals and companies can do that.

    • Derecho64 // February 16, 2010 at 8:59 pm | Reply

      I’d just like to see Watts and D’Aleo provide more evidence for deliberate manipulation. Someone at NASA or NOAA must have done it; let’s see them name names. Otherwise, their accusations are baseless.

      On the other hand, if denialists didn’t make defamatory comments without proof, they’d have to shut up.

  • SteveCase // February 16, 2010 at 11:33 pm | Reply

    When I first was introduced to this issue, it was under the banner on some blog as “The Bolivia Effect” The claim was that there are no reporting stations in Bolivia when there used to be, and now there’s an erroneous “Pink” glow on the world temperature map where Bolivia is.

    With respect to what I just read above, how does that so called “Bolivia Effect” fit into the critique?

    [Response: No relation.]

    • carrot eater // February 17, 2010 at 1:13 am | Reply

      That one’s a weird argument. Since GHCN has no current stations in Bolivia, GISS’s algorithm relies on points outside Bolivia to construct the record there. Which is OK if not optimal; temperatures don’t know about political borders, and the distances involved are not overly long.

      When the weather stations all around Bolivia are warmer than normal, then that gets interpolated in. When the surrounding stations are colder, then relative coolness gets put in. For some reason, the skeptics like to pick out one month where the map looks like the former, and then they go bananas.

      Incidentally, a handful of Bolivian stations have recently begun reporting, so if they continue reporting, that bare spot will be gone at some point in the future.

  • SteveCase // February 17, 2010 at 1:27 pm | Reply

    Carrot Eater,

    Thanks for the response, here’s that blog I read:

    Musings from the Chiefio

    http://chiefio.wordpress.com/2010/01/08/ghcn-gistemp-interactions-the-bolivia-effect/

    GHCN – GIStemp Interactions – The Bolivia Effect

    January 8, 2010 by E.M.Smith

    • dhogaza // February 17, 2010 at 3:36 pm | Reply

      Just in case you haven’t figured it out, Chiefio is more accurately described by adding a “d”, “i”, and “t” to his handle in appropriate locations:

      Chiefidiot.

      Seriously.

      The man doesn’t understand the difference between an anomaly and actual temperature, for instance. Or why anomalies are what are looked at.

    • carrot eater // February 17, 2010 at 3:53 pm | Reply

      Don’t worry, I knew where you were getting that from.

      EM Smith is highly prone to confusing himself, and then getting wrapped up in conspiracy theories.

      dhogaza: Smith is befuddled by the GISS method of combining stations. That they are aligned with each other by adding an offset is completely lost on him.

  • Tim Curtin // February 18, 2010 at 10:08 am | Reply

    There is mathematically no distinction between anomalies and actual absolute temperatures as even GISS makes plain by saying divide anomaly by 100 and add 14 to get actual. Those are scalars, and regressing either absolutes or anomalies against say [CO2] will produce the same outcome. Try it.

    [Response: You win the "bag of hammers" award.]

  • Ray Ladbury // February 18, 2010 at 1:51 pm | Reply

    Tim, while it’s true that you can get the trend from either temperatures or anomalies, that isn’t what micro-Watts and D’Aleo are doing. Their analysis is risibly wrong!

  • Dan Satterfield // February 19, 2010 at 12:42 am | Reply

    Tamino, Exc. post as usual. Unfortunately their readers are far too gone to be convinced by simple things like hard facts.

    Dan

  • Mark A. York // February 19, 2010 at 3:13 am | Reply

    Well, dropping out of the Chico, CA television market to try to blog NASA out of business is stupid on its face. I mean really.

  • celeri eater // February 19, 2010 at 5:27 am | Reply

    When a station X is dropped off in 1990 (example), are the readings made by this X station before 1990 also pulled off the reference mean ?

    • carrot eater // February 19, 2010 at 11:43 am | Reply

      In GISS: so long as it has at least 20 years of overlap with the other stations in the area, it remains part of the record. In GISS, there is no fixed reference mean, in a sense, until all the stations are already combined.

      If the station that is dropped had a different trend from its neighbors, and assuming that trend was correct, then dropping that station will introduce an error, as you will be undersampled by a bit more than you were before.

  • sean // February 21, 2010 at 2:04 pm | Reply

    The analysis is not that stations counted as “rural” has changed as a percentage, rather that a disproportionate number stations which are “rural, at higher latitudes and altitudes”. has disappeared. They will be from the rural count, but the rural count also contains many stations which are less remote or of frankly disputed ruralness. Mountaintops are relatively free from urban expansion, – except perhaps for ski resorts. The arguement is expanded in E.M Smith reference.

    Sean

    [Response: The quote is "The number of stations that dropped out tended to be disproportionally rural."

    The way to avoid a false trend due to urban/suburban/rural status is for the proportions of urban/suburban/rural stations to remain constant. That has not happened, but instead of a disproportionate reduction of rural stations (as implied by the authors), the opposite has occured.]

  • sean // February 21, 2010 at 10:22 pm | Reply

    The article argues the classifications used in the counts incorrectly assigns rural status to stations which have in fact become urban since the population figures were estimated in 1980.

    The real counter argument is in Spencers recent postings on work using NOAA-merged International Surface Hourly (ISH) dataset.
    “.. seems to indicate that the rapid decrease in the number of stations included in the GHCN database in recent years has not caused a spurious warming trend in the Jones dataset “.

    [Response: The article also argues that "The number of stations that dropped out tended to be disproportionally rural." That's a direct quote, and they imply that it has contributed to a warming bias. But they got the facts wrong.]

  • Didactylos // February 22, 2010 at 1:21 am | Reply

    I’ve noticed this tendency by the more scientific deniers before. They make some sweeping and unsubstantiated claim, then they use it as the basis of all sorts of specific arguments that logically rely on the earlier claim.

    Les Hatton, for example, dismissed the entire discipline of climate modelling in a single throwaway sentence, then used this as the basis for attacking all sorts of conclusions that use modelling in any form. No wonder he’s unpublishable!

    There are many more examples.

  • jp // February 23, 2010 at 8:16 pm | Reply

    Thank you for the analysis. Can anybody comment on why the number of stations decreased? What was the real reason that stations were eliminated?

    • carrot eater // February 23, 2010 at 9:58 pm | Reply

      When they put together the database, they scoured various archives and asked around for whatever historical data they could find.

      Then, to keep the database up-to-date going forward, the different countries of the earth were asked to electronically submit a monthly report for each station. This is called a CLIMAT report. The various countries don’t send in CLIMATs for every station within their borders; it takes a bit of effort to put the report together. There’s about 1200 stations that get regularly updated in this way. But that isn’t close to the number of stations they originally fed into the database using archived data.

      It is maybe time to re-scour to collect the data for the non-reporting stations, at least in parts of the world with a low station density. There are also a handful of stations that now issue CLIMATs and don’t make it in to GHCN, but this I think is small compared to the overall numbers.

    • carrot eater // February 24, 2010 at 2:40 pm | Reply

      and by the way: NOAA is working on a new version of the GHCN, for this year. It sounds like it’ll have more data filled in.

      • Kevin McKinney // February 24, 2010 at 4:01 pm

        That’s good. I’ve been encountering a persistent poster alleging that “Canada has only one Arctic temperature station.”

        That’s nonsense, of course, in several ways. (Two are the facts that there are many weather stations in the Canadian Arctic, and that thirty or so appear to have contributed to GHCN over the years.)

        But I’ve not found it easy to tell which of the many stations actually had data for which years. I think it’s possible to extract this from the sources available online, but as far as I could tell, it would take much longer than I could feasibly take.

        It’d be great to see more data filled in for more recent years, especially if we could easily access a station map or something similar.

      • carrot eater // February 24, 2010 at 4:31 pm

        I haven’t had time to really look into Canada, so I’ll bombard with a bunch of links that could be helpful, for anybody wanting to understand the flow of data.

        Obviously, from GHCN v2.mean and v2.temperature.inv, you can see which stations are up-to-date. Of course, the NASA GISS mapper will help you map them.

        To see a listing of when each station had its last report in GHCN, listed with country, see here.
        http://moyhu.blogspot.com/2010/02/updating-ghcn-stations-arent-dying.html

        So that tells you what stations are actively being added to GHCN.

        Then, if you want to see if any stations are being electronically reported, but for some reason not added to GHCN, you can go up a step to the GSN.

        Map of GSN station locations:
        http://www.wmo.ch/pages/prog/gcos/documents/GSN_Station_Map.png
        Map of stations, with actual and updated data:
        http://ds.data.jma.go.jp/gmd/tcc/climatview/index.html
        Tabulated data:
        http://gosic.org/gcos/GSN-data-access.htm
        The original CLIMAT forms
        http://www.ogimet.com/gclimat.phtml.en

        And finally, Canada’s own station list. Presumably, they issue CLIMATs for those station labeled as such.
        http://climate.weatheroffice.gc.ca/prods_servs/wmo_volumea_e.cfm

        So by cross-referencing all of these data sources, you can see if some Canadian stations are being lost somewhere.

        Oh, and this guy made a map of which stations are in GHCN, and are regularly updated, and have very few missing months over the last 30 years (when you set the standards this strictly, it’s a little sparse!):

        http://climatewtf.blogspot.com/2010/02/stations-frequent-reporting-stations-of.html

        Please delete if it tries to post twice.

  • Barton Paul Levenson // February 23, 2010 at 9:31 pm | Reply

    jp.

    I assume it was a matter of appropriations.

  • Zeke Hausfather // February 24, 2010 at 4:19 pm | Reply

    Kevin,

    Michael Tobis wrote a quick script for me awhile back that goes through the GSN data and finds the months for which data is available over the past 10 years for each station. You can find it here: http://drop.io/2pqk4vg/asset/rehmm-zip

    The results from a few weeks back are in longresult.dat

    Hope that helps!

  • sean // February 24, 2010 at 11:11 pm | Reply

    Wow, some monthly data is online, with the data dribbling in months late, How very 20th Century.

    It is after all just very simple low data rate data-logging problem. A quick Google will show you there are off the shelf solutions to provide solar power, measure with traceable instrumentation, and upload the data automatically to the internet. You could even just have two people in a call centre phone folks up if they are more than a few days late. A Romanian call, as they are good at languages.

    Hosting the database itself – 1500 stations, 20 octets per day, less than a MegaOctet per month. Even 200 years of this would not be big database.

  • dhogaza // February 25, 2010 at 2:50 am | Reply

    It is after all just very simple low data rate data-logging problem. A quick Google will show you there are off the shelf solutions to provide solar power, measure with traceable instrumentation, and upload the data automatically to the internet

    I’ve been to developing countries. I’ve noticed that money’s a problem in those countries. They probably feel they have more pressing things to spend money on than to maximize the amount of weather data that’s automatically sent to the CLIMAT reporting system every day.

    I also don’t hear working scientists complaining that they don’t have enough stations to provide “good enough” coverage for their global temperature products. If there’s another massive effort to get more records into the system, the results going to be little wiggly changes to those global temp products is all.

  • Kevin McKinney // February 25, 2010 at 4:51 am | Reply

    Thanks for the tips!

    Just in time for my day off. . .

  • John Goetz // February 26, 2010 at 6:21 am | Reply

    I see a few problems with some of the charts above.

    First, it looks like in the case of a single station with multiple records, the multiple records are being counted as multiple stations rather than as a single station. For example, the first graph implies there were 12K to 15K stations reporting each year from 1951 to 1989. But as we can see from GISS’s plot at
    http://data.giss.nasa.gov/gistemp/station_data/
    the peak was right around 6000 stations. So a bunch of stations are being double (or more) counted due to multiple discrete and overlapping records being kept in GHCN.

    I have a copy of v2.mean from July 6, 2009. Counting discrete stations and accounting for those multiple, overlapping records – and using the R/S/U flags to identify Rural, Small Town, Urban (consistent with GISS except for the bright lights index used in the US) – I do see an increase in urban stations relative to rural. Here are some data points:

    1980: 53% R, 20% S, 27% U and 31% airports
    1989: 52% R, 21% S, 27% U and 33% airports
    1992: 50% R, 21% S, 29% U and 36% airports
    1993: 45% R, 23% S, 33% U and 40% airports
    2007: 32% R, 19% S, 49% U and 51% airports

    (Station count in 2007 was 1491.)

    Now, that may not have any effect on the temperature trend analysis done in other posts (I believe I read somewhere on here you were combining multiple station records, albeit a little differently from GISS, so single stations are not being counted repeatedly). However, the conclusion that relative urban station counts are dropping and rural counts are rising is not true.

    Of course, one does not want to place too much weight on those flags, as a station that is urban today may have at one time been rural – and vice versa. Crawfordsville, IN is a good example of a station that was urban for most of its history, being located right in the middle of town at the power plant. Then in the 1990s it was moved out to a farm, well outside of town. Now that entire record is considered “rural”, and it is used to homogenize today’s nearby urban stations.

    A similar caution should be placed on using the airport flag to denote a station as being located at an airport, particularly for that part of the record that extends back into the 1800s. :-)

Leave a Comment