Corrections to the Mann et al (1998) Proxy Data Base and Northern Hemisphere Average Temperature Series



UPDATE: July 1 2004: The Corrigendum in Nature today (July 1, 2004) by Professors Mann, Bradley and Hughes is a clear admission that the disclosure of data and methods behind MBH98 was materially inaccurate. The text acknowledges extensive errors in the description of the data set. Even more important is the new online Supplementary Information (SI) site, which concedes for the first time that key steps in the computations behind MBH98 were left out of (and indeed conflict with) the description of methods in the original paper.
These items were published on the instruction of the Editorial Board of Nature in response to a Materials Complaint that we filed in November 2003. That our complaint was upheld and the Corrigendum was ordered represents a vindication of our view that, prior to our analysis, there had been no independent attempt to verify or replicate this influential but deeply flawed study, something which was forestalled, at least in part, by inadequate and inaccurate disclosure of data and methods.
This is only the first step in resolving the dispute we initiated last fall. The Corrigendum and the SI contain the gratuitous claim that the errors, omissions and misrepresentations in MBH98 do not affect their results. If this were true, then a simple constructive proof could have been provided, showing before and after calculations. This is conspicuously missing from the Corrigendum and the new SI. We have done the calculations and can assert categorically that the claim is false. We have made a journal submission to this effect and will explain the matter fully when that paper is published.
Further, detailed comments on the Corrigendum and new SI will be released shortly.

UPDATE: June 28 2004: We are receiving daily inquiries from around the world about the status of this project, and we appreciate the continued interest. The previous update was, evidently, premature in timing, though accurate in describing our intention to vindicate our original conclusions. We have had some indication that a technical glitch related to manuscript production has been resolved. Sometime in the next month we anticipate being able to publicly resolve a portion of the current debate.

UPDATE: May 3 2004: We anticipate being able to make a comprehensive update within the next few weeks, which will include information about several forthcoming papers. We will provide a detailed explanation of the differences between our results and those of MBH98. We will show why the explanation offered by Professor Mann (see notes below regarding the three "key indicators") does not work, since we can get our results using the full NOAMER roster and the SWM roster (the latter doesn't make any difference anyway). There are some core methodological differences between their results and ours which drive the changes. There is also another key series involved, but not one that MB&H referred to. We stand by our original conclusions, and are now in a position to explain more exactly what went wrong in MBH98.

NOTE (MAY 3 2004): The web site remains down for reasons we cannot ascertain. The site has been renewed but Steven McIntyre has not been able to access it to re-post the contents, despite complaints to the ISP. We will keep trying. NB: July 5, is back up and updated.

NOTE (APRIL 7 2004): The web site is temporarily down due to technical issues. (Well, an unpaid bill actually.) The domain has been reactivated and the content will be reloaded in a few days.

UPDATE: March 19, 2004 In response to a couple of requests for an update, here is a quick one. A few papers are undergoing review at some journals, including Steve's and my contribution(s), and there is not a dull page to be found in any of them. Unfortunately that's all we can report. While we are waiting for the review process to complete, Mr. McIntyre has been winning squash tournaments and I have started bagpiping lessons.

UPDATE: January 22, 2004 Despite the long quiet on this page, the past 7 weeks have been very busy for us. A number of people have written to ask about progress on Part II, while others have interpreted the 7 week gap as a sign that maybe we ran out of material. No, there is a lot of material, and the challenge has been to sift through it and put it into coherent form. There are now some new journals involved in handling material that arose from our paper, and we have held back releasing any of the Part II contents connected to these review processes. Professor Mann's response focuses on the role of 3 (out of 22) key indicators available in the 15th century portion of the data base. His calculations show that without these series the MBH98 results would look like ours, and his assertion is that we improperly "omitted" the series in question. Our response will establish that the series in question are in fact inadmissible. Of course the discovery that the 1998 conclusions rest so sensitively on only 3 series already points to the lack of robustness of this famous graph. But there is much more to be said, when the time comes.

UPDATE: December 1 We are continuing to work on Part II of our response, which has required a detailed examination of Professor Mann's ftp site, hence the delay. We also traveled to Washington DC on November 18, to present a briefing on Capitol Hill, sponsored by the Marshall Institute and the Competitive Enterprise Institute, on our work to date. A transcript will be published in a few days, we're told. Some interested experts at a European climate lab had privately criticized us for what they regarded as an insufficiently wide circle of reviewers for the E&E paper. We offered to them that they could review Part II before its release, on the condition that if they found errors they could hold us to public account to rectify them, but if the document checks out they would have to issue a statement saying so. After considering it for a week they declined the offer, saying they don't have time to do the review, and would prefer to follow the debate's progress in journals.

UPDATE: November 13 In the note posted below we comment on the deletion of pcproxy.mat and pcproxy.txt from the ftp site in question. We mistakenly thought pcproxy.txt was still at the site because it appeared on-screen under an exact address call. That was a browser cache copy. The file itself has been deleted from the ftp site. Also, the time quoted (Nov 11 1:46 PM) is local Toronto time.

UPDATE: November 11 Our response to the replies thus far from Professor Mann and his colleagues will be presented in three parts. Our overarching goal is to ascertain exactly what data and what computational steps were used by MBH98, so as to focus in as quickly as possible on the real sources of differences between our results. But along the way there are a few new isses that must also be dealt with.
Part 1, available here in PDF format responds to the claim that the data we audited was prepared in April 2003 in response to McIntyre's request to Mann, and that we ought to have gone to Professor Mann's ftp site instead. We show that the data file we were sent was in existence long before April 2003 and had we gone to the ftp site we would have found it contains the same data anyway. We also discuss some other pertinent file identity issues. This document, by establishing the practical equivalence between Professor Mann's ftp site and the data file we were sent, returns our focus to the basic question of data quality and sets the stage for the subsequent parts in which we will extend our existing critique.
Part 2 will present a detailed examination of the contents of Professor Mann's FTP site, in light of the claim that it is the official repository for the MBH98 data. This document has been sent to some colleagues for their comments and will be made available shortly thereafter.
Part 3, now under way, will seek to resolve the outstanding differences between our computational methods and those of MBH. Completion of this part will be contingent on our receiving the specific computer programs MBH used, and we are seeking this disclosure.

UPDATE: November 6 2003. Mr. McIntyre has a cold. Mr. McKitrick is going to an economics workshop in Manitoba for a couple of days, to discuss the question "Does the possibility of climate change imply that we should wash our laundry in cold water?" His presentation, if you are interested, is here. So there won't be any updates until next week.

UPDATE: November 4 2003. Professors Mann, Bradley and Hughes have revised their reply to our paper, see here.
They have also corrected some errors in their goodness-of-fit calculations.

UPDATE: November 3 2003. Professors Mann, Bradley and Hughes have made a more detailed reply to our paper, available here in PDF.
We will have a response prepared shortly.

UPDATE: October 29 2003. Professor Mann has made a preliminary reply to our paper. His reply and our response are available here in MSWord and here in PDF.


These questions summarize the results of our audit of the data set. Answers to these questions are required to settle the contradiction between the original and corrected results.


1.       Does the database contain truncations of series 10, 11 and 100? (and of the version of series 65 used by MBH98)?


2.       Are the 1980 values of series #73 through #80 identical to 7 decimal places? Similarly for the 1980 values of series #81-83?  And for the 1980 values of series #84 and #90-92? What is the reason for this?


3.       Where are the calculations of principal components for series in the range #73-92 that would show that these have been collated into the correct year? Do you have any working papers that show these, and if so, would you make them FTP or otherwise publicly available?


4.       Do the following series contain "fills": #3, #6, #45, #46, #50-#52, #54-#56, #58, #93-#99?


5.       How did you deal with missing closing data in the following series: #11, #102, #103, #104, #106 and #112?


6.       What is the source for your data for series #37 (precipitation in grid-box 42.5N, 72.5W)?  Did you use the data from Jones-Bradley Paris, France and if so, in which series?  More generally, please provide, identifications of the exact Jones-Bradley locations for each of the series #21-42. Where are the original source data?


7.       Did you use summer (JJA) data for series #10 and #11 rather than annual data. If so, why?


8.       Does your dataset contain obsolete data for the following series: #1, #2, #3, #6, #7, #8, #9, #21, #23, #27, #28, #30, #35, #37, #43, #51, #52, #54, #55, #56, #58, #65, #105 and #112?


9.       Do you use the following listed proxies: fran003, ital015, ital015x, spai026 and spai047?  If so, where?


10.   Did you commence your calculation of principal components after the period in which all dataset members were available for the following series: #69-71, #91-92, #93-95, #96-99?


11.   What is the basis for inclusion of some tree ring sites within a region in regional principal component calculations and others as individual dataset components?


12.   Did you commence your calculation of principal components before the period in which all dataset members were available for the following series: #72-80, #84-90? If so, please describe your methodology for carrying out these calculations in the presence of missing data and your justification for doing so?


13.   What is the explained variance under your principal component calculation for the period of availability of all members of your selected dataset?  Would you please make  your working papers that show this FTP or otherwise publicly available?






Go to publications page
Go to Ross McKitrick's homepage