Breaking the hockey stick
The famous graph that supposedly shows that recent temperatures are the highest in a thousand years has now been shown by careful analysis to have been based on faulty data
January 27, 2005
Few people dispute that the earth is getting warmer, but there are people -- so-called "climate skeptics" -- who question whether the change is historically unique and whether it is the result of human activity. These skeptics are generally outsiders, reviled by "true" climate researchers.
On the one hand, Michael Mann, the first author of the two noted hockey-stick papers (in Nature in 1998 and in Geophysical Research Letters in 1999), is the unofficial king of climate research. In 2002, Scientific American included him as one of the top 50 visionaries in science. On the other hand, the two Canadian skeptics are outsiders: Ross McKitrick is a professor of economics and Stephen McIntyre is a mineral exploration consultant -- which Mann likes to call a conflict of interest.
Climate skeptics are most prolific on the Internet, a platform for novices, the scatterbrained and the experienced alike. Not surprisingly, the climate researchers whom we consulted (predominantly Dutch) presumed the work of the two Canadians to be unconvincing. We at Natuurwetenschap & Techniek were initially skeptical about these skeptics as well. However, McIntyre and McKitrick have recently had an article accepted by Geophysical Research Letters -- the same journal that published Mann's 1999 article. This, together with the positive responses of the referees to that article, quickly brought us around.
Even Geophysical Research Letters, an eminent scientific journal, now acknowledges a serious problem with the prevailing climate reconstruction by Mann and his colleagues. This undercuts both Mann's supposed proof that human activity has been responsible for the warming of the earth's atmosphere in the 20th century and the ability to place confidence in the findings and recommendations of the influential Intergovernmental Panel on Climate Change (IPCC). The political implication is a serious undermining of the Kyoto Protocol with its worldwide agreements on reducing emissions of carbon dioxide and other greenhouse gases.
In their two seminal papers, Mann and his colleagues purported to reconstruct Northern Hemisphere temperatures for the last thousand years. Since 1000, temperatures gradually decreased (the shaft of the hockey stick), only to increase sharply from 1900 onwards (the blade).The implication is obvious: Human interference caused this trend to change. McIntyre and McKitrick merely attempted to replicate this oft-quoted study. In doing so, they identified mistake after mistake. They also discovered that this fundamental reconstruction had never actually been replicated by the IPCC or any other scientist. In their replication, basically derived from the same data, temperatures in the 15th century were just as high as they are today -- an outcome that takes the edge off the alarmist scenario of anthropogenic global warming. The criticism by the Canadians is mostly technical in nature: They claim that Mann and his colleagues have misused an established statistical method -- principal component analysis (PCA) -- so that their calculations simply mined data for hockey-stick shaped series and that Mann's results are statistically meaningless.
The scientists that we consulted did not immediately recognize the implications of Mann's eccentric method, suggesting the possibility he himself may not have been aware of the apparent mistake. However, in response to our inquiries, Mann denies any errors and rejects any criticism in strident terms.
Up to January, 2005, none of McIntyre and McKitrick's findings had been published by major scientific journals. Thus, in the opinion of established climate researchers, there was no reason to take them seriously. Climate researchers were quite comfortable in their consensus and repeatedly referred to this "consensus" as a basis for policy. The official expression of the consensus comes from the IPCC. This group, under the flag of the United Nations, comes out with a bulky report every five years on the state of affairs in climate research. Hundreds of climate researchers from every corner of the world contribute to it. In the third report in 2001, Mann himself was a lead author of the chapter on climate reconstructions.
Mann's hockey-stick graph was the only climate reconstruction to make it to the IPCC "Summary for Policy Makers." Its conclusion read: "It is likely that, in the Northern Hemisphere, the 1990s was the warmest decade and 1998 the warmest year during the past thousand years." This statement has been used by governments the world over to promote the Kyoto Protocol.
Stephen McIntyre first came across the hockey stick in late 2002. The Canadian government used the graph to promote the Kyoto treaty. McIntyre explains by telephone: "When I first saw the graph, it reminded me of dot.com profit forecasts, which were also hockey sticks. It was a compelling graphic, but, in the mineral exploration industry, my own field, compelling graphics are one of the techniques used to interest investors in financing mineral exploration."
McIntyre has scrutinized promotional graphics and large data sets for years. "From my own experience, I thought that the graphic looked excessively promotional," he said. "A trick of mining promoters is to overemphasize some isolated results. I wondered if this had been the case with the hockey stick as well. I thought that it would be interesting to look at the data underlying this graphic -- as though I was looking at drill core from an exploration project. The interest was simply personal; I had no intention of writing academic articles and never expected what happened afterward."
McIntyre sent an e-mail to Michael Mann in spring 2003, asking him for the location of the data used in his study. "Mann replied that he had forgotten the location," he said. "However, he said that he would ask his colleague Scott Rutherford to locate the data. Rutherford then said that the information did not exist in any one location, but that he would assemble it for me. I thought this was bizarre. This study had been featured in the main IPCC policy document. I assumed that they would have some type of due-diligence package for the IPCC on hand, as you would have in a major business transaction. If there was no such package, perhaps there had never been any due diligence on the data, as I understood the term. In the end, this turned out to be the case. The IPCC had never bothered to verify Mann, Bradley and Hughes' study."
Despite billions of dollars spent on climate research, academic and institutional researchers had never bothered to replicate Mann's work either. In 2003, McIntyre tackled the job and, from an unusual hobby, the task has since grown to become almost a full-time occupation. On an Internet forum for climate skeptics, he met Ross McKitrick, professor of economics at the University of Guelph, just outside of Toronto. Since meeting in person in September of 2003, the two have been working on the project together. McIntyre does most of the research and McKitrick asks questions and assists in the writing of papers.
Reliable temperature measurements have only been available since around 1850. Before this period, researchers have to rely on indirect indicators, or "proxies," such as tree rings, ice cores, sedimentary layers and corals, of which tree rings are the most commonly used. Scientists studying tree rings will summarize the growth at one site into a single index or chronology, which might start, for instance, at 1470 and end at 1980.
Mann's study is the best known of the multi-proxy studies. For a realistic reproduction of the temperature in the entire Northern Hemisphere, Mann and others attempt to have a relatively even geographic distribution of proxies. This posed a difficulty. The majority of proxies were tree-ring "chronologies," especially from the U.S. Southwest. To achieve more even geographic distribution (and avoid being swamped by North American tree-ring data), Mann used principal component analysis to summarize networks of tree-ring sites, the largest of which was in North America. The 1998 article reported the use of 112 proxy series.
However, for some reason, Mann and his colleagues did not accurately document the data they had actually used. McIntyre says: "Of the series and sites listed in the original documentation, 35 were not actually used. To further confuse matters, in November, 2003, over five years after publication, Mann stated that they had actually used 159 series, instead of the 112 mentioned in his Nature article or in Rutherford's e-mail."
We decided to ask Dr. Eduardo Zorita of the GKSS Research Center in Geesthacht, Germany, who has also recently examined the calculations behind the hockey stick. His response: "This is the first time that I've heard of the number 159. In our analysis of the hockey stick, we do not use the actual data, but a series of pseudo proxies, proxies we take from our simulations. We have always assumed 112 pseudo proxies."
McIntyre decided to check the PC calculations for tree-ring networks, by doing fresh calculations with original data from the World Data Center for Paleoclimatology (WDCP). His results were very different from Mann's. He and McKitrick then sent the full data set (originally downloaded from Mann's FTP site from the address provided by Rutherford) back to Mann for confirmation that this was actually the data set used. In response, Mann stated that he did not have the time to answer this or any other request.
McIntyre and McKitrick then tried to replicate Mann's Northern Hemisphere temperature calculations from scratch. The results largely coincided with the hockey stick, except for the 15th century, when their calculated temperatures were considerably higher than Mann's and were even higher than corresponding estimates in the 20th century. McIntyre emphasized: "We did not claim to have discovered a warm medieval period; we only stated that, given the many defects in the study, it could not be used to assert that the 1990s were the warmest years of the past millennium."
Their findings were published in the interdisciplinary journal Energy and Environment in October, 2003. Mann's early responses were quite unexpected. McIntyre: "Mann stated that we had used the wrong data and somehow we failed to notice errors in the data. This was outrageous, as we had downloaded the data from his own FTP site from the location provided by his own colleague, Scott Rutherford; we had described countless errors in great detail and had re-collated over 300 series to avoid these problems. Now, according to Mann, we should have taken the data off a different address at his FTP site, but this new address had never been mentioned in any publication or even on his own Web site."
A little later, Mann and his colleagues said that they had used a step-wise procedure to deal with missing data, while McIntyre and McKitrick had not. McIntyre says: "This was when the figure of 159 series first appeared. There is no mention of this stepwise method in his Nature article. A PCA calculation fails if there is any missing data."
But McIntyre and McKitrick were most intrigued by the attribution by Mann and his colleagues of the difference in results to three "key indicators" -- most notably a North American data series -- showing that, with different handling of these three series, they also obtained high early-15th-century results. McIntyre and McKitrick decided, for the time being, to concentrate on the years 1400 to 1450, the period with the biggest discrepancies.
"Mann's own response showed that his temperature reconstruction for the first half of the 15th-century depended on [data] from the North American network. We decided to find out everything that we could about these three indicators."
Because of the discrepancy between the published methodology and the methods actually used, the ambiguity over the data sets, and the sudden claim that 159 series had to be used, McIntyre and McKitrick requested original source code from Mann in order to fully reconcile their results. Mann refused. But McIntyre did make an interesting find at Mann's FTP site -- a Fortran program of about 500 lines for the calculation of tree-ring series, virtually the only source code on the entire site. They carefully studied the script and found a highly unusual procedure that had not been mentioned in the Nature article.
McIntyre says: "The effect is that tree-ring series with a hockey-stick shape no longer have a mean of zero and end up dominating the first principal [data] component; in effect, Mann's program mines for series with a hockey-stick shape."
At our request, Dr. Mia Hubert of the Katholieke Universiteit Leuven in Belgium, who specializes in robust statistics, checked to see if Mann's unusual standardization influenced the climate reconstruction. She confirms: "Tree rings with a hockey-stick shape dominate the PCA with this method."
McIntyre and McKitrick decided to perform another check. Using computer simulations of so-called "red noise," they generated networks of artificial tree-ring data over the period of 1400 to 1980. Red noise is commonly used in climatology and oceanography. McIntyre says: "If we used Mann's method on red noise, we consistently obtained hockey sticks with an inflection at the start of the 20th century. We have repeated the simulation thousands of times and in 99% of the cases, the result of the PCA was a hockey stick."
Mann's climate reconstruction methodology would have yielded a hockey-stick graph from any tree-ring data set entered into the model, as long as there is sufficient red noise.
The two Canadians are no longer just one voice crying in the wilderness. On Oct. 22, 2004, in Science, Dr. Zorita and his colleague Dr. Hans von Storch, a specialist in climate statistics at the same institute, published a critique of a completely different aspect of the 1998 hockey-stick article. After studying McIntyre's finding at our request, Von Storch agrees that "simulations with red noise do lead to hockey sticks. McIntyre and McKitrick's criticism on the hockey stick from 1998 is entirely valid on this particular point."
Tomorrow: Part II. The Gaspe trees and the file marked "censored."; Excerpted from Natuurwetenschap & Techniek, a Netherlands science journal. Translated by Angela den Tex.
© National Post 2005
CanWest Interactive, a division of
CanWest Global Communications Corp. All rights reserved.
Optimized for browser versions 4.0 and higher.