A Reexamination of Climate Change Issues

Professor Scott Armstrong Exposing Inaccuracies in Polar Bear Studies

 November 23, 2009


In January 2008, forecasting expert Professor Scott Armstrong of the University of Pennsylvania gave Congressional testimony showing how U.S. Government studies projecting future declining polar bear populations due to shrinkling Arctic ice are seriously flawed through their relying on complex sets of assumptions, and should not be considered to be scientific forecasts.

Scott Armstrong is the founder of the Journal of Forecasting and the International Journal of Forecasting, which are the two main journals that address the subject of forecasting.  Armstrong is also a founder of the International Institute of Forecasters and of the International Symposium on Forecasting.

Mr Armstrong has published more than 50 peer-reviewed papers on forecasting.  His first book on the subject, Long-Range Forecasting, was published in 1978 and revised in 1985.  He is the editor of Principles of Forecasting: A Handbook for Researchers and Practitioners (2001) and the founder and co-director of the forecastingprinciples.com website.  He has been working on forecasting problems for 48 years.









Transcript of Scott Armstrong’s Congressional Testimony

Following are portions of a transcript of Professor J. Scott Armstrong's testimony to the Senate Committe on Environment and Public Works, January 30, 2008.  Armstrong explained how U.S. Government studies projecting future declining polar bear populations due to Arctic ice growth are seriously flawed by relying on complex sets of assumptions, and should not be considered to be scientific forecasts.

View a six minute video of Professor Armstrong's testimony

View the full transcript of Armstrong's testimony


...

My name is Scott Armstrong.  I am a professor at the Wharton School of the University of Pennsylvania.

The decision on whether or not to list polar bears under the Endangered Species Act rests heavily on forecasting.  I am addressing this Committee today as a forecasting expert.

I am a founder of the Journal of Forecasting and the International Journal of Forecasting, which are the two main journals that address the subject of forecasting.  I am also a founder of the International Institute of Forecasters and of the International Symposium on Forecasting.

I have published more than 50 peer-reviewed papers on forecasting.  My first book on the subject, Long-Range Forecasting, was published in 1978 and revised in 1985.  I am the editor of Principles of Forecasting: A Handbook for Researchers and Practitioners (2001) and the founder and co-director of the forecastingprinciples.com website.  I have been working on forecasting problems for 48 years.

...


Audit of Polar Bear forecasting

We conducted forecasting audits of two of the nine administrative reports that were prepared in 2007 to “...Support U.S. Fish and Wildlife Service Polar Bear Listing Decision.”  We selected the reports Amstrup et al. and Hunter et al. as they appeared to be the primary forecasting documents.

Our concern was to establish whether the reports’ forecasts of the polar bear population over the balance of the 21st Century were the product of scientific procedures.

We found that the authors of both reports made complex sets of assumptions in order to derive their polar bear population forecasts.  Critically, the authors assumed on the basis of general circulation model (GCM) forecasts that summer sea ice will diminish in the Arctic over the rest of the 21st Century.  Green and Armstrong (2007) audited the procedures used by GCM climate modelers and found that their forecasts were not scientific.  Forecasting sea ice conditions adds an additional level of complexity to the problem and the GCMs fail to reproduce actual sea ice conditions even when historical periods are simulated.

What if predictions about Arctic sea ice conditions over the 21st century turned out to be right?  Would the polar bear forecasts then be useful?  To assess this, each of the three authors of Armstrong, Green, and Soon (2008) used the forecasting audit software that is available on forecastingprinciples.com.  ...

On average, the authors properly applied only 12% of relevant principles.  In what occupations would work that follows 12% of proper procedures be considered acceptable?

...

Another example of what happens when principles are contravened

How did the authors of the Hunter et al. administrative report obtain forecasts of a rapidly diminishing polar bear population?

Consider again Exhibit 1.  The labels that I will add show that these data represent the ice-free days in the Southern Beaufort Sea.  None of the data shown in the exhibit were used by the authors to forecast the number of ice-free days over the 21st Century.  Instead, they relied on a climate model (GCM) scenario of dramatically increasing numbers of ice-free days.  This is in sharp contrast to what the data in the Exhibit seem to indicate.

Exhibit 3 shows the data with labels.  It is reproduced from Figure 3 on page 26 of Regeher et al (2007), one of the nine USGS administrative reports.

Hunter and her colleagues then used only the five years of data indicated by the filled-in circles and polar bear population estimates for those years to conclude that there was a strong causal relationship between the number of ice-free days and the bear population growth rate.

Finally, they combined the GCM-originated ice-free-day scenario with their estimate of the effect of ice-free days (based on 5-years of data) in order to project the polar bear population over a period of nearly a century (Hunter, Figure 6).  Their process ignored other influences on bear fertility and mortality.


Fully disclosed and peer reviewed

We have sought peer review from researchers and we are continuing to do so.  The latest version of the working paper is always available at publicpolicyforecasting.com.

We sent our paper to all those whom we cited asking them if we had cited them correctly.  This led to some changes and to a few suggestions on how to better describe what they had done.


Conclusions

To date, there are no scientific forecasts of the polar bear population over the 21st Century.  Nor are there any forecasts to suggest that a decision to list them would produce benefits.  I urge further study.