Politicians in particular and the public in general should be careful about supporting policies based on one or two economic studies. Given the nature of social “sciences” such as economics, this is likely to be true for a long time.
The latest lesson is last month’s brouhaha over a flawed study by Harvard professors Kenneth Rogoff and Carmen Reinhart.
The best lesson to take from the study is not about its immediate assertion that high national debts slow economic growth — an assertion that is inherently political in today’s climate. It is rather that while the discipline of economics can provide valuable insights, it is still highly limited in what it can prove or disprove.
The controversy is over the professors’ 2010 journal article, “Growth in a Time of Debt,” in which they argue that historical data demonstrate that increasing levels of government debt slow economic growth, with a particular bend point when debt hits 90 percent of gross domestic product. While they examined 44 countries for periods across 200 years, these conclusions are drawn largely from a group of 20 countries from 1946-2009.
The paper was highly influential, cited by U.S. Republicans favoring spending cuts and Europeans favoring such cuts combined with tax increases — all with the goal of reducing debt. But there were criticisms of it from the start.
Then last month, graduate students at another school, unable to replicate the results claimed by Rogoff and Reinhart, found the authors had made a typographical mistake that excluded a few key numbers when averaging a column in a spreadsheet. Moreover, the Harvard professors had chosen to not use some available data for one country that, when included, changed the outcome. And then there was the question of how the countries were “weighted” in aggregating the results.
Correct for the error, add the unused data and it turns out there is no dramatic drop-off in growth at the 90 percent of GDP threshold. Rogoff and Reinhart acknowledged the spreadsheet error, defended their exclusion of certain data and the weighting technique and argued they still thought their general assertion was correct. The polemic goes on.
I was troubled by the whole thing, because Rogoff is perhaps the economist whose work I most admire. He has done important theoretical work and is able to comment on real-world issues with insight. But the study had serious problems even before the error and excluded data were discovered. And these problems pop up with distressing frequency in other economic research.
The first was the assumption that if high debt were correlated with slow growth, the debt must be the factor slowing the growth. Every student in intro statistics courses is taught that “correlation is not causation,” and yet, here was an article accepted by the most respected economics journal in the country based in great part on that fallacy.
One can also make plausible arguments turning the assertion upside-down: that slow growth causes high debt levels. And there may be an unmentioned third factor causing both — World War II. Several of the 20 countries, including the United States, U.K., New Zealand and Canada, all fought in the war. All emerged with very high national debts compared with GDP.
And all suffered a post-war slump for a year or two as millions were demobilized from the military and production shifted from armaments to civilian goods. Here you have several data points, nations and years, with high debt and slow growth. But was the debt causing the slow growth in those transition years of 1946 and 1947? Clearly not.
Another fundamental flaw was the implicit assumption that nations are homogeneous and that the average of the experiences of many nations of widely varying sizes and economic systems can serve as a valid predictor of what may happen to a specific large nation like ours.
This is a flawed extension of statistical analysis in the natural sciences. In trials of a new medicine, it is better to have 10,000 participants than 1,000. One can control the experiment: Some people got the medicine, and some did not. This number was cured, and that number was not. Yes, people may vary in age, weight, nutrition and other health factors, but such variations are relatively small and can be controlled for.
Logic might seem to indicate it is similarly useful to study 20 nations instead of three or four. But are nations like New Zealand and the Netherlands really part of the same cohort?
Half a century ago, prosperity in these small nations depended very highly on the pace of economic growth in their dominant trading partners, the U.K. and Germany respectively.
If their booms and busts were driven in large part by demand elsewhere, does examining their debt-growth relationships really provide any guidance for the largest nation in the world, one whose currency still is the dominant one used for international trade and reserves?
Also, the 90 percent threshold identified in the study was a variation of the fallacy of false precision. Why exactly 90 percent? Why not 83 or 102 percent?
This specific number was not identified via some analysis. Rather, researchers began by setting up arbitrary intervals of debt-to-GDP percentages, 0 to 30, 30 to 60, 60 to 90, 90 and above.
They then found no relationship in the first two brackets but did as one passed from the third to the fourth, again while using the errors subsequently identified.
But other researchers have now repeated the study using different arbitrary brackets, i.e. 20-40-60 or 25-50-75, and have not found any distinct bend point.
It is somewhat like the error often made about Social Security being a scam because it was set up with a retirement age of 65 when life expectancy was under 63. But that expectancy largely was so low because of high infant mortality.
Some people died at two and some at 90. The expectancy did not imply most people would die near age 62. Many millions were in their 70s and even 80s.
Similarly, finding an average of 90 percent debt-GDP for 20 nations does not mean every nation at that threshold also experienced slow growth. Some may have been sluggish at 50 percent, and some did just fine over 120 percent.
One could go on. The original study was not inherently bad, nor was there any deceptive intent. It just should have been viewed skeptically instead of being touted as definitive proof that austerity measures should be implemented now.