What is the Economic Policy Uncertainty Index Really Telling Us?

By Mike Konczal |

Conservatives have crafted a measurement that uses their own rhetoric as evidence to support their economic talking points.

Do you want to see a magic trick? It doesn’t involves cards, fire, or anyone levitating. Instead I’m going to show you a set of Republican talking points magically turn into an economic index — an index that Republicans then use to argue for their policies.

Mitt Romney’s economics team of Hubbard, Mankiw, Taylor, and Hassett have rapidly turned around an economic policy sheet titled “The Romney Program for Economic Recovery, Growth, and Jobs.” Matt Yglesias has a post on the issue of sluggish growth and Dylan Matthews has one on their review of the stimulus literature. Brad DeLong takes the deep dive through the entire piece here.

I’m interested in something I haven’t seen people critically discuss enough, and that is the “policy uncertainty index.” The Romney plan argues that “uncertainty over policy – particularly over tax and regulatory policy – limited both the recovery and job creation. One recent study by Scott Baker and Nicholas Bloom of Stanford University and Steven Davis of the University of Chicago found that this uncertainty reduced GDP by 1.4 percent in 2011 alone, and that restoring pre-crisis levels of uncertainty would add 2.3 million jobs in 19 months.” This appears to be a new talking point for the candidate’s team, as the same language was in a Wall Street Journal editorial by Hubbard over the weekend.

Let’s take a critical look at this paper, “Measuring Economic Policy Uncertainty,” which also has its own website, as it is likely to come up again in the election season. There are two sets of issues, one related to what the index actually shows and another related to the construction of the index itself.

Interpreting the Index

First off, does the paper show what Romney’s team claims? Matt O’Brien notes that the big run-up in uncertainty in 2011 is a function of the battle over the debt ceiling. This is very obvious from the graph of their index:



I personally think we can blame that fiasco on House Republicans. But even if you think the Democrats share some of the blame, it has nothing to do with Dodd-Frank or Obamacare. But Romney’s team is using this uncertainty issue to call for repealing both.

That said, the rate is elevated starting around 2009. Why is that? The uncertainty index consists of three parts. The first a news search for articles on policy uncertainty, which we’ll return to in a minute. The second part has to do with disagreements among economic forecasters. And the last part is “the number of federal tax code provisions set to expire in future years.” Tax code provisions set to expire are weighted by the formula 0.5^((T+1)/12), where T is the number of months until the tax code expires. That means these provisions weigh more in the analysis as they get closer to expiring — those with more time left have weights approaching 0, and those close to expiration approach 1.

And of course, as the paper notes, “An important recent example involves the Bush-era income tax cuts originally set to expire at the end of 2010.” The way the weighting works is that it jumps in the two years before expiration, which means the tax cuts scheduled to expire at the end of 2010 really start to matter for the index starting in late 2008, when President Obama is elected.

Watch that again. George W. Bush’s economic advisors, like Glenn Hubbard, pass a series of tax cuts in the early 2000s that are set to expire 10 years out. When Obama gets into office the deadline starts to approach, creating “uncertainty” in this index. Then people like Hubbard blame President Obama for all that uncertainty caused by the design of the Bush tax cuts. Brilliant.

A Magic Trick

But now for that magic trick. How do they construct the search of newspaper articles for their index, which generates a lot of the movement?

Their news search index is constructed with four steps. They first isolate their search to a set of articles from 10 major newspapers (USA Today, the Miami Herald, the Chicago Tribune, the Washington Post, the Los Angeles Times, the Boston Globe, the San Francisco Chronicle, the Dallas Morning News, the New York Times, and the Wall Street Journal). They then search articles for the term “uncertainty” or “uncertain.” They then filter again for the word “economic” or “economy.” With economic uncertainty flagged, they then filter again for one of the following words to identify government policy: “policy,” “‘tax,” “spending,” “regulation,” “federal reserve,” “budget,” or “deficit.”

See the problem? We don’t know what specific stories are in their index; however, we can use their search terms listed above to find which articles would have likely qualified. Let’s take a story from their first listed paper, USA Today“Obama taking aim at GOP pledge on campaign trail,” from August 28, 2010 (for the rest of this post, I’m going to underline the words in quotes that would trigger inclusion in their policy uncertainty index):

Brendan Buck, a spokesman for the House GOP lawmakers who crafted the pledge, said “it’s laughable that the president would try to lecture anyone on spending.” [….] Buck said the pledge was developed to address voter worries about high unemployment and record levels of government spending and debt.

“While the president has exploded federal spending and ignored Americans who are asking, ‘Where are the jobs?’, the pledge offers a plan to end the economic uncertainty and create jobs, as well as a concrete plan to rein in Washington’s runaway spending spree,” Buck said.
Spokespeople for the conservative movement tell reporters that President Obama’s policies are causing economic uncertainty. Reporters write it down and publish it. Economic researchers search newspapers for stories about economic uncertainty and policy, and create a policy uncertainty index out of those talking points. The conservative movement then turns around and points to the policy uncertainty index as scientifically justifying their initial talking points about Obama and uncertainty as well as the need to implement their policies. Taa-daa! Magic.
Two Other Issues

It’s amazing how much of the GOP rhetoric you find when trying to replicate this index. With that in mind, there are two additional issues with the index, one empirical and the other theoretical. Let’s start with this story, likely caught in their index, USA Today‘s “Minority leader accuses Obama, aides of ‘job-killing,'” from August 28, 2010: “House Minority Leader John Boehner of Ohio used a speech in Cleveland to blame Obama’s spendingtax and regulatory policies for creating uncertainty and stalling economic growth.”

Let’s pretend, after this story came out, that reporters follow up by asking a lot of experts what they think, and those experts say “There’s little evidence to support Boehner’s idea that uncertainty over regulation and policy are contributing to economic weakness.” What happens? Do they cancel? No, the uncertainty index flags it as more economic uncertainty.

If Boehner, upon reading that story, went out the next day and gave a quote to a reporter that said “I no longer think that uncertainty caused by regulation is contributing to our economic problems,” that would be flagged as more uncertainty!

Which is to say that the empirical problems with this measure of policy uncertainty always bias the results upward. Data is never perfect, so it is important to understand which way it is likely to bias. The noise machine of talking points biases this index upwards, but any stories pushing back against this uncertainty meme would also push the index upwards.

There’s also the theoretical issue. Their story is one of a weak economy created by government policy uncertainty, of “taxes, government spending and other policy matters.” Last fall, the authors wrote an editorial for Bloomberg arguing that their model showed that “harmful rhetorical attacks on business and millionaires,” the NLRB’s actions against Boeing, and Obamacare were all major factors in the weak recovery. These all point to the supply side of the economy.

But what about uncertainty from lack of demand? Consider a story that begins with “Keynesian economists argue that the economy today is weak because businesses are uncertain about future customers and workers are uncertain about their future jobs, and the textbook response to this situation is expansionary monetary and fiscal policy.” This would be flagged in their index as a problem of government policy, though it is a story of weak aggregate demand.

This isn’t a hypothetical. Let’s look at another story likely captured by their index, USA Today, “Retail sales drop for first time in 5 months,” August 13, 2008:

Retail sales fell in July, the weakest performance in five months, as shoppers shunned autos and other big ticket items. [….] Analysts said the poor showing in July, the last month for bulk mailings of stimulus checks, raised concerns about consumer spending going forward.

“Cautious and uncertain consumers are watching their wallets and with the back-to-school shopping season under way, that does not bode well for retailers,” said Joel Naroff, chief economist for Naroff Economic Advisors. [….] The disappointing performance of retail sales meant that the consumer sector, which accounts for two-thirds of total economic activity, got off to a weak start at the beginning of the third quarter.

As the economy is going into freefall, as the worst recession since the Great Depression is starting, as the Great Moderation is coming to an end and the violence of the business cycle and a prolonged downturn shows its ugly head again, consumers are reducing consumption because of economic uncertainty. Yet this index reads this as just another example of out-of-control government policy and records it as such. The index will see stories about demand uncertainty as stories about supply, which means it will have trouble telling any accurate story about the Great Recession and our current troubles.

(I have a follow up post, taking apart the rest of the index, here.)

Follow or contact the Rortybomb blog:



Mike Konczal is a Fellow with the Roosevelt Institute, where he works on financial reform, unemployment, inequality, and a progressive vision of the economy. His blog, Rortybomb, was named one of the 25 Best Financial Blogs by Time magazine. Follow him on Twitter @rortybomb.