Scott Sumner  

The problem with "shocks"

Competition, technocrat-style... Meritocracy Bleg...

I recently presented a paper discussing how economists think about the "stance" of monetary policy. That is, what do economists mean by terms like "easy money" and "tight money"? It turns out that the entire subject is just a big muddle. My paper quoted a number of big names like Friedman, Mishkin and Bernanke, all saying that changes in the monetary base (QE) and interest rates are not reliable indicators, and then suggesting all sorts of alternatives. In other words, there was absolutely no agreement. My views were closest to those of Bernanke, who recommended looking at NGDP growth or inflation (I prefer NGDP growth.)

I'll return to this issue in a moment, but first I'd like to digress and discuss how economists think about economic "shocks." A shock is some sort of exogenous event (not explained by the model), which impacts the macroeconomy. It might be a shift in monetary or fiscal policy, or some sort of supply-side disturbance like political instability, war, natural disaster, or imposition of 90% marginal tax rates. The shock affects other macro variables over a period of time.

Macroeconomists have developed a technique called "Vector Auto Regression" (VAR) for estimating the impact of shocks. Thus you could collect lots of data and then try to statistically estimate the impact of a change in government spending, or a change in monetary policy, on variables like inflation and RGDP. In general, I haven't kept up with this field, but unless I'm mistaken there is good reason to be skeptical of this entire line of research. To see why, we need to return to the initial discussion of the stance of monetary policy. How can economists be expected to measure the impact of monetary policy, if they don't even know what monetary policy is?

Even worse, the same problem affects fiscal policy, even if we can agree as to what fiscal policy is. For instance, suppose we can agree that the cyclically-adjusted budget deficit is a reasonably objective measure of the stance of fiscal policy. So we go out and do a VAR study of the effect of fiscal shocks, holding monetary policy constant. But wait, didn't we just decide that we don't even know what monetary policy is? In that case, how can we hold it constant? Is it real interest rates, nominal rates, the base, M2, trade-weighted exchange rates, NGDP growth, or something else? And if we use the Bernanke/Sumner criterion (say expected NGDP growth) then it's pretty likely that VAR studies of fiscal policy will show almost no effect on NGDP due to monetary offset.

In the past I've played the devil's advocate and suggested that the concept of inflation is not well-defined and not very useful. This might seem to be a similar exercise in nihilism. In fact it's far worse. Even I accept that if inflation during 1980 is measured at 22% in Britain, 13% in the US, and 5% in Germany, then it's reasonable to assume the actual rates of inflation (if we could measure it accurately), would line up in roughly the same way. Here I'm making a much more radical claim. It's not just that I think interest rates are an imperfect measure of the stance of monetary policy, I think they are downright perverse, that is, over any substantial period of time a tight money policy leads to low interest rates, and vice versa.

Would there be any way to save the study of macroeconomic shocks? I think so, but to do so we'd have to radically re-think what we mean by monetary shocks. In a better world we'd have highly liquid and subsidized prediction markets for NGDP, RGDP, inflation and unemployment. These would give us investor expectations in real time. Instead of monetary, fiscal, and various real shocks, we'd just have nominal and various real shocks. After all, in economics the term 'nominal' basically means measured in money terms. Instead of VAR studies we'd look at market reactions to policy shocks in real time.

For instance, suppose the Fed stuns the markets today by hinting that rates will rise in June. Stocks and other asset prices fall sharply within minutes of the announcement. A study could examine the impact of the surprise announcement on the various macro futures prices. Suppose his comment led to an immediate 0.3% fall in 2 year NGDP growth expectations, and a 0.2% fall in 2 year RGDP growth expectations (implying a 0.1% fall in inflation expectations.) That study would show the size of the monetary shock, and how its effects were likely to be partitioned between real GDP and inflation. It would replace VAR tests with a market test. Likewise, we could look at the impact of a fiscal policy surprise on NGDP growth expectations, which would provide a fairly direct measure of central bank incompetence (where zero effect is "competence.")

As I indicated, I am not well informed on the modern VAR literature. Perhaps the problems I've discussed have been fixed. But the papers I have looked at recently don't give me much confidence. Nor does the existence of a "price puzzle." (Early VAR studies seemed to show that tight money raised inflation.) Nor am I reassured when I talk to other economists about how they would measure the stance of monetary policy. And you can't solve these problems by looking at interest rate movements not predicted by the model, for instance.

I'd appreciate any thoughts from people with more expertise in this area than I have.

Comments and Sharing

COMMENTS (19 to date)
Njnnja writes:

I am not familiar with the literature on monetary policy, but I do a lot of work with cointegrated time series and there are cousins of VAR models that would seem to handle the kind of problems that you point out. For example, a VECM should be able to handle the cointegration between fiscal policy and monetary policy, and adding a Markovian regime switching model should handle the problem that sometimes low interest rates are associated with low inflation and tight monetary policy but sometimes tight monetary policy is effected by the central bank increasing short rates.

I find these models to be very powerful, and the irf from them quite descriptive, but one does have to be very careful in selecting the proper causal model ahead of time. It is very easy to just start running a bunch of analyses and finding one that (over)fits the data but isn't really describing how the process works.

Jeff writes:

It's easy to get confused when reading VAR-based papers. The typical VAR exercise concludes with plots of impulse-response functions to various shocks, including policy shocks. But the response to a policy shock tells you next to nothing about what a good policy is. It only tells you what has been the response in the past to unanticipated changes in money or the fed funds rate, which is not the same thing at all.

Most of us think a good monetary policy is one that moves expectations to where the policy maker wants them to be. One way to do this is to have a clearly understood policy, like the Taylor rule. However, once a Taylor rule is followed for a few years, the coefficients of the VAR will reflect that the funds rate responds to the unemployment and inflation rates. So in terms of the VAR, at least, policy is entirely anticipated. If you want to evaluate how good that policy is, what you want to look at is how the VAR responds to nonmonetary shocks, because those are the responses that incorporate the policy reactions.

Another way of looking at this is to realize that if you think you have a good policy, you won't change it. So policy shocks tend not to happen very often, and it's unclear why anyone should even care about the VAR's predicted response to something that isn't going to happen.

In your example, you are asking to see what happens to expectations when the Fed does something crazy. Well, maybe not crazy, but at least unexpected. But if we think the Fed is not crazy, and they suddenly do something that makes us think maybe they are, we can't know how much of the change in expectations is due to "policy" and how much is due to new doubts about the competence of the central bankers. I don't really see a way around this.

Scott Sumner writes:

njnnja, But you haven't answered the question---how do you identify monetary shocks?

Jeff, Very good comment. One can think about policy reform as a two stage process. The first stage is getting policy to the point where the path of AD is about what the Fed wants (perhaps through NGDP futures targeting), and then see where the regime seems to fall short (if it does.)

If there are obvious problems, we could then change the AD target path.

But it's too hard to solve both problems at once using econometric tools such as VAR. By "both problems," I mean wrong instrument rule and wrong policy goal.

Michael Byrnes writes:

Are you getting at the same issue Nick Rowe wrote about here:

John Hall writes:

Identification (of shocks) is important in structural VAR models, but I never saw anything that says identification is a requirement for VAR models. I have found most of the discussion of shocks confusing. I find it much easier to think in terms of the conditional distribution (e.g. what is the conditional mean of this VAR given nominal GDP is forecast to grow 5%).

Jeff writes:

@John Hall,

Identification is important if you're trying to do policy analysis. If you're just trying to do forecasting, the Doan-Litterman-Sims BVAR is hard to beat.

Njnnja writes:

The identification of monetary shocks falls out of the fitting of the model. The cointegration allows for the sort of "double-dependence" where we know that, say, changing the fed funds rate affects inflation measures, but inflation measures affect changing the fed funds rate. So if your policy instrument is "Fed Funds Rate", then monetary shocks are changes in the Fed Funds Rate, and you can look at how changes in the fed funds rate impacts your other variables by looking at the irf results. Further, different monetary policy regimes would be interpreted through the expected states of the Markov transitions.

This paper seems to do exactly what I was thinking; namely, assign one regime to "high rates = tight money" and another to "high rates = loose money". Of course that is the most straightforward way to think about it so I'm not taking any credit for a particularly creative idea.

Further, at the time the paper was written (early 2000's), the only policy instrument they are looking at is the fed funds rate, so the effect of policy shocks that they look at are limited to irf's on fed funds rates (see figure 4). Of course, if one were to do this analysis again today one would include as variables things like the language of forward guidance and the values of "dots" or planned asset purchases in the Fed statements and minutes.

Jose Romeu Robazzi writes:

If I were to do this, I would try to identify a certain (low) number of variables usually perceived as associated with monetary policy and would try do build a (VAR) model of NGDP and the first diferences of these variables...

Scott Sumner writes:

Michael, That's one problem, but it's not the one I'm getting at.

Njnnja, I have no interest in how changes in the fed funds rate impact inflation or any other variable. I'm interested in how monetary shocks affect the economy. So if you start by trying to use interest rates to identify monetary shocks, then as far as I'm concerned all is lost before you have even begun.

You said:

"The identification of monetary shocks falls out of the fitting of the model."

No model can identify monetary shocks until you first figure out what you mean by "monetary shocks." There is the "how to measure it" problem, and then there is the "what are we trying to measure" problem. We haven't even reached the point of solving the what are we trying to measure problem.

For instance, you might look at interest rate movements not predicted by the model. By why would you assume that that variable has anything to do with monetary shocks?

Njnnja writes:

I'm not using "interest rates" in general to identify monetary shocks. I am thinking of a simple model, where the fed can't talk, can't do qe, can't do anything, except set a number, say between 1 and 10, that indicates it's policy stance. Confounding this is the fact that sometimes, "1" refers to a loose monetary policy, and other times, "1" refers to a tight money policy.

As I'm sure you've figured out, that number that it sets is the Fed Funds rate. But it's not important as an interest rate per se, rather, it is only important because it is (in the simplified model) the only observable and quantifiable measure of actions taken by the central bank. And therefore, it is a good measure of endogenous monetary shock, since, the fed being the monopoly supplier of base money, and it having no other way of effecting that policy, a change to the fed funds rate or holding of the fed funds rate constant *is* an endogenous monetary shock.

It's like doing a regression on a treatment group, where you know that one group was exposed to the treatment, but you don't know if they followed it, and you don't know if the non-treatment group didn't follow it. So you don't know if the Fed is tightening or loosening from some theoretical perspective, and you don't really care. All you care about are observables, and in the simplified model, the number that they set is the only observable about monetary policy you can have.

For example, if the fed thought it was tightening by increasing the fed funds rate, but the parameterization of the model says that it was in the regime where *decreasing* the fed funds rate is associated with tightening, then in fact, it was not tightening.

Next you can expand from the simplified model to incorporate other actions (or inactions) that the central bank can take. But in 2004, when the study was done, I wouldn't be so quick to say that the fed funds rate wasn't in fact the best way for the fed to transmit monetary shocks.

Nick Rowe writes:

I think Jeff's comment is spot on.

Jeff: is what you said there well-understood by the people who do VARs? Or were those your own thoughts?

Scott: good post. But I'm not sure that prediction markets solve the underlying problem any better than VARs. In both cases, the effect of a shock will depend on what the central bank is holding constant.

Scott Sumner writes:

Njnnja, You said:

"I'm not using "interest rates" in general to identify monetary shocks. I am thinking of a simple model, where the fed can't talk, can't do qe, can't do anything, except set a number, say between 1 and 10, that indicates it's policy stance. Confounding this is the fact that sometimes, "1" refers to a loose monetary policy, and other times, "1" refers to a tight money policy.
As I'm sure you've figured out, that number that it sets is the Fed Funds rate."

The Fed sets lots of numbers, including reserve requirements. I don't think that changes in reserve requirements describe the stance of monetary policy, nor do interest rates. The best indicator would be NGDP futures prices.

Your mistake is assuming that just because the Fed targets the fed funds rate over six month periods, it can be used to identify the stance of monetary policy. That's not true, because the changes in monetary policy that matter are those that occur over many months, and the interest rate is endogenous over that time frame. Nick Rowe has some good posts explaining that concept.

Nick, I agree about Jeff's comment.

Regarding prediction markets, I believe they can be useful in predicting the effect of a given change in NGDP growth expectations, on RGDP growth expectations, as I indicated in my example. Of course if the Fed were targeting NGDP expectations then you'd have no nominal shocks to measure. But in that case the markets could still predict the impact of a real shock on RGDP growth, holding NGDP constant.

Jeff writes:

@Scott and Nick:

Stop it, you two, I'm blushing.

In answer to Nick's question, I think most of the VAR modelers do understand this, but I often get the feeling that we're in lamppost territory. Somebody comes up with a cool new technical innovation that let's him identify something that couldn't be previously identified, and then tries to make the case that it's really telling us something important, rather than the other way around.

But don't get me started on what's wrong with the profession.

Njnnja writes:
Your mistake is assuming that just because the Fed targets the fed funds rate over six month periods, it can be used to identify the stance of monetary policy. That's not true, because the changes in monetary policy that matter are those that occur over many months, and the interest rate is endogenous over that time frame.
I'm not sure what you mean by "the interest rate." As you are aware, there are plenty of "interest rates." Do you mean Fed Funds? Spot 3M LIBOR? 10yr Treasury? 2s/10s spread? ATM forward 2x1 swaption rate? When I say "Fed Funds" I mean "Fed Funds", not any other market interest rate (Although I think that ultimately I would include the entire OIS curve as reflective of forward FF)

I am not assuming that the fed funds rate can be used to identify changes in monetary policy; I am hypothesizing that based on the fact that, as you say, there isn't much that the Fed did back in 2004 that could have been used to indicate their stance on monetary policy. But they did control the Fed Funds rate, and to say that there is no relationship between Fed Funds and monetary policy is just silly. The problem is that it is a complex, non linear relationship. But if the Fed increased Fed Funds 100bps tomorrow (ceteris paribus) that is clearly an indication of monetary policy, either a tightening (if inflation expectations are still low), a loosening (if inflation expectations have jumped much higher), or a commitment to remain the same (if inflation expectations have gone up about 100 bps).

Supporting my hypothesis is the fact that the paper showed 2 significant cointegration vectors between FF, output, and price level. And even more powerfully, it showed the non-montonic relationship that you are frequently concerned about between Fed Funds and monetary policy stance, where in, say, the 1990's, higher FF rates are associated with lower output and a lower future price level, but in times like in the 70's, a higher FF rate is actually associated with a higher future price level. And given the relatively straightforward approach of the paper, it is probably a pretty robust result that would hold under a more sophisticated analysis.

Anyways I agree that while an NGDP futures market would be a pretty good metric to measure monetary policy stance, one shouldn't be too quick to ignore the data that we already have. Or even worse, make the claim that the Fed Funds rate has no relationship to monetary policy stance, just because the relationship is complex and a difficult one to model.

PS I realize that my earlier link was broken so let me try again

Scott Sumner writes:

Jeff, Just thinking out loud, perhaps my problem is that I'm looking for a metric for monetary shocks, when there is none--or rather there are many. There are monetary policy tools, and then various shocks like unexpected inflation, NGDP, exchange rates, M2 changes, etc. In some cases the relationship is strictly positive---the easier the money the higher the NGDP growth. In other cases it is complex, tighter money can lead to either higher or lower interest rates (where "tight money" is defined as monetary policy that slows NGDP growth.)

I forget, what's the term for a function where an increase in X leads to an increase in Y, for all values of X (or vice versa)? Is it "monotonic"?

Njnnja, I agree it is wrong to say there is no relationship between monetary policy and interest rates, there is a complex one, as you say. My point is that interest rates are not monetary policy, they are an effect of monetary policy. Or in the case of IOR, an instrument of monetary policy.

Monetary policy also affects the price of zinc, but no one goes around describing monetary policy by referring to changes in the price of zinc.

I'm interested in how we measure monetary shocks, and interest rates are a horrible indicator--we need something else.

(My previous comment had a typo, I meant rates are targeted for 6 weeks, not 6 months. After that they are endogenous. After that they mostly reflect what's going on with the economy.)

Jeff writes:

If there is a policy instrument the central bank controls, you might define a "monetary shock" as the difference between where the bank actually set it and where the public thought the bank would set it. If you're going to call something a "shock", it has be unexpected.

If that's the definition, I don't know why we would care about monetary shocks, except perhaps to criticize the central bank for shocking people. I don't see why we should think that the response to unexpected actions by the Fed is very informative about reactions to expected actions.

If an action is completely expected, there is no reaction to it at all. If policy is always expected, there's never a reaction to the policy itself.

But even a completely known policy rule does affect how the economy responds to other shocks. For example, if one policy rule was "hold the aggregate price level constant", versus, say, an NGDP level path, then we would expect that an adverse supply shock would reduce real output more in the first case than in the second.

Scott Sumner writes:

Jeff, I don't want to use the instrument setting, because it is too ambiguous. A cut in the Fed funds target could be easy or tight money. If we go to unexpected changes in the instrument, that's slightly better, but only slightly. It tells you that on that day money was either easier or tighter than expected. But what we care about is whether over a 6 month period money is easier or tighter than expected. And for that interest rates are useless.

We don't care about unexpected actions by the Fed, we care about unexpected moves in NGDP. Or perhaps I should say that we only care about the Fed to the extent that they create unexpected moves in NGDP.

I agree with your last paragraph.

Jeff writes:

Scott, I think we're in agreement. I agree with you that a market-based measure of expected NGDP growth is probably the best guide we can get for monetary policy. Whether or not it is an accurate measure of policy doesn't really matter, since we can't seem to even define what we mean by tight or loose policy. The real question is, what should the Fed do? And it seems to me that your proposal to target a path for the level of expected NGDP is probably the best we can come up with.

Jose Romeu Robazzi writes:

I understood from the discussion that it would be nice to find antecedent indicators that affect NGDP.
I am sure people have alread suggested these, but the two things that come to my mind are capital markets new issuance, and credit growth (or credit conditions indicators )...

Comments for this entry have been closed
Return to top