Adam is optimistic about the spread of RCTs into economics, and (like most people in development) I share his optimism. But there is a strong resistance to the rise of experimental and quasi-experimental methods in economics. In development we mainly argue over RCTs, but in economics as a whole the focus is on “natural” experiments, where circumstances generate effectively random variation in a causal variable. One of my favorite examples of this is Doug Almond’s paper on the effect of maternal fasting during early pregnancy on a child’s health and economic outcomes, which relies in part on data from right here in Michigan. His analysis relies on the fact that fasting during Ramadan is not required for pregnant women, but it’s common for women not to realize they are pregnant early on, and fast anyway.
Recent economics Nobel-winner Chris Sims is not a fan of natural experiments in economics; indeed, he won the prize in part for his work on vector autoregression models, which typify the model-everything, nothing-is-exogenous school of thought. This short comment is a pretty accessible overview of what he doesn’t like about the quasi-experimental approach, and focuses mostly on papers that analyze the deterrent effect of the death penalty. I highly recommend it for applied stat-heads like myself.
However, I was pretty let down by Sims’ complete failure to back up his central claim: “Because we are not an experimental science, we face difﬁcult problems of inference.” We’re not? Says who, and why? Claiming that economics is not an experimental science seems to imply that we can learn nothing from experiments, which I think is obviously false. Perhaps by that statement he is implying the weaker claim that there are some important things economists cannot learn from experiments – but by that standard medicine, with its utter failure to run an RCT on the effect of smoking on health, is not an experimental science either.
Hat tip: Dan Hirschman