Can Economists Be Trusted? Are There Any Wrong Answers in Economics? - Part II

Mark Thoma:

"I wasn't completely happy with my discussion in Can Economists Be Trusted? Are There Any Wrong Answers in Economics?. I talked about cherry picking results to serve political aims, but I didn't talk about or come down hard enough on the misrepresentation of results for political purposes.
So I'm glad to see Andrew Gelman continuing the discussion:

"Can economists be trusted?," by Andrew Gelman: Mark Thoma has an interesting discussion of the challenge that the economics profession, and individual economists, have when they give policy recommendations.
Mark's basic point goes as follows. Consider the following four stages of a model:
(a) assumptions about fundamental principles of how the world works,(b) normative principles (that is, fundamental goals, views about how the world should be),(c) conclusions about the likely effects on policy,(d) recommendations about policies.
In any rigorous economic model, there should be a mapping leading from (a) to (c). Further reasoning (possibly mathematical modeling, as in cost-benefit analysis) will take you from (b) and (c) to (d).
That's all fine. But Mark's point is that the reasoning can go the other way too: start with (b) and (d), and then you can figure out what (c) needs to be, and then you can go back one more step and figure out what model (a) you need to get started! Even if economists are not doing this reasoning-from-conclusions-to-assumptions explicitly, you could well believe it's going on implicitly as well as being induced by various pressures such as the selection of what research results to report and even what problems to work on.
This is inevitable, and I discuss it in ... Bayesian Data Analysis. We call it the garbage-in-garbage-out problem: If you can come with any decision you'd like by just altering the inputs of your analysis, then what's the point of decision analysis (or, by extension to the above-linked example, economic modeling) at all?
My answer is something that I call "institutional decision analysis," which has two principles:
1. It can be a good idea to provide reasoning to justify your decisions. ...[A]n institution--whether it be a business, a government agency, a nonprofit organization, or some other grouping--often needs some path of bread crumbs connecting assumptions to recommendations. ...
2. As Mark noted, an overall decision recommendation on anything important is likely to be so dependent on assumptions to such an extent that it's probably fair to say that the analyst is reasoning from conclusions to assumptions (from (d) to (c) and then to (a), in my above notation). But, even then, formal decision analysis can be useful in making relative recommendations. This is the point that we made in our article about decision making for home radon. In the economics context, this might suggest that economists of different political persuasions could still give useful recommendations about how to spend money or cut taxes, or where in the economy such policies would make more or less sense.


There's always a temptation to act as a lawyer - to use theory and empirical evidence, your own if necessary, to make the best possible case for the policies you would like to see enacted.

Lobbyists certainly do this, consultants do this in some cases (though they ought to advise their clients of the full spectrum of evidence), politicians don't hesitate to shade things to make themselves look good, and some think tanks are also fully engaged in this type of activity (in a few cases, to the point of blatantly misrepresenting theory or evidence in order topromote their point of view ).

But lawyer like advocacy for preferred policies is not how academic economists ought to present evidence to policymakers and to the public. This does not mean economists should always end up in some wishy-washy, on the one hand, on the other hand position due to unavoidable uncertainty involved in the decision-making process, they can still come to firm policy recommendations. And in this regard point 1 above - connecting assumptions to recommendations - is helpful in explaining the basis for the decision (I'd add the connection between the empirical evidence and the recommendations as well). But I'd also like to see a bit more than that, including how robust the recommendations are to changes in the assumptions, whether there are other common assumptions that lead to different outcomes, and if so, why these assumptions were ruled out, and the strength of the empirical evidence being used to support the policy position.