Is evidence-based institutional analysis a possibility?

Chris Blattman writes:

But Acemoglu is right that institutional and political change are more important and the evidence-based crowd have done very little here. Most of that evidence is about anti-corruption or election monitoring or other things that I doubt change politics very much.

Meanwhile all the good political economy research (like Acemoglu’s) has no clear implication for social and political change in the world. There is a big disconnect. These scholars have mostly ignored this gap either because… I don’t know why. Maybe it’s too treacherous or hard, or they don’t find it interesting enough, or they are cynical about policy change. I don’t know. Someone explain it to me.

Blattman is spot on.

I think that students of institutions and institutional development have not joined the evidence-based crowd for two main reasons:

  • Politics I: Much of the evidence-based research out there eschews politics, instead focusing on the technical aspects of problems. Works that explicitly deal with political scenarios exist, but are rare. Part of the reason this is the case is that agencies that finance impact evaluations and other kinds of evidence-based policy research agendas have incentives to remain as apolitical as they can (you need host country government permission to do research in the first place …..)
  • Politics II: The other reason is that it is almost impossible to engage in politically relevant big-picture-development research while remaining apolitical. You see this in splits among macroeconomists in the United States (Macro questions make it really hard for researchers to shed off their normative priors). In the same vein, the best placed people to carry out evidence-based studies of institutions and how to change them are often professors in universities in the developing world — the problem is they do not do enough research due to a lack of resources and/or the relevant skill sets; and their own governments often neglect them. Regardless of their nationality, the most visible development economists in universities in the North Atlantic often lack the political connections or the bandwidth to engage in host-country politics; and are thus limited in the extent to which they can effectively study the most vexing policy questions out there.

These reasons are not due to anyone’s fault, just how research is currently financed and structured.

A possible way to get around these problems could be MBA-style case studies of reform programs from across the globe that can then be retooled by Comparativist country specialists — incoming Stanford CDDRL director Frank Fukuyama has very exciting ongoing work on this front.

On a tangentially-related point, I think that works that combine technical brilliance and deep local knowledge (think Bates’ lesser-read books on the Zambian Copperbelt) are about to come in vogue again. It used to be that only a few grad school programs (at least in political science) emphasized technical competence out of econ envy to match the economists. This is getting more commonplace, thereby establishing a new baseline (the data revolution is also helping a great deal by increasing the scope of country-specific studies of macro questions). And once a critical mass is achieved then the comparative advantage will favor those who are both technically competent and can also speak intelligently about how policy dovetails with local politics. The title “country-specialist” will soon no longer be synonymous with “qualitative research”; and more students will be primed to value good qualitative research.


One thought on “Is evidence-based institutional analysis a possibility?

  1. There are two problems here.

    The first has gotten some recent bling: lack of anything resembling reliable data. A lot of what we have is the barest of estimates and the rest tends to be guesses. This makes for some real problems when you try to make a connect between political analysis and “evidence-based research”. These could be smoothed out by cooperation by governments, but many of them aren’t interested in doing too much of anything besides justifying their present course. You don’t need reliable data for that.

    The second is the lack of scope for findings that are based on sound data effectively analyzed. Since these are usually the result of local projects and often involve largely qualitative evidence, even the best data has limited relevance over the horizon. The Bates work mentioned above is illustrative. So you know something about mining in Zimbabwe. What makes you think the data are relevant to mining in Katanga?

    How to get around this? I think one way would be to quit relying so much on reliable data. Yes, I just said that. The basic rule here is that any data is preferable to none. Go out and collect as much data as you can in whatever way you can. Then stick it into modern data analysis apps and convert it into more reliable stuff. Stratify it so they the sample resembles the actual population in question. Then randomly select from that. This won’t cure the “the government won’t cooperate” problem, but it could make for some better conclusions,


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s