More on the apparently *transient* effects of unconditional cash transfers

Berk Ozler over at Development Impact has a follow up post on GiveDirectly’s three-year impacts. The post looks at multiple papers analyzing results from the same cash transfer RCT in southwestern Kenya:

First, on the initial studies:

On October, 31, 2015, after the release of the HS (16) working paper in 2013, but before the eventual journal publication of HS (16), Haushofer, Reisinger, and Shapiro released a working paper titled “Your Gain is My Pain.”  In it, they find large negative spillovers on life satisfaction (a component of the psychological wellbeing index reported in HS 16) and smaller, but statistically significant negative spillovers on assets and consumption. The negative spillover effects on life satisfaction, at -0.33 SD and larger than the average benefit on beneficiaries, imply a net decrease in life satisfaction in treated villages. Furthermore, the treatment (ITT) effects are consistent with HS (16), but the spillover effects are not. For example, the spillover effect on the psychological wellbeing index in Table III of HS (16) is approximately +0.1, while Table 1 in HRS (15) implies an average spillover effect of about -0.175 (my calculations: -0.05 * (354/100)). There appear to be similar discrepancies on the spillovers implied for assets and consumption in the HRS (15) paper and HS (16). I am not sure what to make of this, as HRS (15) is an unpublished paper – there must [be] a good explanation that I am missing. Regardless, however, these findings of negative spillovers foreshadow the three-year findings in HS (18), which I discuss next.

Then on the three-year findings:

As I discussed earlier this week, HS (18) find that if they define ITT=T-S, virtually all the effects they found at the 9-month follow-up are still there. However, if ITT is defined in the more standard manner of being across villages, i.e. ITT=T-C, then, there is only an effect on assets and nothing else.

… As you can see, things have now changed: there are spillover effects, so the condition for ITT=T-S being unbiased no longer holds. This is not a condition that you establish once in an earlier follow-up and stick with: it has to hold at every follow-up. Otherwise, you need to use the unbiased estimator defined across villages, ITT=T-C.

To nitpick with the authors here, I don’t buy that [….] lower power is responsible for the finding of no significant treatment effects across villages. Sure, as in HS (16), the standard errors are somewhat larger for across-village estimates than the same within-village estimates. But, the big difference between the short- and the longer-term impacts is the gap between the respective point estimates in HS (18), while they were very stable (due to no/small spillovers) in HS (16). Compare Table 5 in HS (18) with Appendix Table 38 and you will see. The treatment effects disappeared, mainly because the differences between T and C are much smaller now, and even negative, than they were at the nine-month follow-up.

And then this:

If we’re trying to say something about treatment effects, which is what the GiveDirectly blog seems to be trying to do, we already have the estimates we want – unbiased and with decent power: ITT=T-C. HS (18) already established a proper counterfactual in C, so just use that. Doesn’t matter if there are spillovers or not: there are no treatment effects to see here, other than the sole one on assets. Spillover estimation is just playing defense here – a smoke screen for the reader who doesn’t have the time to assess the veracity of the claims about sustained effects.

Chris has a twitter thread on the same questions.

Bottom line: we need more research on UCTs, which GiveDirectly is already doing with a (hopefully) better-implemented really long-term study.

 

 

A Kenyan Scientist Joins the Worm Wars

For those new to the deworming debate, see here, here, here, here and here (Macartan’s response is worth reading in detail). The famous original deworming study was conducted in western Kenya, so it’s nice to get the perspective of Kenyan scientist Dr. Charles Mwandawiro. Dr Mwandawiro is the Chief Research Officer and Assistant Director of Partnership and Collaboration at the Kenya Medical Research Institute (KEMRI). He writes:

I have studied and seen myself the negative effects chronic parasitic worm infections have on childhood development. Children with severe or recurring infections have impaired growth and cognitive development because the worms lodge in their bodies, stealing the nutrients a child is able to take in. Heavy infections can result in serious clinical disease. To combat infection and give our children a chance at good health, many countries, including Kenya, run school-based mass deworming programmes that have been shown to be a simple and cost-effective strategy to reduce the disease burden of parasitic worms in school-age children, the group at highest risk.

Safe, low-cost drugs are available to treat intestinal worm infections and are the standard of medical care. The World Health Organization (WHO) recommends periodic mass treatment in areas where worm infections are above certain thresholds. Some have challenged this WHO policy, accepting that those who are known to be infected should be treated, but questioning whether the existing evidence base is strong enough to support mass treatment.

Let me say unequivocally: Mass school-based deworming works. Just three years ago, Kenya launched a national deworming program. Prevalence of parasitic worms has been reduced from 35% to 17% and as low as 6% right after a deworming round. Our focus in the National Deworming Programme in Kenya is on the reduction of infection and possibly even elimination of the public health threat of worms.

More on this here.

The WormWars are a fantastic lesson on the complications of policymaking. Contributors have weighed in from different perspectives: Does school-based mass deworming work in reducing the prevalence of parasitic worms?; what is the opportunity cost of deworming kids, thereby improving their developmental prospects?; Does a kid’s health trump everything?; did the Busia intervention increase school attendance?; did the authors adequately address the methodological challenges involved in the study? What is a policymaker to make of all of this?

Because of the complicated nature of the questions involved, the original study is being asked to bear more weight than it can withstand. Like Macartan, I think the focus should be on the school attendance outcomes, which was the primary goal of the original study. This, of course, does not mean we should completely disregard the very important questions relating to the health and developmental prospects of kids in locales with high prevalence rates of parasitic worms. Because of the long-term effects of malnutrition on cognitive development, it is reasonable to make the case that deworming kids should trump most competing uses of resources.

Policymakers, if you can, Please. Deworm. All. The. Kids.

H/T Chris Blattman

Potholes potholes potholes!

It is a key road that links western Kenya, Uganda, Rwanda and the eastern DRC to the Kenyan port of Mombasa. But the state of the Kisumu-Busia “highway” does not exemplify its economic importance to the wider east African region.  Potholes, dangerously narrow stretches, and encroachment by vendors are some of the many things that are wrong with the Kisumu-Busia highway. The many accidents that occur on the road tell it all. Last Tuesday I witnessed the aftermath of an accident in the town of Ugunja in Ugenya when on a visit to my aunt’s in Got Osimbo. A tanker swerved while trying to avoid oncoming traffic. As always happens, locals rushed to the scene with containers to siphon away fuel. The fuel caught fire and burnt many stalls that line the road in Ugunja town and a section of the famous St. Michael’s Hotel. As far as I know there was only one fatality – thanks to the fact that the tanker was carrying diesel and not the more inflammable petrol. It is not that long ago when similar accidents in Sidindi and Sachangwan caused the death of dozens of people who were trying to loot fuel.

I can’t stop asking myself: HOW HARD CAN IT BE? How hard can it be for the four countries that depend on this key road to get their act together and construct a proper road?