More on the apparently *transient* effects of unconditional cash transfers

Berk Ozler over at Development Impact has a follow up post on GiveDirectly’s three-year impacts. The post looks at multiple papers analyzing results from the same cash transfer RCT in southwestern Kenya:

First, on the initial studies:

On October, 31, 2015, after the release of the HS (16) working paper in 2013, but before the eventual journal publication of HS (16), Haushofer, Reisinger, and Shapiro released a working paper titled “Your Gain is My Pain.”  In it, they find large negative spillovers on life satisfaction (a component of the psychological wellbeing index reported in HS 16) and smaller, but statistically significant negative spillovers on assets and consumption. The negative spillover effects on life satisfaction, at -0.33 SD and larger than the average benefit on beneficiaries, imply a net decrease in life satisfaction in treated villages. Furthermore, the treatment (ITT) effects are consistent with HS (16), but the spillover effects are not. For example, the spillover effect on the psychological wellbeing index in Table III of HS (16) is approximately +0.1, while Table 1 in HRS (15) implies an average spillover effect of about -0.175 (my calculations: -0.05 * (354/100)). There appear to be similar discrepancies on the spillovers implied for assets and consumption in the HRS (15) paper and HS (16). I am not sure what to make of this, as HRS (15) is an unpublished paper – there must [be] a good explanation that I am missing. Regardless, however, these findings of negative spillovers foreshadow the three-year findings in HS (18), which I discuss next.

Then on the three-year findings:

As I discussed earlier this week, HS (18) find that if they define ITT=T-S, virtually all the effects they found at the 9-month follow-up are still there. However, if ITT is defined in the more standard manner of being across villages, i.e. ITT=T-C, then, there is only an effect on assets and nothing else.

… As you can see, things have now changed: there are spillover effects, so the condition for ITT=T-S being unbiased no longer holds. This is not a condition that you establish once in an earlier follow-up and stick with: it has to hold at every follow-up. Otherwise, you need to use the unbiased estimator defined across villages, ITT=T-C.

To nitpick with the authors here, I don’t buy that [….] lower power is responsible for the finding of no significant treatment effects across villages. Sure, as in HS (16), the standard errors are somewhat larger for across-village estimates than the same within-village estimates. But, the big difference between the short- and the longer-term impacts is the gap between the respective point estimates in HS (18), while they were very stable (due to no/small spillovers) in HS (16). Compare Table 5 in HS (18) with Appendix Table 38 and you will see. The treatment effects disappeared, mainly because the differences between T and C are much smaller now, and even negative, than they were at the nine-month follow-up.

And then this:

If we’re trying to say something about treatment effects, which is what the GiveDirectly blog seems to be trying to do, we already have the estimates we want – unbiased and with decent power: ITT=T-C. HS (18) already established a proper counterfactual in C, so just use that. Doesn’t matter if there are spillovers or not: there are no treatment effects to see here, other than the sole one on assets. Spillover estimation is just playing defense here – a smoke screen for the reader who doesn’t have the time to assess the veracity of the claims about sustained effects.

Chris has a twitter thread on the same questions.

Bottom line: we need more research on UCTs, which GiveDirectly is already doing with a (hopefully) better-implemented really long-term study.

 

 

Are researchers scared of bursting the unconditional cash transfers bubble?

This is from Berk Ozler over at Development Impact (which you should all be reading religiously, btw):

……. An increasing number of studies show short-term effects of cash transfers dissipating over time, at varying speeds of decay. But, more on that below… What did surprise me is that I had to read the transcript of the interview to find out about this new finding (no working paper yet, it seems, but here is an abstract). No one was tweeting about the massive four-year effects disappearing: remember that women almost doubled their income compared to the control group five years earlier. It’s not news that these effects are gone?

We are all guilty. If the quote had been about the durability of the effects of cash transfers – even at half of the short- and medium-term levels – many of my tweeps would be shouting it from the rooftops. Why? Because, we disseminate evidence that reinforces our view of the world, but choose to ignore or rationalize away stuff that does not. That may help to keep oneself sane these days, but a good public academic it does not make. Most of us think we’re better than that but we are fooling ourselves. Yes, many of you will politely retweet one of my posts about this or that hype about cash transfers, but deep down you know what you think: unconditional cash transfers are great and there is not a thing any researcher can do about it…

Even in the most favorable interpretation of these new findings, however, the fact remains that there is no treatment effect of cash transfers on beneficiary households other than a sizeable increase in non-land assets, which are held mostly in improved roofs and livestock. This new paper and Blattman’s (forthcoming) work mentioned above join a growing list of papers finding short-term impacts of unconditional cash transfers that fade away over time: Hicks et al. (2017), Brudevold et al. (2017), Baird et al. (2018, supplemental online materials). In fact, the final slide in Hicks et al. states: “Cash effects dissipate quickly, similar to Brudevold et al. (2017), but different to Blattman et al. (2014).” If only they were presenting a couple of months later…

Cash transfers do have a lot of beneficial effects – depending on the target group, accompanying interventions, and various design parameters, but that discussion is for my next post…

Quick thoughts:

  1. Give Directly and the research agenda around their interventions have been great for showing the many ways in which targeted welfare provision can be structured to increase levels of household consumption and investment. I am curious to see the economic impact of their UBI project being rolled out in Kenya. Also, I don’t think that they would deliberately under-publicize unfavorable research findings — see here. Looking forward to the full range of research findings from their previous interventions.
  2. In addition to increasing household consumption (direct cash), we should also be thinking about ways to improve the provision of public goods and services — perhaps by doing the two together.

The other implication here (attributable to Justin Sandefur) is that may be cash transfers would work if they were part of a permanent welfare system.

But are the Malawis of this world (fiscally and politically) really ready for this? Should Malawian policymakers instead be spending their precious time worrying about agricultural productivity and getting their jobless youth into factory work?

All to say that more research is needed on cash transfers, especially with a focus on the political economy implications of such policy proposals and in conjunction with some public goods component.

more from Siaya: Siaya District Hospital is a disgrace

If anyone knows anyone important in the ministry of health and medical services please let them know that the x-ray division at the Siaya district hospital is not even trying – especially the bespectacled guy who apparently recently went on a trip overseas (I know this because he spent like two hours total chatting with random visitors about it). While visiting the hospital I saw what patients at the hospital have to go through on a daily basis. There was the woman who had come in for two days straight to get a dental x-ray. The previous day the x-ray guy sent her home because he had to leave after a three hour lunch break. The next morning when she returned she was made to wait for more than two hours because the said guy had to help out a nurse’s pregnant friend who wanted to have an ultra sound done. The most annoying bit of it all is that the waiting area is right outside the guy’s office and so we could hear all their conversations. The said guy then took another hour doing some paperwork and chatting with friends who popped in about every fifteen minutes. Je, huu ni ungwana???

I obviously got annoyed by all this and in my naivety dialed the hospital hotline which is listed under the poster with the costs of all the x-rays outside the doctor’s office. The lady on the other side casually told me that they would take care of the delay and backlog with the x-rays and immediately hung up. Nothing was done. And to imagine that these guys live on taxpayers’ money. I have contacted Mr. Anyang’ Nyong’o himself and I am still waiting to hear from him (Mr. Nyong’o is very good with email, the only other time I wrote him he wrote back within a week).

jkia has free internet!

The last time I had free wireless at an airport was in Hartford, Connecticut. I am therefore absolutely delighted to be able to blog as I wait for my flight to London tonight. I am not looking forward to the long hours in pressurized steel tubes – as one of my pals calls them – and the long layover in the infamous Heathrow. Although there is no chance of missing my connecting flight to San Francisco, I am bracing myself for the possibility of not having my luggage when I arrive there.

Already missing home. It has been a fun one month, most of which I spent in rural Kenya. Being in Nairobi has been fun too. I am glad I got to be here for the referendum, the promulgation of the new constitution and the release of the 2009 national census results. Kenya is a lot of things, but lately it has been trying tooth and nail to put its best foot forward, the al-Bashir fiasco notwithstanding (I am one of those optimists who are hoping that Kenya was playing smart diplomacy by allowing the genocidaire president to come here in exchange of his honoring the January 9th secession referendum for Southern Sudan).

back in business…

Dear readers, sorry for the long silence. I blame it on the super-slow internet connectivity offered by Safaricom in western Kenyan. Back here in Nairobi things are a bit more normal. Over the next few days I shall post a series of blogs that I wrote but never published for one reason or another.

As promised, here is the comparison between Siaya and Bondo.

Siaya is the older town, and has been district headquarters in the wider Alego and Ugenya areas for quite some time. Bondo on the other hand is a “younger” town (I use ‘young’ here loosely, I don’t quite know when it was founded) and only became district headquarters due to the influence of the Odinga family which calls it its home town. A son of the older Odinga, Raila Odinga, is Kenya’s Prime Minister.

The contrast between the two towns is a lesson in African/Kenyan political economy of development. Siaya, lacking a big man in the capital (Aringo was forever in the opposition and Yinda and Weya are total non-starters) has lagged behind Bondo in many respects. Bondo has a teachers’ training college that has recently been made into  a university college. It has a nice tarmac road linking it to Kisumu, the provincial capital. Right now the roads within Bondo are being done. The road linking Bondo and Siaya is also being done. But within Siaya nothing is happening as far as the roads are concerned, despite the sizable motorcycle and matatu traffic. The road linking Siaya and Kisumu is full of pot holes. The lesson here is that it matters to have someone high up in Nairobi.

The similarities, to me at least, are more interesting. Both towns are filled with small scale shops selling the same stuff – small dukas and “supermarkets”, bicycle and motorcycle repair shops, hardware stores, mobile airtime shops etc. Diversification has not taken root in either town. Even the good roads do not appear to have helped Bondo in this respect. There is also a serious shortage of green vegetables in both. I am told the shortages in kales in particular last between August and March (Kenyan entrepreneurs, are you reading?). I asked if there is a close place nearby where one can source vegetables but was told that the best way to go about it is to “import” the stuff all the way from Nairobi.

My two-day focus on this vegetable thing has taught me that the problem appears to be the fact that all the vegetable farming in this area is rain-fed and are therefore highly seasonal. No one has so far bothered to go large-scale and use some irrigation. Digging a borehole or pumping water from a nearby stream is not that hard. The cost of digging a borehole in the village is no more than $ 800. It is not the cost that is prohibitive, it is collective action that remains elusive – even in this ethnically homogenous part of Kenya.

Just as an aside, my experiences in Siaya so far have made me come to the conclusion that poverty and underdevelopment are, to a large extent, a state of mind. I have mingled with a few local people from Nairobi’s upper middle class and even they seem to traffic in the notion that it is the duty of government and God to run their lives.