Hi Elisabeth,

Thanks for your question! My name is Ariella Park, and I have taken over management of the Urban Services Initiative since Thomas’ initial posts. I am responding to your questions below, but please feel free to follow up if you would like any clarification.

Current state of affairs:
USI’s active grant-making activities have ended, as we have committed all our research funds for projects. Our portfolio of sub-grants has expanded since Thomas’ initial post to include 28 funded evaluations. However, many of our funded research projects span several years and are still ongoing. To this end, USI has received an extension through 2018 so that we can continue to support our sub-grants. As mentioned below, we also received a small pot of additional funding in 2016 to fund research and policy outreach activities in Bangladesh.

A handful of our projects have wrapped up, but only two have publicly available papers so far. (And pilot projects may never produce an academic paper as they are designed to be feasibility studies.) Links to the available papers are below; the first is a hygiene-focused paper.
Handwashing and Habit Formation: www.hbs.edu/faculty/Publication%20Files/...a2a-9f8dab3dccc2.pdf
Prepaid Electricity: Better Service Delivery for the Poor?: www.nber.org/papers/w22895

Biggest successes: overall, we saw consistent and high demand and interest for USI’s research agenda, as evidenced by the number of proposals submitted and funded.
USI has completed seven RFPs since its inception, awarding approximately $3.8 million to 28 research studies. Seven of these projects are focused on the WSH sector. Thirteen funded projects were pilot proposals, and three of them were subsequently granted funding from USI for full evaluations based on successful piloting results.
USI also held three conferences in Sri Lanka (2012), South Africa (2013), and Nepal (2014) to develop new research partnerships between J-PAL research affiliates and development practitioners. A fourth evidence dissemination conference is planned for 2018.
In January 2016, USI successfully secured a small pot of funding from an anonymous donor to fund projects specifically in Bangladesh.

You can see a list of all our funded projects here: www.povertyactionlab.org/USI/evaluations

Main challenges:
In terms of overall learnings across the portfolio, it’s a bit early to say, as the majority of our projects are still ongoing. I think it will be interesting to see what common learnings we will be able to draw across all of these projects, which tackle a wide variety of issues in the urban space.

Dear Thomas,

I was just looking over some of the Gates Foundation funded projects that finished last year and wondered how they were going. For example this one, for which you have provided such comprehensive and in-depth information, was completed October last year (unless it got an extension). It's title was: "Urban Services Initiative - Use of randomized evaluations to understand barriers to provision of urban services"

Did I get it right that this program funded five sub-grants which did work in India, Kenya and Zambia? Have they all produced their final reports or expected publications by now? If yes, can they be found on your website?

Looking back over this program what would you like to highlight in terms of:
- Current state of affairs
- Biggest successes
- Main challenges / frustrations
?

Probably some of the five haven gone on further and received more funding? Were there any main learnings across those five?
(Perhaps there is even a final report available for sharing?)

Thanks a lot in advance for sharing your insights.

Regards,
Elisabeth

Hi Sarah,

Thank you for following up on my previous post.

To answer your question on government support of successful programs, since USI is a relatively new initiative and randomized evaluations often span multiple years, only two USI-funded projects have been completed to date. Both of these are pilots, which serve more as proof-of-concept studies than research that produces results on whether the intervention should be continued or brought to scale.

Outside of USI, other research conducted by J-PAL affiliated researchers that found particular programs to be effective have been scaled up with the support of government actors. To date, 202.72 million people have been reached through the scale-up of programs evaluated by J-PAL affiliates and found to be effective (www.povertyactionlab.org/scale-ups). For example, chlorine dispensers at community water sources, first evaluated by J-PAL affiliates in rural Kenya, now reach 4.65 million people in Kenya, Malawi, and Uganda, and is a good example of an intervention that focuses on behavior change such as habit formation and social norming (www.povertyactionlab.org/scale-ups/chlor...ya-malawi-and-uganda). J-PAL affiliates’ research has also influenced policy changes for which the impacts are more difficult to quantify; for instance, pollution audits in Gujarat and their effect on reduced industrial emissions and the effect this has on overall health and well-being (www.povertyactionlab.org/news/gujarat-ad...hey-reduce-pollution).

In response to your question about government support for implementing evidence-informed programs and policies, oftentimes a program or policy is scaled by the implementing partner of the evaluation, including governments, because of their initial interest in learning whether to invest in it or not. And as I alluded to in my previous post, J-PAL also works to establish long-standing relationships with governments to institutionalize the use of evidence and encourage continued government support of effective programs. As one example, the government of Tamil Nadu and J-PAL signed a Memorandum of Understanding (MoU) in November 2015 (www.povertyactionlab.org/about-j-pal/new...mil-nadu-partnership). Over the year leading up to the MOU, officials from nine government departments, 15 J-PAL affiliates, and numerous J-PAL staff engaged in dialogue to review pressing local problems, existing programs, and evidence from other studies in order to develop a number of possible solutions to rigorously test, and, if successful, scale up throughout the state. To date, more than a dozen new pilots studies and randomized evaluations have emerged from this partnership with the government of Tamil Nadu, the findings of which will surely inform their future policies.

Concerning your first question on political will, I think I would be able to provide you a more thorough answer if you could clarify your question a little further for me. Thank you again and please feel free to raise any additional questions on my response.

All the best,
Thomas

Hi Thomas,
Thanks for the additional background on the comprehensive nature of the research program. I look forward to seeing results from the studies that are underway. In the types of studies that look at habit formation, or motivations to invest in water or sanitation at a household level, is there a consideration of how political will from different levels of government or NGOs from one area to another can play a role?
A related question, have you found that government actors are interested in supporting the kind of successful programs that the trials have identified (perhaps some results would be less politically appealing than others)?
Best,
Sarah

Dear Sarah,
Thank you for your thoughtful questions and for your interest in USI.

To answer your first question, the outcomes of the projects will be made available as research progresses. In addition, USI requires all funded projects to be registered on the American Economic Association (AEA) registry of randomized evaluations and to publish their data within a specified period of time after project completion. Additionally, as working papers and journal publications become available, these will also be accessible on the USI website.

On your second question of incorporating different approaches into the research questions, we can think about approach in terms of both discipline and methodology. J-PAL is a network of affiliated researchers, who are united by their use of randomized evaluations to answer critical questions concerning poverty alleviation. J-PAL affiliates focus on randomized evaluations because, designed and implemented well, they are the most rigorous way to estimate the causal impact of the program. But importantly, the design of J-PAL affiliates’ research incorporates knowledge from multiple disciplines and research methods.

J-PAL affiliates ensure their research is locally embedded. J-PAL affiliates possess considerable experience in the contexts in which they work. They have invested in developing strong partnerships with local institutions, and through long-term collaboration with implementing partners, J-PAL affiliates develop expertise in the social, cultural, and political economy environments in which they work.

Prior to beginning a project, J-PAL affiliates conduct needs assessments and qualitative research that inform the study design. They additionally seek the expertise of other disciplines during their background research. As an example from USI, Daniel Bennett and Simone Schaner’s study on non-communicable diseases in India incorporates knowledge from medicine and public health in the research questions and interventions.

At its core, J-PAL affiliates’ research is theory-driven and addresses hypothesized barriers as supported by extensive exploratory and qualitative work prior to the randomized evaluation. The theory that underlies the study design both motivates the research question and provides a framework for interpreting the results. As you are probably aware, economic publications generally devote substantial attention to the theoretical framework and the potential channels that drive the results.

All of this creates comprehensive research beyond the randomized evaluation component of the study, advancing our knowledge and contributing to the generalizability of results. I hope this response adequately answers your question.

All the best,
Thomas

Hi Thomas,
Thanks for sharing this information, it seems like an ambitious initiative that will provide some critical information for decision-makers (will outcomes of these sub-grants be shared on the website?). I had one comment on the approach. Given the broad focus on identifying underlying factors that affect demand of WSH services it seems like there could be fruitful collaborative opportunities to build on these findings by working with researchers with other disciplinary perspectives. For instance, although the initiative largely employs randomized controlled trials, I noticed many of the research questions, e.g. political economy questions, could also be investigated through other approaches (from the website: “Through the use of randomized evaluations, the initiative aims to better understand and address important barriers to the provision of critical urban services, including collective action failures, political economy constraints, low willingness to pay for services, and difficulties that may arise from the presence of a mix of transient and permanent populations.”). I don’t know if this is being considered at any stage of the project but collecting other types of evidence could build on the results of these trials to provide a greater understanding of why certain results were obtained and contribute to more comprehensive findings. Curious to hear your thoughts!
Best regards,
Sarah

I would like to introduce to you today a quite large (nearly 5 Mio USD) grant that I am leading with funding by the Bill & Melinda Gates Foundation. This grant has a number of sub-grants, some of which deal with sanitation (but not all of them).

Title of grant: Urban Services Initiative (USI)

• Name of lead organization: Abdul Latif Jameel Poverty Action Lab (J-PAL), www.povertyactionlab.org/
• Primary contact at lead organization: Thomas Chupein, Policy Manager, J-PAL, www.povertyactionlab.org/chupein
• Grantee location: Cambridge, Massachusetts, USA
• Developing country where the research is being or will be tested: Various in Asia and Africa
• Start and end date: 10/14/2011 – 10/31/2015
• Grant type: (e.g. Global Challenges Explorations, Reinvent the Toilet Challenge, Other) Other
• Grant size in USD: $4,874,457
• (see www.gatesfoundation.org/How-We-Work/Quic...s/2011/11/OPP1037202)
• Funding for this research currently ongoing (yes/no): Yes

Short description of the project:
Through use of randomized evaluations, the initiative aims to better understand and address important barriers to the provision of critical urban services.

More information about this methodology (randomized evaluations) is available here:
www.povertyactionlab.org/methodology

Goal(s):
To identify and rigorously evaluate, through randomized controlled trials, innovative methods designed to improve the welfare of the urban poor in Asia and Africa. USI covers a broad range of urban issues including water, sanitation, and hygiene, migrant integration and livelihoods, energy and the environment, transportation, housing and infrastructure, and delivery of health and education services.

Objectives:
Encourage innovative projects designed to address the challenge of delivering urban services, and use randomized evaluations to test the effectiveness of those innovations in the field; Establish strong research teams, and increase capacity for some developing countries’ researchers to design and conduct rigorous, randomized evaluations to test the effectiveness of their proposed solutions; Disseminate knowledge learned from USI research to policymakers and donors at the local, national, and international levels, so that effective solutions are promoted and scaled-up.

Research or implementation partners: Please see www.povertyactionlab.org/usi for more information on each of the eleven individual research projects’ institutional partners (also see below).

Links, further readings – results to date: www.povertyactionlab.org/usi

Current state of affairs:
On-going grantmaking for pilot and full randomized evaluations; eleven on-going research projects in Africa and Asia (see below).

Biggest successes so far:
Successfully executed four competitive rounds of grantmaking, awarding over $1 million to eleven unique research projects; held three conferences in Sri Lanka (2012), South Africa (2013), and Nepal (2014) to develop new research partnerships between J-PAL research affiliates and development practitioners. The most recent one took place in Nepal (see: www.povertyactionlab.org/event/usi-nepal-matchmaking-conference)

Main challenges / frustration:
See: www.povertyactionlab.org/USI/evaluations


Those projects under the USI programme with a focus on sanitation include the following five:

(1)
Wastewater as a Collective Action Problem: Effluent Trading for Water Quality in Urban India

Researchers: Rohini Pande, Michael Greenstone, Nick Hagerty, Nicholas Ryan, Anant Sudarshan
Partner(s): Indian state pollution regulator, and a common effluent treatment plant

Location: India
Timeline: 2014-2016
Type of Project: Full Study

The density that defines cities exacerbates collective action problems: my garbage litters your street, my sewage taints your drinking water. Households and firms often do not see the true cost that their waste imposes on others, which leads to excessive discharge. The resultant pollution of common resources, like waterways and reservoirs, imposes high costs on downstream residents. Researchers will investigate whether markets for pollution—specifically, the discharge of industrial effluent in urban India—can provide better incentives for conservation. Such trading markets have never been used to manage water pollution in India. Researchers will partner with an Indian state regulator and a common effluent treatment plant to set up a cap-and-trade system for effluent in a large group of industrial plants. Markets theoretically yield efficient water conservation, provided there are clear property rights and low transaction costs. Researchers will conduct a randomized evaluation of the initial allocation of discharge permits to test this fundamental relationship. This trial will provide policy guidance on the scope for market instruments to address collective action problems in public services with externalities from overuse.


(2)
Handwashing and Habit Formation

Researchers: Atonu Rabbani, Reshmaan Hussam, Giovanni Reggiani, Natalie Rigol
Partner(s):Society for Health and Demographic Surveillance, India

Location: India
Timeline: 2014-2015
Type of Project: Pilot Study

This project addresses poor hand hygiene, a leading driver of child mortality via bacterial and viral contamination and resulting diarrhea and acute respiratory infection. Public health campaigns focused on handwashing with soap have consistently failed to generate long term behavioral change, despite the effectiveness of the practice in clinical studies in improving health. In collaboration with the MIT Media Lab, researchers have developed an innovative measurement tool for handwashing that generates the precise data and feedback loop required to study and nurture systematic behavioral change via the habit loop; the project then employs this device across a large sample of households and nursery schools along with a series of incentive-driven interventions intended to generate sustained habit formation.
This pilot study will take place in West Bengal, India. The study intervention will target mothers and children below age five directly in their homes and through government-funded nurseries. The pilot study will randomize households into one of four treatment arms: 1) handwashing with soap and information, 2) treatment 1 plus monitoring, 3) treatment 2 plus incentives, and 4) treatment 3 plus self-commitment. Each treatment will be cross-randomized with a reminder treatment via text message. Immediate outcomes include frequency and timing of handwashing; long term outcomes include changes in household valuation for soap and detailed child-level health measures.


(3)
Creating a Toilet Habit
This sub-grant has been introduced on the forum here: forum.susana.org/forum/categories/71-beh...ya-j-pal-and-sanergy

Researchers: Mushfiq Mobarak, Judy Chevalier, Johann Caro Burnett
Partner(s): Sanergy

Location: Kenya
Timeline: 2013-2015
Type of Project: Full Study

Public health externalities from unhygienic sanitation remain a significant development challenge, even in areas where hygienic latrines are accessible or affordable. We hypothesize that behaviors like open defecation may persist because they represent ingrained habits that are difficult to change. Inspired by findings from psychology and neuroscience, we propose field experiments that are designed to instill a revised habit of community toilet use among the slum population of Nairobi. Our partner, Sanergy has created a network of hygienic latrines in Nairobi, but face a challenge of low demand for the toilets. Habit loops have been successfully created by private sector firms to increase demand for many household products and behaviors such as brushing regularly with Pepsodent toothpaste, or spraying Febreze air freshener. We propose to create such a loop for Sanergy toilets using a combination of economic incentives and a marketing campaign that is attentive to psychological cues and rewards. The experiments are designed to separate habit formation from other closely related models of risk aversion and learning.

(4)
Demand for Sanitation in Kenyan Urban Slums

Researchers: Paul Gertler, Sebastian Galiani
Partner(s): Athi Water and Sanitation Board, Nairobi, Kenya; The World Bank; Water and Sanitation Program

Location: Kenya
Timeline: 2014-2016
Type of Project: Full Study

We propose to study the demand for household connection to municipal sewage systems in informal slums in Nairobi, Kenya. Governments are investing in expensive sewerage systems to bring sanitation services to the household door. The cost-effectiveness of these investments depends on the number of households that connect to the sanitation systems. However, there are large fixed costs to connect to sewage systems including both the costs charged by the utility investment in household sanitation facilities, and pipes to connect from the house to the network. We propose to use a RCT to estimate price elasticity of the demand for connections, and the extent to which the price elasticity depends on information about the relationship between sanitation and health. We also consider complications related to collective action in multi-household compound connections, and resident versus non-resident landlords. Results from this study are critical to developing pricing/subsidy and information campaign policies to cost-effectively improve connectivity.

(5)
Encouraging the Adoption of Improved Sanitation Solutions in Lusaka

Researchers: Muthoni Ngatia, William Pariente, Roland Rathelot
Partner(s): Water and Sanitation for the Urban Poor (WSUP)

Location: Zambia
Timeline: 2014-2016
Type of Project: Full Study

We propose to partner with Water and Sanitation for the Urban Poor (WSUP) to carry out a research study to test different strategies to encourage the urban poor in three peri-urban areas in Lusaka to build pour-flush latrines that connect to sewerage services. The study further proposes to study the public health implications of having various proportions of a community connected to modern sewerage solutions and to gain a better understanding of low-income urban housing markets.

As this post is quite long I also attach the same content in a pdf file.

Any questions? Please post them here.

Regards,
Thomas


*Thomas Chupein*

Policy Manager, J-PAL <povertyactionlab.org/> 30 Wadsworth St, E53-334, Cambridge, MA 02142
USA
Follow us on Twitter <twitter.com/#%21/JPAL_Global>, Facebook <www.facebook.com/JPAL.Global>