The Benefits of Shared Measurement Systems

Lessons from the MWA-Ethiopia ProgramIRC SymposiumApril 9-11, 2013

Implementing in Ethiopia Since 2004

5 MWA Members, 4 Local PartnersCommon Policies & Strategies

EKHC

3

The Challenge:The Challenge:

How do you foster learning that leads to improvement in an environment when

there is often little room (time, resources) for it?

What to MonitorWhat to Monitor

• Outputs = what org. does– # of water points constructed– # of hygiene trainings given

• Outcomes = how target groups respond to what you did– # households using water point– WASHCOM makes needed repairs– Handwashing behavior change– # households constructing latrine

Each org uses own methods to monitor. Reports designated

indicators to Secretariat

Each org uses own methods to monitor. Reports designated

indicators to Secretariat

Use MEL Framework tools to monitor key outcomes.

Report to Secretariat

Use MEL Framework tools to monitor key outcomes.

Report to Secretariat

‘Core’ Indicators for Output, Outcomes, and Impact:• % of functional systems; • % of households using an improved water source; • % of households with access to 15 liter per day from a protected

water source; • % of households with access to water for productive uses; • % of households with access to improved latrines; • # of communities with open defecation free (ODF) status, • % of households with soap and water at a hand washing station; • % of people with knowledge of critical hand washing times.

6

Common MEL System

7

Outcome monitoring toolsOutcome monitoring toolsIndicator category

Type of info collected

ToolData

collected byScope

Water schemes

Functionality; maintenance; governance; etc.

Water Scheme Monitoring Form

Govt officer or

NGO staff ‡

All intervention water schemes*

Community promotion

Adoption of latrines, handwashing facilities, water treatment; etc.

Household WASH Checklist

Health Extension

Workers† or hired

enumerators

Sample of hh in kebeles that received WASH promotion

CLTSProgress toward kebele ODF declaration 

CLTS Monitoring Form

NGO staff

1 CLTS form for each triggered kebele

Outcome monitoring toolsOutcome monitoring tools

Indicator category

Type of info collected ToolData

collected by

Scope

Schools

School outcomes: WASH materials put out; latrine cleanliness; capacity to sustain; etc.Students outcomes: observed handwashing; latrine use; knowledge

School Monitoring Form (optional) Rapid Student Survey §

NGO staffAll

intervention schools

Health facilities

Facility outcomes: WASH materials made available; latrine cleanliness; capacity to sustain; etc.

Health Facility Monitoring Form

NGO staffAll

intervention facilities

Data Reflection Tool (DaRT):Data Reflection Tool (DaRT):A Qualitative ToolA Qualitative Tool

1. Summarize information you gathered, and 1-2 most interesting pieces of information you learned.

2. Describe things that indicated your activities have prompted target group to progress toward achieving the relevant outcome indicators.

3. Describe things that suggest your activities have NOT prompted the expected response.

4. Is there anything your organization can do to address issues in #3? Do you have plans to take action?

5. Describe any potential threats to sustainability that you observed and what you plan to do to address them.

6. What would you like to ask or discuss with other MWP partners?

Designed to prompt reflection & discussion

Costs of Monitoring• $150,000 USD to develop MEL Framework

with the assistance of Emory University Center for Global Safe Water

• $1,500 per partners per data collection exercise

• $45,000 for data collection overall3% of total grant budget dedicated to M&E

Challenges

• Endless (?) Debate over indicators• Perceived costs & time required to do data

collection• Translation in numerous languages• No provision for post-program monitoring or

evaluation

Successes• Time series data collected by 9 partners in

same format. • Partner level and program level analysis• Coordination with local and national

government for information and data exchange

• Early identification of weak areas (governance, sanitation progress)

MWA Theory of Change