Measuring and evaluating systems change is complex - but doable

The bigger picture

A growing number of organisations have accepted that water and sanitation services will only last if the systems behind them are strong – the governance, the financing, the institutions, the accountability. This creates a difficult problem: how do you know if what you’re doing is really working?

Most organisations have tools and approaches in place to measure infrastructure. They count taps and toilets and people reached. Systems strengthening looks for different signals: government officials planning differently, budgets better aligned, the right institutions talking to each other. These changes are harder to see, slower to emerge, and rarely attributable to a single intervention.

The result is a paradox: as more programmes claim to take a systems approach, fewer can convincingly demonstrate whether their interventions are effective and how that ultimately leads to people receiving better services.

How we worked

Over the past few years, we've worked alongside a range of organisations to help them design, monitor and evaluate their system strengthening programmes – including Welthungerhilfe, delivering the FCDO-funded WASH Systems for Health programme.

We use tools like results chains, outcome harvesting and building block assessments that we’ve built and refined with partners over years – part of a growing body of innovation, research and knowledge. They help capture, anticipate and measure how people and organisations across the system are changing their behaviour, and test whether a programme is contributing to these changes. These in turn help the partners to adjust what they do.

In India, Ruchika Shiva, Director, Riva Asia was working alongside Welthungerhilfe's team at the midpoint of their first programme phase of the Global WASH Programme. The team was doing good work at the community level, but the evaluation was surfacing that engagement at the State level would be needed to achieve systemic change.

Biju Dangol, Welthungerhilfe's Technical Expert for water, sanitation and hygiene said: “Working with Riva has been a valuable experience. Their technical expertise and collaborative approach, particularly in the mid-term study on systems strengthening and the development of the proposal, provided valuable insights that strengthened our programme direction. We highly value this partnership and look forward to continuing our collaboration.”

What changed

Welthungerhilfe entered phase two of the programme with a clearer theory of how system change happens, more specific intermediate outcomes to track, and a stronger practice of reflective adaptation. 

There are still important methodological gaps that we need to better address: for example, showing not just that institutions are functioning better, but that this translates into people getting cleaner, more reliable water. That is where the learning is still live.

What's next

A systems approach is now widely accepted. What comes next is harder: proving it delivers. More programmes will need ways to show that governance, financing and institutional reform are actually changing outcomes, not just activity – and someone willing to ask uncomfortable questions about what the evidence really shows. That's part of what our research and innovation work is for. It's also something we're still working through, alongside partners. Contact us if you are working through similar issues, we'd be keen to talk.