This blog was written by Dr Divya Nambiar, Principal Consultant and Hub Lead, Skills, Livelihoods and Education Systems and Phalasha Nagpal, Consultant, Skills, Livelihoods, and Education Systems at Oxford Policy Management (OPM). It primarily draws on OPM’s Symposium at the September 2023 UKFIET conference.*

Oxford Policy Management’s symposium titled, Results-based financing (RBF) in the education and skills development sectors: Perspectives from academia, policy, and practice, at the UKFIET 2023 conference, addressed the following questions through a stimulating panel discussion: What does it take to make a results-based financing (RBF) programme successful? What lessons can we draw to inform future programmes?

OPM is the independent evaluator for several RBF programmes in the education sector – spanning a range of areas including: early childhood education, improving learning outcomes in primary and secondary grades, and enhancing skills and livelihoods across countries such as Sierra Leone, Tanzania, Rwanda, and India. While our teams have looked deep within our individual projects -adapting, iterating, and learning on the go; we have rarely had the chance to step back and look across these projects collectively in collaboration with our partners and draw out our shared learning. The UKFIET September 2023 conference gave us a chance to do precisely this.

Our panel consisted of OPM colleagues and our project partners – the British Asian Trust, Education Outcomes Fund and FCDO, who shared insights from three different RBF programmes:

  • The Skill Impact Bond, which provides short-term skills training and employment to 50,000 young men and women across India and is the largest Development Impact Bond (DIB) in the world.
  • Education Programme for Results (EPforR) programme, which seeks to strengthen the public education system in Tanzania with a focus on equity, access, and quality.
  • Sierra Leone Education Innovation Challenge (SLEIC), which focuses on improving literacy and numeracy outcomes for over 134,000 pupils across 325 primary schools in Sierra Leone.

With golden Oxford sunlight streaming into the hall at the Examination Schools, we shared our collective experience in the field of RBF – starting with highlighting the similarities and differences across the three programmes. We discussed some of the initial challenges in setting up such programmes; the process of building trust and effective partnerships; the crucial role of evaluators in generating data and evidence (to inform payments and to drive change); the importance of government support, and examples of effective collaborations for improving learning outcomes for children, strengthening education systems, or providing skills and jobs to youth.

It was equally inspiring to engage with our audience of over 100 practitioners, scholars, and policymakers from across the globe, who brought in examples from their country contexts and raised important questions on sustainability and scale of RBF programmes.

Four key insights from our panel discussion are summarised below.

First, RBF programmes raise the bar of success. Unlike traditional programmes that define success in terms of inputs (such as budget allocations), RBF redefines success in terms of outcomes achieved. It goes a step further, by providing financial incentives for outcomes achieved, incentivising stakeholders to change the way they approach, design, and implement such programmes. For example, from measuring the budget outlays for skills training schemes, RBF initiatives like the Skill Impact Bond measures how many young people were certified, employed, and retained in their jobs.

Second, meaningful partnerships, empathy, and mutual respect, are drivers of change. RBF requires a coalition of diverse partners – governments, foundations, evaluators, implementors – who are aligned in their common goals. During such collaborations, the different stakeholders identify common challenges and work collectively to find solutions. This is only possible by recognising mutual strengths, building consensus, and discovering ways to cooperate in order to drive sustainable change. The sense of empathy fostered because of these collaborations is a key driver of success in RBF programmes. In the EPforR project in Tanzania, it was the strong relationship and trust between the government (the key implementation partner) and other  programme stakeholders that led the programme to sustain beyond Phase I (2014-2021) through the launch of Phase II (2022-2027).

Third, evaluation and evidence generation take centre stage. Payments under RBF programmes are based on the credible verification and evaluation of results achieved – making the evaluator’s role crucial. Equally, the focus on achieving outcomes expands this role to that of a learning partner: generating evidence, documenting lessons learned, and feeding this back for programme strengthening. In doing so, evaluators create evidence that goes beyond traditional verification, often for the first time on themes such as youth empowerment, sustainable livelihoods, and quality of learning. It is crucial to prioritise evidence generation and learning – as this supports scalability and drives systems change. For example, in the Skill Impact Bond, our study found that undernutrition among women belonging to marginalised scheduled tribes in India led to instances of fainting on the factory floor. In response to this, the programme partners are already brainstorming potential solutions to address such emerging challenges..

Fourth, balancing achievement of outcomes with a focus on equity and social inclusion is crucial. The focus on increasing average results alone could come at the risk of undermining outcomes for the most vulnerable and marginalised learners and youth. In SLEIC and the Skill Impact Bond, additional financial incentives are provided to service providers for female participants to ensure that young girls’ and women’s outcomes are prioritised. Panellists agreed that such considerations must be integrated into the RBF structure at the design phase itself.

The overall consensus was that RBF is a tool or mechanism like any other, and we need to be circumspect around their suitability in any given situation, country, and context. This mechanism is still new in the education and skills space, and we are yet to fully understand how these programmes perform. In this regard, consolidating proof of concept and test cases from the sector over time can help understand their on-ground effectiveness. Equally, there is merit in learning from RBF projects in the health sector, which is more mature in terms of employing innovative finance, while also appreciating the sector-based differences between education and health.

Reflecting on these key insights from the symposium discussion and our own practical experience of assessing various RBF programmes, we summarise the key ingredients necessary for designing, implementing, and evaluating such projects in the education sector, below.

Key ingredients for designing, implementing, and evaluating RBF programmes in the education sector

Must have Good to have Avoid
Clarity on what success means – in terms of clearly defined outcome and measurable indicators that are pre-agreed by all stakeholders Pre-existing rapport and relationship with partners Designing programme hastily in short time frame as that can lead to inefficiencies during implementation
Partnerships premised on trust and empathy committed to a common goal Time to iterate and evolve evaluation design and build consensus with partners Formulating complex and difficult to measure targets for informing payments
Strong government role/ buy-in for the evaluation to ensure sustained implementation and for impact at scale Government invested in having a strong learning agenda and committed to using evaluation insights for future programme design Under-budgeting for evidence creation and adaptive learning
Committed stakeholders who are willing to commit time to iterate and adapt for achieving results  Early involvement and participation of the MEL partner to align methodology with measurement of payment linked indicators. Sidelining social equity and inclusion considerations which may lead to exclusion of marginalised/ vulnerable individuals in the achievement of outcomes
Detailed contracting arrangements with clearly defined roles, responsibilities, liabilities, and payment clauses Leverage templates and good practices for contracting. A dedicated transaction manager to convene the partnership can help with the continuous & efficient management of the programme. Overlapping roles and responsibilities of different stakeholders as it can lead to inefficiencies and reduce accountability

 

*Our sincere thanks to our panellists: Sourovi De (OPM), Abha Thorat-Shah (British Asian Trust), Miléna Castellnou (Education Outcomes Fund), and John Lusingu (FCDO), for generously sharing their rich experience with us. We are also grateful to our team members Anusha Ghosh  (OPM) and Sadashiv Nayanpally (OPM) for all their support in organising this symposium.