Four lessons from evaluations of the education response to Ebola

This blog was written by Joe Hallgarten, Principal Consultant, Education Development Trust. It is  part of a Global Partnership for Education series about learning from the education response after the Ebola outbreak in West Africa. It looks at what some of the evaluations have shown, and highlight four findings that can help today’s response to the coronavirus outbreak. It was originally published on the Global Partnership for Education website on 20 April 2020.

Education recommendations provoked by the coronavirus pandemic appear to be spreading as rapidly as the virus itself. However, few appear to be based on evidence of school systems that have actually experienced a shutdown.

In truth, until a month ago it seemed that much of the education world wasn’t especially curious about the effects of disease outbreaks such as H1N1 or Ebola, despite the extraordinary efforts of those directly involved. Our collective complacency now appears somewhat misguided, as we all rapidly try to learn from the past in order to adapt to our current reality.

To inform the education sector’s responses to COVID-19, DFID’s Knowledge for Development program has commissioned a number of rapid reviews. My report focuses on efforts to mitigate the educational impact of previous disease outbreaks, concentrating on school-age learners. Almost all the literature relates to Ebola, and its impact in Guinea, Liberia and Sierra Leone.

As we might expect, these disease outbreaks have negative impacts on children’s learning, safety and wellbeing. The table below, adapted from ACAP’s model, summarizes them.

Unsurprisingly, poorer and more vulnerable children are disproportionately affected by disease outbreaks.

They are more likely to be learning less, to be eating less, and to be victims of increases in home and community violence and child exploitation. Girls also bear a greater negative cost, with rises in sexual abuse, teenage pregnancy and early marriage.

Distance learning solutions, whether low- or high-tech, often indirectly discriminate against girls due to power dynamics within families. And when schools do re-open, poorer children and girls are less likely to return, with increased dropout rates caused partly by fear, stigmatization and, in some cases, deliberate exclusion.

Mitigating actions

Our review divided mitigating actions into three categories:

• Amelioration – Crisis responses while most or all schools are closed
• Reparation – Recovery responses when schools reopen
• Preparation – Mitigating the educational risks of future outbreaks.

Unfortunately, across all three areas, we found limited evidence on actual impact of interventions or different approaches to action, coordination, funding or prioritization. However, lessons can be learnt from how various interventions have negotiated fluid political economies, and have been attuned to cultural and social factors, highly adaptive when supported by donor flexibility, and based on integrated responses, multi-sector collaboration and integration.

Moving away from the cold steel of an evidence review, I can offer four tentative suggestions. In highlighting examples of practice, these ideas build on earlier blogs on Sierra Leone, as well as Education Development Trust’s think piece for policymakers.

1. Digital didn’t kill the radio stars.

Our review found no evidence that online learning, screen or mobile-phone based technologies had played a positive role in supporting at-home learning during Ebola. Save the Children’s review of ed-tech in protracted crises found that the general dearth of high-quality impact evidence is amplified in such contexts.

The claim-based rhetoric around this being ‘ed tech’s moment’ appears somewhat misplaced in low-income contexts. In these environments, the evidence on radio is far more promising. UNICEF estimate that over 1 million children were reached through radio education during Ebola.

In Kailahun, one of the poorest districts in Sierra Leone, an existing project – ‘Getting Ready for School’ – rapidly redesigned itself to become a radio program: Pikin to Pikin Tok (Child to Child Talk). 36 existing ‘young facilitators’ created content in three languages and radios were distributed to another 252 facilitators who created listening groups. The final evaluation showed high levels of child engagement, and strong agreement from adults that the program had contributed to children’s learning. This program, according to its initiators, ‘illustrates how investment in smaller organizations, already operating successfully and which have built relationships of trust with their communities and authorities, can produce results during and after a humanitarian crisis’.

2. Remote teacher learning might be more feasible and impactful than remote child learning.

Given the evidence emerging on the efficacy of some professional learning programs in protracted conflicts, we surprisingly found no example of approaches that supported distance teacher training and professional development in disease contexts.

The increasing pervasiveness of internet-enabled devices amongst teachers means that technology-enabled professional development is probably making faster progress in low-income settings than child-centred edtech.

Education Development Trust’s programs in Kenya and Rwanda are already using smartphones and SD cards loaded with videos of teaching practices to support training, coaching and communities of practice. Where infrastructures and relationships are already in place, like in these programmes, school closures might provide an opportunity to trial remote approaches to improving teacher skills, blend these with face-to face training when teachers are allowed to meet together again, and – most importantly – weave this into classroom-based continuous professional development when schools re-open.

3. Education can’t wait, but nor can psychosocial support.

During Ebola, children, especially those quarantined, suffered from social isolation and post-traumatic stress, and there were reports of greater personal and family frustrations. While most education interventions aim to improve both wellbeing and learning, it may be that the former needs to take priority in disease-related emergencies.

A small number of informal learning programs that included psychosocial support demonstrated sustained impact. In Liberia, for example, an established peacebuilding education and advocacy program re-assigned 241 young volunteers to support health education during and after Ebola-related school closures. The volunteers’ pre-Ebola training and existing understanding of these communities’ contexts were crucial to their successful engagement.

Another psychosocial arts program for children in Liberia demonstrated statistically significant decreases in reported symptoms of poor mental health. Meanwhile, in North Kivu in the Democratic Republic of Congo, school leaders and teachers were trained in what the WHO has described as ‘psychological first aid’.

One gender-focused program in Sierra Leone rapidly adapted to become a one-hour daily class in life skills, sexual and reproductive health and vocational learning. The classes were held in safe spaces in villages and served 4,700 adolescent girls. After the crisis, school enrollment rates fell by only 8% in villages which had received the classes, compared to 16% in villages which had not received the intervention.

4. Good data, evidence and evaluation are sound long-term investments.

The dearth of good quality impact evidence, while understandable in both crisis and recovery contexts, renders the task of initiating an evidence-based educational response to COVID-19 in low and lower-middle income countries more challenging.

Using the richer evidence base on efforts to mitigate the impact of other emergencies (such as conflicts and natural disasters) on the education sector, we need to learn more about what is most relevant, transferrable and adaptable to disease-related emergencies. While capacity and resourcing pressures will inevitably prioritize action over research, donors who can ringfence sufficient evaluation funding within new rapid responses, or empower existing program evaluations to adapt to changing predicaments, may find that these efforts have a good long-term social return on investment if and when the next outbreak comes our way.

Beyond evaluations, crises such as disease outbreaks can provide an opportunity for an enhanced focus on the data-strengthening aspects of reform. Perhaps we can be optimistic that the three countries most affected by Ebola may, as a result, have established and sustained an improved culture of data use which may serve them well in the current crisis. Despite the pressures on resource reallocation, it is vital to ensure that data-strengthening remains a priority.

Education Development Trust is working actively with DFID and others to understand more broadly what best practice might look like in distance learning, remote teacher professional development, and workforce planning, as we adapt our own programs and continue to support ministries of education, school leaders and teachers to make smart, evidence-informed choices at this critical time.