Developing mobile learning for children experiencing displacement: Lessons from Pakistan

This blog was written by Dr. Allyson Krupar and Farman Khan, Save the Children. It is based on a presentation given at the 2023 UKFIET conference.

Does app-based remote learning improve children’s skills during educational disruptions?

Recent research about the prospects of using educational technologies to support learning in low-resource and crisis-affected contexts has proposed that app-based learning, within specific parameters could benefit children’s skills. Based on ongoing research and identified gaps by practitioners, particularly during COVID-19, but relevant to all crisis-affected environments including areas affected by climate-related disasters, Save the Children developed Learning Tree. Learning Tree is an offline app-based learning platform for children to access literacy, numeracy, and social and emotional learning content tailored to children’s skill levels. It also provides teacher professional development content to support teachers experiencing displacement, to support educational continuity and decrease learning loss during educational disruptions.

Piloting Learning Tree

Despite improvements in education access in Pakistan, disparities persist, with access and learning quality dependent on socio-economic status. In 2020 alone, Pakistan experienced three category 3 emergencies (floods, earthquake, and an avalanche), in addition to the global pandemic, causing temporary displacement and disruption in education systems. Save the Children Pakistan identified the need for a mobile offline application, accessible with locally-available technology and resources, that could be deployed to help children remain connected with educational content. From 2021 to 2022, we piloted the app using pre-existing Return to Learning curricula, which had been previously found to support children’s learning during educational disruptions in Lebanon and Colombia through face-to-face implementation. We adapted the curriculum to remote learning, emphasising social and emotional skills and gamification to create interactive and accessible learning content for children ages 6 to 14.

We assessed the pilot in a mixed-method and quasi-experimental design with quantitative learning outcomes data from the Holistic Assessment of Learning and Development Outcomes (HALDO) and in-app user data, as well as qualitative data from children, caregivers, and teachers to understand and compare learning outcomes for those who used the app with those who did not, as well as understand the feasibility and implementation of the app in its first-ever implementation. We piloted in girls-only Accelerated Learning Centers (ALCs) in Shikarpur, Sindh, Pakistan due to programmatic needs and disruptions in the area. During school closures, ALCs, along with Learning Tree, provide children an opportunity to continue their learning and educational activities as well as support caregivers to enhance children’s learning at home and teachers to support children’s reintegration into the classroom when schools reopen. Half of the school communities were assigned to use the Learning Tree app and half to the comparison group for the pilot, sampling from 25 schools in each group, with a total of 250 children in each group.

Results

We found that children who used Learning Tree had significantly higher skill gains in all domains when controlling for background variables and baseline scores.

These results signify that the Learning Tree pilot increased children’s literacy, numeracy, socio-emotional learning (SEL), and executive functioning skills, as measured by HALDO. We also found that in-app data helped us understand caregiver/child and teacher interactions and time-on-task in the app. Given that the application works offline, in-app data only synced for 150 child/caregiver users’, with an average of 13 minutes of interaction on each module per user. Children interacted with questions in the app as well, answering 182 questions, 45% correctly. Qualitative feedback from caregivers emphasised the quality of the stories, interactions, and games as specifically relevant for their children, and an overall positive perception of the content and use.

Lessons learned and next steps

With positive results also come lessons learned. Overwhelmingly, children, caregivers and teachers identified a need for more teacher guidance, both for teachers to be trained in the app and to feel comfortable working with it with children. Mobile phones themselves and internet access also were identified as potential pain points to expand the use of the app, given that only about 50% of the households surveyed had access to smart phones. There were also language issues within the app, with some translation issues not being in the local dialect and some English words included and too advanced for learners. More technical recommendations included reducing the size of the app and the frequency that the app would sync with new content, as well as having a vertical option for users to use it on their phone. Specific content like numeracy and stories were identified both as strengths and opportunities to further develop the app in these areas with more number games and problems and more stories for children to read.

Future programming and policy

• Given positive results, we are testing Learning Tree in Guatemala and Colombia, specifically with highly mobile populations.
• In Pakistan, the team held events and workshops with key education in emergencies stakeholders to discuss how partners can use the app in programming.

Program design

• Given the lower gains in SEL skills, more specific targeted SEL content can be strengthened as stand-alone modules, in addition to the current integration of SEL throughout modules. 
• We identified possible spill-over effects due to caregivers outside the intervention group accessing the app, future programming should be sensitive to who has access to the app.
• Learning Tree requires some face-to-face interaction with children, caregivers and/or teachers to implement. Our ongoing pilots with highly mobile populations have identified touch-points exist, including existing child friendly spaces, transit centres, or temporary learning spaces to enhance product delivery and guidance to users. 

Application design

• Guidance, help functionality, or how-to’s should be built into the app in child-friendly language.
• Telecommunications partnerships can be leveraged to enhance network access in emergency responses.
• Further background data collected in the app could help implementers understand challenges children face, such as risk factors related to household chores, child labour, and forced marriage. 

These promising results and lessons learned are currently being used to further develop the Learning Tree app, contribute to future development of remote educational programming in Pakistan and other crisis-affected environments, and help shape future programming for expansion to populations on the move.