More than the sum of its parts: Internationally-comparable citizen-led assessments to ensure no child is left behind

This blog was written by Professor Pauline Rose, Director of the Research for Equitable Access and Learning (REAL) Centre at the Faculty of Education, University of Cambridge. It has also been published on the PAL Network blog website.

From local and national to internationally-comparable data on learning

Since the advent of the Sustainable Development Goals (SDGs), there has been a heated debate on how to track learning outcomes in ways that ensure no one is left behind. These debates have raised legitimate concerns that assessing learning is too often undertaken through high-stakes approaches that can be detrimental to children. Members of the People’s Action for Learning (PAL) Network have been at the forefront of developing approaches that adopt community-led approaches to assessing basic literacy and numeracy. Their approaches are straightforward to implement, and simple to communicate the results. Assessing children in households rather than schools is beneficial both in including children who are not in school, as well as removing the dangers of high-stakes testing.

Findings from analysis of the data have highlighted the extremely low levels of learning that have persisted over at least a decade in countries such as rural India, Pakistan, Kenya, Tanzania and Uganda. Rather than adopting a high-stakes testing approach, these assessments aim to promote an innovative approach to social change that is citizen-driven and accountable to the public. These have led to national-level advocacy alongside local-level actions, for example providing additional support to those who have not learnt the basics.

An important feature of the PAL Network approach is that it has been nationally and locally driven. This has huge benefits in terms of affecting change within communities themselves. However, a drawback has been that the data produced by each country survey, while similar in approach, has not been comparable at an international level… until now. It is exciting to see the new assessment tool developed by the PAL Network – International Common Assessment of Numeracy (ICAN) – an assessment of foundational numeracy, providing data on early grades of primary school. The principles of engagement in the development of assessments within countries continues with ICAN, which was developed through a collaborative effort between PAL Network member organisations in 13 low- and middle-income countries across Africa, Asia and America. It is open-source, easy-to-use, available in 11 languages, and most importantly, it will offer international comparability of results aligned to the global SDG 4.1.1 (a) on the proportion of children and young people learning in grades 2/3.

Learning from PAL Network data

Through my research, together with others in the REAL Centre, we have had the opportunity to analyse data collected by PAL Network members. We have already found it to be a tremendous resource, enabling a robust understanding of who is learning, the schools they are in and, through this, identifying some of the areas that are needed for improvement if the education SDG is to be achieved. I outlined some of my findings in a 2015 UNICEF blog: Achieving the education SDG: Start early, stay the course, which discussed how to make our global learning crisis a thing of the past by 2030. The data have been vital in showing the multi-dimensional challenges to learning, that disadvantage starts early, and how disadvantages intersect as children develop through the education system. For example, data from rural India showed that there is a performance gap at age 7 between poorer children who are first-generation learners, and richer children whose parents attended school. Gender begins to interact with socio-economic status by age 8 and widens gaps for poor girls whose parents have not been to school, compared with their male counterparts. By age 11, poor girls will only reach the same chance of learning as rich girls achieved three years earlier (see Figure 1).

A key message from our analysis of the data is that learning inequalities start young – even before a child goes to primary school – and are largely driven by disparities in wealth. Wealth interacts with other forms of disadvantage, such as gender, where a child lives, and whether they have a disability. Alongside my colleague Ben Alcott, we highlighted some of this analysis in a DFID think piece paper: How can Education Systems become more Equitable by 2030? The graphs below show how learning disparities are enforced and grow throughout primary school years (see Figure 2). Analysis of these data have played an important role in DFID’s 2018 Education Policy, which has a strong focus on tackling intersecting disadvantage from the early grades of primary school.

Work by Nidhi Singal and Ricardo Sabates, in collaboration with colleagues at ASER Pakistan, has shown how school systems fail children with disabilities in particular: even if they are able to access schools, their opportunities to learn remain lower than their counterparts.

One of the most contentious areas of education policy in recent years has been the role of private schools. PAL Network data has helped us to understand who benefits from attending private schools, identifying that the wealth divide. This is highlighted in work with Ben Alcott and Rob Gruijters.

We have already found huge value in the individual country data but the internationally-comparable ICAN data will equate to more than the sum of its parts. Being able to take the data created by each country and also compare across countries will be even more beneficial and fruitful for really understanding progress towards the SDGs and the extent to which gaps are or are not being closed, and so what actions need to be taken. It will be a vital resource for the repository of learning data collated by the UNESCO Institute for Statistics – the main source for tracking progress towards the SDGs.

Learning loss as a result of COVID-19

The availability of ICAN couldn’t be more timely. In the light of the global COVID-19 pandemic, it is anticipated that many of those children who were already struggling to learn the basics are most likely to suffer losses in learning. Having these comparable data will make it easier to understand the changes as a result of COVID-19, in contexts which had extremely low learning levels prior to the pandemic. The PAL Network will be in a strong position to build on their already-existing vast experience of collecting data on children’s learning, whether they are in school or not, and using this to identify and advocate for actions that will be needed locally, nationally and globally.

As a researcher, I am excited at the prospect of using and learning from the data generated by ICAN, and keen to collaborate with others – including in the PAL Network – to further our commitments to leaving no one behind.