This blog was written by Fergal Turner (Oxford MeasurEd), with the session being run in collaboration with Emily Tusiime (Aga Khan Foundation East Africa), Devotha Mlay and Martin Ariapa (RELI), and Maurits Spoelder (Education Cannot Wait).
Conferences can often be intense. We float around in a sea of new knowledge, stimulating discussions and caffeine-fuelled catch ups, often with little time to sit quietly and reflect on what we are learning. This year at the September 2023 UKFIET conference, we hosted a creative session with the aim of creating a space for reflection and creativity at the end of a busy week, back in person. We wanted to take the subject of learning assessment, a serious and complex topic at the heart of many of the conferences discussions, and address it with a sense of fun and unstructured creativity. By doing this we wanted to disrupt the unimodality of conference knowledge transfer, collectively learning from each other through the medium of art!
The question for discussion was simple: How can we make sure that a range of perspectives are considered in decisions about what should be measured and how?
The impetus for this question comes from three reflections based on our work on supporting our partners in building tools and systems for assessment. In our work we passionately believe that embedding the right voices throughout the process of developing assessments is essential for ensuring that i) measures are valid, ii) data is credible and useful for answering questions of policy, programming and practices, and iii) we are acting ethically and giving voice to those who have the greatest stake in the results of our measures.
To investigate this question, we teamed up with three of our partners who work on assessment with diverse groups of stakeholders.
Group one was hosted by Schools2030, who work closely with teachers in ten countries to define priority domains for holistic learning, and to develop teacher-centred measurement tools. Group two was hosted by the Regional Education Learning Initiative (RELI) who work with adolescents and communities across Kenya, Tanzania and Uganda to develop household assessments of life-skills and values. Group three was hosted by Education Cannot Wait, who work across eight countries with humanitarian systems to develop system level assessments of holistic learning in crisis contexts.
Each of these groups had a simple task – spend an hour drawing something that represented the challenges and opportunities of working with teachers, adolescents, and within humanitarian architectures.
The conversations and their results were reflective, discursive, and in line with the session format, creative!
Group 1: Teachers as a bridge and assessment as a lane
The first group was hosted by Emily Tusiime, the East Africa regional assessment lead for the Aga Khan Foundation. The discussion focused on the role that teachers play in supporting the attainment of learning outcomes. It was discussed that assessment plays an important role in creating a bridge to link low levels of learning to high levels of learning, as it is intertwined with instruction in teachers’ day-to-day practice.
To ensure that assessment is an integrated part of instruction, teachers need trust, a culture of reflection, technical capacities in assessment, and a supportive school environment. Without trust, assessment data can become a tool of scrutiny rather than empowerment. Without a culture of reflection, teachers won’t collaboratively engage with data from assessments. Without technical support, teachers who are less data literate will struggle to fully engage in assessment, and without a supportive school environment, trust, a culture of reflection and collaboration are not possible.
Group 2: On track to community-driven assessment
Our second discussion group was hosted by Devotha Mlay and Martin Ariapa representing RELI’s Assessment of Lifeskills and Values in East Africa (ALiVE) programme.
In their map, the journey was represented by a train track. The destination was more relevant tools, changes in practice, and nurturing holistic learning. The stops along the way were allowing adolescents and communities to board the train, by including them in (1) the design of measures and (2) reflecting on tool validity. Along the way, participation could be derailed, first, by a lack of time, and secondly, by a lack of resources. Supporting the train’s journey was a bridge, with pillars built on shared processes, shared tools, communication and training. All these pillars supported the train of participation in reaching its destination.
Group 3: A more aligned future for crisis response
Our final group was hosted by Maurits Spoelder of Education Cannot Wait. The focus of the group’s discussion was imagining a future in which different state and non-state actors collaborated across the humanitarian, development, and peace nexus.
To do this, the discussion focused on describing the current reality, and imagining a future, ideal reality. In the current reality, state and non-state systems often act separately in emergency response, with little input from teachers, schools, and communities. In the ideal future, there are concentric circles of collaboration, in which learners and schools are at the centre of developing assessment systems. Around them are schools and communities, supported by state and non-state systems, and a global eco-system. This positions assessment as a long-term activity which should span the humanitarian, development and peace nexus. The pathway from present to future is paved with shared tools, leadership, shifts in mindsets, money, collaboration, and time. However, always in the background there are new crises which can distract from long-term thinking and collaboration.
Where to now?
This was a creative exercise in trying to understand how best we can broaden participation in assessment processes across a range of contexts. Reflecting generally on the discussions, we conclude that there is a growing recognition that, considering what decisions data will inform should be the first step in designing assessment systems. We cannot consider this question of why, without considering the question of by who and for who. Specifically, it is important that efforts to develop assessment tools and systems should involve:
- Teachers as the primary point of contact with learners in the education system. Involving teachers can create a clear link between evidence from assessment and pedagogical practices in the classroom.
- Communities as a primary stakeholder in education. Involving communities in assessments can ensure that what is being measured is what is most relevant to children, young people, and the communities they live in.
- Complex systems of actors in crisis-affected contexts. Where systems of education have become disrupted, it is important to recognise the wide range of different voices who need to contribute to the development of assessments in order for them to be relevant and robust.
Across the three discussions, it was clear that collaboration, trust and openness between organisations working on assessment are central enablers. Conversely, a lack of time, a lack of resources, and a lack of mutual trust are the biggest threats.
This session represented an important opportunity to bring together a wide range of actors from across the globe and across education systems to collectively reflect on how best to formulate the importance of involving, teachers, communities and actors across the humanitarian and development settings in the process of defining and measuring learning outcomes. The reflections from each of the groups captures a growing recognition that the question of who is key in the discussion on how we measure learning.
