This blog was written by Louise Yorke and Janice Kim, Research Associates at the Research for Equitable Access and Learning (REAL) Centre, University of Cambridge. Yorke is also researcher for the RISE Programme Ethiopia Team and Kim for the World Bank Early Learning Partnerships (ELP) programme. Additional authors are Belay Hagos Hailu, Associate Professor of Education at Institute of Educational Research (IER) in Addis Ababa University, and Chanie Ejigu Berhie, Lecturer at the Center for Food Security Studies, College of Development Studies, Addis Ababa University. Berhie is also project coordinator for RISE Ethiopia and ELP at Policy Studies Institute (PSI). This blog was originally published on the RISE Programme website on 16 April 2021.
While in-person data collection has not been possible during the COVID-19 crisis, distance research methods (surveying online or via phone) have been offering a way to collect data during the pandemic. However, these approaches raise some additional ethical and practical challenges and considerations, including evaluating whether and how to continue research. While a number of useful resources have provided important guidelines and suggestions for researchers undertaking this type of research, they have generally been developed in high-income countries. Less focus is given to the considerations that arise when undertaking research in low- and middle-income countries – although this gap is now starting to be filled by a growing body of research, for example by the World Bank, Young Lives, J-PAL, IDInsight and Innovations for Poverty Action.
In this blog, we reflect on some of the practical and ethical issues that arose during implementation of two related research projects that took place in Ethiopia during the COVID-19 school closures and aimed to understand the impact of the crisis on the education system.
- Phone surveys with school principals and teachers in 168 primary schools, as part of the RISE Ethiopia research programme, supported by the Cambridge Alborada research fund; and
- Phone surveys with 480 parents and primary caregivers, largely living in rural areas as part of the Early Learning Partnership (ELP) Systems Research programme.
These reflections may have relevance for other researchers conducting similar types of research and can contribute to the wider debate on best practice.
- Designing distance research
The RISE Ethiopia and ELP projects aimed to understand what was taking place during the pandemic to support students’ learning and what was needed for school re-opening. In particular, both projects sought to inform the Government’s COVID-19 response strategy, which was an important consideration in deciding to undertake the research during the pandemic. Given that we had established good levels of communication and engagement with the Government and donors prior to the pandemic, we were particularly well placed to provide rapid information about the impact of the pandemic on the education system.
Conducting phone surveys was the best choice for our research, given the constraints on travel and in-person interaction posed by the pandemic. Also, a relatively higher share of the population have access to a mobile phone and it is a preferred mode of communication in Ethiopia – 56 percent of the population have access to a mobile phone, compared with 19 percent who have access to the internet. The strength of conducting the research by phone was the ability to rapidly collect high-quality real-time data, while also helping to ensure the safety of the fieldworkers and participants. A challenge was that phone conversations are generally not appropriate for lengthy surveys, which required special considerations.
We engaged in an iterative process of instrument development with local research partners and aimed to keep our interview to a maximum of 30 minutes. We predominantly included closed-ended questions in order to be able to reach a relatively large number of respondents and to compare responses across different parts of the country (as well as across another survey in Rwanda, which was developed in parallel). We sought to ensure that questions were simple and easily understood by the respondent. We also sought to combine different types of questions (i.e. yes/no, rank) to allow for different types of information to be collected and to make the survey less monotonous for the respondent. Through a few open-ended questions at the end, we provided participants with the opportunity to share any further information that had not been captured during the interview. Given that many of our participants were likely to be in stressful and precarious circumstances, we decided that we would not collect any sensitive information through our phone surveys.
- Identifying and recruiting participants in distance research
Although distance research has the ability to transcend geographical boundaries, one of the main challenges is identifying and recruiting participants. A particular advantage of our respective studies was that the target participants had previously participated in and were already familiar with the study, which perhaps contributed to the high response rate (85 percent in the RISE Ethiopia study and 98 percent in the ELP study). In adddtion, the fieldworkers who conducted the previous face-to-face surveys were recruited to conduct the phone surveys. We believe that this also helped with the high responses rates, as familiriatiy with the field teams created confidence on the part of the target survey participants.
The potential for selection bias is important to consider – those who had access to a mobile phone may be substantially different from those who did not; and those who responded to the survey were likely to be different from those who were not able to. For this reason, we found it was important to keep track of those who responded, to help identify any potential bias at the sampling and analysis stage. For RISE Ethiopia, we invited school principals from each of the 168 schools that had been previously included in the research. For ELP, we contacted a sub-sample of our 2019 baseline survey and adopted purposive sampling associated with urban-rural locations, regions, school enrolment status, gender, and caregivers’ literacy – given our interest in understanding how different population groups have been affected by school closures. We were mindful of confirming the identity of the participants and the fact that failure to do so may compromise the integrity of the research, for example, by including participants who are not part of the target group, or inadvertently including children or vulnerable participants. No participants under the age of 18 years old were included in our study.
- Obtaining informed consent and ethical clearance
As with all research, getting informed consent is an important process, but this can be more challenging through distance research, especially in determining whether the participants have understood what they are consenting to and that they have the opportunity to ask any additional questions. We informed participants why they have been selected for the study, what their participation would entail and any risks of participating in phone survey (e.g. the risk that their interview may be overheard). While it is generally advised that consent should be written unless there is strong rationale for other forms of consent (e.g. verbal consent due to low literacy levels), in our study we sought verbal consent from our participants. This has been an established practice throughout our research, due to the sensitivities around signing official documents in Ethiopia. We sought additional consent to record the open-ended questions and recording was only carried out where consent was given, otherwise the enumerator took notes by hand.
We were also careful to ensure participants were informed that they were free, for any reason and at any time, to withdraw their consent and stop participating in the study. A challenge of distance research is that it may not be immediately clear that a participant has withdrawn from a study (e.g. they may exit by ending the phone call). As such, we provided guidelines for the fieldworkers to follow up with participants if this happened to be the case. Efforts were also made to schedule the interviews for a time that suited the participants. In terms of compensation for survey participation, we provided all school-level stakeholders and parents with 100 ETB (approx. £2) phone credit. We followed a strict procedure for providing this type of compensation and made sure that it was not used to coerce the participants to take part. Participants were informed of the provision of phone credit, only after fully informed verbal consent was obtained. Ethical clearance was obtained from the ethical review board of Addis Ababa University and the Faculty of Education at the University of Cambridge for this study.
- Building trust and rapport
Building trust and rapport is important for any research study, which again becomes more challenging in distance research. It is necessary to consider how to build trust and convey credibility to the participants and how to put them at ease (e.g. by making contact before conducting the research). In both studies, we advised the enumerators that the same person who carries out the interview should make the initial contact with the participants.
Another challenge in distance research is that researchers generally have less control over the environment – participants may be distracted or engaged in some other activity (e.g. watching TV). It is important to control the research setting and/or ensure that all participants are exposed to the same environmental conditions, as this may lead to invalid conclusions about data. As this was not the case in our research, we provided the participants with some suggestions for taking part, i.e. finding a quiet place with no distraction if possible.
Difficulties also arise in conducting distance research due to loss of visual cues and body language. During the phone interviews, we advised the enumerators to engage in active listening, make sure the respondent fully understood the question and to provide clarification if needed. We reminded them that pauses in the conversation could have multiple meanings (e.g. the participant may not have understood the question, or they may be formulating their answer).
- Conducting the interview
Prior to the interviews taking place, comprehensive and detailed training was provided for all fieldworkers involved in the data collection. A detailed fieldworker manual was also provided. The fieldworkers’ training included an overview of the aims and objectives of the phone survey study, the approach taken, key responsibilities of fieldworkers, safeguarding concerns, as well as issues around ethical behaviour, codes of conduct, and whistleblowing. High attention was given to the physical and psychological wellbeing of fieldworkers and they all had direct contact with the survey coordinator throughout the data collection.
We were attentive to the fact that we are in a pandemic and normal routines have been significantly disrupted. In general, people may feel worried or anxious, or have additional burdens. Some may be living in unsecure or unsafe circumstances which we are not aware of. We also considered that some participants may be enthusiastic to talk, or grateful for the chance to discuss some issues. To try to minimise risk to the participants, we reminded the fieldworkers to make every effort to put them at ease, to try and make the experience as relaxed and enjoyable as possible and not to make assumptions about their identity. We also reminded the enumerators to be prepared for the unexpected throughout the phone call, including the participant getting interrupted, needing to end the call suddenly, loss of connectivity, or battery dying, and to be prepared to reschedule the interview for a more suitable time if this is required. Once the conversation was finished, we asked the fieldworkers to thank the participants for their time.
- Disseminating research findings
All of our research strives to strengthen the link between knowledge, evidence and social impact. In the first year of the COVID-19 pandemic, there was (and remains) urgency in getting the information out as quickly as possible to inform key stakeholders. Our distance research was conducted in August 2020, and analysed and published in September 2020 as we intended to inform the Government and donors before school re-opening in Ethiopia, which was planned for October 2020. We used multiple communication channels to share our findings. With a series of blog entries (RISE, ELP) and research reports (RISE, ELP), we held meetings and workshops with high-level government officials, experts and donors to share what we found.
The findings of our research highlight two urgent issues:
- Despite significant efforts by the Government to support remote learning, many pupils had little or no education during the closure period.
- Although it is vital to re-open schools, huge challenges remain to make schools COVID-safe environments. In most cases, schools lack soap and running water, and facilities were not equipped to apply social-distancing measures.
Our findings were presented in a workshop organised by the Ministry of Education, intended for gathering ideas for the preparations of schools re-opening. Representative experts and directors from regional education bureaus and the Ministry of Education attended the workshop. The following recommendations from the study by the RISE team were well taken by the Ministry of Education as inputs for the preparations of re-opening of schools:
- Preparations on physical distancing and hygienic materials in schools;
- Sensitisation of the community to resend their children back to school;
- Classroom rearrangement to create more space (e.g. a shift system);
- The need for making up of classes to compensate lost learning.
This research draws attention to COVID-19’s impact on the education systems in Ethiopia, which may be relevant to other contexts in sub-Saharan countries
