This blog was written by Jasmine Dhingra, Isha Shingte and Garima Grover from Central Square Foundation. It was originally published on the RISE Programme website on 23 January 2023 as the first in a series. Links to other RISE blogs in this series drawing on lessons from applying the Education Systems Diagnostic tool are listed at the bottom of the page.
The RISE Education Systems Diagnostic is a tool for identifying strategic priorities for education system reform. In the first blog in this series, we hear from Central Square Foundation, one of the organisations that piloted the Diagnostic in 2022.
Context of the Study
Central Square Foundation works on improving learning outcomes with many state governments across India. One such partnership with a state in Northern India focuses on designing system-led programs, influencing policy-making, and developing interventions and knowledge products based on research and international best practices.
Education features in the concurrent list in the Constitution of India, i.e., both the centre and states legislate on it. The central government develops nationwide laws, policies, and missions to define the national vision of education across the country and through national initiatives and schemes provides a part of funding towards it.
The state with which we partnered has over 6 million students in close to 20 thousand schools, with more than 125 thousand teachers in the government education system, and several layers of management, which makes it complex. At every administrative level, specific actors are handling the operations of these schools. Since 2017, the state has implemented a series of educational reforms. Over the last five years, these initiatives seem to have helped the state do better than other states and improve its past performance across multiple indicators such as the Performance Grading Index 2019-20, National Achievement Survey 2021, etc.
The state’s aspiration to sustain and continuously improve learning levels, keenness to undertake a deeper diagnostic exercise, receptivity to new insights, and strong political will to change the way of functioning motivated us to seek a deeper insight into the education system by piloting a system-wide diagnostic in the state. In addition, the recent change in state government presented a great opportunity and a strong impetus for piloting such a diagnostic. The new government anchored good quality education in its election manifesto and promised to undertake a structured and systemic approach to education reforms. For this purpose, Central Square Foundation collaborated with the Research on Improving Systems of Education (RISE) Programme to pilot the RISE Systems Diagnostic Framework in this Indian state.
Diagnostic Study Framework and Approach
The diagnostic is anchored around the RISE Education Systems Framework, which is summarised in a 5×4 matrix and lays out the key relationships of an education system, elements that characterise them, and interactions between them. The four key relationships it studies are Politics (between citizens and the highest executive, legislative, and fiduciary authorities), Compact (between the highest executive, legislative, fiduciary authorities and the education authority), Management (between education authority and frontline providers), and Voice and Choice (between service recipients and frontline providers). The accountability relationships are analysed using features of Delegation, Finance, Information, Support, and Motivation.
The approach of the Diagnostic study includes:
- Identifying the key actors in the education system,
- Specifying the relationships between them,
- Identifying the primary alignment(s) of the relationships,
- Exposing incoherences within the system, and identifying priorities for reform.
The RISE diagnostic study is a process to facilitate and support actors in selecting a high-level strategic reform priority to improve learning, based on the latest education systems research. The pilot study has been undertaken in collaboration with the state’s Department of School Education. The study has two objectives:
- Diagnose: Facilitate government use of systems thinking to diagnose the components of the education system that are not working together as well as those that could deliver learning
- Prioritise: Facilitate government prioritisation of one or two key areas of the system for reform, to create better alignment for improving learning outcomes.
The objective of the pilot was to study all four relationships; however, currently, the framework provides limited design and guidance pertaining to the relationship of ‘Politics’. In addition, given the volatile political scenario in the state (an election had just concluded and a new government formed), we decided to exclude ‘Politics’.
Applying the Framework
The following illustration (Fig. 1) briefly describes our approach to conducting the diagnostic:
Figure 1: Approach to the diagnostic
Contextualising/Adapting the framework
Making a contextually informed diagnosis process is crucial as the same tools/process may not always show the same effect when applied in different contexts. The team first synthesised the existing literature relevant to the RISE framework alongside other content relevant to the Indian and state education systems and learning.
The initial synthesis suggested a clear set of relationships, their misalignment, possible (in)coherences that affect learning, and key adoptions required to make this approach a more contextual fit with the research design. We then calibrated the approach by including and excluding a few aspects (enumerated further). This calibrated model was then reviewed for the feasibility of studying coherence for learning at all levels. Contextualising the framework significantly helped us in identifying some of the points of progress specific to the Indian Education System.
Further, are our reflections from the processes of Desk Review and Primary Research.
Desk Review
- Identifying Principal – Agent: During our desk research, we utilised the framework to identify the key stakeholders across the three relationships.
Figure 2: The principal-agent relationship
The Principal-Agent approach was helpful in doing the same; however, in the education system we studied, many system actors were playing both the role of a Principal and an Agent. In such cases, the diagnostic team decided to study the relationships such as Compact and Management from multiple Principal – Agent respects.
- Included additional aspects to the study: During our desk research, we identified that there are some aspects which may prove significant in terms of their impact on learning; however, they have not been included in the original framework as they may not be widely applicable to other contexts. During the desk review and while identifying preliminary points of study for primary research, the diagnostic team decided to include aspects such as motivation and support of the non-teaching staff at the state level under management/compact relationships in addition to studying teachers’ mindsets and beliefs and including students as key stakeholders.
- Analysis and developing hypothesis(es): While undertaking the analysis of our desk research to develop the hypotheses, the team found the guiding questions and sub-element descriptions provided in the framework useful to identify how and where each of the elements’ alignment or incoherence lay, helping us break down the areas of further study for our primary research.
- Unpacking terminologies and definitions for Primary Research: Certain concepts and definitions have been interpreted by the diagnostic team or assumed as per the state’s context. Some of the examples include:
- Concepts such as Account v. Accounting, Spider v. Starfish and Management v. Governance. For instance, Spider is interpreted as the system being centralised and Starfish indicates decentralisation among the levels following the State Authority.
- Alignments where the term ‘Partial Alignments’ in sub-elements or elements indicates that parts of the alignment were true in our context, but not completely. The diagnostic team identified partial alignments to retain nuances of description that were or weren’t true in order to depict a correct picture of the system.
- Terminologies such as Good Quality Teaching, Financial Experimentation, etc wherein no exact definition exists in the system or in the RISE framework. For instance, Financial Experimentation has been interpreted as Innovation in budget planning and expenditures.
Primary Research
- Sampling Design: One of the major steps undertaken during the planning of primary research was sampling design and approach. In order to have a substantial representation of identified stakeholder groups, especially from the frontline, parents, and students situated in districts, respondents were identified based on appropriate qualitative sampling methods. In addition, to account for variations in different regions of the state, based on proximity to the international border, demographic differences due to rural, urban, and migrant populations, as well as performance in learning assessment surveys, the team identified districts that would be representative of the state.
- Tools for FGDs and Interviews: The next critical step undertaken was to align and develop tools for conducting FGDs and Interviews with respondents. Taking the framework to the ground involved certain complexities. Considering the range of stakeholders across various levels and to avoid any biases, it was decided that the framework would not be explained to them as is but contextualised to make the areas of discussion applicable and comprehensible to the relevant stakeholders. However, discussions had to be facilitated in a way that made the diagnostic team able to infer alignments and incoherences in the system from what seemed to be routine regular conversations. Hence, the tools and worksheets provided by RISE weren’t adopted and instead the team developed its own set of tools for FGDs and interviews. Developing the tools involved significant time commitment as each of the elements and sub-elements questions were further broken down or articulated to suit the context of various stakeholders.
- Analysis of Data: The analysis of data was divided across various steps, from synthesising the information across FGDs and interviews based on sub-elements, further summarising into elements, and combining the findings of both Desk Review and Primary Research. This entire process included an extensive amount of information which helped us understand many nuances of the system.
- Application of learnings: We took action on some of the pressing issues that surfaced during the research to develop more informed interventions in areas such as Teacher Professional Development, Teaching Learning Material and Academic Strategies, etc.
Prioritisation and Way Forward
As we conclude both phases of research, we are in the process of compiling the findings and drafting recommendations for the state. Through this report, we aim to provide highly localised insights surfacing the gaps and points of progress for the state in the short, mid and long term. The findings from the study would benefit the state to identify the incoherences in the system and in the coming months, the state plans to prioritise its interventions on the basis of the recommendations provided.
The multi-faceted approach of this study has helped us capture and analyse the nuanced information of the systemic drivers. We appreciate the RISE team for the opportunity and guidance throughout to be able to contextualise the framework as per the Indian context.
———————————————-
This is the first in a series of blogs from the RISE team outlining experience of how the Education Systems Diagnostic has been applied in different settings. Other blogs include:
· The JPC-VERSO Experience of Working on the RISE Diagnostic Tool
