Since 2015 there have been intensive efforts to design metrics intended to measure and track countries’ progress towards the 17 agreed on Sustainable Development Goals and their respective targets.
But is this the kind of evidence that will help countries plan action on the ground to meet these ambitious objectives? Not necessarily. Here are two examples from the field of education.
Globally comparable measurements of learning may not answer questions about equity. The SDG 4 focus on equity is reflected in references to all children in the relevant targets. For example, SDG 4.1 states: “By 2030, ensure that all girls and boys complete free, equitable and quality primary and secondary education leading to relevant and effective learning outcomes.”
International large scale assessments (ILSAs, such as PISA, TIMSS and others) are designed to generate comparable measurements of children’s learning. Although they address different age groups and/or curricular subjects, a design feature they have in common is that the assessments are conducted in school. In many developing countries, however, not all children are in school. Even when they are enrolled in school, many attend irregularly. And when they do attend, they may be going to schools that are not on any official list of schools. While evidence generated via school-based data collection may be well suited to the western country contexts where these models originated, in developing countries this type of assessment cannot answer the question of whether all children are learning.
Globally comparable measurements of learning show what children can’t do, but action on the ground needs to build on what children can do. Testing older children is easier than testing younger ones, and pen-paper assessments are far easier to administer than oral ones. For these reasons, international large scale assessments test older children using written formats. But there is by now plenty of evidence showing that across the developing world, children fail to acquire foundational reading and math abilities in the early years of schooling; these deficits only grow over time. Addressing this issue requires periodic assessments of foundational learning from very early on. Moreover, assessing reading requires one-on-one oral assessment – there’s no other way to gauge a child’s reading ability. Getting back a blank written test paper provides no clues as to where the problem lies.
There are alternatives. The Citizen-Led Assessment (CLA) model is designed to generate evidence on scale around the key issue of foundational learning, while at the same time informing and engaging a wide variety of stakeholders. Because the latter objective is as important as the former, assessment tools are designed to be ‘user-friendly’ rather than ‘expert-friendly’ – simple to understand and use by almost anyone. Beginning in 2005 as the Annual Status of Education Report (ASER) in India, the model has spread organically over time to include fourteen countries on three continents, now collectively known as the People’s Action for Learning (PAL) network. At the core of the CLA model is a set of principles rather than a fixed set of indicators, and each member country adapts the model as needed. For example, among these principles is the commitment to measure foundational skills (each country decides what this means in their context), to use simple tools and processes so that ‘ordinary citizens’ can participate, and to test in the household rather than in school (so that all children are included). While the data is not comparable across countries, it does generate both evidence and widespread discussion around the core question common to all countries that participate in the network: are our children learning?
Questions: Across the developing world and in all sectors how do we not only collect data, but also model the process of using evidence to understand today’s ground realities – in other words, how do we build capacity for evidence-based decision making?
How do we design metrics and methods that are clear and understandable for large numbers of people to understand, engage with, and act on?
Dr. Suman Bhattacharjea, Director of Research ASER Centre shared this article at the Decolonising Development: Whose Voice, Whose Agenda? Online conference held on 22nd to 24th May 2017. The Online conference was organised by Leeds University Centre for Global Development (CGD) and INTRAC.