Introducing ICAN (International Common Assessment of Numeracy)- as a Global Learning Metric

The Learning Crisis has been on the global landscape for over a decade. It is the most critical challenge to the attainment of the Sustainable Development Goal (SDG) 4, the successor to Millennium Development Goal 2. SDG 4 is a radical departure from access to learning.  However, the global impasse has been on how to measure ‘learning’ such that it is clearly defined and understood by all for early years, lower primary, upper primary and lower secondary as a Global Learning Metric.

Whilst upper primary (grades 4/5/6) and lower secondary (Grade 8) are often found to be formally part of national and international assessments in many countries, it is early years and lower primary learning measures that continue to elude many. Yet, both are critical foundational metrics for successful transitions to primary, upper primary and post primary levels.

What began as a provocation whilst addressing basic learning challenges in South Asia by Pratham, one of the largest NGOs in India, the citizen led assessment tool was rapidly transformed as an at scale measure for learning improvement at the national level (ASER “impact” India) and, subsequently across 14 countries of Latin America, Africa and Asia (2006/7-2020). The group of countries voluntarily came together to repurpose themselves as the People’s Action for Learning (PAL) Network in 2015.

The PAL Network established its secretariat in Nairobi to ideate, give voice and initiate capacity building of member countries and to build robust quality assured systems for learning and action measurement. The simple learning assessment tools were adapted across three continents and country contexts to carry common value propositions, viz.: citizen led, household based, one on one for each child, leave no child behind -the equity principle; simple visual open source tools for amplified sharing of results that would trigger actions.

However, the multi-country/continent movement was unable to ensure complete comparability of the learning tools and outcomes to capture globally agreed minimum proficiency levels for both language and numeracy. PAL Network members never hesitated to reach out to global place holders including the UNESCO Institute of Statistics(UIS) the custodian for SDG 4 data/metrics, the World Bank etc. for feedback on its work.

By 2017 Citizen Led Assessments (CLAs) were accepted as a legitimate source of data and measure by UIS. The network was invited to lead and participate in the Global Alliance to Monitor Learning (GAML) as an advisory body to inform the global architecture for defining and measuring SDG 4 indicators, which includes the UN Statistical Commission through Technical Cooperation Group (TCG) and Inter-Agency Groups (IAGs). For the PAL Network, its aspiration to be truly comparable remained unfinished business until the September 2018.

Game Changing Moment- the ideation of ICAN 

The meeting convened by UIS for ‘consensus building on proficiency levels with performance descriptors’ in UNESCO Paris in September 2018 inviting all leading global stakeholders in assessments became a game changer for the PAL Network, where two of its core members were invited (UIS Concept Note-Sept. 2018). The meeting’s objective was to report on three education levels (in Grade 2 or 3 (4.1.1a), at the end of primary education (4.1.1b), and at the end of lower secondary education (4.1.1c) in two subject areas (Reading and Mathematics) as specified in indicator 4.1.1’. The consensus framing helped the PAL Network to analyze its citizen led tools threadbare from each country context, map the missed learning proficiencies for the numeracy tool first and then literacy. After 18 months of hard core reviews, addition of new items, pilots and adjustments the PAL Network is proudly presenting ICAN (International Common Assessment of Numeracy) to the world. ICAN has been tested in 13 member countries across three continents. What is ICAN? Does it have the potential to become a Global Learning Metric for SDG 4.1.1a? If so, how? Will this metric have backward linkages to early years? Will ICAN be applicable during and post COVID-19 contexts?

What is ICAN-its Nuts and Bolts?

Developed and implemented by the PAL Network member countries, ICAN is a cross national, comparative assessment tool, covering diverse regions of the global South. It is an assessment of foundational numeracy using common items, providing data on early grades/lower primary. It further highlights gaps in foundational numeracy even for older children. The Assessment Domains were decided based on literature review and other international assessments as agreed in the global consensus meeting convened by UIS (September 2018). ICAN measures all agreed proficiencies that must be reflected in curriculum, syllabus, textbooks and beyond pedagogies. These proficiencies for numeracy for grades 2 and 3, include the following:

Number, Counting, Number Recognition, Number Operations, Word problems, Space, shape and measurement spread across three domains, each with its own topics and tasks.

PAL Network member countries prepared the tool for ICAN spread across eleven languages and agreeing to retain the core consistent principles that distinguish the network.

1. Same assessment tool is used for all children in the age group of 5-16.
2. Most tasks pegged at up to grade 3-level curriculum.
3. Method of administration.
   • Oral and one to one: Children were allowed to use paper and pencil to work out problems where needed.
   • Children enrolled in school but also those un-enrolled and present in the household.
   • Progressive: Not all tasks were given to all children. Similar to most existing PAL Network assessments cut- offs were decided based on the findings of the field pilot. Children who were not performing on number operations tasks at lower levels were asked to attempt tasks at higher difficulty level.

Adaptive for the Best in Each Child: The ICAN assessment process incorporated various elements aimed at ensuring that it captured the best that each child can do. Surveyors were trained to build rapport with children to create a relaxed and encouraging environment. The surveyors were trained to speak slowly and clearly to ensure that all children are able to fully understand the expectation from the task. Children were given the time they needed to do each task. The assessment process was adaptive to the child’s ability, so that she did not have to attempt all levels. Thus, the child’s comfort and the commitment to accurately record her best possible response was at the core of the test design.

Does it have the potential of becoming a Global Learning Metric (GLM)? The principles of comparability, frequency and adherence to global consensus based proficiencies and descriptors make ICAN a hot favorite as a GLM. ICAN is being scaled further and leveraged as an innovation, supported by the Global Partnership for Education (GPE). ICAN will provide valid and reliable estimates of the status of children’s schooling and foundational abilities for comparison and monitoring trends across the PAL Network countries aligned with the requirements of SDG 4.1 (For numeracy now and literacy by 2021). For the Network implementing ICAN with adherence to wider framework of domains and skills in country assessments, and Data Quality Standards Framework, ICAN will strengthen the pitch for CLAs in existing and new regions.

How will ICAN bring value addition to transitions from pre-primary to upper primary and post primary? The domains and tasks that ICAN covers for numeracy (literacy to be completed by 2021) spans early years or pre-primary and upper primary foundational competencies.

COVID-19 and Implications for ICAN and SDG 4.1.1

The global disruption to education and learning by the apocalyptic COVID-19 has been an unimaginable scenario for all the architects of SDGs/SDG 4 globally. Not only has the pandemic flipped negative to positive phrases such as social distancing, lockdown etc. but it has redefined spaces; creating shifts in learning from school-based to home /community based; from face to face learning to distance, ed-tech hybrid, blended learning at an unimagined speed and with instant adjustments! These shifts are here to stay; new social assets have been created in the ‘new normal’; and, adjustments will continue.

There are implications for the global, regional and national groups working on indicator definitions of SDG 4 and a ‘new role for data/assessments” to factor not just a freak pandemic but a possibility for recurring viruses at scale up to 2030 and beyond (UIS 2020). The relevance of ICAN amidst shifts from an Education System to a Learning System (Education Commission 2020) cannot be over emphasized as would the competencies, which must be self-paced for each learner, sometime at home and sometime at school; children perhaps more vulnerable amidst equity/digital/learning gaps, but also more resilient, with more agency across flexible approaches, when provided with enabling opportunities; the child who was silenced in regimented schools becomes lively and innovative.

Rukmini Banerji, CEO of Pratham Education Foundation and founder of CLAs, rightly asserts that ‘In this context ICAN not only provides an ideal opportunity to re-visit the learning crisis and re-engage in new and stronger ways to ensure every child is in school and learning well. Through ICAN, we can look forward to families and communities, teachers and schools working together for rebuilding our children’s futures’.