PAL-ELANA: Advancing Early Learning Assessment Across 12 Countries

By: Muhammad Usman, Program Manager, PAL Network Secretariat

Introduction:

The Early Language & Literacy and Numeracy Assessment (ELANA) project is a groundbreaking initiative aimed at transforming how we measure the foundational learning in the Global South. In partnership with 12 PAL Network members (ASER Centre, ASER Nepal, Facilidade-ICDS, FUPECG – ÁBACOenRed, MIA, IID, ITA/ASER Pakistan, LARTES-IFAN, OMAES, Usawa Agenda, Uwezo Tanzania, and Uwezo Uganda), ELANA seeks to evaluate and enhance foundational learning outcomes for young children aged 4-10 years. This blog post provides an overview of ELANA’s inception, the development and field trials of the assessment tool, and our plans for the current phase i.e. final rollout.

Inception of ELANA:

ELANA was conceived in 2020 with the goal of addressing significant educational disparities in the Global South. Recognizing the critical importance of early childhood education, we set out to create a comprehensive assessment tool that would measure early language, literacy, and numeracy skills. This tool is designed to provide actionable data that can inform policy decisions and educational strategies, ultimately helping to bridge the gap in foundational learning.

ELANA is being developed by three consortium partners: the PAL Network, Pratham/ASER India, and the Australian Council for Educational Research (ACER). Funded by the KIX initiative of the Global Partnership for Education (GPE), Foreign Commonwealth and Development Office (FCDO), and the Hewlett Foundation, the project is currently being implemented by PAL Network’s member organizations in 12 countries.

ELANA builds upon the Citizen Led Assessments (CLAs) movement and the PAL Network’s award-winning assessment initiative,  International Common Assessment of Numeracy (ICAN). Like CLAs, ELANA is conducted orally with one child at a time and does not assume that children can read. It is designed to be conducted in households to reach all children between 4 and 10 years old, not just those enrolled in or attending school. ELANA maintains the common feature of providing comparable data across different contexts in which PAL Network members work.

Overview of the ELANA Tool:

The ELANA tool is a meticulously developed assessment framework that evaluates key early learning competencies set by the UNESCO’s Global Proficiency Framework. It includes a range of activities and tasks designed to assess children’s proficiency in language, literacy, and numeracy. The tool has been tailored to be culturally relevant and accessible, ensuring it accurately reflects the diverse educational contexts of the participating countries.

ELANA expands the ICAN tool by adding set of numeracy skills measured in five domains: Numbers, Geometry, Data, Measurement, and Patterns. Additionally, it includes a brand-new language and literacy component, assessing aspects ranging from oral language to reading comprehension skills. Implemented in nine different languages (English, Spanish, French, Portuguese, Urdu, Hindi, Bangla, Nepali, and Kiswahili), the tool followed a rigorous and innovative process of item writing, review, and translation-adaptation. Prioritizing pre-primary and early primary grade competencies for the children, ELANA provides data on the learning continuum from pre-primary skills to foundational literacy and numeracy.

Multi-Stage Adaptive Testing (MSAT):

A unique and significant component of ELANA is its Multi-Stage Adaptive Testing (MSAT) design. This method enhances the accuracy and efficiency of the assessment by tailoring it to the child’s ability level. The assessment is divided into three stages:

• Stage 1: The child is given a core set of items that span all competency levels. This stage establishes a baseline of the child’s abilities.
• Stage 2: Based on the child’s performance in Stage 1, they are directed to a set of items that are either at a higher or lower competency level.
• Stage 3: This final stage further refines the assessment by categorizing the child into high, medium, or low competency levels, providing a precise estimation of their skills.

Figure 2: ELANA MSAT Design

The MSAT approach ensures that each child is assessed accurately and efficiently, minimizing frustration and maximizing the reliability of the data collected. This adaptive testing method leverages technology to provide a better assessment experience, making the process both child-friendly and environmentally sustainable by using tablets instead of paper.

Field Trials:

To ensure the effectiveness and reliability of the ELANA tool, we conducted three rigorous field trials.

• Field Trial 1: The initial trial, conducted in 2021, focused on testing the feasibility and adaptability of the assessment tool in diverse educational settings. The pilot was conducted in five schools per country in all the participating countries. Feedback from this trial was crucial in refining the tool’s components and ensuring its cultural relevance.
• Field Trial 2: The second trial, conducted in 2022, expanded the scope to a larger sample size (45 villages per country), allowing us to test the tool’s reliability and validity across different regions. This phase provided valuable data on the tool’s consistency and its ability to capture accurate learning outcomes.
• Field Trial 3: The final trial, conducted in 2023, in 15 villages per country, aimed at fine-tuning the assessment procedures, testing the MSAT design and training methodologies. By this stage, the ELANA tool had undergone several iterations, incorporating feedback from educators, field workers, and children. The success of this trial confirmed the tool’s readiness for large-scale implementation.

Plans for Final Rollout:

We are now in the exciting rollout phase of the ELANA project. Our goal is to collect data from 180 communities per country, divided over three districts, across all 12 participating countries.

Here are the key milestones and current status of the rollout:

May-June 2024	Training of Project Management Teams (PMTs): We have successfully completed the training of our country teams. These teams are crucial for on-ground logistics, training of the filed enumerators, overseeing the data collection and monitoring process and ensuring the integrity of the assessment.
June 2024	Data Collection Initiation: Nicaragua has already begun their first round of data collection in the municipality of San Dionisio. Half of the participating countries will commence their surveyor training in the week starting June 24th, with the remaining countries starting from the week commencing July 1st onwards.
June-August 2024	Data Collection Timeline: We aim to complete data collection by August. This comprehensive effort will provide a robust dataset that reflects the early learning status of children across diverse educational landscapes.
September-October 2024	Data Cleaning and Analysis: Post data collection, we will undertake a rigorous process of data cleaning and analysis. This phase is critical for ensuring the accuracy and reliability of our findings.
November 2024	Report Writing and Dissemination: We plan to compile our findings into a detailed report, scheduled for release in November. This report will be disseminated to stakeholders, policymakers, and educational practitioners to inform and guide future educational strategies.

The ELANA project represents a significant step forward in understanding and improving early childhood education in the Global South. By providing comprehensive data on early learning outcomes, ELANA will empower educators and policymakers to make informed decisions that can enhance the educational experiences and achievements of young children. We are excited to continue this journey and look forward to the transformative impact that ELANA will have on early education.