EnglishFrenchPAL NetworkSpanish
Last month, senior data analysts and leaders from the PAL Network of citizen-led assessment organizations met in Nairobi, Kenya to discuss implementation of the newly created network-wide Data Quality Standards Framework. The meeting marked exactly one year since network members collectively decided to focus on actively supporting each other to strengthen systems for ensuring robust data. In the opening plenary, Dr. Wilima Wadhwa from ASER Centre, India talked about the importance of PAL Network members establishing systems to ensure robust and reliable data: “When you release data highlighting government failure, the first thing that will be attacked is your methodology. If someone can cast doubt on your methodology or say your data is unreliable or that the sampling hasn’t been done properly and isn’t representative, then people will find a reason not to engage with the findings.”
Three key design features of citizen-led assessments have limited the extent to which the model is accepted as a robust and reliable assessment in national and international policy circles. The first is the involvement of citizen-volunteers in conducting the assessment, which is used to cast doubt on the quality and reliability of the data. The second is the simplicity of the assessment tools, which is seen as evidence of a lack of sophistication and rigor. Lastly, the same foundational assessments are given to every child within a sampled household, regardless of their age or grade. This is used to cast doubt on the scope of citizen-led assessments as they do not provide information about learning competencies of children who are able to complete the highest level of the basic assessment.
Catering to the realities of global South countries
The citizen-led assessment model was designed carefully and deliberately to cater to the realities found in global South countries. The design reflects a philosophy that is different from that of standard school-based assessments, as citizen-led assessments train volunteers to assess children regardless of their schooling status, using simple tests and tools, sitting one-on-one with the child, in their homes. What first appears as a very simple assessment is backed up by highly-sophisticated processes designed to ensure that the data generated are reliable. This includes systematic processes for sampling, partner and volunteer selection, training, monitoring and recheck. In addition, careful data cleaning and other methods are used to validate the data.
“When we first started talking about network-wide data quality standards, it was because different countries [within the network] have done different things [to ensure quality data]” Dr Wadhwa explained. “Documenting and strengthening these processes and sharing best practices is one of the reasons the network exists. As more countries start to conduct citizen-led assessments and the network has grown, network members have requested to learn from each other. We are very proud of the citizen-led assessment movement, because it has grown organically over time, without being prescriptive. The commitment to strengthening our data has also grown organically. It was this shared focus that brought us together a year ago to draft a standards framework. We are back here now to say – OK. Now we have this document, how can we support each other to implement it?”
Contribution of citizen-led assessment data to monitoring SDG4
Over the past three years, the PAL Network has advocated for the inclusion of an early grade indicator in SDG4. To obtain a clear picture of progress, data used to measure this indicator must include all children, regardless of schooling status. In the wake of the World Education Forum 2015, PAL Network released a public statement to the Technical Advisory Group on SDG4, demonstrating the suitability of citizen-led assessments to track the acquisition of foundational skills for all children. This was followed by an Open Letter to the Inter-Agency and Expert Group (IEAG) in March 2016 with an urgent appeal to retain the draft indicator for the percentage of children at Grades 2/3 who have learned the basics, and a position statement on SDG4 in July 2016.
Since May 2016, network members have been active participants in the Global Alliance to Monitor Learning (GAML), ensuring that citizen-led assessment data is included as a complementary source of data for reporting against the SDG4 Indicator 4.1.1. Data from citizen-led assessments are included in the UIS Catalogue of Learning Assessments (CLA), marking an important step towards the inclusion of learning data for children in developing countries who are likely to be found in non-traditional settings.
Ensuring no child is left behind
To ensure that no child is left behind, active efforts are needed to identify the most disadvantaged children. Identifying these children requires an understanding of where they are most likely to be found. Across the developing world, the hardest to reach children are often found in the hardest to reach areas, and are not likely to be in school. Citizen-led assessments have significantly improved knowledge of the inequalities that persist in educational access and acquisition of foundational reading and numeracy skills, in contexts where a significant proportion of children are out-of-school or attending irregularly.
The newly created PAL Network Data Quality Standards Framework will help member countries improve and ensure technical rigour, whilst allowing flexibility to accommodate the diversity of processes and adaptations to local context that is central to the citizen-led assessment model. The DQSF will be accompanied by implementation and monitoring plans, where member countries will support each other to meet the minimum required standards, ensuring that the PAL Network continues to be recognized as making an important and robust contribution to understanding learning progress for all children.