EnglishKenyaPAL Network
Learning assessments remain paramount in providing data that is crucial in informing progress towards achieving Sustainable Development Goal 4. PAL Network, that brings together member countries conducting citizen-led learning assessments and actions, is such an organization committed to generate better data. Two years ago, when PAL Network began implementing its three-year strategy (2017 – 2019), it focused among other things to ensure the data generated is of high quality.
The commitment to produce robust data compelled country leaders and data analysts to meet in Aurangabad, India, for the network’s first workshop on data quality in January 2017. A year later, the data quality workshop, dialogue among members, and collaborative efforts resulted in a network-wide Data Quality Standards Framework (DQSF) which was later published through a blog post on the UNESCO Institute of Statistics (UIS) official website. The PAL DQSF is a living document that guides PAL Network member organizations to ensure technical rigor to generate robust evidence. While emphasis is on the methodological precision, the imperative is allowing flexibility to accommodate the diversity of processes and adaptations to local context that is central to the citizen-led assessment approach.
The PAL Network DQSF is underpinned by key quality concepts for learning assessments, as outlined in the UNESCO Principles of Good Practice in Learning Assessment (GP-LA). Although these principles are directed towards agencies with or associated with national governments, the concepts are expressions of fundamental principles specifically related to large-scale learning assessments. The PAL DQSF has ten segments namely; survey tools, test development, sampling, recruitment, training, data collection, field monitoring, data management, analysis and reporting; covering the full cycle of citizen-led assessments.
Anchored on GP-LA, the DQSF presents a three-tier set of technical standards that ensure assessments across the Network yields high quality, robust and reliable data. Tier A Standards are the minimum principles that each country must meet to be considered a member of the PAL Network. Tier B standards are the desirable requirements recommended for each member country once they have achieved the minimum standards. The last tier, which are C standards representing broader goals of best practices, should resources allow, including measures to improve comparability across PAL Network assessments.
To ensure that DQSF is utilized, the PAL Network members developed a two phased implementation plan to monitor its adherence. These steps, sequential in nature, are; self-reporting and peer monitoring. Self-monitoring, which is the first step in implementing DQSF, requires members to gauge and track themselves if they are adhering or not to each standard. For members not adhering, the self-reporting tool allows them devise measures that bolster adherence.
The diagram below shows an overview of the network wide adherence to the DQSF.
Additionally, it highlights progress made by members against tier A standard for each of the DQSF’s ten segments. Traffic light colors are used to symbolize results where 70% and above shows good progress and less than 50% adherence indicate slow progress. Tier A standards doesn’t only focus on actions that improve quality of data, but also emphasizes the need to document all processes.
The second step of the DQSF implementation plan is the peer-monitoring or process audit visits. Member countries are provided with a peer –monitoring tool that documents three key steps, namely, pre-survey, during the survey and post-survey processes that are deemed vital for production of quality data.
The DQSF has not only guided members on how to produce quality data, but will also be crucial in directing various processes related to network wide assessments. As such, key process of the upcoming common assessment that aims to generate comparable data across PAL countries, is funneled by the DQSF. The DQSF informed the creation of survey tools and the sampling frame and will continue to guide members through partner and volunteer recruitment, field implementation, data management, analysis and reporting.
Lastly, results emerging from the two implementation steps has revealed some new learning horizons for the network. Evidently, management of data is an area worth improving. One way of addressing this challenge is to develop an instruction manual to help members document and strengthen data processes. We call all organizations in the learning assessment markets to focus on better data to monitor progress towards realization of SDG 4.