EnglishFrenchPAL NetworkSpanish
Challenges to educational quality
Improving the quality of education is of great urgency today. This is due to the growing recognition that attending school does not necessarily equate to learning. The COVID-19 crisis made problems that education systems face globally more evident. Several authors agree on the pressing need to strengthen students’ basic skills like reading, writing and arithmetic. This is to not only prevent learning losses but to also improve the quality of teaching and learning. Emphasis has been placed on the need to implement proven impact methodologies such as Pratham’s Teaching at The Right Level – TaRL.
Colombia faces particular challenges in delivering quality education, demonstrated for instance by the country’s underperformance in PISA. This issue is especially acute in rural areas when connectivity is low, resulting in a major rural-urban disparity in learning. Unfortunately, information is not yet available about potential learning losses during the past year for Colombia. Neither is there up-to-date information about children’s basic skills levels by age or grade, such as those available in various countries in the Global South, for example, the Citizen-Led Assessment (CLA) or ASER.
Literacy+
In Colombia, with the non-profit organisation GLOT, we have adapted the Teaching at the Right Level (TaRL) impact methodology of Pratham (India) to the national context. We have done this through our Literacy+ programme, which seeks to evaluate and improve children’s literacy and numeracy skills.
Since 2019, we have implemented several small-scale pilots of TaRL in low-income communities and rural schools. With the onset of the COVID-19 pandemic and subsequent lockdowns in 2020, we adapted the Literacy+ programme to the new context of remote learning using the Kolibri app. Kolibri is a free, offline and open-source platform by Learning Equality that allows implementations in any context, including areas without internet access. Although Kolibri is mostly associated with young students’ learning processes, we pair it with our Literacy+ programme to train trainers, namely teachers and volunteers. Our Kolibri channel contains activities and materials that equip trainers with the skills and knowledge needed to implement the TaRL methodology in the classroom.
Although Kolibri is mostly associated with young students’ learning processes, we pair it with our Literacy+ programme to train trainers, namely teachers and volunteers. Our Kolibri channel contains activities and materials that equip trainers with the skills and knowledge needed to implement the TaRL methodology in the classroom.
Kolibri pilot evaluation
In 2021, we evaluated Kolibri with a researcher from the University of Sussex and the support of SUMMA (the Laboratory for Research and Innovation in Education for Latin America and the Caribbean). The evaluation aimed to test the user and learning experience to improve the model, following the Education Endowment Foundation’s (EEF) Implementation and Process Evaluation Handbook, which organises research questions into specific categories. The categories we used for the pilot were evidence of promise, feasibility of delivery and readiness for trial.
Although these categories are associated with the use of large samples for evaluation processes, this pilot was carried out with only 13 teachers of a rural area near the Medellín municipality. Despite not being a statistically representative sample, this study allows us to obtain significant insights at an early stage to refine the model in preparation for scaling up for a trial with more teachers. This pilot evaluation is focused on the training of educators with Kolibri phase, rather than the whole process of implementation with students.
The pilot study used both qualitative and quantitative research methods. Semi-structured interviews, focus groups and surveys were utilised. Document analysis and data were collated to respond to the key research questions for each category. Evidence of promise looked at issues such as whether teachers acquired the required skills to develop TaRL activities. Feasibility of delivery was concerned with questions including whether the Kolibri user experience was satisfactory. Readiness for trial included doing research to determine whether the intervention materials, procedures, or even the logical framework needed changing.
Regarding context, the pilot started at the end of April 2021 at the peak of COVID-19, lockdown and social crisis (clashes) in Colombia. Teachers’ devices were used for the pilot. However, it was impossible for our implementers to travel and install the Kolibri channel on them due to the broader social context. The pilot used a Kolibri model implementation where the installation and configuration took place online.
Findings and considerations to improve the user and learning experience with Kolibri
Through the pilot, there were issues relating to online configuration. For instance, some teachers failed to install Kolibri due to weak internet connectivity. It was also a time-consuming process, suggesting that it is better to use a model that allows an administrator to travel to different communities to install and configure the learners’ facilities.
As the teachers’ own devices were used, some participants’ devices did not meet the minimum requirements for the optimum use of Kolibri. There was the risk of data loss from teachers’ personal devices if they uninstalled the programme, or the programme did not run properly. For this reason, it is prudent to prioritise using school devices if available or providing learners with tablets or computers. The latter has financial implications for implementers and partner institutions that should be considered. Alternatively, other learning tools such as WhatsApp could be used for the learning process.
Some content and resources on Kolibri did not display on participants’ devices, although they appeared to have been downloaded correctly. Some teachers reported a couple of typos in the content while others suggested increasing the font size of some of the resources, which are considerations for improving the text.
As the training programme proposes a process in which synchronous meetings were scheduled at the end of each module to strengthen concepts and create a learning community among teachers, connectivity issues posed a challenge when face-to-face meetings were not possible during the weeks of Colombian social distress. For the effective implementation of the TaRL methodology in schools, fostering learning communities through embedding collaboration into the teachers’ schedules would allow them to continue improving their understanding and use of TaRL. This would be done through discussion, reflection and peer and mentor feedback. Then, spaces to develop practical activities beyond the theory given on Kolibri would enable the teachers to enhance their understanding of the approach and build their confidence to implement it effectively.
Once the training programme finished, the teachers believed TaRL provided a contextually-responsive solution to the specific learning gaps they identified in their schools. This meant that despite the technological barriers they faced, the teachers remained highly motivated to engage with the programme as they found Kolibri easy to use. As the learning process seeks to equip teachers to effectively implement the TaRL approach within schools, it highlights the importance of correctly evaluating learning objectives and reflecting that in the logical framework.
These findings and implications cannot be generalised; however, we cannot wait until we grow to start collecting evidence. This pilot is a starting point for the bigger implementation of our Literacy+ programme, and hopefully, for similar programmes seek to benefit from using Kolibri. By utilising this sort of accessible technology and harnessing teachers’ skills in a collaborative environment, interventions of this kind can potentially scale their impact and reach students in non-connected areas. This would ensure inclusive learning and educational justice at a time when it is desperately needed.
For feedback or questions: diana@glot.org
References
Banerji, R. (2020). COVID-19 Makes Teaching at the Right Level Even More Pressing. The Research on Improving Systems of Education (RISE). https://riseprogramme.org/blog/covid-19-TaRL
EEF and the University of Manchester. 2019. Implementation and process evaluation (IPE) for interventions in education settings: An introductory handbook. https://educationendowmentfoundation.org.uk/public/files/Evaluation/Setting_up_an_Evaluation/IPE_Handbook.pdf
GLOT & SUMMA, 2021. Literacy+ program 2021 pilot report.
Learning Equality. (2020). Implementation models for distance learning with Kolibri v0.14. https://docs.google.com/document/d/1zXisj-GRzFqeJWXqPUyg8mKfSyilaE8oYPm_hHAH0Sw/edit#heading=h.oe4f88dtb4ct
OECD. (2018). Pisa Colombia 2018. OECD Economics Department Working Papers. https://www.oecd.org/pisa/publications/PISA2018_CN_COL.pdf
Soler, M, Morlà-Folch, T, García-Carrión, R and Valls, R (2019) ‘Transforming rural education in Colombia through family participation’, Journal of Social Science Education, 18(4), pp.67- 80, doi: 10.4119
WIDE, nd. World Inequality Database on Education. Colombian Report. https://www.education-inequalities.org/countries/colombia#?dimension=community&group=all&year=latest
1119
