Feasible measurement of learning in emergencies: lessons from Uganda

A new assessment tool aims to provide a rapid, holistic understanding of displaced learners’ needs.

Education assessments in emergencies currently focus on access or aim to understand teachers’ experience and qualifications rather than closely examine children’s experiences and learning.[1] Moreover, many international and national assessments are lengthy and require highly trained data collectors, who are often unavailable in the immediate aftermath of displacement. The lack of assessments that are rigorous yet feasible to undertake in education in emergencies (EiE) programming means there is often limited information about gaps in children’s learning and the impact of EiE programmes.

The Holistic Assessment of Learning and Development Outcomes (HALDO) is designed to fill such knowledge gaps.[2] It enables practitioners to understand the needs of learners aged 4–12 across four domains: literacy, numeracy, social and emotional learning (SEL), and executive functioning.[3] HALDO also includes questions about demographic and household characteristics, socio-economic status, home learning environment, and disability.[4]

In 2018, HALDO was piloted in 32 education centres and schools managed by Save the Children and Windle International Uganda in the Rwamwanja and Kyangwali refugee settlements in western Uganda. Given the recent influx of refugees from the Democratic Republic of Congo (DRC) to both settlements and their diverse language backgrounds (24 mother tongue languages were identified), schools have found accommodating learners particularly challenging.

Challenges to implementation

We found that HALDO offered a reliable, valid measurement of the different domains of children’s learning and development in these settlements. In this article we focus on three challenges experienced during the pilot which relate to its practical application.

Language: In discussion with community organisers, we identified English, Kiswahili and Kinyabusha/Kinyarwanda as the most appropriate languages for the assessment. This required a majority of assessors to speak all three languages fluently since schools were not separated by language. Additionally, assessors found that the Kiswahili translation did not use child-friendly language; rather than allowing the assessors to reword questions during the assessments, we needed instead to revise the wording for all items during the training, to ensure consistency and use of appropriate terminology. In recognition of language diversity, those undertaking future assessments should conduct a more comprehensive language mapping before translating the tool, create a sampling frame based on the languages spoken, engage assessors that speak the languages fluently, and recommend that translators work with community members to incorporate child-friendly language.

Assessor training: We piloted a ‘rapid assessor training’ model that can be used in EiE programmes that have limited time and resources. The pilot in Uganda highlighted the challenges with this model. Notably, the outcome of deploying such a training model can be to damage the reliability of assessors’ scoring, since there may have been insufficient time for them to develop a harmonised understanding of the assessment and its use. The pilot helped us to identify specific questions with which assessors struggled – like timed tasks and deciding on appropriate and inappropriate responses for SEL questions – allowing us to modify the rapid assessor training to better support assessors’ understanding in relation to these areas and how responses should be scored. 

Facilitator feedback: One aim of HALDO is to provide teachers and facilitators with aggregated information about the learning gaps of children with whom they are working, and the pilot underlined the importance of this. One way to support this provision of feedback is to connect the assessment results with tailored learning content to ensure that the learning and development needs that have been identified are accompanied by explicit instruction on the appropriate teaching materials required to address them.

Continuing refinement of HALDO includes creating guidance on how to better adapt and translate HALDO and improving assessor training so that the tool can be rapidly deployed in the early onset of displacement. Through further testing and analysis, we will use HALDO findings to inform our recommendations for teachers and facilitators working with children in situations of forced displacement to ensure that the education being provided meets the children’s needs more effectively.

 

Nikhit D’Sa ndsa@savechildren.org
Director, Research, Evaluation, and Learning

Allyson Krupar akrupar@savechildren.org
Senior Specialist, Learning Research

Clay Westrope cwestrope@savechildren.org
Senior Specialist, Learning Research

Save the Children www.savethechildren.org   

 

[1] For more on ‘good learning’ in displacement, see Krupar and Anselme in this issue.

[2] For more information about HALDO, email learningassessment@savechildren.org.

[3] Literacy measures vocabulary, letter identification, reading accuracy and reading comprehension. Numeracy measures counting, number identification, addition and subtraction, and word problems. SEL measures self-concept and empathy. Executive functioning measures short-term memory and working memory through questions about number sequences.

[4] HALDO includes three questions about disability which draw on the Washington Group Short Set on Functioning bit.ly/WG-SS-2017

 

Disclaimer
Opinions in FMR do not necessarily reflect the views of the Editors, the Refugee Studies Centre or the University of Oxford.
Copyright
FMR is an Open Access publication. Users are free to read, download, copy, distribute, print or link to the full texts of articles published in FMR and on the FMR website, as long as the use is for non-commercial purposes and the author and FMR are attributed. Unless otherwise indicated, all articles published in FMR in print and online, and FMR itself, are licensed under a Creative Commons Attribution-NonCommercial-NoDerivs (CC BY-NC-ND) licence. Details at www.fmreview.org/copyright.