Who Is Missing from Learning Assessments?
Who Is Missing from Learning Assessments?
Why representation is the first question we should ask before interpreting assessment results
Learning assessments are often described as nationally representative. The phrase is reassuring: it suggests that the results tell us something about children across an entire country.
But representative of whom?
The answer depends on how children enter the assessment sample, who is eligible, who is present on the day of testing and who ultimately completes each part of the assessment. Publicly available household assessment microdata make these processes visible.
School participation matters
The most obvious distinction is between school-based and household-based assessments. School-based assessments describe children who are attending sampled schools. Household assessments begin with children living in sampled households, regardless of whether they are enrolled.
This distinction matters because school participation varies enormously across countries and across childhood. In Chad, for example, 86% of seven-year-olds are out of school. Where substantial numbers of children are out of school, a school-based assessment cannot describe national learning among all children. The gap may become particularly important during adolescence as dropout rises.
The consequences extend beyond who is counted. Children who are out of school score, on average, between 11 and 54 percentage points lower in mathematics than children who are enrolled, depending on the country. A school-based assessment does not just miss these children numerically — it misses some of the clearest evidence of educational disadvantage in the entire dataset.
Representation is also about school type
Even among enrolled children, assessment samples may differ in the types of schools they represent. Non-government schools educate a substantial share of children in some African countries and very few in others.
Comparisons between household and school assessments show that apparently similar grade samples can contain very different proportions of children attending non-government schools. In Senegal, for example, 27% of Grade 2 children attend non-government schools according to the household-based ICAN-ICAR survey, compared with only 15% according to the school-based PASEC survey of the same grade. Understanding the sampling frame is therefore essential before treating two assessment populations as equivalent.
The intended sample is not always the realised sample
Representative sampling does not guarantee a representative analytical sample. Household interviews may be completed while an eligible child is unavailable; children may be excluded from a reading assessment because no suitable language is available; or they may decline particular tasks.
In The Gambia, for example, approximately 19% of Grade 2 children were excluded from the reading assessment because no suitable assessment language was available to them — before they ever had the chance to attempt it.
These mechanisms matter because non-response can be selective. In our analysis of MICS, children who completed the reading assessment scored, on average, 43 percentage points higher in mathematics than children who did not. Reading results therefore describe a selected subgroup rather than exactly the same population represented by the mathematics assessment.
Ask who is represented before asking who performs better
These examples point to a simple principle: representation is not a single feature established when a sample is drawn. It is the outcome of a sequence of decisions and behaviours involving sampling, school participation, eligibility, language and response.
Before comparing learning outcomes across countries, programmes or domains, researchers should first establish whether the populations being compared are themselves comparable.
Household assessment microdata are especially valuable because they allow us to examine these questions directly. They remind us that the first question in any assessment analysis should not be 'How well did children perform?' but 'Which children are represented in this result?'