Reliability of Assessment of Reading Ability in Three Countries

Over the last few decades, there have been a number ofnational and international programmes to assessperformance in key subjects. For example theInternational Association for Evaluation ofEducational Attainment (IEA), International Assessmentof Educational Progress (IAEP), and the ThirdInternational Mathematics and Science Survey (TIMSS)programs have aimed to compare performance betweenparticipating countries.Such exercises have different requirements from thosetests, such as public examinations, which aim toassess the attainment of individuals. National andinternational assessments aim to cover a very widerange of materials, and often use the technique ofmultiple matrix sampling to do so.The investigation is based on generalizability-typeanalyses of three national data sets at two ages, andlooks not only at the variance components arising fromsampling of schools, and of pupils within them, butalso at the variation between different assessmentinstruments, and between items within assessmentinstruments. If interpreted with care, such resultscan be of value in the design of future studies. Thispaper concentrates largely on the precision ofestimates of overall means, but analyses of the typedescribed could also be used to compare theperformance of subpopulations.