Sampling weights in multilevel modelling: an investigation using PISA sampling structures

Abstract Background Standard methods for analysing data from large-scale assessments (LSA) cannot merely be adopted if hierarchical (or multilevel) regression modelling should applied. Currently various approaches exist; they all follow generally a design-based model of estimation using the pseudo maximum likelihood method and adjusted weights corresponding hierarchies. Specifically, several different to scaling sampling in models are promoted, yet no study has compared them provide evidence which performs best therefore preferred. Furthermore, software programs implement algorithms, leading results. Objective In this study, we determine based on simulation, procedure showing smallest distortion actual population features. We consider estimation, optimization acceleration methods, weights. Three scenarios have been simulated statistical program R. The analyses performed with two packages LSA data, namely Mplus SAS. Results conclusions simulation results revealed three weighting performing retrieving true parameters. One implies only level (here: final school weights) is because its simple implementation most favourable one. This finding clear recommendation researchers multilevel (MLM) when or similar structure. Further, found little differences performance default settings used, package providing slightly more precise estimates. Different algorithm starting accelerating could cause these distinctions. However, it emphasized that recommended approach, both perform equally well. Finally, techniques student investigated. They nearly identical use Programme International Student Assessment (PISA) 2015 illustrate practical importance relevance assessment models.