Bias in the measurement of child attributes in educational research: Measurement bias in multilevel data

Suzanne Jak
Department of Pedagogical & Educational Sciences, FMG, University of Amsterdam

Project: Project financed by University of Amsterdam

Project running from: 1 January 2009 – 1 January 2014

Supervisor: dr F.J. Oort (University of Amsterdam)

Summary:
Background
The measurement of child attributes brings about problems because informants (e.g., the children themselves, their parents, their teachers, etc.) may have different frames of reference when answering test or questionnaire items. Such different frames of reference may result in measurement bias, so that observed differences and changes in test scores do not reflect true differences and changes in child attributes. Measurement bias thus complicates all research into child attributes (e.g., evaluation of intervention effects, sex differences, cultural differences, relationships with explanatory variables).

Objectives
We will extend existing structural equation modelling (SEM) procedures for the detection
of measurement bias with procedures for bias detection in multilevel data, continuous and discrete.

We will investigate the feasibility of these new procedures, by applying them in secondary analyses of educational data, investigating the impact of measurement bias on the results of testing substantive hypotheses in educational research, and investigating different ways to account for apparent measurement bias.

Method
We will first investigate measurement bias in existing data sets of our department by means of secondary analyses. When we find measurement bias, we will account for this bias, and investigate whether the test results of the original hypotheses are different from the test results that are obtained when measurement bias is accounted for. Dependent on our findings, we may modify the SEM procedures, and further investigate the latent variable modelling procedures with simulated data, e.g., to investigate power, effect size indices, and the impact of measurement bias. This approach will be used with various sets of multilevel data, and various sets of discrete data.

Relevance
We will obtain additional knowledge of:
(1) the psychometric properties of several measurement instruments that are commonly applied in educational research,
(2) the extent of measurement bias in educational research,
(3) the impact of possible measurement bias on substantive conclusions,
(4) the robustness of educational research to possible measurement bias. Moreover, the research project is psychometrically relevant because it extends and further develops procedures for testing measurement bias in multilevel data, continuous and discrete. Methods to detect measurement bias and to account for measurement bias will result in stronger substantive conclusions.

Date of defence:  27 September 2013

Title of thesis:  Cluster bias: Testing measurement invariance in multilevel data. S. Jak (2013, September 27). Universiteit van Amsterdam (112 pag.). Supervisor(s): prof.dr. F.J. Oort & prof.dr. C.V. Dolan.