University of Groningen
Faculty of Behavioural and Social Sciences
Heymans Institute

Supervisors:
prof.dr. T.A.B. Snijders, dr. A. Boomsma
dr. M.A.J. van Duijn, dr. M.E. Timmerman

On 23 June 2014 Janke ten Holt defended her thesis entitled

A comparison between factor analysis and item response theory modeling in scale analysis

Summary
Factor analysis (FA) and item response theory (IRT) modeling are compared (a) theoretically by examining the model definitions and considering the most commonly applied estimation methods for the models, (b) empirically by reviewing journal articles on scaling research to show what is actually done in practice, (c) in model estimation performance by means of a Monte Carlo simulation study to investigate the robustness of either approach against violations of distributional assumptions, and (d) in practice by applying the models to empirical data sets.

In a simulation study, the effects of four explanatory variables (latent variable (LV) distribution, item response distribution, scale strength, and sample size) on model estimation are investigated by applying four selected scaling models on samples of generated data: FA of the sample covariance matrix (FA-lin), FA of the estimated polychoric correlation matrix (FA-poly), the graded response IRT model (IRT-grm), and the nonparametric Mokken IRT model extended to polytomous items (IRT-mok).

For normal data it is shown that both FA-poly and IRT-grm perform well. Under conditions of a skew-normal LV, the performance of all parametric models deteriorates compared to normal LV conditions, most notably when combined with skewed item variables. IRT-grm performs best in such circumstances. Nonnormal LV and item distributions do not pose any estimation problems for IRT-mok.

Guidelines are provided for applied researchers employing scale analysis: Take a large enough sample size, use available substantive knowledge, inspect the sample data, choose a model based on specific characteristics of the data, assess model fit, and use model-estimated LV scores.

Project

A comparison between factor analysis and item response theory modeling in scale construction
In practice, the two most important approaches for scale construction are either based on factor analytic or item response theory modeling. The proposed research deals with a comparative analysis of both methods, which differ in their assumptions, but share similar principles and coinciding goals. The stability and sensitivity of scales obtained with each method, are topics of investigation. The analysis of empirical data is part of the research design. The aim is to develop, so far lacking, practical recommendations for the applied researcher in a perhaps unified approach.

This project was financed by NWO.