Linear logistic test models for rule-based item generation

Hanneke Geerlings (PhD student)
Department of Research Methodology, Measurement, and Data Analysis (OMD)
Twente University

Project:project at Twente University

Project running from:1 September 2007 – 1 September 2011

Supervisors: Prof. dr C.A.W. Glas & prof. dr W. J. Van der Linden

Summary:
This project is embedded in a larger project called ‘Rule-based Item Generation of Algebra Word Problems Based upon Linear Logistic Test Models for Item Cloning and Optimal Design’ that is funded by the Deutsche Forschungsgemeinschaft (German Research Foundation). The project is a collaboration between the Universities of Münster and Twente. In this project, techniques from cognitive analysis, item response theory (IRT), hierarchical modeling, and optimal design theory are combined to develop procedures for automated item generation and test assembly for the testing of basic mathematical competencies in early secondary education, as can be assessed with algebra word problems. It will also be investigated how the models and procedures should be optimized and generalized when they are applied in computerized adaptive testing, testing for diagnosis, and large-scale educational assessments. The final goal is the development of a software program which adaptively generates tailor-made items for algebra word problems based on optimal design, linear-logistic test models, and models for test item cloning. The sub-project presented here focuses on the statistical aspects of the project. Starting point is the classical version of the linear-logistic test model (e.g., Fischer, 1995). This model will be extended through incorporating random effects as well as interaction effects. The hierarchical model for item cloning will be provided with a structure for the item parameters developed in other sub-projects. The parameters of the model will be estimated in a Bayesian framework, by means of Markov Chain Monte Carlo (MCMC) computation. If time allows, estimation in a frequentist framework (by means of Marginal Maximum Likelihood, MML, estimation) can also be considered. The result will be used in the application of optimal design techniques for automated test assembly from pools of item families. The selection criteria will be based on the hyperparameters that describe the item families instead of the usual lower-level parameters of the discrete items. Both information-based and Bayesian criteria for item selection will be studied.

Date of defence:  23 March 2012

Title of thesis:  Psychometric methods for automated test design (ISBN: 978-90-365-3330-0)

Promotores:  Prof. dr C.A.W. Glas, Prof. dr W. J. Van der Linden