The Joint Model of Two-Parameter Logistic and Response Time Model for Computer-Based Tests

The Joint Model of Two-Parameter Logistic and Response Time Model for Computer-Based Tests
  IJETT-book-cover           
  
© 2024 by IJETT Journal
Volume-72 Issue-1
Year of Publication : 2024
Author : Ninik Zuroidah, Kumaidi, Samsul Hadi, Kusaeri, Syahrul Ramadhan
DOI : 10.14445/22315381/IJETT-V72I1P112

How to Cite?

Ninik Zuroidah, Kumaidi, Samsul Hadi, Kusaeri, Syahrul Ramadhan, "The Joint Model of Two-Parameter Logistic and Response Time Model for Computer-Based Tests," International Journal of Engineering Trends and Technology, vol. 72, no. 1, pp. 117-129, 2024. Crossref, https://doi.org/10.14445/22315381/IJETT-V72I1P112

Abstract
The purpose of this research is to produce a test measurement model that can explain the realistic conditions of the computer-based test by considering the response time. This research is a measurement model development consisting of two main analyses, such as 1) developing a joint model of two-parameter logistic and response time model and 2) examining the accuracy of the parameter model by using standard deviation and testing model fit by using DIC statistics. The model is implemented on empirical data from the National Selection test results of 1559 New Learners of Madrasah Aliyah Negeri (MAN) Insan Cendekia Ministry of Religion of the Republic of Indonesia in 2019 in the field of Mathematics consisting of 15 items with a time limit of 45 minutes. Estimation of parameter models used the Bayesian MCMC method with R2WinBUGS Software. The results of this research show that the joint model of Two Parameter Logistic and response time model produces a hierarchical structure model with three (3) groups of parameters, i.e., person parameters, item parameters, and rho parameters. Measurement models involving response time can improve the accuracy of model parameters compared with Two Parameter Logistic models that are separated from response time. The empirical data in this research fit better with the joint model of Two Parameter Logistic with response time characterized by a smaller DIC value than the Two Parameter Logistic model separated by response time. Therefore, it proves that the joint model of the Two Parameter Logistic with response time is more suitable to be implemented for computer-based and time-constrained tests.

Keywords
Response accuracy, Response time, Two parameter logistic, Bayesian MCMC, CBT.

References
[1] Ariel Rubinstein, “Response Time and Decision Making: An Experimental Study,” Judgment and Decision Making, vol. 8, no. 5, pp. 540-551, 2013.
[CrossRef] [Google Scholar] [Publisher Link]
[2] Wim J. Van Der Linden, “A Hierarchical Framework for Modeling Speed and Accuracy on Test Items,” Psychometrika, vol. 72, pp. 287-308, 2007.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Paul De Boeck, and Minjeong Jeon, “An Overview of Models for Response Times and Processes in Cognitive Tests,” Frontiers in Psychology, vol. 10, pp. 1-11, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Yulia A. Dodonova, and Yury S. Dodonov, “Faster on Easy Items, More Accurate on Difficult Ones: Cognitive Ability and Performance on a Task of Varying Difficulty,” Intelligence, vol. 41, no. 1, pp. 1-10, 2013.
[CrossRef] [Google Scholar] [Publisher Link]
[5] Jean-Paul Fox, and Sukaesi Marianti, “Person-Fit Statistics for Joint Models for Accuracy and Speed,” Journal of Educational Measurement, vol. 54, no. 2, pp. 243-262, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[6] Ronald K. Hambleton, and Hariharan Swaminathan, Item Response Theory Principles and Applications, Kluwer-Nijhoff Pub, pp. 1- 332, 1989.
[Google Scholar] [Publisher Link]
[7] Ronald K. Hambleton, Hariharan Swaminathan, and H. Jane Rogers, Fundamentals of Item Response Theory, SAGE Publications, pp. 1-174, 1991.
[Google Scholar] [Publisher Link]
[8] Gerard J.P. Van Breukelen, “Psychometric Modeling of Response Speed and Accuracy with Mixed and Conditional Regression,” Psychometrika, vol. 70, pp. 359-376, 2005.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Wim J. Van Der Linden, and Ronald K. Hambleton, Item Response Theory: Brief History, Common Models, and Extensions, Handbook of Modern Item Response Theory, pp. 1-28, 1997.
[CrossRef] [Google Scholar] [Publisher Link]
[10] Yi-Hsuan Lee, and Haiwen Chen, “A Review of Recent Response-Time Analyses in Educational Testing,” Psychological Test and Assessment Modeling, vol. 53, no. 3, pp. 359-379, 2011.
[Google Scholar] [Publisher Link]
[11] D. Thissen, “Latent Trait Scoring of Timed Ability Tests,” Computerized Adaptive Testing Conference, Wayzata, MN, USA, pp. 1- 467, 1979.
[Google Scholar] [Publisher Link]
[12] Wim J. Van Der Linden, “Conceptual Issues in Response-Time Modeling,” Journal of Educational Measurement, vol. 46, no. 3, pp. 247-272, 2009.
[CrossRef] [Google Scholar] [Publisher Link]
[13] Andrew Gelman et al., Bayesian Data Analysis, 1st ed., Chapman and Hall/CRC, pp. 1-552, 1995.
[CrossRef] [Google Scholar] [Publisher Link]
[14] David W. Hosmer, Jr. Stanley Lemeshow, and Rodney X. Sturdivant, Applied Logistic Regression, Wiley, pp. 1-528, 2013.
[Google Scholar] [Publisher Link]
[15] Prathiba Natesan, “A Review of Bayesian Item Response Modeling: Theory and Applications,” Journal of Educational and Behavioral Statistics, vol. 36, no. 4, pp. 550-552, 2011.
[CrossRef] [Google Scholar] [Publisher Link]
[16] Ioannis Ntzoufras, Bayesian Modeling Using WinBUGS, Wiley, pp. 1-520, 2011.
[Google Scholar] [Publisher Link]
[17] J.K. Lindsey, Statistical Analysis of Stochastic Processes in Time, Cambridge University Press, pp. 1-338, 2004.
[Google Scholar] [Publisher Link]
[18] Heri Retnawati, Item Response Theory and its Applications: For Researchers, Measurement and Testing Practitioners, Graduate Students, Yogyakarta, Nuha Medika, pp. 1-200, 2014.
[Google Scholar] [Publisher Link]
[19] Frederic M. Lord, Melvin R. Novick, and Allan Birnbaum, Statistical Theories of Mental Test Scores, Information Age Pub, pp. 1- 568, 2008.
[Google Scholar] [Publisher Link]
[20] Deborah L. Schnipke, and David J. Scrams, Exploring Issues of Examinee Behavior: Insights Gained from Response-Time Analyses, 1 st ed., Computer-Based Testing, pp. 1-30, 2002.
[Google Scholar] [Publisher Link]
[21] T. Loeys, Yves Rosseel, and Kristof Baten, “A Joint Modeling Approach for Reaction Time and Accuracy in Psycholinguistic Experiments,” Psychometrika, vol. 76, pp. 487-503, 2011.
[CrossRef] [Google Scholar] [Publisher Link]