logo logo European Journal of Educational Research

EU-JER is is a, peer reviewed, online academic research journal.

Subscribe to

Receive Email Alerts

for special events, calls for papers, and professional development opportunities.

Subscribe

Publisher (HQ)

Eurasian Society of Educational Research
Eurasian Society of Educational Research
7321 Parkway Drive South, Hanover, MD 21076, USA
Eurasian Society of Educational Research
Headquarters
7321 Parkway Drive South, Hanover, MD 21076, USA
classical test theory differential aptitude test item parameter modern test theory

Study Item Parameters of Classical and Modern Theory of Differential Aptitude Test: Is it Comparable?

Farida Agus Setiawati , Rizki Nor Amelia , Bambang Sumintono , Edi Purwanta

This study aimed to find the Classical Test Theory (CTT) and Modern Test Theory (MTT) item parameters of the Differential Aptitude Test (DAT) and exam.

T

This study aimed to find the Classical Test Theory (CTT) and Modern Test Theory (MTT) item parameters of the Differential Aptitude Test (DAT) and examined their comparability of them. The item parameters being studied are difficulty level and discrimination index. 5.024 data of the result sub-test DAT were documented by the Department of Psychology and Guidance and Counselling bureau. The parameter of classical and modern test items was estimated and correlated by examining the comparability between parameters. The results show that there is a significant correlation between item parameter estimates. The Rasch and IRT 1-PL models have the highest correlation toward CTT regarding the item difficulty level. In contrast, model 2-PL has the highest correlation toward CTT in the item discrimination index. Overall, the study concluded that CTT and MTT were comparable in estimating item parameters of DAT and thus could be used independently or complementary in developing DAT.

Keywords: Classical test theory, differential aptitude test, item parameter, modern test theory.

cloud_download PDF
Cite
Article Metrics
Views
290
Download
340
Citations
Crossref
2

Scopus
1

References

Abed, E. R., Al-Absi, M. M., & Abu shindi, Y. A. (2016). Developing a numerical ability test for students of education in Jordan: An application of Item Response Theory. International Education Studies, 9(1), 161. https://doi.org/10.5539/ies.v9n1p161

Adedoyin, O. O., Nenty, H. J., & Chilisa, B. (2020). Investigating the invariance of item difficulty parameter estimates based on CTT and IRT. International Journal of Educational Research and Reviews, 7(11). https://bit.ly/3JFG5T3

Ahmadi, A., & Thompson, N. A. (2012). Issues affecting Item Response Theory fit in language assessment: A study of differential item functioning in the Iranian National University entrance exam. Journal of Language Teaching and Research, 3(3), 401–412. https://doi.org/10.4304/jltr.3.3.401-412

AL-khadher, M. M. A., & Albursan, I. S. (2017). Accuracy of measurement in the classical and the modern test theory: An empirical study on a children intelligence test. International Journal of Psychological Studies, 9(1), 71–80. https://doi.org/10.5539/ijps.v9n1p71

Andrich, D. (2011). Rating scales and Rasch measurement. Expert Review of Pharmacoeconomics & Outcomes Research, 11(5), 571–585. https://doi.org/10.1586/erp.11.59

Avvannavar, S. M., Ambrose, B., & Chandavarkar, M. (2013). Determination of effectiveness of aptitude test to improve sincerity in the recruitment process. International Journal of Management, 4(4), 75–81.

Awopeju, O. A., & Afolabi, E. R. I. (2016). Comparative analysis of classical test theory and item response theory based item parameter estimates of senior school certificate mathematics examination. European Scientific Journal, 12(28), 263. https://doi.org/10.19044/esj.2016.v12n28p263

Baker, F. B. (2001). The basics of item response theory (2nd ed.). ERIC Clearinghouse on Assessment and Evaluation. https://eric.ed.gov/?id=ED458219

Bennet, G. K., Seashore, H. G., & Wesman, A. G. (1956). The Differential Aptitude Test: An overview. The Personnel and Guidance Journal, 35(2), 81–91. https://doi.org/10.1002/j.2164-4918.1956.tb01710.x

Bichi, A. A., Embong, R., Talib, R., Salleh, S., & Bin Ibrahim, A. (2019). Comparative analysis of classical test theory and item response theory using chemistry test data. International Journal of Engineering and Advanced Technology, 8(5C), 1260–1266. https://bit.ly/40Ifc8k

Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental mesurement in the human sciences. In Applying the Rasch Model (Third ed.). Routledge. https://doi.org/10.4324/9781315814698

Boone, W. J., & Scantlebury, K. (2006). The role of Rasch analysis when conducting science education research utilizing multiple-choice tests. Science Education, 90(2), 253–269. https://doi.org/10.1002/sce.20106

Boopathiraj, C., & Chellamani, K. (2013). Analysis of test items on difficulty level and discrimination index in the test for research in education. International Journal of Social Science & Interdisciplinary Research, 2(2), 189–193. https://bit.ly/3lzCVYQ

Brennan, R. L. (2010). Generalizability theory and classical test theory. Applied Measurement in Education, 24(1), 1–21. https://doi.org/10.1080/08957347.2011.532417

Cohen, R. J., & Swerdlik, M. E. (2018). Psychological testing and assessment. In G. K. Zammit & J. W. Hull (Eds.), Guidebook for clinical psychology interns (pp. 121-134). Springer. https://doi.org/10.1007/978-1-4899-0222-1_8

Courville, T. G. (2004). An empirical comparison of Item Response Theory and Classical Test Theory item/person statistics. Texas A&M University. https://core.ac.uk/download/pdf/147123147.pdf

Çıkrıkçı, N. (2002). A study of Raven Standard Progressive Matrices Test’s item measure under classic and item response models: An empirical comparison. Journal of Faculty of Education Sciences, 35(1), 71-80. https://doi.org/10.1501/Egifak_0000000055

D’Oliveira, T. C. (2004). Dynamic spatial ability: An Exploratory analysis and a confirmatory study. The International Journal of Aviation Psychology, 14(1), 19–38. https://doi.org/10.1207/s15327108ijap1401_2

Dardick, W. R., & Mislevy, R. J. (2016). Reweighting data in the spirit of Tukey: Using Bayesian Posterior Probabilities as Rasch Residuals for studying misfit. Educational and Psychological Measurement, 76(1), 88–113. https://doi.org/10.1177/0013164415583351

de Ayala, R. J. (2009). The theory and practice of item response theory. The Guildford Press.

DeMars, C. (2010). Item Response Theory: Understanding statistics measurement. Oxford University Press. https://doi.org/j3rq

Dewberry, C. (2011). Aptitude testing and the legal profession. In Aptitude tests currently used in the professional services sector (Issue June). https://bit.ly/3JAUaAT

du Toit, M. (Ed.). (2003). IRT from SSI. Scientific Software International, Inc.

Ekpo, E. O., Egbonyi, I. C., & Bassey, S. W. (2016). Transformation from classical test theory (CTT) to item response theory (IRT) in research instrument validation. Journal of Educational Research, 1(4), 104–117. https://bit.ly/40wyFZ3

Eleje, L. I., Onah, F. E., & Abanobi, C. C. (2018). Comparative study of Classical Test Theory and Item Response Theory using diagnostic quantitative economics skill test item analysis results. European Journal of Educational & Social Sciences, 3(1), 57–75. https://bit.ly/3TQjKa5

Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates, Inc.

Fan, X. (1998). Item response theory and classical test theory: An empirical comparison of their item/person statistics. Educational and Psychological Measurement, 58(3), 357–381. https://doi.org/10.1177/0013164498058003001

Hakstian, A. R., & Bennet, R. W. (1978). Validity studies using the comprehensive ability battery (CAB): II. Relationship with the DAT and GATB. Educational and Psychological Measurement, 38(7), 1003–1015. https://doi.org/10.1177/001316447803800419

Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and appliction (Vol. 1). Springer.

Hashmi, M., Zeeshan, A., Saleem, M., & Akbar, R. (2012). Development and validation of an aptitude test for secondary school mathematics students. Bulletin of Education and Research, 34(1), 65-76. https://bit.ly/3ZGmmZc

Hernandez, R. (2009). Comparison of the item discrimination and item difficulty of the Quick-Mental Aptitude Test using CTT and IRT methods. The International Journal of Educational and Psychological Assessment, 1(1), 12–18. https://bit.ly/3JCh9Mb

Hu, Z., Lin, L., Wang, Y., & Li, J. (2021). The integration of Classical Testing Theory and Item Response Theory. Psychology, 12(9), 1397–1409. https://doi.org/10.4236/psych.2021.129088

Kohli, N., Koran, J., & Henn, L. (2015). Relationships among Classical Test Theory and Item Response Theory frameworks via Factor Analytic Models. Educational and Psychological Measurement, 75(3), 389–405. https://doi.org/10.1177/0013164414559071

Košir, K., & Pečjak, S. (2007). Personality, motivational factors and difficulties in career decision-making in secondary school students. Psychological Topics, 16(1), 141–158. https://hrcak.srce.hr/file/32320

Linacre, J. M. (2012). Winsteps help for Rasch analysis. Winsteps. https://www.winsteps.com/winman/copyright.htm

MacDonald, P., & Paunonen, S. V. (2002). A Monte Carlo comparison of item and person statistics based on Item Response Theory versus Classical Test Theory. Educational and Psychological Measurement, 62(6), 921–943. https://doi.org/10.1177/0013164402238082

Magno, C. (2009). Demonstrating the difference between Classical Test Theory and Item Response Theory using derived test data. In The International Journal of Educational and Psychological Assessment, 1(1), 1-11. https://bit.ly/3Z9TSXE

Mahakud, G. C. (2013). Is it essential to measure intelligence along with aptitude test for career guidance. Research World-Journal of Arts, Science & Commerce, 4(1), 92–102. https://bit.ly/3Z9TiZY

Mankar, J., & Chavan, D. (2013). Differential aptitude testing of youth. International Journal of Scientific and Research Publications, 3(7), 1–6. https://bit.ly/3K0BImK

Marais, A. C. (2007). Using the differential aptitude test to estimate intelligence and scholastic achievement at grade nine level (Issue June) [University South Africa]. https://bit.ly/3lBZ5d1

Masriah, Z., Nursalim, M. M., & Fitriani, A. (2018). Persepsi mahasiswa terhadap jurusan perguruan tinggi dan konsep diri dengan kesesuaian minat memilih [Effect of student perceptions of college majors and self-concept on the suitability of interest in choosing]. Anfusina: Journal of Psychology, 1(1), 61–76. https://doi.org/10.24042/ajp.v1i1.3639

Mead, A. D., & Meade, A. W. (2010). CTT and IRT 1Test Construction using CTT and IRT with unrepresentative samples. Paper Presented at the Annual Meeting of the Society for Industrial and Organizational Psychology in Atlanta GA, 56. https://bit.ly/3naYBeg

Mehta, G., & Mokhasi, V. (2014). Item analysis of multiple choice questions-An assessment of the assessment tool. International Journal of Health Sciences and Research, 4(7), 197–202. https://bit.ly/42KzWxF

Mitra, N. K., Nagaraja, H. S., Ponnudurai, G., & Judson, J. P. (2009). The levels of difficulty and discrimination indices in type a multiple choice questions of pre-clinical semester 1 multidisciplinary summative tests. International EJournal of Science Medicine Education, 3(1), 2–7. https://doi.org/10.56026/imu.3.1.2

Muhid, A., Yusuf, A., Kusaeri, Novitasari, D. C. R., Asyhar, A. H., & Ridho, A. (2020). Determining scholastic aptitude test as predictors of academic achievement on students of Islamic School in Indonesia. New Educational Review, 61, 211–221. https://bit.ly/3ZJOafu

Petrillo, J., Cano, S. J., McLeod, L. D., & Coon, C. D. (2015). Using Classical Test Theory, Item Response Theory, and Rasch Measurement Theory to evaluate patient-reported outcome measures: A comparison of worked examples. Value in Health, 18(1), 25–34. https://doi.org/10.1016/j.jval.2014.10.005

Philip, A., & Odunayo, O. B. (2017). Application of Item Characteristic Curve (ICC) in the selection of test items. British Journal of Education, 5(2), 21–41. https://bit.ly/409wvif

Pollard, B., Dixon, D., Dieppe, P., & Johnston, M. (2009). Measuring the ICF components of impairment, activity limitation and participation restriction: An item analysis using Classical Test Theory and Item Response Theory. Health and Quality of Life Outcomes, 7, Article 41. https://doi.org/10.1186/1477-7525-7-41

Progar, S., Socan, G., & Slovenija, M. P. (2008). An empirical comparison of Item Response Theory and Classical Test Theory. Horizons of Psychology, 17(3), 5–24. https://bit.ly/3JGdr3Y

Pyari, P., Mishra, K., & Dua, B. (2016). A study of impact of aptitude in mathematics as stream selection at higher secondary level. Issues and Ideas in Education, 4(2), 141–149. https://doi.org/10.15415/iie.2016.42011

Qasem, M. A. N. (2013). A comparative study of Classical Theory (CT) and Item Response Theory (IRT) in relation to various approaches of evaluating the validity and reliability of research tools. IOSR Journal of Research & Method in Education, 3(5), 77–81. https://doi.org/10.9790/7388-0357781

Saputro, M. (2017). Analisis faktor-faktor yang mempengaruhi keputusan mahasiswa dalam memilih program studi [Analysis of the factors that influence student decisions in choosing a study program]. Jurnal Pendidikan Informatika Dan Sains, 6(1), 83–94. http://bit.ly/3MAr8or

Sayyah, M., Vakili, Z., Alavi, N. M., Bigdeli, M., Soleymani, A., Assarian, M., & Azarbad, Z. (2012). An item analysis of written multiple-choice questions: Kashan University of Medical Sciences. Nursing and Midwifery Studies, 1(2), 83–87. https://core.ac.uk/download/pdf/143838611.pdf

Setiawati, F. A., Izzaty, R. E., & Hidayat, V. (2018a). Evaluasi karakteristik psikometrik tes bakat [Evaluation of psychometric property of the aptitude test]. Humanitas, 15(1), 46–61. https://bit.ly/3ZC7laV

Setiawati, F. A., Izzaty, R. E., & Hidayat, V. (2018b). Analisis respon butir pada tes bakat skolastik [The item response theory analysis of scholastic test]. Jurnal Psikologi/Psychology Journal, 17(1), 1–17. https://doi.org/10.14710/jp.17.1.1-17

Setiawati, F. A., Izzaty, R. E., & Hidayat, V. (2018c). Items parameters of the space-relations subtest using Item Response Theory. Data in Brief, 19, 1785–1793. https://doi.org/10.1016/j.dib.2018.06.061

Shah, A. F., & Raza, M. A. (2009). The impact of parents ’ education towards the science aptitude of the students at elementary level in Southern Punjab. Pakistan Journal of Social Sciences, 29(1), 117–125. https://bit.ly/3n9qGme

Stemler, S. E., & Naples, A. (2021). Rasch measurement v. Item Response Theory: Knowing when to cross the line. Practical Assessment, Research and Evaluation, 26, Article 11. https://doi.org/10.7275/v2gd-4441

Sumintono, B., & Widhiarso, W. (2015). Aplikasi permodelan Rasch pada assessment pendidikan [Rasch modeling application in educational assessment]. Trim Komunikata.

Thomas, M. L. (2011). The value of Item Response Theory in clinical assessment: A review. Assessment, 18(3), 291–307. https://doi.org/10.1177/1073191110374797

Thorpe, G. L., & Favia, A. (2012). Data analysis using item response theory methodology: An introduction to selected programs and applications. Psychology, 20, 1–34.

Tractenberg, R. E. (2010). Classical and modern measurement theories, patient reports, and clinical outcomes. Contemporary Clinical Trials, 31(1), 1–3. https://doi.org/10.1016/S1551-7144(09)00212-2

Yang, F. M., & Kao, S. T. (2014). Item response theory for measurement validity. Shanghai Archives of Psychiatry, 26(3), 171–177.  

...