The Development of a Four-Tier Diagnostic Test Based on Modern Test Theory in Physics Education
Edi Istiyono
,
Wipsar Sunu Brams Dwandaru
,
Kharisma Fenditasari
,
Made Rai Suci Shanti Nurani Ayub
,
Duden Saepuzaman
Diagnostic tests are generally two or three-tier and based on classical test theory. In this research, the Four-Tier Diagnostic Test (FTDT) was develo.
- Pub. date: January 15, 2023
- Pages: 371-385
- 381 Downloads
- 421 Views
- 0 Citations
Diagnostic tests are generally two or three-tier and based on classical test theory. In this research, the Four-Tier Diagnostic Test (FTDT) was developed based on modern test theory to determine understanding of physics levels: scientific conception (SC), lack of knowledge (LK), misconception (MSC), false negatives (FN), and false positives (FP). The goals of the FTDT are to (a) find FTDT constructs, (b) test the quality of the FTDT, and (c) describe students' conceptual understanding of physics. The development process was conducted in the planning, testing, and measurement phases. The FTDT consists of four-layer multiple-choice with 100 items tested on 700 high school students in Yogyakarta. According to the partial credit models (PCM), the student's responses are in the form of eight categories of polytomous data. The results of the study show that (a) FTDT is built on the aspects of translation, interpretation, extrapolation, and explanation, with each aspect consisting of 25 items with five anchor items; (b) FTDT is valid with an Aiken's V value in the range of 0.85-0.94, and the items fit PCM with Infit Mean Square (INFIT MNSQ) of 0.77-1.30, item difficulty index of 0.12-0.38, and the reliability coefficient of Cronbach's alpha FTDT is 0.9; (c) the percentage of conceptual understanding of physics from large to small is LK type 2 (LK2), FP, LK type 1 (LK1), FN, LK type 3 (LK3), SC, LK type 4 (LK4), and MSC. The percentage sequence of MSC based on the successive material is momentum, Newton's law, particle dynamics, harmonic motion, work, and energy. In addition, failure to understand the concept sequentially is due to Newton's law, particle dynamics, work and energy, momentum, and harmonic motion.
Keywords: Developing test, Four-Tiers Diagnostic Test, modern test theory.
References
Adodo, S. O. (2013). Effects of two-tier multiple choice diagnostic assessment items on students’ learning outcome in basic science technology (BST). Academic Journal of Interdisciplinary Studies, 2(2), 201. https://doi.org/10.5901/ajis.2013.v2n2p20
Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45(1), 131–142. https://doi.org/10.1177/0013164485451012
Anderson, L. W., & Krathwohl, D. R. (2010). Kerangka landasan untuk pembelajaran, pengajaran, dan asesmen [Foundational framework for learning, teaching, and assessment]. Pustaka Pelajar.
Apino, E., & Retnawati, H. (2016). Creative problem solving to improve students’ higher order thinking skills in mathematics instructions. In W. Warsono (Ed.), Proceeding of 3rd International Conference on Research, Implementation and Education of Mathematics and Science (pp. 339-346). Universitas Negeri Yogyakarta.
Azwar, S. (2012). Reliabilitas dan validitas [Reliability and validity] (1st ed.). Pustaka Pelajar.
Azwar, S. (2019). Konstruksi tes kemampuan kognitif [Cognitive ability test construction]. Pustaka Pelajar.
Azwar, S. (2020). Reliabilitas dan validitas [Reliability and validity] (4th ed.). Pustaka Pelajar.
Baker, M., Rudd, R., & Pomeroy, C. (2001). Relationships between critical and creative thinking. Journal of Southern Agricultural Education Research, 51(1), 173–188.
Bauldry, S. (2015). Structural equation modeling. In J. D. Wright (Ed.), International encyclopedia of the social & behavioral sciences (2nd ed., pp. 615–620). Elsevier. https://doi.org/10.1016/B978-0-08-097086-8.44055-9
Bennett, D. M. (2014). The U.N. convention on the rights of persons with disabilities and U.K. mental health legislation. British Journal of Psychiatry, 205(1), 76–77. https://doi.org/10.1192/bjp.205.1.76a
Caleon, I. S., & Subramaniam, R. (2010). Do students know What they know and what they don't know? Using a four-tier diagnostic test to assess the nature of students' alternative conceptions. Research in Science Education, 40(3), 313–337. https://doi.org/10.1007/s11165-009-9122-4
Cetin-Dindar, A., & Geban, O. (2011). Development of a three-tier test to assess high school students' understanding of acids and bases. Procedia-Social and Behavioral Sciences, 15, 600–604. https://doi.org/10.1016/j.sbspro.2011.03.147
Crocker, L. (2012). Introduction to measurement theory. In J. Green, G. Camilli & P. Elmore (Eds.), Handbook of complementary methods in education research (pp. 371–384). Routledge. https://doi.org/10.4324/9780203874769
Crumb, L. N. (1983). The classification of biographical dictionaries in reference collections using the library of congress classification system. Cataloging & Classification Quarterly, 3(1), 41-44. https://doi.org/10.1300/J104v03n01_03
Fariyani, Q. (2015). Pengembangan four-tier diagnostic test untuk mengungkap miskonsepsi fisika siswa sma kelas X [Development of four-tier diagnostic test to reveal the physics misconceptions of class X high school students]. Journal of Innovative Science Education, 4(2), 41–49. https://l24.im/oACm4
Ghozali, I., & Fuad, H.. (2005). Structural equation modeling: Teori, konsep, dan aplikasi dengan program LISREL 8.54 [Structural equation modeling: Theory, concepts, and applications with programs LISREL 8.54]. Badan Penerbit Unversitas Diponegoro [Diponegoro University Publishing Agency].
Gierl, M. J. (2007). Making diagnostic inferences about cognitive attributes using the rule-space model and attribute hierarchy method. Journal of Educational Measurement, 44(4), 325–340. https://doi.org/10.1111/j.1745-3984.2007.00042.x
Grigorovitch, A. (2014). Children's misconceptions and conceptual change in Physics Education: The concept of light. Journal of Advances in Natural Sciences, 1(1), 34–39. https://doi.org/10.24297/jns.v1i1.5037
Gurel, D., Eryilmaz, A., & McDermott, L. C. (2015). A review and comparison of diagnostic instruments to identify students' misconceptions in science. Eurasia Journal of Mathematics Science and Technology Education, 11(5), 989-1008. https://doi.org/10.12973/eurasia.2015.1369a
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Sage.
Holme, P. (2015). Modern temporal network theory: A colloquium. European Physical Journal B, 88, Article 234. https://doi.org/10.1140/epjb/e2015-60657-4
Huda, C., Sulisworo, D., & Toifur, M. (2017). Analisis buku ajar termodinamika dengan konsep technological pedagogical and content knowledge (TPACK) untuk Penguatan kompetensi belajar mahasiswa [Analysis of thermodynamics textbooks with technological pedagogical concepts and content knowledge (TPACK) for strengthening student learning competencies]. Jurnal Penelitian Pembelajaran Fisika, 8(1), 1–7. https://doi.org/10.26877/jp2f.v8i1.1330
Istiyono, E., Mardapi, D., & Suparno, S. (2014). Pengembangan tes kemampuan berpikir tingkat tinggi fisika (pysthots) peserta didik SMA [Development of higher order thinking physics (pyshots) tests for high school students]. Jurnal Penelitian dan Evaluasi Pendidikan, 18(1), 1–12. https://doi.org/10.21831/pep.v18i1.2120
Kandil İngeç, Ş. (2009). Analyzing concept maps as an assessment tool in teaching physics and comparison with the achievement tests. International Journal of Science Education, 31(14), 1897–1915. https://doi.org/10.1080/09500690802275820
Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview. Theory into Practice, 41(4), 212–218. https://doi.org/10.1207/s15430421tip4104_2
Lane, S., Raymond, M. R., & Haladyna, T. M. (Eds.). (2016). Handbook of test development (2nd ed.). Routledge.
Lissitz, R. W., & Samuelsen, K. (2007). Further clarification regarding validity and education. Educational Researcher, 36(8), 482–484. https://doi.org/10.3102/0013189x07311612
Mardapi, D. (2008). Teknik penyusunan instrumen tes dan nontes [Techniques for preparing test and non-test instruments]. Mitra Cendekia.
Mardapi, D. (2017). Pengukuran penilaian dan evaluasi pendidikan [Measurement of educational assessment and evaluation] (2nd ed.). Parama Publishing.
Mayers, A. M., Khoo, S. T., & Svartberg, M. (2002). The Existential Loneliness Questionnaire: Background, development, and preliminary findings. Journal of Clinical Psychology, 58(9), 1183–1193. https://doi.org/10.1002/jclp.10038
Miller, M. D., Linn, R. L., & Gronlund, N. E. (2009). Measurement and assessment in teaching (10th ed.). Pearson.
Prahani, B. K., Amiruddin, M. Z., Suprapto, N., Deta, U. A., & Cheng, T. H. (2022). The trend of physics education research during COVID-19 pandemic. International Journal of Educational Methodology, 8(3), 517-533. https://doi.org/10.12973/ijem.8.3.517
Pratama, G. S., & Retnawati, H. (2018). Urgency of higher order thinking skills (HOTS) content analysis in mathematics textbook. Journal of Physics: Conference Series, 1097, Article 12147. https://doi.org/10.1088/1742-6596/1097/1/012147
Putranta, H., & Supahar, S.. (2019). Development of Physics-Tier Tests (PysTT) to measure students' conceptual understanding and creative thinking skills: A qualitative synthesis. Journal for the Education of Gifted Young Scientists, 7(3), 747–775. https://doi.org/10.17478/jegys.587203
Ratnaningdyah, D. (2018). Mengungkap miskonsepsi fisika dengan metode the three-tier test [Revealing physics misconceptions with the three-tier test method]. In Prosiding seminar nasional program pascasarjana universitas PGRI Palembang [Proceedings of the national seminar on the postgraduate program Palembang PGRI university] (pp. 533–540). Palembang University.
Retnawati, H. (2014). Teori respons butir dan penerapannya [Item response theory and its application] (1st ed.). Nuha Medika.
Retnawati, H. (2016a). Analisis kuantitatif instrumen penelitian (panduan peneliti, mahasiswa, dan psikometrian) [Quantitative analysis of research instruments (researcher, student, and psychometrical guide)]. Parama Publishing.
Retnawati, H. (2016b). Validitas reliabilitas dan karakteristik butir [The validity of the reliability and characteristics of the item]. Parama Publishing.
Roberts, B. W., Walton, K. E., & Viechtbauer, W. (2006). Patterns of mean-level change in personality traits across the life course: A meta-analysis of longitudinal studies. Psychological Bulletin, 132(1), 1–25. https://doi.org/10.1037/0033-2909.132.1.1
Rosyid, Jatmiko, B., & Supardi, Z. A. I. (2013). Meningkatkan hasil belajar fisika menggunakan model orientasi IPA (PBL berbasis multirepresentasi) pada konsep mekanika di SMA [Improving physics learning outcomes using a science orientation model (multi-representation-based PBL) on the concept of mechanics in high school]. Jurnal Pendidikan Fisika, 2(3), 1–12. https://l24.im/6VmLE
Saepuzaman, D., & Karim, S. (2016). Desain pembelajaran student’s conceptual construction guider berdasarkan kesulitan mahasiswa calon guru fisika pada konsep gerak parabola [Student's conceptual construction guider learning design based on difficulty of prospective physics teacher students on parabolic motion concept]. Jurnal Penelitian & Pengembangan Pendidikan Fisika, 2(2), 79-86. https://doi.org/10.21009/1.02211
Saepuzaman, D., Retnawati, H., & Istiyono, E. (2021). Can innovative learning affect student HOTS achievements?: A meta-analysis study. Pegem Journal of Education and Instruction, 11(4), 290–305. https://doi.org/10.47750/pegegog.11.04.28
Saepuzaman, D., Utari, S., & Nugraha, M. G. (2019). Development of basic physics experiment based on science process skills (SPS) to improve conceptual understanding of the preservice physics teachers on Boyle's law. Journal of Physics: Conference Series, 1280, Article 052076. https://doi.org/10.1088/1742-6596/1280/5/052076
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. https://doi.org/10.1037/0033-2909.86.2.420
Silung, S. N. W., Kusairi, S., & Zulaikah, S. (2016). Diagnosis miskonsepsi siswa sma di kota malang pada konsep suhu dan kalor menggunakan three tier test [Diagnosis of misconceptions of high school students in malang city on the concept of temperature and heat using the three tier test]. Jurnal Pendidikan Fisika dan Teknologi, 2(3), 95-105. https://doi.org/10.29303/jpft.v2i3.295
Subali, B., & Suyata, P. (2012). Pengembangan item tes konvergen dan divergen dan penyelidikan validitasnya secara empiris [Development of convergent and divergent test items and investigation of their empirical validity]. Diandra Pustaka Indonesia.
Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan RASCH pada assessment pendidikan [Application of RASCH modeling in educational assessment]. Trim komunikata Bandung
Suwarto. (2013). Belajar tuntas, miskonsepsi, dan kesulitan belajar [Complete learning, misconceptions, and learning difficulties]. Jurnal Hasil Riset, 22(1), 85 – 95. https://l24.im/XrUQIp
Tiandho, Y. (2018). Miskonsepsi gaya gesek pada mahasiswa [Frictional force misconceptions on undergraduate student]. Jurnal Pendidikan Fisika dan Keilmuan, 4(1), 1-9. https://doi.org/10.25273/jpfk.v4i1.1814
Tsui, C. Y., & Treagust, D. (2010). Evaluating secondary students’ scientific reasoning in genetics using a two-tier diagnostic instrument. International Journal of Science Education, 32(8), 1073-1098. https://doi.org/10.1080/09500690902951429
Yang, T. C., Chen, S. Y., & Hwang, G. J. (2015). The influences of a two-tier test strategy on student learning: A lag sequential analysis approach. Computers and Education, 82, 366–377. https://doi.org/10.1016/j.compedu.2014.11.021
Yanuike, A. W., Setyarsih, W., & Kholiq, A. (2017). Penggunaan Phet simulation dalam ecirr untuk mereduksi miskonsepsi siswa pada materi fluida dinamis [The use of Phet simulation in ecirr to reduce students' misconceptions on dynamic fluid materials]. Inovasi Pendidikan Fisika, 5(3), 161–164. https://l24.im/nl0b
Zukhruf, K. D., Khaldun, I., & Ilyas, S. (2017). Remediasi miskonsepsi dengan menggunakan media pembelajaran interaktif pada materi fluida statis [Remediation of misconceptions by using interactive learning media on static fluid materials]. Indonesian Journal of Science Education/Jurnal Pendidikan Sains Indonesia, 4(1), 56–68. https://l24.im/mFY
Zulfikar, A., Saepuzaman, D., Novia, H., Setyadin, A. H., Jubaedah, D. S., Sholihat, F. N., Muhaemin, M. H., Afif, N. F., Fratiwi, N. J., Bhakti, S. S., Amalia, S. A., Hidayat, S. R., Nursani, Z., Hermita, N., Costu, B., & Samsudin, A. (2019). Reducing eleventh-grade students' misconceptions on gravity concept using PDEODE∗E-based conceptual change model. Journal of Physics: Conference Series, 1204, Article 012026. https://doi.org/10.1088/1742-6596/1204/1/012026