Psychometric and Structural Evaluation of the Physics Metacognition Inventory Instrument
Haeruddin Haeruddin
,
Zuhdan Kun Prasetyo
,
Supahar Supahar
,
Elisa Sesa
,
Gazali Lembah
The purpose of this study is to evaluate the psychometric and structural instruments of the Physics Metacognition Inventory (PMI) developed by Taasoob.
- Pub. date: January 15, 2020
- Pages: 215-225
- 744 Downloads
- 1136 Views
- 6 Citations
The purpose of this study is to evaluate the psychometric and structural instruments of the Physics Metacognition Inventory (PMI) developed by Taasoobshirazi, Bailey, and Farley (2015). The PMI consists of 26 items in six factors. The English and Indonesian versions were tested on an English course (N = 37) in the Geophysics study program at Tadulako University. The trials were conducted separately within a two-week interval. The data collected from 364 students of the Physics Education Department, University of Tadulako were analyzed using the Exploratory Factor Analysis (EFA). Later, data were collected from 351 students of some Indonesian universities which have physics education study programs, and the data were analyzed using the Confirmatory Factor Analysis (CFA). The EFA result reveals six factors based on the rotation result with the maximum loading factor. The CFA result shows the RMSEA values of .018, 2 (284) = 316.32 (χ2 / df = 1,11), GFI = .93, CFI = .99, AGFI = .92 and NFI = .93 which meet the cut-off statistic value, and therefore, the model is considered fit, with the Construct Reliability Estimation (CR) of .93, Composite Reliability of = .95, and maximum reliability of Ω = .96. The results obtained reveal that the PMI scale has good, valid and reliable psychometric properties. Therefore, PMI can be used to measure the level of metacognition of students when solving physics problems. Future studies using PMI are also discussed.
Keywords: Psychometric evaluation, physics metacognition inventory, problem solving.
References
Akben, N. (2018). Effects of the Problem-Posing approach on students’ problem solving skills and metacognitive awareness in science education. Research in Science Education, 48(1), 1–23. https://doi.org/10.1007/s11165-018-9726-7
Balta, N., Mason, A. J., & Singh, C. (2016). Surveying Turkish high school and university students’ attitudes and approaches to physics problem solving. Physical Review Physics Education Research, 12(1), 010129. https://doi.org/10.1103/PhysRevPhysEducRes.12.010129
Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 3186–3191. https://doi.org/10.1097/00007632-200012150-00014
Bowen, N. K., & Guo, S. (2013). Structural equation modeling (Vol. 15). New York, NY: Oxford University Press.
Brown, A. L. (1978). Knowing when, where, and how to remember: A problem of metacognition. Advances in Instructional Psychology, 1(1), 225–253.
Chi, M. T. H. (2006). Two approaches to the study of experts’ characteristics. In K. A. Ericsson, N. Charness, P. J. Feltovich, & R. R. Hoffman (Eds.), The Cambridge Handbook of Expertise and Expert Performance (pp. 21–30). New York, NY: Cambridge University Press.
Colthorpe, K., Sharifirad, T., Ainscough, L., Anderson, S., & Zimbardi, K. (2018). Prompting undergraduate students’ metacognition of learning: implementing ‘meta-learning’ assessment tasks in the biomedical sciences. Assessment & Evaluation in Higher Education, 43(2), 272–285. https://doi.org/10.1080/02602938.2017.1334872
Dinsmore, D. L., Alexander, P. A., & Loughlin, S. M. (2008). Focusing the conceptual lens on metacognition, self-regulation, and self-regulated learning. Educational Psychology Review, 20(4), 391–409. https://doi.org/10.1007/s10648-008-9083-6
Flavell, J. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist, 34(10), 906–911. https://doi.org/10.1037/0003-066x.34.10.906
Geldhof, G. J., Preacher, K. J., & Zyphur, M. J. (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19(1), 72–91. https://doi.org/10.1037/a0032138
Ghanizadeh, A. (2017). The interplay between reflective thinking, critical thinking, self-monitoring, and academic achievement in higher education. Higher Education, 74(1), 101–114. https://doi.org/10.1007/s10734-016-0031-y
Haeruddin, Prasetyo, Z. K., & Supahar. (2019). The Development of a Metacognition Instrument for College Students to Solve Physics Problems. International Journal of Instruction, 13(1), 1–16. Retrieved from http://www.e-iji.net/volumes/359-onlinefirst
Hair Jr, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2014). Multivariate data analysis (7th ed.). Harlow, United Kingdom: Pearson Education Limited.
Harrison, G. M., & Vallin, L. M. (2017). Evaluating the metacognitive awareness inventory using empirical factor-structure evidence. Metacognition and Learning, 13(1), 15–38. https://doi.org/10.1007/s11409-017-9176-z
Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16(3), 235–266. https://doi.org/10.1023/B:EDPR.0000034022.16470.f3
Hoyle, R. H. (2012). Handbook of structural equation modeling. New York, NY: The Guilford Press.
Jacobs, J. E., & Paris, S. G. (1987). Children’s metacognition about reading: issues in definition, measurement, and instruction. Educational Psychologist, 22(3), 255–278. https://doi.org/10.1080/00461520.1987.9653052
Jacobse, A. E., & Harskamp, E. G. (2012). Towards efficient measurement of metacognition in mathematical problem solving. Metacognition and Learning, 7(2), 133–149. https://doi.org/10.1007/s11409-012-9088-x
Jahangard, Z., Soltani, A., & Alinejad, M. (2016). Exploring the relationship between metacognition and attitudes towards science of senior secondary students through a structural equation modeling analysis. Journal of Baltic Science Education, 15(3), 340–349.
Joreskog, K. G., & Sorbom, D. (1993). LISREL 8 : Structural Equation Modeling With the SIMPLIS Command Language. Uppsala, Sweden: Scientific Software International, Inc.
Kamata, A., Turhan, A., & Darandari, E. (2013, April). Estimating reliability for multidimensional composite scale scores. Paper presented at the annual meeting of American Educational Research Association, Florida State University, Chicago, USA.
Kipnis, M., & Hofstein, A. (2008). The inquiry laboratory as a source for development of metacognitive skills. International Journal of Science and Mathematics Education, 6(3), 601–627. https://doi.org/10.1007/s10763-007-9066-y
Koch, A. (2001). Training in metacognition and comprehension of physics texts. Science Education, 85(6), 758–768. https://doi.org/10.1002/sce.1037
Kryjevskaia, M., Stetzer, M. R., & Grosz, N. (2014). Answer first: Applying the heuristic-analytic theory of reasoning to examine student intuitive thinking in the context of physics. Physical Review Special Topics - Physics Education Research, 10(2), 020109. https://doi.org/10.1103/PhysRevSTPER.10.020109
Mansyur, J., Lestari, W., Werdhiana, I. K., & Rizal, M. (2018). Students’ Metacognition Skills in Physics Problem Solving Based on Epistemological Beliefs. In D. K. Walanda, M. Basir, N. Mery, K. Manurung, Lukman, J. Mansyur, … & Syamsuriwal (Eds.), First Indonesian Communication Forum of Teacher Training and Education Faculty Leaders International Conference on Education 2017 (pp. 28–31). Palu, Indonesia: Tadulako University.
Meijer, J., Sleegers, P., Elshout-Mohr, M., van Daalen-Kapteijns, M., Meeus, W., & Tempelaar, D. (2013). The development of a questionnaire on metacognition for students in higher education. Educational Research, 55(1), 31–52. https://doi.org/10.1080/00131881.2013.767024
Mundilarto. (2003). Kemampuan mahasiswa menggunakan pendekatan analitis kuantitatif dalam pemecahan soal fisika [Students’ ability to use quantitative analytical approaches in solving physics problems]. Journal of Mathematics and Science Education/Jurnal Pendidikan Matematika Dan Sains, 3(VIII), 137–142.
Ozturk, N. (2017). Assessing metacognition: Theory and practices. International Journal of Assessment Tools in Education, 4(2), 134–134. https://doi.org/10.21449/ijate.298299
Pallant, J. (2011). SPSS survival manual a step by step guide to data analysis using the SPSS program (4th ed.). Sydney, Australia: Allen & Unwin.
Pathuddin, Budayasa, I. K., & Lukito, A. (2019). Metacognitive activity of male students: Difference field independent-dependent cognitive style. Journal of Physics: Conference Series, 1218(1), 1–4. https://doi.org/10.1088/1742-6596/1218/1/012025
Patton, M. Q. (2014). Qualitative research & evaluation method (4th ed.). Los Angeles, CA: Sage Publications.
Retnawati, H. (2016). Analisis kuantitatif instrumen penelitian: Panduan peneliti, mahasiswa, dan psikometrian. [Quantitative analysis of research instruments: A guide for researchers, students and psychometrics]. Yogyakarta, Indonesia: Parama Publishing.
Sart, G. (2014). The effects of the development of metacognition on project-based learning. Procedia - Social and Behavioral Sciences, 152, 131–136. https://doi.org/10.1016/j.sbspro.2014.09.169
Schellings, G., & van Hout-Wolters, B. (2011). Measuring strategy use with self-report instruments: Theoretical and empirical considerations. Metacognition and Learning, 6(2), 83–90. https://doi.org/10.1007/s11409-011-9081-9
Schellings, G., Van Hout-Wolters, B. H. A. M., Veenman, M. V. J., & Meijer, J. (2013). Assessing metacognitive activities: The in-depth comparison of a task-specific questionnaire with think-aloud protocols. European Journal of Psychology of Education, 28(3), 963–990. https://doi.org/10.1007/s10212-012-0149-y
Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, Vol. 19, pp. 460–475. https://doi.org/10.1006/ceps.1994.1033
Schumacker, R. E., & Lomax, R. G. (2016). A begginer’s guide to structural equation modeling (4th ed.). New York, NY and London, UK: Routledge Taylor & Francis Group.
Souza, A. C. de, Alexandre, N. M. C., & Guirardello, E. de B. (2017). Propriedades psicomericas na avaliacao de instrumentos: avaliacao da confiabilidade e da validade [Psychometric properties in instrument evaluation: reliability and validity evaluation]. Epidemiology and Health Services: Brazilian Health System Journal/Epidemiologia e Servicos de Saude : Revista Do Sistema Unico de Saude Do Brasil, 26(3), 649–659. https://doi.org/10.5123/S1679-49742017000300022
Sperling, R. A., Howard, B. C., Miller, L. A., & Murphy, C. (2002). Measures of children’s knowledge and regulation of cognition. Contemporary Educational Psychology, 27(1), 51–79. https://doi.org/10.1006/ceps.2001.1091
Srinivasan, D. P., & Pushpam, M. A. M. (2016). Exploring the influence of metacognition and metaemotion strategies on the outcome of students of IX Std. American Journal of Educational Research, 4(9), 663–668. https://doi.org/10.12691/education-4-9-3
Taasoobshirazi, G., Bailey, M., & Farley, J. (2015). Physics metacognition inventory part II: Confirmatory factor analysis and rasch analysis. International Journal of Science Education, 37(17), 2769–2786. https://doi.org/10.1080/09500693.2015.1104425
Taasoobshirazi, G., & Farley, J. (2013). Construct validation of the physics metacognition inventory. International Journal of Science Education, 35(3), 447–459. https://doi.org/10.1080/09500693.2012.750433
Unlu, Z. K., & Dokme, I. (2019). Adaptation of physics metacognition inventory to Turkish. International Journal of Assessment Tools in Education, 6(1), 125–137. https://doi.org/10.21449/ijate.483104
Uyar, R. O., Yilmaz Genc, M. M., & Yasar, M. (2018). Prospective preschool teachers’ academic achievements depending on their goal orientations, critical thinking dispositions and self-regulation skills. European Journal of Educational Research, 7(3), 601–613. https://doi.org/10.12973/eu-jer.7.3.601
Veenman, M. V. J. (2011). Learning to self-monitor and self-regulate. In R. E. Mayer & P. A. Alexander (Eds.), Handbook of Research on Learning and Instruction (pp. 197–218). New York, NY: Routledge Taylor & Francis Group.
Veenman, M. V. J., Kok, R., & Blöte, A. W. (2005). The relation between intellectual and metacognitive skills in early adolescence. Instructional Science, 33(3), 193–211. https://doi.org/10.1007/s11251-004-2274-8
Widhiarso, W., & Ravand, H. (2014). Estimating reliability coefficient for multidimensional measures: A pedagogical illustration. Review of Psychology, 21(2), 111–121.
Winne, P. H., & Perry, N. E. (2000). Measuring Self-Regulated Learning. In M. Boekaerts, M. Zeidner, & P. R. Pintrich (Eds.), Handbook of Self-Regulation (pp. 531–566). New York, NY: Academic Press.
Winston, K. A., Van Der Vleuten, C. P. M., & Scherpbier, A. J. J. A. (2010). An investigation into the design and effectiveness of a mandatory cognitive skills programme for at-risk medical students. Medical Teacher, 32(3), 236–243. https://doi.org/10.3109/01421590903197035
Yuberti, Latifah, S., Anugrah, A., Saregar, A., Misbah, & Jermsittiparsert, K. (2019). Approaching problem-solving skills of momentum and impulse phenomena using context and problem-based learning. European Journal of Educational Research, 8(4), 1217–1227. https://doi.org/10.12973/eu-jer.8.4.1217