Developing Students' Critical Thinking: Examining the Influence of Learning Management Approaches Through Meta-Analysis and Propensity Score Matching
Critical thinking is a skill that enables individuals to keep pace with changes and enhances crucial competencies for contemporary competitiveness. Ma.
- Pub. date: July 15, 2024
- Online Pub. date: May 01, 2024
- Pages: 1391-1409
- 282 Downloads
- 1378 Views
- 0 Citations
- #Critical thinking
- # learning management approaches
- # meta-analysis
- # propensity score matching
- # research synthesis.
Critical thinking is a skill that enables individuals to keep pace with changes and enhances crucial competencies for contemporary competitiveness. Many researchers have studied learning management approaches to develop students' critical thinking, resulting in a substantial body of knowledge but lacking clear systematic summaries. The researchers aimed to (a) examine the effect sizes and research characteristics influencing students' critical thinking, and (b) compare the effect sizes of learning management approaches after adjusting with propensity score matching from 108 graduate research published between 2002 and 2021. Data were collected using research characteristics recording forms and research quality assessment questionnaires. Effect sizes were calculated using Glass's method and analyzed through random effect, fixed effect, and regression meta-analysis. Findings revealed that (a) research on developing learning management approaches influences students' critical thinking at a high level (d ̅ = 1.669), with nine research characteristics, including the field of publication, courses, total duration, teacher learning process, learning media, measurement and evaluation, research design, research statistics, and research quality, statistically significantly influencing students' critical thinking, and (b) after adjustment, inquiry-based learning significantly influences students' critical thinking. Recommendations for developing students' critical thinking include learning activities that encourage problem exploration, expanding thinking through collaborative analysis, and applying diverse media and activity sheets tailored to context suitability.
critical thinking learning management approaches meta analysis propensity score matching research synthesis
Keywords: Critical thinking, learning management approaches, meta-analysis, propensity score matching, research synthesis.
References
Abrami, P. C., Bernard, R. M., Borokhovski, E., Waddington, D. I., Wade, C. A., & Persson, T. (2015). Strategies for teaching students to think critically: A meta-analysis. Review of Educational Research, 85(2), 275-314. https://doi.org/10.3102/0034654314551063
Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M. A., Tamim, R., & Zhang, D. (2008). Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis. Review of Educational Research, 78(4), 1102-1134. https://doi.org/10.3102/0034654308326084
Adesope, O. O., Trevisan, D. A., & Sundararajan, N. (2017). Rethinking the use of tests: A meta-analysis of practice testing. Review of Educational Research, 87(3), 659-701. https://doi.org/10.3102/0034654316689306
Akinbadewa, B. O., & Sofowora, O. A. (2020). The effectiveness of multimedia instructional learning packages in enhancing secondary school students’ attitudes toward Biology. International Journal on Studies in Education, 2(2), 119-133. https://doi.org/10.46328/ijonse.19
Allen, M. (2020). Understanding the practice, application, and limitations of meta-analysis. American Behavioral Scientist, 64(1), 74-96. https://doi.org/10.1177/0002764219859619
Alsaleh, N. J. (2020). Teaching critical thinking skills: Literature review. Turkish Online Journal of Educational Technology, 19(1), 21-39. https://bit.ly/49QkEcI
Amineh, R. J., & Asl, H. D. (2015). Review of constructivism and social constructivism. Journal of Social Sciences, Literature and Languages, 1(1), 9-16. http://bit.ly/3v23J8v
Austin, P. C. (2009). The relative ability of different propensity score methods to balance measured covariates between treated and untreated subjects in observational studies. Medical Decision Making, 29(6), 661-677. https://doi.org/10.1177/0272989X09341755
Badhiwala, J. H., Karmur, B. S., & Wilson, J. R. (2021). Propensity score matching: A powerful tool for analyzing observational nonrandomized data. Clinical Spine Surgery, 34(1), 22-24. https://doi.org/10.1097/BSD.0000000000001055
Bai, H. (2011). A comparison of propensity score matching methods for reducing selection bias. International Journal of Research and Method in Education, 34(1), 81-107. https://doi.org/10.1080/1743727X.2011.552338
Balta, N., & Sarac, H. (2016). The effect of 7e learning cycle on learning in science teaching: a meta-analysis study. European Journal of Educational Research, 5(2), 61-72. https://doi.org/10.12973/eu-jer.5.2.61
Benedetto, U., Head, S. J., Angelini, G. D., & Blackstone, E. H. (2018). Statistical primer: Propensity score matching and its alternatives. European Journal of Cardio-Thoracic Surgery, 53(6), 1112-1117. https://doi.org/10.1093/ejcts/ezy167
Berkhout, S. W., Haaf, J. M., Gronau, Q. F., Heck, D. W., & Wagenmakers, E.-J. (2024). A tutorial on Bayesian model-averaged meta-analysis in JASP. Behavior Research Methods, 56, 1260-1282. https://doi.org/10.3758/s13428-023-02093-6
Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (Eds.). (2021). Introduction to meta-analysis. John Wiley and Sons. https://bit.ly/3SGZV4M
Borenstein, M., & Higgins, J. P. T. (2013). Meta-analysis and subgroups. Prevention Science, 14, 134-143. https://doi.org/10.1007/s11121-013-0377-7
Card, N. A. (2012). Applied meta-analysis for social science research. The Guilford Press. https://bit.ly/3HWcMuP
Chu, S. K. W., Reynolds, R. B., Tavares, N. J., Notari, M., & Lee, C. W. Y. (2017). 21st century skills development through inquiry-based learning: From theory to practice. Springer. https://doi.org/10.1007/978-981-10-2481-8
Cooper, H., Hedges, L. V., & Valentine, J. C. (2019). The handbook of research synthesis and meta-analysis (3rd ed.). Russell Sage Foundation. https://doi.org/10.7758/9781610448864
Czodrowski, P. (2014). Count on kappa. Journal of Computer-Aided Molecular Design, 28, 1049-1055. https://doi.org/10.1007/s10822-014-9759-6
Donoghue, G. M., & Hattie, J. A. C. (2021). A meta-analysis of ten learning techniques. Frontiers in Education, 6, Article 581216. https://doi.org/10.3389/feduc.2021.581216
Dowdy, A., Tincani, M., & Schneider, W. J. (2020). Evaluation of publication bias in response interruption and redirection: A meta‐analysis. Journal of Applied Behavior Analysis, 53(4), 2151-2171. https://doi.org/10.1002/jaba.724
Dwyer, C. P., Hogan, M. J., & Stewart, I. (2014). An integrated critical thinking framework for the 21st century. Thinking skills and Creativity, 12, 43-52. https://doi.org/10.1016/j.tsc.2013.12.004
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315, 629-634. https://doi.org/10.1136/bmj.315.7109.629
Eisenkraft, A. (2003). Expanding the 5e model. The Science Teacher, 70(6), 56-59. https://bit.ly/3GLas9A
Elder, L., & Paul, R. (2021). Critical thinking: Tools for taking charge of your learning and your life (4th ed.). Rowman & Littlefield.
Ennis, R. H. (1987). A taxonomy of critical thinking dispositions and abilities. In J. B. Baron & R. J. Sternberg (Eds.), Teaching thinking skills: Theory and practice (pp. 9-26). W H Freeman/Times Books/ Henry Holt & Co.
Ennis, R. H. (1993). Critical thinking assessment. Theory Into Practice, 32(3), 179-186. https://doi.org/10.1080/00405849309543594
Ennis, R. H. (2015). Critical thinking: A streamlined conception. In M. Davies, & R. Barnett, (Eds.), The Palgrave handbook of critical thinking in higher education (pp. 31-47). Palgrave Macmillan. https://doi.org/10.1057/9781137378057_2
Ennis, R. H. (2018). Critical thinking across the curriculum: A vision. Topoi, 37, 165-184. https://doi.org/10.1007/s11245-016-9401-4
Esterhuizen, T. M., & Thabane, L. (2016). Con: Meta-analysis: some key limitations and potential solutions. Nephrology Dialysis Transplantation, 31(6), 882-885. https://doi.org/10.1093/ndt/gfw092
Facione, P. A. (2023). Critical thinking: What it is and why it counts. Insight Assessment. https://bit.ly/44g6a4L
Ferguson, C. J., & Brannick, M. T. (2012). Publication bias in psychological science: Prevalence, methods for identifying and controlling, and implications for the use of meta-analyses. Psychological Methods, 17(1), 120-128. https://doi.org/10.1037/a0024445
Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5(10), 3-8. https://doi.org/10.3102/0013189X005010003
Green, P. (2015). Teaching critical thinking for lifelong learning. In M. Davies, & R. Barnett (Eds.), The Palgrave handbook of critical thinking in higher education (pp. 107-121). Palgrave Macmillan. https://doi.org/10.1057/9781137378057_7
Harris, H., & Horst, S. J. (2016). A brief guide to decisions at each step of the propensity score matching process, Practical Assessment, Research, and Evaluation, 21(4), 1-11. https://doi.org/10.7275/yq7r-4820
Harrison, F. (2011). Getting started with meta‐analysis. Methods in Ecology and Evolution, 2(1), 1-10. https://doi.org/10.1111/j.2041-210X.2010.00056.x
Haukoos, J. S., & Lewis, R. J. (2015). The propensity score. JAMA, 314(15), 1637-1638. https://doi.org/10.1001/jama.2015.13480
Heinrich, C., Maffioli, A., & Vazquez, G. (2010). A primer for applying propensity-score matching. Inter-American Development Bank. https://bit.ly/3HXEuHw
Higgins, J. P., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta‐analysis. Statistics in Medicine, 21(11), 1539-1558. https://doi.org/10.1002/sim.1186
Hitchcock, D. (2017). Critical thinking as an educational ideal. In On reasoning and argument. (Vol. 30, pp. 477-497). Springer. https://doi.org/10.1007/978-3-319-53562-3_30
Indah, R. N., & Kusuma, A. W. (2016). Factors affecting the development of critical thinking of Indonesian learners of English language. Journal of Humanities and Social Science, 21(6), 86-94. https://bit.ly/4aoSkQg
Itsarangkul Na Ayutthaya, T., & Damrongpanit, S. (2022a). A meta-analysis of instructional management models affecting creative thinking development. European Journal of Educational Research, 11(4), 2069-2085. https://doi.org/10.12973/eu-jer.11.4.2069
Itsarangkul Na Ayutthaya, T., & Damrongpanit, S. (2022b). A meta-analysis of instructional management model for students' creative thinking development: An application of propensity score matching. European Journal of Educational Research, 11(4), 2429-2444. https://doi.org/10.12973/eu-jer.11.4.2429
Jacobs, G. M., & Renandya, W. A. (2019). Student centered cooperative learning: Linking concepts in education to promote student learning. Springer. https://doi.org/10.1007/978-981-13-7213-1
Kane, L. T., Fang, T., Galetta, M. S., Goyal, D. K. C., Nicholson, K. J., Kepler, C. K., Vaccaro, A. R., & Schroeder, G. D. (2020). Propensity score matching: a statistical method. Clinical Spine Surgery, 33(3), 120-122. https://doi.org/10.1097/BSD.0000000000000932
Karampelas, K. (2023). Critical thinking in national primary science curricula. Eurasian Journal of Science and Environmental Education, 3(2), 51-60. https://doi.org/10.30935/ejsee/13271
Kennedy, T. J., & Sundberg, C. W. (2020). 21st century skills. In B. Akpan, & T. J. Kennedy (Eds.), Science education in theory and practice (pp. 479-496). Springer. https://doi.org/10.1007/978-3-030-43620-9_32
Kocak, O., Coban, M., Aydin, A., & Cakmak, N. (2021). The mediating role of critical thinking and cooperativity in the 21st century skills of higher education students. Thinking Skills and Creativity, 42, Article 100967. https://doi.org/10.1016/j.tsc.2021.100967
Lai, E. R. (2011). Critical thinking: A literature review. Research Reports. https://bit.ly/42zDaUG
Lambert, C. T., & Guillette, L. M. (2021). The impact of environmental and social factors on learning abilities: a meta‐analysis. Biological Reviews, 96(6), 2871-2889. https://doi.org/10.1111/brv.12783
Lampert, M., & Graziani, F. (2009). Instructional activities as a tool for teachers' and teacher educators' learning. The Elementary School Journal, 109(5), 491-509. https://doi.org/10.1086/596998
Lee, W.-S. (2013). Propensity score matching and variations on the balancing test. Empirical Economics, 44, 47-80. https://doi.org/10.1007/s00181-011-0481-0
Lee, Y. H. (2019). Strengths and limitations of meta-analysis. The Korean Journal of Medicine, 94(5), 391-395. https://doi.org/10.3904/kjm.2019.94.5.391
Mahapoonyanont, N. (2010). Factors related to critical thinking abilities; a meta-analysis. Procedia Social and Behavioral Sciences, 9, 986-990. https://doi.org/10.1016/j.sbspro.2010.12.272
McPeck, J. E. (Ed.). (2016). Critical thinking and education. Routledge. https://doi.org/10.4324/9781315463698
Moallem, M., Hung, W., & Dabbagh, N. (Eds.). (2019). The Wiley handbook of problem-based learning. John Wiley & Sons Inc. https://doi.org/10.1002/9781119173243
Morgan, C. J. (2018). Reducing bias using propensity score matching. Journal of Nuclear Cardiology, 25(2), 404-406. https://doi.org/10.1007/s12350-017-1012-y
Moust, J., Bouhuijs, P., & Schmidt, H. (2021). Introduction to problem-based learning (4th ed.). Routledge. https://doi.org/10.4324/9781003194187
Muthma’Innah, M., Dahlan, J. A., & Suhendra, S. (2019). Ability of mathematical critical thinking–what about learning cycle 7e model? Journal of Physics: Conference Series, 1157(3), Article 032129. https://doi.org/10.1088/1742-6596/1157/3/032129
Nicol, C., Yakuba, E., & Habinshuti, G. (2020). An overview of learning cycles in science inquiry-based instruction. African Journal of Educational Studies in Mathematics and Sciences, 16(2), 67-81. https://doi.org/10.4314/ajesms.v16i2.5
Niu, L., Behar-Horenstein, L. S., & Garvan, C. W. (2013). Do instructional interventions influence college students’ critical thinking skills? A meta-analysis. Educational Research Review, 9, 114-128. https://doi.org/10.1016/j.edurev.2012.12.002
Oliveira, L. B., Díaz, L. J., Carbogim, F. C., Rodrigues, A. R. B., & Püschel, V. A. A. (2016). Efetividade das estratégias de ensino no desenvolvimento do pensamento crítico de graduandos de Enfermagem: uma metanálise [Effectiveness of teaching strategies on the development of critical thinking in undergraduate nursing students: a meta-analysis]. Journal of School of Nursing – University of São Paulo/Revista da Escola de Enfermagem da USP, 50(2), 355-364. https://doi.org/10.1590/S0080-623420160000200023
O'Reilly, C., Devitt, A., & Hayes, N. (2022). Critical thinking in the preschool classroom-a systematic literature review. Thinking Skills and Creativity, 46, Article 101110. https://doi.org/10.1016/j.tsc.2022.101110
Paul, R., & Elder, L. (2005). A guide for educators to critical thinking competency standards. Foundation for Critical Thinking. https://bit.ly/48gtYFY
Paul, R., & Elder, L. (2020). The miniature guide to critical thinking concepts and tools. Rowman and Littlefield. https://bit.ly/3OHHBHr
Rannikmäe, M., Holbrook, J., & Soobard, R. (2020). Social constructivism—Jerome Bruner. In B. Akpan, & T. J. Kennedy (Eds.), Science education in theory and practice (pp. 259-275). Springer. https://doi.org/10.1007/978-3-030-43620-9_18
Reiffel, J. A. (2020). Propensity score matching: the ‘devil is in the details’ where more may be hidden than you know. American Journal of Medicine, 133(2), 178-181. https://doi.org/10.1016/j.amjmed.2019.08.055
Ripley, D. (2015). An examination of flipped instructional method on sixth graders’ mathematics learning: Utilizing propensity score matching. University of Nevada. https://bit.ly/49cwezm
Rosenbaum, P. R. (2020). Modern algorithms for matching in observational studies. Annual Review of Statistics and Its Application, 7, 143-176. https://doi.org/10.1146/annurev-statistics-031219-041058
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41-55. https://doi.org/10.1093/biomet/70.1.41
Rothstein, H. R. (2008). Publication bias as a threat to the validity of meta-analytic results. Journal of Experimental Criminology, 4, 61-81. https://doi.org/10.1007/s11292-007-9046-9
Rubin, D. B. (1997). Estimating causal effects from large data sets using propensity scores. Annals of Internal Medicine, 127(8-2), 757-763. https://doi.org/29d
Rücker, G., Schwarzer, G., Carpenter, J. R., & Schumacher, M. (2008). Undue reliance on /2 in assessing heterogeneity may mislead. BMC Medical Research Methodology, 8, Article 79. https://doi.org/10.1186/1471-2288-8-79
Sainani, K. L. (2012). Propensity scores: uses and limitations. PM&R, 4(9), 693-697. https://doi.org/10.1016/j.pmrj.2012.07.002
Saiz, C., & Rivas, S. F. (2016). New teaching techniques to improve critical thinking. The Diaprove methodology. Educational Research Quarterly, 40(1), 3-36.
Saylor, J. G., Alexander, W. M., & Lewis, A. J. (1981). Curriculum planning for better teaching and learning (4th ed.). Holt, Rinehart and Winston.
Sedgwick, P., & Marston, L. (2015). How to read a funnel plot in a meta-analysis. BMJ, 351, Article h4718. https://doi.org/10.1136/bmj.h4718
Slavin, R. E. (2014). Aprendizaje cooperativo y rendimiento académico: ¿por qué funciona el trabajo en grupo? [Cooperative learning and academic achievement: Why does groupwork work?]. Annals of Psychology/Anales De Psicología, 30(3), 785-791. https://doi.org/10.6018/analesps.30.3.201201
Staffa, S. J., & Zurakowski, D. (2018). Five steps to successfully implement and evaluate propensity score matching in clinical research studies. Anesthesia and Analgesia, 127(4), 1066-1073. https://doi.org/10.1213/ANE.0000000000002787
StataCorp. (2023). Stata 18 meta-analysis reference manual: Release 18. Stata Press. https://www.stata.com/manuals/meta.pdf
Sternberg, R. J. (1986). Critical thinking: Its nature, measurement, and improvement. National Inst. of Education.
Stone, D. L., & Rosopa, P. J. (2017). The advantages and limitations of using meta-analysis in human resource management research. Human Resource Management Review, 27(1), 1-7. https://doi.org/10.1016/j.hrmr.2016.09.001
Suardana, I. N., Redhana, I. W., Sudiatmika, A. A. I. A. R., & Selamat, I. N. (2018). Students’ critical thinking skills in chemistry learning using local culture-based 7e learning cycle model. International Journal of Instruction, 11(2), 399-412. https://doi.org/10.12973/iji.2018.11227a
Tang, J.-L., & Liu, J. L. (2000). Misleading funnel plot for detection of bias in meta-analysis. Journal of Clinical Epidemiology, 53(5), 477-484. https://doi.org/10.1016/S0895-4356(99)00204-8
Thongnuypram, C., & Sopheerak, S. (2013). Factor influencing the critical thinking of teacher students studying at the faculty of education in Suratthani Rajabhat University. Procedia-Social and Behavioral Sciences, 103, 386-391. https://doi.org/10.1016/j.sbspro.2013.10.349
Thornton, A., & Lee, P. (2000). Publication bias in meta-analysis: its causes and consequences. Journal of Clinical Epidemiology, 53(2), 207-216. https://doi.org/10.1016/S0895-4356(99)00161-4
Van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002). Advanced methods in meta‐analysis: multivariate approach and meta‐regression. Statistics in Medicine, 21(4), 589-624. https://doi.org/10.1002/sim.1040
Varoglu, L., Yilmaz, A., & Sen, S. (2023). Effect of 5E learning cycle assisted with concept maps on conceptual understanding. Pedagogical Research, 8(3), Article em0161. https://doi.org/10.29333/pr/13167
Vevea, J. L., Coburn, K. M., & Sutton, A. J. (2019). Publication bias. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (3rd ed., pp. 383-429). Russell Sage Foundation, https://doi.org/10.7758/9781610448864.21
Xu, E., Wang, W., & Wang, Q. (2023). The effectiveness of collaborative problem solving in promoting students’ critical thinking: A meta-analysis based on empirical literature. Humanities and Social Sciences Communications, 10, Article 16. https://doi.org/10.1057/s41599-023-01508-1
Zhao, C., Pandian, A., & Singh, M. K. M. (2016). Instructional strategies for developing critical thinking in EFL classrooms. English Language Teaching, 9(10), 14-21. https://doi.org/10.5539/elt.v9n10p14