Evaluating the Results of PISA Assessment: Are There Gaps Between the Teaching of Mathematical Literacy at Schools and in PISA Assessment?
The problems in education in the countries of the Organization for Economic Cooperation and Development (OECD) vary from country to country. The diffe.
- Pub. date: July 15, 2023
- Pages: 1479-1493
- 530 Downloads
- 1289 Views
- 0 Citations
The problems in education in the countries of the Organization for Economic Cooperation and Development (OECD) vary from country to country. The differences between "upper class" and "lower class" countries in PISA assessment results have led to a research gap. The purpose of this study was to (a) test students' mathematical literacy skills on the Program for International Student Assessment (PISA) test and compare the results using the sum of means across OECD countries; (b) examine the relationship between students' mathematical competence, precision, and self-perception of mathematical literacy skills in the PISA test; and (c) analyze the gaps that exist between the implementation of mathematics instruction in school and the mathematical literacy as measured on the PISA test. This study was designed as a mixed method with an explanatory sequential design. The data collection methods included test procedures, questionnaires, and interviews. The result of this study showed that the overall mean score obtained was below the OECD average. In general, the respondents achieved only level 2 mathematics proficiency. A significant relationship was found between mathematical competence, precision, and self-perception in mathematical skills. On the other hand, there was a gap, namely the difference at the implementation level, where mathematical literacy measured by PISA differed from the measurement of mathematical learning achievement by teachers in school. The results showed that teaching that emphasizes only problem-solving procedures affects low mathematical competence and is not useful enough for students to deal with the PISA mathematics test.
Keywords: Education gaps, mathematical competence, mathematical literacy, PISA assessment.
References
Abramovich, S., Grinshpan, A. Z., & Milligan, D. L. (2019). Teaching mathematics through concept motivation and action learning. Education Research International, 2019, Article 3745406. https://doi.org/10.1155/2019/3745406
Acharya, B. R. (2017). Factors affecting difficulties in learning mathematics by mathematics learners. International Journal of Elementary Education, 6(2), 8–15. https://doi.org/10.11648/j.ijeedu.20170602.11
Adler, R. H. (2022). Trustworthiness in qualitative research. Journal of Human Lactation, 38(4), 598–602. https://doi.org/10.1177/08903344221116620
Ahyan, S., Turmudi, & Juandi, D. (2019). Mathematical literacy of ninth-grade students in solving PISA mathematics problems. International Journal of Innovation, Creativity and Change, 5(6), 483–495. https://bit.ly/3XXZk0m
Aksu, N., Aksu, G., & Saracaloğlu, S. (2022). Prediction of the factors affecting PISA mathematics literacy of students from different countries by using data mining methods. International Electronic Journal of Elementary Education, 14(5), 613–629. https://doi.org/10.26822/iejee.2022.267
Albaladejo, I. M. R., García, M. D. M., & Codina, A. (2015). Developing mathematical competencies in secondary students by introducing dynamic geometry systems in the classroom. Education and Science, 40(177), 43–58. https://doi.org/10.15390/EB.2015.2640
Almarashdi, H. S., & Jarrah, A. M. (2023). Assessing tenth-grade students’ mathematical literacy skills in solving PISA problems. Social Sciences, 12(1), Article 33. https://doi.org/10.3390/socsci12010033
Bem, D. J. (1972). Self-perception theory. Advances in Experimental Social Psychology, 6, 1–62. https://doi.org/10.1016/S0065-2601(08)60024-6
Bøyum, S. (2014). Fairness in education - a normative analysis of OECD policy documents. Journal of Education Policy, 29(6), 856–870. https://doi.org/10.1080/02680939.2014.899396
Castrol, F. G., Kellison, J. G., Boyd, S. J., & Kopak, A. (2010). A methodology for conducting integrative mixed methods research and data analyses. Journal Mix Methods Research, 4(4), 342–360. https://doi.org/10.1177/1558689810382916
Chamorro-Atalaya, O., Gamarra-Bustillos, C., Villanueva-Acosta, V., Samanamud-Loyola, O., Leva-Apaza, A., Tasayco-Jala, A., Torres-Quiroz, A., & Peralta-Eugenio, G. (2022). Self-perception on the acquisition of investigative competencies in the context of virtual learning during Covid-19. International Journal of Information and Education Technology, 12(12), 1417–1423. https://doi.org/10.18178/ijiet.2022.12.12.1766
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches. SAGE Publications.
Ehmke, T., van den Ham, A.-K., Sälzer, C., Heine, J., & Prenzel, M. (2020). Measuring mathematics competence in international and national large scale assessments: Linking PISA and the national educational panel study in Germany. Studies in Educational Evaluation, 65, Article 100847. https://doi.org/10.1016/j.stueduc.2020.100847
Gurkaynak, G., & Gulcu, A. (2012). The effect of mathematica on primary students’ perceptions of properties of three - dimensional geometric objects. Procedia - Social and Behavioral Sciences, 47, 507–514. https://doi.org/10.1016/j.sbspro.2012.06.689
Harju-Luukkainen, H., Vettenranta, J., Ouakrim-Soivio, N., & Bernelius, V. (2016). Differences between students’ PISA reading literacy scores and grading for mother tongue and literature at school: A geostatistical analysis of the Finnish PISA 2009 data. Education Inquiry, 7(4), Article 29413. https://doi.org/10.3402/edui.v7.29413
Hasibuan, S. A., Fauzi, K. M. A., & Mukhtar, M. (2020). Development of PISA mathematical problem model on the content of change and relationship to measure students mathematical problem-solving ability. International Electronic Journal of Mathematics Education, 15(2), Article em0570. https://doi.org/10.29333/iejme/6274
Hopfenbeck, T. N., Lenkeit, J., El Masri, Y., Cantrell, K., Ryan, J., & Baird, J.-A. (2018). Lessons Learned from PISA: A systematic review of peer-reviewed articles on the programme for international student assessment. Scandinavian Journal of Educational Research, 62(3), 333–353. https://doi.org/10.1080/00313831.2016.1258726
Hwang, J., & Ham, Y. (2021). Relationship between mathematical literacy and opportunity to learn with different types of mathematical tasks. Journal on Mathematics Education, 12(2), 199–222. https://doi.org/10.22342/JME.12.2.13625.199-222
Ioane, B. R. (2017). Cultural adaptation of research instruments – The case of materialism scales culturally adapted for use in China. Asian Business Research, 2(2), 7–22. https://doi.org/10.20849/abr.v2i2.148
Johnston-Wilder, S., Baker, J. K., McCracken, A., & Msimanga, A. (2020). A toolkit for teachers and learners, parents, carers and support staff: Improving mathematical safeguarding and building resilience to increase effectiveness of teaching and learning mathematics. Creative Education, 11(8), 1418–1441. https://doi.org/10.4236/ce.2020.118104
Kandeel, R. A. A. (2021). Learners’ mathematics proficiency levels on PISA 2018: A comparative study. International Journal of Instruction, 14(3), 393–416. https://doi.org/10.29333/iji.2021.14323a
Kaur, T., & Prendergast, M. (2022). Students’ perceptions of mathematics writing and its impact on their enjoyment and self-confidence. Teaching Mathematics and Its Applications, 41(1), 1–21. https://doi.org/10.1093/teamat/hrab008
Koyuncu, İ., & Fırat, T. (2020). Investigating reading literacy in PISA 2018 assessment. International Electronic Journal of Elementary Education, 13(2), 263–275. https://doi.org/10.26822/iejee.2021.189
Krawitz, J., Chang, Y.-P., Yang, K.-L., & Schukajlow, S. (2022). The role of reading comprehension in mathematical modelling: Improving the construction of a real-world model and interest in Germany and Taiwan. Educational Studies in Mathematics, 109, 337–359. https://doi.org/10.1007/s10649-021-10058-9
Kusmaryono, I., Ubaidah, N., & Basir, M. A. (2022). It doesn’t mean that students don’t have mathematics anxiety: A case study of mathematics learning with path analysis. European Journal of Educational Research, 11(3), 1683–1697. https://doi.org/10.12973/eu-jer.11.3.1683
Kusmaryono, I., Wijayanti, D., & Maharani, H. R. (2022). Number of response options, reliability, validity, and potential bias in the use of the Likert scale education and social science research: A literature review. International Journal of Educational Methodology, 8(4), 625–637. https://doi.org/10.12973/ijem.8.4.625
Lara-Porras, A. M., Rueda-García, M. D. M., & Molina-Muñoz, D. (2019). Identifying the factors influencing mathematical literacy in several spanish regions. South African Journal of Education, 39(2), Article 1630. https://doi.org/10.15700/saje.v39ns2a1630
Le Hebel, F., Montpied, P., Tiberghien, A., & Fontanieu, V. (2017). Sources of difficulty in assessment: Example of PISA science items. International Journal of Science Education, 39(4), 468–487. https://doi.org/10.1080/09500693.2017.1294784
Lester, J. N., Cho, Y., & Lochmiller, C. R. (2020). Learning to do qualitative data analysis: A starting point. Human Resource Development Review, 19(1), 94–106. https://doi.org/10.1177/1534484320903890
Mathew, J. S. (2017). Self-perception and academic achievement. Indian Journal of Science and Technology, 10(14), 1–6. https://doi.org/10.17485/ijst/2017/v10i14/107586
Melgar, Á. S., Romero, A. A., Orué, L. A. M., Ynjante, Ó. R. E., & Ocaña-Fernandez, Y. (2022). Learning experiences of mathematical situations in university students from the innovation project. International Journal of Health Sciences, 6(S7), 706–715. https://doi.org/10.53730/ijhs.v6ns7.11231
Michener, E. R. (1978). Understanding understanding mathematics. Cognitive Science, 2(4), 361–383. https://doi.org/10.1016/S0364-0213(78)80052-4
Miller, E. R. (2020). Analyzing the cognitive demand of enacted examples in precalculus: A comparative case study of graduate student instructors. Journal of Mathematics and Science: Collaborative Explorations, 16(1), 202–223. https://bit.ly/3KFZPrc
Möbus, V., Jackisch, C., Lück, H. J., du Bois, A., Thomssen, C., Kuhn, W., Nitz, U., Schneeweiss, A., Huober, J., Harbeck, N., von Minckwitz, G., Runnebaum, I. B., Hinke, A., Konecny, G. E., Untch, M., Kurbacher, C., Kahlert, S., Hauser, N., Marmé, F., … Morenz, B. (2018). Ten-year results of intense dose-dense chemotherapy show superior survival compared with a conventional schedule in high-risk primary breast cancer: Final results of AGO phase III iddEPC trial. Annals of Oncology, 29(1), 178–185. https://doi.org/10.1093/annonc/mdx690
Mouli, M. F., Zulkardi, Z., & Putri, R. I. I. (2023). Development of PISA-type questions and activities in a smartphone context. Mathematics Education Journal/ Jurnal Pendidikan Matematika, 17(1), 1–20. https://doi.org/10.22342/jpm.17.1.19386.1-20
Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic analysis: Striving to meet the trustworthiness criteria. International Journal of Qualitative Methods, 16(1), 1–13. https://doi.org/10.1177/1609406917733847
Organization for Economic Co-operation and Development. (2009). PISA data analysis manual: SPSS (2nd ed.). OECD Publishing. https://doi.org/10.1787/9789264056275-en
Organization for Economic Co-operation and Development. (2014). 2014 report on the implementation of the OECD strategy on development. OECD Publishing. https://bit.ly/40L2xkM
Organization for Economic Co-operation and Development. (2017). PISA 2015 assessment and analytical framework: Science, reading, mathematic, financial literacy and collaborative problem solving (Revised ed.). OECD Publishing. https://bit.ly/3VU9f5f
Organization for Economic Co-operation and Development. (2018a). PISA for development assessment and analytical framework: Reading, mathematics and science. OECD Publishing. https://bit.ly/3FrAhvp
Organization for Economic Co-operation and Development. (2018b). PISA 2021 mathematics framework (draft). OECD Publishing. https://bit.ly/3uu8p3k
Perales, R. G., & Ruiz, A. P. (2021). Comparison between performance levels for mathematical competence: Results for the sex variable. Frontiers in Psychology, 12, Article 663202. https://doi.org/10.3389/fpsyg.2021.663202
Polya, G. (1962). Mathematical discovery: On understanding, learning, and teaching problem solving. John Wiley & Sons. https://bit.ly/3JtJ1BP
Prendergast, M., Johnson, P., Fitzmaurice, O., Liston, M., O’Keeffe, L., & O’Meara, N. (2014). Mathematical thinking: Challenging prospective teachers to do more than “talk the talk.” International Journal of Mathematical Education in Science and Technology, 45(5), 635–647. https://doi.org/10.1080/0020739X.2013.868538
Pulkkinen, J., & Rautopuro, J. (2022). The correspondence between PISA performance and school achievement in Finland. International Journal of Educational Research, 114, Article 102000. https://doi.org/10.1016/j.ijer.2022.102000
Rose, E., Hands, B., & Larkin, D. (2012). Reliability and validity of the self-perception profile for adolescents: An Australian sample. Australian Journal of Psychology, 64(2), 92–99. https://doi.org/10.1111/j.1742-9536.2011.00031.x
Scandura, J. M. (1966). Precision in research on mathematics learning: The emerging field of psycho-mathematics. Journal of Science in Teaching, 4(4), 253–274. https://doi.org/10.1002/tea.3660040413
Schleicher, A. (2019). PISA 2018: Insights and interpretation. OECD Publishing. https://bit.ly/3UyjCuw
Stacey, K. (2011). The PISA view of mathematical literacy in Indonesia. Journal on Mathematics Education, 2(2), 95–126. https://doi.org/10.22342/jme.2.2.746.95-126
Stahl, N. A. N., & King, J. R. (2020). Expanding approaches for research: Understanding and using trustworthiness in qualitative research. Journal of Developmental Education, 44(1), 26–28. https://bit.ly/3n7dZbK
Stringer, R. W., & Heath, N. (2008). Academic self-perception and its relationship to academic performance. Canadian Journal of Education, 31(2), 327–345.
Tekkumru-Kisa, M., Stein, M. K., & Schunn, C. (2015). A framework for analyzing cognitive demand and content-practices integration: Task analysis guide in science. Journal of Research in Science Teaching, 52(5), 659–685. https://doi.org/10.1002/tea.21208
Timler, A., McIntyre, F., Rose, E., & Hands, B. (2019). Exploring the influence of self-perceptions on the relationship between motor competence and identity in adolescents. PLoS ONE, 14(11), Article e0224653. https://doi.org/10.1371/journal.pone.0224653
Ulia, N., & Kusmaryono, I. (2021). Mathematical disposition of students’, teachers, and parents in distance learning: A survey. Premiere Educandum : Jurnal Pendidikan Dasar dan Pembelajaran, 11(1), 147–159. https://doi.org/10.25273/pe.v11i1.8869
Vostanis, A., Padden, C., Chiesa, M., Rizos, K., & Langdon, P. E. (2021). A precision teaching framework for improving mathematical skills of students with intellectual and developmental disabilities. Journal of Behavioral Education, 30, 513–533. https://doi.org/10.1007/s10864-020-09394-2
Wise, S. L., & DeMars, C. E. (2005). Low examinee effort in low-stakes assessment: Problems and potential solutions. Educational Assessment, 10(1), 1–17. https://doi.org/10.1207/s15326977ea1001_1
Wisniewski, B., Röhl, S., & Fauth, B. (2022). The perception problem: A comparison of teachers’ self-perceptions and students’ perceptions of instructional quality. Learning Environments Research, 25, 775–802. https://doi.org/10.1007/s10984-021-09397-4
Yeong, M. L., Ismail, R., Ismail, N. H., & Hamzah, M. I. (2018). Interview protocol refinement: Fine-tuning qualitative research interview questions for multi-racial populations in Malaysia. The Qualitative Report, 23(11), 2700–2713. https://doi.org/10.46743/2160-3715/2018.3412
Zaim, M., Refnaldi, R., Zainil, Y., & Ramadhani, F. (2021). PISA reading literacy assessment and senior high school reading literacy assessments: How do they differ? International Journal of Research in Counseling and Education, 5(1), 72-78. https://doi.org/10.24036/00466za0002