Effect of Computer Based Test on Motivation: A Meta-Analysis
This meta-analysis study investigates the effect of computer-based tests on motivation. The random effect model is the basis for calculating the summa.
- Pub. date: October 15, 2023
- Pages: 1583-1599
- 646 Downloads
- 884 Views
- 0 Citations
This meta-analysis study investigates the effect of computer-based tests on motivation. The random effect model is the basis for calculating the summary effect, and it is found that the 30 articles obtained through the PRISMA method, which consist of four parts, namely identification, screening, eligibility, and included, can be generalized. Data analysis was performed using R studio and OpenMEE. Based on the research results, the p-value <.05 was obtained, so it was concluded that using a computer-based test significantly affected motivation. In the moderator analysis, it was found that computer-based tests were most effective for intrinsic motivation for the motivation variable, university students for the participants variable, and the country of Iran for the region variable. Evaluation of publication bias using the funnel plot and regression method shows no publication bias, proving that the 30 articles analyzed represent all existing studies on computer-based tests.
Keywords: Computer-based test, meta-analysis, motivation, R studio.
References
Agustini, K., & Wahyuni, D. S. (2013). Pengaruh penggunaan simulasi binary tree berbasis CAI terhadap motivasi dan hasil belajar matematika diskrit mahasiswa jurusan PTI undiksha [Effect of using CAI-based binary tree simulation on motivation and learning outcomes of discrete mathematics students majoring in PTI undiksha]. Jurnal Pendidikan Indonesia, 2(1), 162–172. https://doi.org/10.23887/jpi-undiksha.v2i1.6369
Ahmad, N. J., Yakob, N., Bunyamin, M. A. H., Winarno, N., & Akmal, W. H. (2021). The effect of interactive computer animation and simulation on students’ achievement and motivation in learning electrochemistry. Jurnal Pendidikan IPA Indonesia, 10(3), 311–324. https://doi.org/10.15294/JPII.V10I3.26013
Akhtar, H., Silfiasari, Vekety, B., & Kovacs, K. (2022). The effect of computerized adaptive testing on motivation and anxiety: A systematic review and meta-analysis. Assessment, 30(5), 1379-1390. https://doi.org/10.1177/10731911221100995
Aremu, B. V. (2021). The use of mobile learning to improve students’ motivation and the achievement of learning outcomes. KIU Journal of Humanities, 6(2), 175–183. http://bit.ly/3QUG2FO
Bailey, D., Almusharraf, N., & Hatcher, R. (2021). Finding satisfaction: Intrinsic motivation for synchronous and asynchronous communication in the online language learning context. Education and Information Technologies, 26, 2563–2583. https://doi.org/10.1007/s10639-020-10369-z
Baragilly, M., & Willis, B. H. (2022). On estimating a constrained bivariate random effects model for meta-analysis of test accuracy studies. Statistical Methods in Medical Research, 31(2), 287–299. https://doi.org/10.1177/09622802211065157
Beisemann, M., Doebler, P., & Holling, H. (2020). Comparison of random-effects meta-analysis models for the relative risk in the case of rare events: A simulation study. Biometrical Journal, 62(7), 1597–1630. https://doi.org/10.1002/bimj.201900379
Boevé, A. J., Meijer, R. R., Albers, C. J., Beetsma, Y., & Bosker, R. J. (2015). Introducing computer-based testing in high-stakes exams in higher education: Results of a field experiment. PLoS ONE, 10(12), Article e0143616. https://doi.org/10.1371/journal.pone.0143616
Borenstein, M., & Higgins, J. P. T. (2013). Meta-analysis and subgroups. Prevention Science, 14, 134–143. https://doi.org/10.1007/s11121-013-0377-7
Bridgeman, B., Lennon, M. L., & Jackenthal, A. (2003). Effects of screen size, screen resolution, and display rate on computer-based test performance. Applied Measurement in Education, 16(3), 191–205. https://doi.org/10.1207/S15324818AME1603_2
Butcher, J. N., Perry, J. N., & Atlis, M. M. (2000). Validity and utility of computer-based test interpretation. Psychological Assessment, 12(1), 6–18. https://doi.org/10.1037/1040-3590.12.1.6
Çakir, R. (2019). Effect of web-based intelligence tutoring system on students’ achievement and motivation. Malaysian Online Journal of Educational Technology, 7(4), 45–59. https://bit.ly/44pNVbW
Card, N. A. (2012). Applied meta-analysis for social science research. The Guilford Press. https://psycnet.apa.org/record/2011-26930-000
Carpenter, R., & Alloway, T. (2019). Computer versus paper-based testing: Are they equivalent when it comes to working memory? Journal of Psychoeducational Assessment, 37(3), 382–394. https://doi.org/10.1177/0734282918761496
Çelen, B. (2019). Motivations for teaching english: A review of journal articles on L2 teacher motivation. In G. Y. Ekşi, L. Guerra, D. Werbińska, & Y. Bayyurt (Eds.), Research trends in english language teacher education and english language teaching (pp. 241–258). University of Évora.
Cheng, L., Ritzhaupt, A. D., & Antonenko, P. (2019). Effects of the flipped classroom instructional strategy on students’ learning outcomes: A meta-analysis. Educational Technology Research and Development, 67(4), 793-824. https://doi.org/10.1007/s11423-018-9633-7
Chua, Y. P. (2012). Effects of computer-based testing on test performance and testing motivation. Computers in Human Behavior, 28(5), 1580–1586. https://doi.org/10.1016/j.chb.2012.03.020
Chua, Y. P., & Don, Z. M. (2013). Effects of computer-based educational achievement test on test performance and test takers’ motivation. Computers in Human Behavior, 29(5), 1889–1895. https://doi.org/10.1016/j.chb.2013.03.008
Davies, R. S., & West, R. E. (2014). Technology integration in schools. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (pp. 841–853). Springer. https://doi.org/10.1007/978-1-4614-3185-5_68
Dettori, J. R., Norvell, D. C., & Chapman, J. R. (2022). Fixed-effect vs random-effects models for meta-analysis: 3 points to consider. Global Spine Journal, 12(7), 1624–1626. https://doi.org/10.1177/21925682221110527
Dewi, A. R. (2020). Students’ perception towards computer-based testing-edubox-and its impact to their English learning motivation at SMP negeri 45 bandung [Doctoral dissertation, Universitas Terbuka]. Universitas Terbuka Digital Archive. http://repository.ut.ac.id/9702/1/cover%20dafis.pdf
Dewi, A. R., Saehu, A., & Budiman, R. (2021). Students’ perception of computer-based testing-edubox–and its impact on their english learning motivation. Indonesian Journal of Learning and Instruction, 4(1), 19–28. https://bit.ly/3HhTSis
Dolan, R., & Burling, K. (2012). Computer-based testing in higher education. In C. Secolsky & D. B. Denison (Eds.), Handbook on measurement, assessment, and evaluation in higher education (pp. 321–335). Routledge.
Drasgow, F., & Mattern, K. (2006). New tests and new items: Opportunities and issues. In D. Bartram & R. K. Hambleton (Eds.), Computer-based testing and the internet (pp. 59–76). John Wiley & Sons Ltd. https://doi.org/10.1002/9780470712993.ch3
Dresel, M., & Haugwitz, M. (2008). A computer-based approach to fostering motivation and self-regulated learning. The Journal of Experimental Education, 77(1), 3–20. https://doi.org/10.3200/JEXE.77.1.3-20
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315(9), 629–634. https://doi.org/10.1136/bmj.315.7109.629
Gholami, M., Changaee, F., Karami, K., Shahsavaripour, Z., Veiskaramian, A., & Birjandi, M. (2021). Effects of multiepisode case-based learning (CBL) on problem-solving ability and learning motivation of nursing students in an emergency care course. Journal of Professional Nursing, 37(3), 612–619. https://doi.org/10.1016/j.profnurs.2021.02.010
Good, V., Hughes, D. E., Kirca, A. H., & McGrath, S. (2022). A self-determination theory-based meta-analysis on the differential effects of intrinsic and extrinsic motivation on salesperson performance. Journal of the Academy of Marketing Science, 50, 586–614. https://doi.org/10.1007/s11747-021-00827-6
Granito, M., & Chernobilsky, E. (2012, October 19). The effect of technology on a student’s motivation and knowledge retention [Paper presentation]. Northeastern Educational Research Association (NERA) Annual Conference, University of Connecticut, United States.
Grund, A., Galla, B. M., & Fries, S. (2022). Achievement motivation in students’ everyday lives: Its relationship to momentary positive and negative activation and the moderating role of mindfulness. Learning and Individual Differences, 97, Article 102176. https://doi.org/10.1016/j.lindif.2022.102176
Hakim, B. M. (2018). Comparative study on validity of paper-based test and computer-based test in the context of educational and psychological assessment among arab students. International Journal of English Linguistics, 8(2), 85–91. https://doi.org/10.5539/ijel.v8n2p85
Hamman, E. A., Pappalardo, P., Bence, J. R., Peacor, S. D., & Osenberg, C. W. (2018). Bias in meta-analyses using Hedges’ d. Ecosphere, 9(9), Article e02419. https://doi.org/10.1002/ecs2.2419
Hariri-Akbari, M., Shokrvash, B., Mahmoodi, F., Jahanjoo-Aminabad, F., Yousefi, B., & Azabdaftari, F. (2018). Conversion of extrinsic into intrinsic motivation and computer based testing (CBT). BMC Medical Education, 18, Article 143. https://doi.org/10.1186/s12909-018-1249-4
Harrer, M., Cuijpers, P., Furukawa, T., & Ebert, D. (2021). Doing meta-analysis with R: A hands-on guide. Chapmann & Hall Press. https://doi.org/10.1201/9781003107347
Haryanto. (2012). Pemrograman visual C++ konsep dasar, algoritma dan aplikasi [Visual C++ programming basic concepts, algorithms and applications]. UNY Press.
Hidayah, H. (2016). Pengaruh penggunaan instrumen tes berbasis wondershare quiz creator dengan paper and pencil test terhadap motivasi belajar siswa pada mata pelajaran PPKn [The effect of using a wondershare quiz creator-based test instrument with a paper and pencil test on student learning motivation in Civics subjects] [Unpublished master's thesis]. Universitas Sebelas Maret.
Hoerunnisa, A., Suryani, N., & Efendi, A. (2019). The effectiveness of the use of e-learning in multimedia classes to improve vocational students’ learning achievement and motivation. Kwangsan: Jurnal Teknologi Pendidikan, 7(2), 123–137. https://doi.org/10.31800/jtp.kw.v7n2.p123--137
Ibrahim, D. S., & Suardiman, S. P. (2014). Pengaruh penggunaan e-learning terhadap motivasi dan prestasi belajar matematika siswa sd negeri tahunan yogyakarta [The effect of the use of e-learning on the motivation and achievement of learning mathematics in elementary school tahunan Yogyakarta]. Jurnal Prima Edukasia, 2(1), 66–79. https://doi.org/10.21831/jpe.v2i1.2645
Istiyono, E., Dwandaru, W. S. B., Erfianti, L., & Astuti, W. (2020). Applying CBT in physics learning to measure students’ higher order thinking skills. Journal of Physics: Conference Series, 1440, Article 012061 https://doi.org/10.1088/1742-6596/1440/1/012061
Jackson, D., & Turner, R. (2017). Power analysis for random-effects meta-analysis. Research Synthesis Methods, 8(3), 290–302. https://doi.org/10.1002/jrsm.1240
Kebritchi, M., Hirumi, A., & Bai, H. (2010). The effects of modern mathematics computer games on mathematics achievement and class motivation. Computers and Education, 55(2), 427–443. https://doi.org/10.1016/j.compedu.2010.02.007
Keter, J. K. (2018). Effects of computer based cooperative mastery learning on secondary school students’ skills acquisition, motivation and achievement in chemistry practicals in bomet county, Kenya [Doctoral dissertation, Egerton University]. Egerton University Institutional Repositary. https://bit.ly/44lGvGS
Khoshsima, H., Hosseini, M., & Toroujeni, S. M. H. (2017). Cross-mode comparability of computer-based testing (CBT) versus paper-pencil based testing (PPT): An investigation of testing administration mode among iranian intermediate EFL learners. English Language Teaching, 10(2), 23-32. https://doi.org/10.5539/elt.v10n2p23
Kotera, Y., Taylor, E., Fido, D., Williams, D., & Tsuda-McCaie, F. (2021). Motivation of UK graduate students in education: Self-compassion moderates pathway from extrinsic motivation to intrinsic motivation. Current Psychology. 42, 10163–10176. Advance online publication. https://doi.org/10.1007/s12144-021-02301-6
Lestari, I. (2022). Pengaruh ujian online terhadap kecemasan, motivasi dan prestasi siswa [The effect of online exams on students' anxiety, motivation and achievement][Master's thesis, Universitas Muhammadiyah Malang]. Universitas Muhammadiyah Malang Digital Archive. https://eprints.umm.ac.id/84571/1/Indah%20Lestari.pdf
Lowyck, J. (2014). Bridging learning theories and technology-enhanced environments: A critical appraisal of its history. In J. M. Spector, M. D. Merrill, J. Elen, & M.J.Bishop (Eds.), Handbook of research on educational communications and technology (pp. 3–20). Springer. https://doi.org/10.1007/978-1-4614-3185-5_1
Luecht, R. M. (2006). Operational issues in computer-based testing. In D. Bartram & R. K. Hambleton (Eds.), Computer based testing and the internet (pp. 91–114). John Wiley & Sons Ltd. https://doi.org/10.1002/9780470712993.ch5
Maqableh, M., Masa’deh, R. M. T., & Mohammed, A. B. (2015). The acceptance and use of computer based assessment in higher education. Journal of Software Engineering and Applications, 8(10), 557–574. https://doi.org/10.4236/jsea.2015.810053
Mehrabi, T., Behzadi, S., Sabouri, F., & Alavi, M. (2016). Assessment the effect of the CBT on motivation of the nursing students. Iranian Journal of Nursing and Midwifery Research, 21(2), 118–123. https://bit.ly/3XD0Rcn
Morris, L. S., Grehl, M. M., Rutter, S. B., Mehta, M., & Westwater, M. L. (2022). On what motivates us: A detailed review of intrinsic v. extrinsic motivation. Psychological Medicine, 52(10), 1801-1816. https://doi.org/10.1017/S0033291722001611
Nasri, M., Shafiee, S., & Sepehri, M. (2021). An investigation of Iranian intermediate EFL learners’ L2 motivation and attitude in a computer-assisted language learning environment. Issues in Language Teaching, 10(1), 355–389. https://doi.org/10.22054/ilt.2021.62359.614
Nikou, S. A., & Economides, A. A. (2016). The impact of paper-based, computer-based and mobile-based self-assessment on students’ science motivation and achievement. Computers in Human Behavior, 55(B), 1241–1248. https://doi.org/10.1016/j.chb.2015.09.025
Ockey, G. J. (2009). Developments and challenges in the use of computer-based testing for assessing second language ability. Modern Language Journal, 93(1), 836–847. https://doi.org/10.1111/j.1540-4781.2009.00976.x
Partovi, T., & Razavi, M. R. (2019). The effect of game-based learning on academic achievement motivation of elementary school students. Learning and Motivation, 68, Article 101592. https://doi.org/10.1016/j.lmot.2019.101592
Piaw, C. Y. (2011, January 15). Comparisons between computer-based testing and paper-pencil testing: Testing effect, test scores, testing time and testing motivation [Paper presentation]. The Informatics Conference, University of Malaya, Malaysia.
Piaw, C. Y. (2012). Replacing paper-based testing with computer-based testing in assessment: Are we doing wrong? Procedia - Social and Behavioral Sciences, 64, 655–664. https://doi.org/10.1016/j.sbspro.2012.11.077
Proske, A., Roscoe, R. D., & McNamara, D. S. (2014). Game-based practice versus traditional practice in computer-based writing strategy training: Effects on motivation and achievement. Educational Technology Research and Development, 62, 481–505. https://doi.org/10.1007/s11423-014-9349-2
Ramdani, D., Susilo, H., Suhadi, & Sueb. (2022). The effectiveness of collaborative learning on critical thinking, creative thinking, and metacognitive skill ability: Meta-analysis on biological learning. European Journal of Educational Research, 11(3), 1607–1628. https://doi.org/10.12973/eu-jer.11.3.1607
Romero, C., Ventura, S., & de Bra, P. (2009). Using mobile and web-based computerized tests to evaluate university students. Computer Applications in Engineering Education, 17(4), 437–477. https://doi.org/10.1002/cae.20242
Ronoh, P. K., Wachanga, S. W., & Keraro, F. N. (2014). Effects of computer based mastery learning approach on students’ motivation to learn biology. Journal of Education and Practice, 5(32), 117–126. http://bit.ly/3wlTkC6
Saepuzaman, D., Retnawati, H., Istiyono, E., & Haryanto. (2021). Can innovative learning affect students’ HOTS achievements?: A meta-analysis study. Pegem Egitim ve Ogretim Dergisi, 11(4), 290–305. https://doi.org/10.47750/pegegog.11.04.28
Safaruddin, Taufan, J., Mahdi, A., Intania, & Silvia, R. (2021). Developing a computer based test application to measure braille literation ability for special teacher. In Ifdil, H. Ardi, Z. Ardi, P. Susanto, & Nofrion (Eds.), Proceedings of the 2nd Progress in Social Science, Humanities and Education Research Symposium (PSSHERS 2020) (pp. 36-38). Atlantis Press. https://doi.org/10.2991/assehr.k.210618.007
Schunk, D. H., & DiBenedetto, M. K. (2020). Motivation and social cognitive theory. Contemporary Educational Psychology, 60, Article 101832. https://doi.org/10.1016/j.cedpsych.2019.101832
Sins, P. H. M., van Joolingen, W. R., Savelsbergh, E. R., & van Hout-Wolters, B. (2008). Motivation and performance within a collaborative computer-based modeling task: Relations between students’ achievement goal orientation, self-efficacy, cognitive processing, and achievement. Contemporary Educational Psychology, 33(1), 58–77. https://doi.org/10.1016/j.cedpsych.2006.12.004
Soimah, I. (2018). Pengaruh media pembelajaran berbasis komputer terhadap hasil belajar IPA ditinjau dari motivasi belajar siswa [The influence of computer-based learning media on science learning outcomes in terms of student learning motivation]. Natural: Jurnal Ilmiah Pendidikan IPA, 5(1), 38–44. https://doi.org/10.30738/natural.v5i1.2559
Suparman, A. R., Rohaeti, E., & Wening, S. (2022). Development of attitude assessment instruments towards socio-scientific issues in chemistry learning. European Journal of Educational Research, 11(4), 1947-1958. https://doi.org/10.12973/eu-jer.11.4.1947
Taranilla, R. V., Tirado-Olivares, S., Cózar-Gutiérrez, R., & González-Calero, J. A. (2022). Effects of virtual reality on learning outcomes in K-6 education: A meta-analysis. Educational Research Review, 35, Article 100434. https://doi.org/10.1016/j.edurev.2022.100434
Tüzün, H., Yılmaz-Soylu, M., Karakuş, T., İnal, Y., & Kızılkaya, G. (2009). The effects of computer games on primary school students’ achievement and motivation in geography learning. Computers & Education, 52(1), 68–77. https://doi.org/10.1016/j.compedu.2008.06.008
van der Linden, W. J. (2006). Model based innovations in computer based testing. In D. Bartram & R. K. Hambleton (Eds.), Computer based testing and the internet issues and advances (pp. 39–58). John Wiley & Sons Ltd. https://doi.org/10.1002/9780470712993.ch2
Wallace, B. C., Lajeunesse, M. J., Dietz, G., Dahabreh, I. J., Trikalinos, T. A., Schmid, C. H., & Gurevitch, J. (2017). OpenMEE: Intuitive, open-source software for meta-analysis in ecology and evolutionary biology. Methods in Ecology and Evolution, 8(8), 941–947. https://doi.org/10.1111/2041-210X.12708
Wu, Q., Wang, J., Lin, X., Han, D., Hu, H., & Gao, H. (2023). Determining the efficacy and safety of acupuncture for the preventive treatment of menstrual migraine: A protocol for a PRISMA-compliant systematic review and meta-analysis. Journal of Pain Research, 16, 101-109. https://doi.org/10.2147/JPR.S394446
Zarei, A. A., & Hashemipour, M. (2015). The effect of computer-assisted language instruction on improving EFL learners’ autonomy and motivation written corrective feedback view project the effect of computer-assisted language instruction on improving EFL learners’ autonomy and motivation. Journal of Applied Linguistics, 1(1), 40–58. https://bit.ly/3wjrjej
Zenisky, A. L., & Luecht, R. M. (2016). The future of computer‑based testing: Some new paradigms. In C. S. Wells & M. Faulkner-Bond (Eds.), Educational measurement from foundations to future (pp. 221–238). The Guilford Press.