A Combination of Context Input Process Product and Kirkpatrick Evaluation Model to Determine the Effectiveness of E-Training for Principals during COVID-19 Pandemic
COVID-19 pandemic forces training for principals to be conducted online. This study aims to evaluate the context, input, process, response, study, beh.
- Pub. date: October 15, 2022
- Pages: 2087-2100
- 687 Downloads
- 1218 Views
- 2 Citations
COVID-19 pandemic forces training for principals to be conducted online. This study aims to evaluate the context, input, process, response, study, behavior, and outcome to determine the effectiveness and make recommendations for training. The approach used in this study was a mixed method with a concurrent embedded design and a qualitative method as the main method. The subjects of this study were the principals, committees, and instructors. Data were collected through interviews, observations, and questionnaires and analyzed using Miles and Huberman's model, descriptive analysis, and a t-test. The results show that e-training is effective in context, input, and process. Response, learning, and attitude assessments prove that knowledge, skills, and attitude have improved. Participants will be able to implement the experience gained and impact school quality improvement. This study contributes to the combination of the two evaluation models proven to produce a complete result. The study for the e-training recommends needed assessment before the training, the activity before the training to acquire the skills in using the learning management system, and the monitoring and evaluation after the training.
Keywords: Context input process product, effectiveness, e-training, evaluation, Kirkpatrick
References
Abdulghani, H. M., Shaik, S. A., Khamis, N., Al-Drees, A. A., Irshad, M., Khalil, M. S., Alhaqwi, A. I., & Isnani, A. (2014). Research methodology workshops evaluation using the Kirkpatrick's model: Translating theory into practice. Medical Teacher, 36(1), 24–29. https://doi.org/ggzfvr
Agarwal, N., Pande, N., & Ahuja, V. (2014). Expanding the Kirkpatrick evaluation model towards more efficient training in the IT sector. International Journal of Human Capital and Information Technology Professionals, 5(4), 19–34. https://doi.org/10.4018/ijhcitp.2014100102
Al-Hamad, N. Q., Rathwan, A. M., & Rababah, E. Q. (2020). Training needs for kindergarten principals in the light of digital age requirements. International Journal of Early Childhood Special Education, 12(2), 125–136. https://doi.org/h9bt
Al-Shanawani, H. M. (2019). Evaluation of self-learning curriculum for kindergarten using Stufflebeam's CIPP model. SAGE Open, 9(1), 1–13. https://doi.org/h9bv
Al hila, A. A., Alhelou, S. E. M., Al Shobaki, M. J., & Abu Naser, S. S. (2017). The impact of applying the dimensions of IT governance in improving e-training case study of the ministry of telecommunications and information technology in Gaza governorates. International Journal of Engineering and Information Systems, 1(8), 194–219. https://bit.ly/3OiSJaC
Alavudeen, S. S., Easwaran, V., Mir, J. I., Shahrani, S. M., Aseeri, A. A., Khan, N. A., Almodeer, A. M., & Asiri, A. A. (2021). The influence of COVID-19 related psychological and demographic variables on the effectiveness of e-learning among health care students in the southern region of Saudi Arabia. Saudi Pharmaceutical Journal, 29(7), 775–780. https://doi.org/10.1016/j.jsps.2021.05.009
Alkali, A. U., & Mansor, N. N. A. (2017). Interactivity and trust as antecedents of e-training use intention in Nigeria: A structural equation modelling approach. Behavioral Sciences, 7(3). https://doi.org/10.3390/bs7030047
Altavilla, G. (2019). Monitoring training to adequate the teaching method in training: An interpretative concepts. Journal of Physical Education and Sport, 19(5), 1763–1766. https://doi.org/10.7752/jpes.2019.s5258
Amin, I., Yousaf, A., Walia, S., & Bashir, M. (2021). What shapes e-learning effectiveness among tourism education students? an empirical assessment during COVID19. Journal of Hospitality, Leisure, Sport and Tourism Education, 30, 1-13. https://doi.org/h9bw
Aziz, S., Mahmood, M., Rehman, Z., & Report, A. (2018). Implementation of CIPP model for quality evaluation at school level: A case study. Journal of Education and Educational Development, 5(1), 189-206.
Baczek, M., Zaganczyk-Baczek, M., Szpringer, M., Jaroszynski, A., & Wozakowska-Kapłon, B. (2021). Student’s perception of online learning during COVID pandemic. Medicine, 100(7), 1–6. https://doi.org/h9bx
Belaya, V. (2018). The use of e-learning in vocational education and training (VET): Systematization of existing theoretical approaches. Journal of Education and Learning, 7(5), 92. https://doi.org/10.5539/jel.v7n5p92
Ben Amara, N., & Atia, L. (2016). E-training and its role inhuman-resources development. Global Journal of Human Resource Management, 4(1), 1–12. https://bit.ly/3OtMEbL
Bernardino, G., & Curado, C. (2020). Training evaluation: A configurational analysis of success and failure of trainers and trainees. European Journal of Training and Development, 44(4–5), 531–546. https://doi.org/gm9rh2
Brauckmann, S., Pashiardis, P., & Ärlestig, H. (2020). Bringing context and educational leadership together: Fostering the professional development of school principals. Professional Development in Education, 1–12. https://doi.org/gjd4kc
Calvo, S., Morales, A., & Wade, J. (2019). The use of MOOCs in social enterprise education: An evaluation of a North–South collaborative FutureLearn program. Journal of Small Business and Entrepreneurship, 31(3), 201–223. https://doi.org/h9bz
Chalikias, M., Raftopoulou, I., Sidiropoulos, G., Kyriakopoulos, G. L., & Zakopoulos, V. (2020). "The school principal's role as a leader in teachers" professional development: The case of public secondary education in Athens". Problems and Perspectives in Management, 18(4), 461–474. https://doi.org/10.21511/ppm.18(4).2020.37
Chang, T. Y., Hsu, M. L., Kwon, J. S., Kusdhany, M. L. S., & Hong, G. (2021). Effect of online learning for dental education in Asia during the pandemic of COVID-19. Journal of Dental Sciences, 16(4), 1095–1101. https://doi.org/h9b2
Cotes, J., & Ugarte, S. M. (2021). A systemic and strategic approach for training needs analysis for the International Bank. Journal of Business Research, 127, 464–473. https://doi.org/10.1016/j.jbusres.2019.05.002
Deodhar, M., & Powdwal, S. (2017). Impact of continuing education programs (CEPs) on LIS professionals in academic libraries in Mumbai, India. Library Management, 38(2/3), 117–130. https://doi.org/h9b3
Dewi, L. R., & Kartowagiran, B. (2018). An evaluation of internship program by using Kirkpatrick evaluation model. Research and Evaluation in Education, 4(2), 155–163. https://doi.org/h9b4
Dhull, I., & Sakshi, M. (2017). Online learning. International Education & Research Journal, 3(8), 32–34. https://bit.ly/3PLwyuA
Divayana, D. G. H., Sanjaya, D. B., Marhaeni, A. A. I. N., & Sudirtha, I. G. (2017). CIPP evaluation model based on mobile phone in evaluating the use of blended learning platforms at vocational schools in Bali. Journal of Theoretical and Applied Information Technology, 95(9), 1983–1995. https://bit.ly/3Ok0E7E
Dorri, S., Akbari, M., & Dorri Sedeh, M. (2016). Kirkpatrick evaluation model for in-service training on cardiopulmonary resuscitation. Iranian Journal of Nursing and Midwifery Research, 21(5), 493–497. https://doi.org/h9b8
Embi, Z., Neo, T.-K., & Neo, M. (2017). Using Kirkpatrick's evaluation model in a multimedia-based blended learning environment. Journal of Multimedia Information System, 4(3), 115–122. https://doi.org/h9bs
Esgaiar, E., & Foster, S. (2019). Implementation of CIPP model for quality evaluation at Zawia University. International Journal of Applied Linguistics and English Literature, 8(5), 106-115. https://doi.org/10.7575/aiac.ijalel.v.8n.5p.106
Finney, T. L. (2019). Confirmative evaluation: New CIPP evaluation model. Journal of Modern Applied Statistical Methods, 18(2), 2–24. https://doi.org/h9b9
Florea, N. V., Duica, A., & Duica, M. (2016). Using models and evaluation planning to improve corporate training activity and trainee performance. Valahian Journal of Economic Studies, 7(1), 45–56. https://bit.ly/3PhjgGM
Foster, C., Rodriguez-Marroyo, J. A., & De Koning, J. J. (2017). Monitoring training loads: The past, the present, and the future. International Journal of Sports Physiology and Performance, 12, 2–8. https://doi.org/gfvxfm
Garg, S., & Sharma, S. (2020). User satisfaction and continuance intention for using e-training: A structural equation model. Vision, 24(4), 441–451. https://doi.org/h9cb
Gokmenoglu, T., Sonmez, E. D., Yavuz, I., & Gok, A. (2021). Turkish ministry of national education school-based disaster education program: A preliminary results of the program evaluation. International Journal of Disaster Risk Reduction, 52, 1-8. https://doi.org/10.1016/j.ijdrr.2020.101943
Gumede, L., & Badriparsad, N. (2021). Online teaching and learning through the students' eyes – Uncertainty through the COVID-19 lockdown: A qualitative case study in Gauteng province, South Africa. Radiography, 28(1), 193–198. https://doi.org/10.1016/j.radi.2021.10.018
Gurmu, T. G. (2020). Primary school principals in Ethiopia: Selection and preparation. Educational Management Administration and Leadership, 48(4), 651–681. https://doi.org/h9cc
Hakan, K., & Seval, F. (2011). CIPP evaluation model scale: Development, reliability and validity. Procedia - Social and Behavioral Sciences, 15, 592–599. https://doi.org/fft545
Hanaysha, J., & Tahir, P. R. (2016). Examining the effects of employee empowerment, teamwork, and employee training on job satisfaction. Procedia - Social and Behavioral Sciences, 219, 272–282. https://doi.org/h9cd
Hassan, A., Hassan, J., & Yen, T. A. (2020). E-Training and development, motivation and employee performance among academicians: Case study of academicians in UniMAP. Journal of Physics: Conference Series, 1529, 1-8. https://doi.org/h9cf
Heydari, M. R., Taghva, F., Amini, M., & Delavari, S. (2019). Using Kirkpatrick's model to measure the effect of a new teaching and learning methods workshop for health care staff. BMC Research Notes, 12(1), 1–5. https://doi.org/h9cg
Hofmeister, C., & Pilz, M. (2020). Using e-learning to deliver in-service teacher training in the vocational education sector: Perception and acceptance in Poland, Italy and Germany. Education Sciences, 10(7), 1–17. https://doi.org/10.3390/educsci10070182
Hussein, E., Daoud, S., Alrabaiah, H., & Badawi, R. (2020). Exploring undergraduate students' attitudes towards emergency online learning during COVID-19: A case from the UAE. Children and Youth Services Review, 119, 1-20. https://doi.org/gmwcj7
Indra, R., Kustati, M., Saregar, A., Warnis, Nelmawarni, & Yusuf, Y. Q. (2020). The effect of principals' leadership towards effective learning at an Indonesian secondary school. European Journal of Educational Research, 9(3), 1063–1074. https://doi.org/10.12973/eu-jer.9.3.1063
Ismail, A., Zaharudin, R., Hashim, N., & Ariffin, J. (2020). The impact of e-NPQEL on the continuance intention of using e-training among aspired school leaders in Malaysia. International Journal of Interactive Mobile Technologies, 14(19), 109–123. https://doi.org/10.3991/ijim.v14i19.15965
Jain, G., Sharma, N., & Shrivastava, A. (2021). Enhancing training effectiveness for organizations through blockchain-enabled training effectiveness measurement (BETEM). Journal of Organizational Change Management, 34(2), 439–461. https://doi.org/10.1108/JOCM-10-2020-0303
Jiang, N., Sumintono, B., Perera, C. J., Harris, A., & Jones, M. S. (2018). Training preparation and the professional development of principals in Henan Province, China: Formal and informal learning. Asia Pacific Education Review, 19(1), 41–51. https://doi.org/10.1007/s12564-017-9513-6
Jones, C., Fraser, J., & Randall, S. (2018). The evaluation of a home-based paediatric nursing service: Concept and design development using the Kirkpatrick model. Journal of Research in Nursing, 23(6), 492–501. https://doi.org/h9ch
Kamal, K., Alaghbari, M., & Atteia, M. (2016). E-training & employees' performance a practical study on the ministry of education in the kingdom of Bahrain. Journal of Resources Development and Management, 18, 1–8. https://bit.ly/3PBx7Y2
Karimnia, A., & Kay, E. (2015). An evaluation of the undergraduate TEFL program in Iran: A multi case study. International Journal of Instruction, 8(2), 83–98. https://doi.org/10.12973/iji.2015.827a
Kirkpatrick, D. L., & Kirkpatrick, J. D. (2007). Implementing the four levels. Berrett-Koehler Publishers. https://bit.ly/3IKUL25
König, J., Jäger-Biela, D. J., & Glutsch, N. (2020). Adapting to online teaching during COVID-19 school closure: Teacher education and teacher competence effects among early career teachers in Germany. European Journal of Teacher Education, 43(4), 608–622. https://doi.org/ghvsw8
Le, L. K., Tran, T., Le, H. T. T., & Le, T. T. T. (2021). A study on factors affecting school principals' competencies in Vietnam's mountainous provinces. Management in Education, 35(4), 174–181. https://doi.org/gg83r4
Lemay, D. J., Bazelais, P., & Doleck, T. (2021). Transition to online learning during the COVID-19 pandemic. Computers in Human Behavior Reports, 4, 1-9. https://doi.org/gm2jb3
Li, J., Qin, C., & Zhu, Y. (2021). Online teaching in universities during the covid-19 epidemic: A study of the situation, effectiveness and countermeasures. Procedia Computer Science, 187, 566–573. https://doi.org/h9cj
Liao, S.-C., & Hsu, S.-Y. (2019). Evaluating a continuing medical education program: New world Kirkpatrick model approach. International Journal of Management, Economics and Social Sciences, 8(4), 266–279. https://doi.org/h9ck
Lippe, M., & Carter, P. (2018). Using the CIPP model to assess nursing education program quality and merit. Teaching and Learning in Nursing, 13(1), 9–13. https://doi.org/h9f6
Ludwikowska, K. (2018). The effectiveness of training needs analysis and its relation to employee efficiency. Zeszyty Naukowe Politechniki Poznańskiej Organizacja i Zarządzanie, 77, 179–193. https://doi.org/h9f7
Maqableh, M., & Alia, M. (2021). Evaluation online learning of undergraduate students under lockdown amidst COVID-19 Pandemic: The online learning experience and students' satisfaction. Children and Youth Services Review, 128, 1-11. https://doi.org/gn3cw2
Maudsley, G., & Taylor, D. (2020). Analysing synthesis of evidence in a systematic review in health professions education: Observations on struggling beyond Kirkpatrick. Medical Education Online, 25(1), 1-10. https://doi.org/h9f8
Mestry, R. (2017). Empowering principals to lead and manage public schools effectively in the 21st century. South African Journal of Education, 37(1), 1–11. https://doi.org/10.15700/saje.v37n1a1334
Miles, M. B., Huberman, A. M., & Saldana, J. (2014). Qualitative data analysis, a methods sourcebook, (4th ed.). Sage Publications. https://bit.ly/3aK8kCz
Mohebbi, N., Akhlaghi, F., Yarmohammadian, M. H., & Khoshgam, M. (2011). Application of CIPP model for evaluating the medical records education course at master of science level at Iranian medical sciences universities. Procedia - Social and Behavioral Sciences, 15, 3286–3290. https://doi.org/10.1016/j.sbspro.2011.04.287
Mohmmed, A. O., Khidhir, B. A., Nazeer, A., & Vijayan, V. J. (2020). Emergency remote teaching during Coronavirus pandemic: The current trend and future directive at Middle East College Oman. Innovative Infrastructure Solutions, 5(3), 1–11. https://doi.org/10.1007/s41062-020-00326-7
Mpofu, M., & Hlatywayo, C. K. (2015). Training and development as a tool for improving basic service delivery; the case of a selected municipality. Journal of Economics, Finance and Administrative Science, 20(39), 133–136. https://doi.org/10.1016/j.jefas.2015.10.004
Nácher, M. J., Badenes-Ribera, L., Torrijos, C., Ballesteros, M. A., & Cebadera, E. (2021). The effectiveness of the gokoan e-learning platform in improving university students' academic performance. Studies in Educational Evaluation, 70, 1-9. https://doi.org/10.1016/j.stueduc.2021.101026
Nasser, M. M., & Ja'ashan, H. (2020). The challenges and prospects of using e-learning among EFL Students in Bisha University. Arab World English Journal, 11(1), 124–137. https://doi.org/10.24093/awej/vol11no1.11
Neyazi, N., Arab, P. M., Farzianpour, F., & Mahmoudi Majdabadi, M. (2016). Evaluation of selected faculties at Tehran University of medical sciences using CIPP model in students and graduates point of view. Evaluation and Program Planning, 59, 88–93. https://doi.org/f87672
Norberg, K. (2019). The Swedish national principal training programme: A programme in constant change. Journal of Educational Administration and History, 51(1), 5–14. https://doi.org/gd8vh4
Park, J., Liu, D., Yi, M. Y., & Santhanam, R. (2019). GAMESIT: A gamified system for information technology training. Computers and Education, 142, 1-43. https://doi.org/10.1016/j.compedu.2019.103643
Park, Y., & Jo, I. H. (2019). Factors that affect the success of learning analytics dashboards. Educational Technology Research and Development, 67(6), 1547–1571. https://doi.org/10.1007/s11423-019-09693-0
Paull, M., Whitsed, C., & Girardi, A. (2016). Applying the Kirkpatrick model: Evaluating an interaction for learning framework curriculum intervention. Issues in Educational Research, 26(3), 490–507.
Purinton, T., & Khalil, D. (2016). Adaptations of international standards on educational leadership preparation in Egypt. Educational Considerations, 43(3), 2–9. https://doi.org/10.4148/0146-9282.1015
Quraishi, U., & Aziz, F. (2016). Selection process for principals in northern Pakistan: Key issues and new directions. Asian Journal of Social Sciences and Humanities, 5(2), 192–206. https://cutt.ly/RX1GfdX
Rachmawati, Y., & Suyatno, S. (2021). The effect of principals' competencies on teachers' job satisfaction and work commitment. Participatory Educational Research, 8(1), 362–378. https://doi.org/10.17275/per.21.21.8.1
Ragsdale, J. W., Berry, A., Gibson, J. W., Herber-Valdez, C. R., Germain, L. J., & Engle, D. L. (2020). Evaluating the effectiveness of undergraduate clinical education programs. Medical Education Online, 25(1), 1-6. https://doi.org/h9f9
Ruskanda, L. (2018). Implementation of the Kirkpatrick model training program evaluation. International Journal of Advanced Research, 6(8), 878–892. https://doi.org/10.21474/ijar01/7590
Sahni, J. (2020). Managerial training effectiveness: An assessment through Kirkpatrick framework. TEM Journal, 9(3), 1227–1233. https://doi.org/10.18421/TEM93-51
Salmah, S., Ahmad, A., & Idkham, A. M. (2020). The effect of principals competencies on teachers' performance of public schools in Makassar city. Universal Journal of Educational Research, 8(12), 7825–7832. https://doi.org/10.13189/ujer.2020.082571
Schukro, C., Emich, M., Fritzer-Szekeres, M., Strametz-Juranek, J., & Sponder, M. (2019). Paper based training diaries for monitoring of performance progress due to long-term physical activity. Polish Archives of Internal Medicine, 129(10), 679–685. https://doi.org/10.20452/pamw.14975
Shi, X. L. (2018). Research on performance evaluation system of college entrepreneurship education level based on CIPP model. Educational Sciences: Theory & Practice, 18(5), 1494–1506. https://doi.org/10.12738/estp.2018.5.046
Shih, Y. C. D., & Yuan, Y. P. (2019). Evaluating an English elite program in Taiwan using the CIPP model. Journal of Asia TEFL, 16(1), 200–219. https://doi.org/gmvdnq
Smith, M. G., & Rains, L. (2020). Evaluation of an accredited training program on implementation of point-of-care testing in community pharmacies. Journal of the American Pharmacists Association, 60(6), 140–144. https://doi.org/10.1016/j.japh.2020.04.015
Sopha, S., & Nanni, A. (2019). The CIPP model: Applications in language program evaluation. Journal of Asia TEFL, 16(4), 1360–1367. https://doi.org/gmvdnp
Sriprasertpap, K. (2015). The development of online training model for Srinakharinwirot University in Thailand. Procedia - Social and Behavioral Sciences, 197, 1913–1917. https://doi.org/h9hm
Steele, L. M., Mulhearn, T. J., Medeiros, K. E., Watts, L. L., Connelly, S., & Mumford, M. D. (2016). How do we know what works? A review and critique of current practices in ethics training evaluation. Accountability in Research, 23(6), 319–350. https://doi.org/gh7hqb
Stufflebeam, D. L., & Zhang, G. (2017). The CIPP evaluation model how to evaluate for improvement and accountability. Guilford Publications. https://bit.ly/3yPIc15
Sunita, D., & Ajeya, J. (2010). Training need assessment: A critical study. Advances in Management, 3(1), 59–65. https://bit.ly/3oW3Yf5
Tuna, H., & Başdal, M. (2021). Curriculum evaluation of tourism undergraduate programs in Turkey: A CIPP model-based framework. Journal of Hospitality, Leisure, Sport and Tourism Education, 29, 1–9. https://doi.org/10.1016/j.jhlste.2021.100324
Ukkonen-Mikkola, T., & Varpanen, J. (2020). Integrated initial and continuing training as a way of developing the professional agency of teachers and student teachers. Teaching and Teacher Education, 96, 103189. https://doi.org/10.1016/j.tate.2020.103189
Welsh, E. T., Wanberg, C. R., Brown, K. G., & Simmering, M. J. (2003). E-learning: Emerging uses, empirical results and future directions. International Journal of Training and Development, 7(4), 245–258. https://doi.org/c64w5j
Winingsih, L. H., & Sulistiono, A. A. (2020). Factors influencing the principal's leadership and its impact on learning quality and learning outcome. Journal of Educational and Social Research, 10(2), 143–156. https://doi.org/10.36941/jesr-2020-0034
Wolor, C. W., Solikhah, S., Fidhyallah, N. F., & Lestari, D. P. (2020). Effectiveness of e-training, e-leadership, and work life balance on employee performance during COVID-19. Journal of Asian Finance, Economics and Business, 7(10), 443–450. https://doi.org/h9gb
Yu-Fong Chang, J., Wang, L. H., Lin, T. C., Cheng, F. C., & Chiang, C. P. (2021). Comparison of learning effectiveness between physical classroom and online learning for dental education during the COVID-19 pandemic. Journal of Dental Sciences, 16(4), 1281–1289. https://doi.org/10.1016/j.jds.2021.07.016
Zahro, S., & Wu, M. C. (2016). Implementing of the employees training evaluation using kirkpatrick's model in tourism industry a case study. International Journal of Innovation and Applied Studies, 17(3), 1042–1049. https://bit.ly/3zchwsE
Zainab, B., Bhatti, M. A., & Pangil, F. (2017). An examination of the factors affecting the adoption of e-training in the Nigerian civil service sector. Global Business and Organizational Excellence, 36(4), 33–42. https://doi.org/10.1002/joe.21788
Zalat, M. M., Hamed, M. S., & Bolbol, S. A. (2021). The experiences, challenges, and acceptance of e-learning as a tool for teaching during the COVID-19 pandemic among university medical staff. PLOS ONE, 16, 1–12. https://doi.org/gmb98b