logo logo European Journal of Educational Research

EU-JER is is a, peer reviewed, online academic research journal.

Subscribe to

Receive Email Alerts

for special events, calls for papers, and professional development opportunities.

Subscribe

Publisher (HQ)

Eurasian Society of Educational Research
Eurasian Society of Educational Research
7321 Parkway Drive South, Hanover, MD 21076, USA
Eurasian Society of Educational Research
Headquarters
7321 Parkway Drive South, Hanover, MD 21076, USA
context input process product effectiveness e training evaluation kirkpatrick

A Combination of Context Input Process Product and Kirkpatrick Evaluation Model to Determine the Effectiveness of E-Training for Principals during COVID-19 Pandemic

Yari Dwikurnaningsih , Marinu Waruwu , Krisma Widi Wardani

COVID-19 pandemic forces training for principals to be conducted online. This study aims to evaluate the context, input, process, response, study, beh.

C

COVID-19 pandemic forces training for principals to be conducted online. This study aims to evaluate the context, input, process, response, study, behavior, and outcome to determine the effectiveness and make recommendations for training. The approach used in this study was a mixed method with a concurrent embedded design and a qualitative method as the main method. The subjects of this study were the principals, committees, and instructors. Data were collected through interviews, observations, and questionnaires and analyzed using Miles and Huberman's model, descriptive analysis, and a t-test. The results show that e-training is effective in context, input, and process. Response, learning, and attitude assessments prove that knowledge, skills, and attitude have improved. Participants will be able to implement the experience gained and impact school quality improvement. This study contributes to the combination of the two evaluation models proven to produce a complete result. The study for the e-training recommends needed assessment before the training, the activity before the training to acquire the skills in using the learning management system, and the monitoring and evaluation after the training.

Keywords: Context input process product, effectiveness, e-training, evaluation, Kirkpatrick

cloud_download PDF
Cite
Article Metrics
Views
506
Download
598
Citations
Crossref
2

Scopus
0

References

Abdulghani, H. M., Shaik, S. A., Khamis, N., Al-Drees, A. A., Irshad, M., Khalil, M. S., Alhaqwi, A. I., & Isnani, A. (2014). Research methodology workshops evaluation using the Kirkpatrick's model: Translating theory into practice. Medical Teacher, 36(1), 24–29. https://doi.org/ggzfvr

Agarwal, N., Pande, N., & Ahuja, V. (2014). Expanding the Kirkpatrick evaluation model towards more efficient training in the IT sector. International Journal of Human Capital and Information Technology Professionals, 5(4), 19–34. https://doi.org/10.4018/ijhcitp.2014100102

Al-Hamad, N. Q., Rathwan, A. M., & Rababah, E. Q. (2020). Training needs for kindergarten principals in the light of digital age requirements. International Journal of Early Childhood Special Education, 12(2), 125–136. https://doi.org/h9bt

Al-Shanawani, H. M. (2019). Evaluation of self-learning curriculum for kindergarten using Stufflebeam's CIPP model. SAGE Open, 9(1), 1–13. https://doi.org/h9bv

Al hila, A. A., Alhelou, S. E. M., Al Shobaki, M. J., & Abu Naser, S. S. (2017). The impact of applying the dimensions of IT governance in improving e-training case study of the ministry of telecommunications and information technology in Gaza governorates. International Journal of Engineering and Information Systems, 1(8), 194–219. https://bit.ly/3OiSJaC

Alavudeen, S. S., Easwaran, V., Mir, J. I., Shahrani, S. M., Aseeri, A. A., Khan, N. A., Almodeer, A. M., & Asiri, A. A. (2021). The influence of COVID-19 related psychological and demographic variables on the effectiveness of e-learning among health care students in the southern region of Saudi Arabia. Saudi Pharmaceutical Journal, 29(7), 775–780. https://doi.org/10.1016/j.jsps.2021.05.009

Alkali, A. U., & Mansor, N. N. A. (2017). Interactivity and trust as antecedents of e-training use intention in Nigeria: A structural equation modelling approach. Behavioral Sciences, 7(3). https://doi.org/10.3390/bs7030047

Altavilla, G. (2019). Monitoring training to adequate the teaching method in training: An interpretative concepts. Journal of Physical Education and Sport, 19(5), 1763–1766. https://doi.org/10.7752/jpes.2019.s5258

Amin, I., Yousaf, A., Walia, S., & Bashir, M. (2021). What shapes e-learning effectiveness among tourism education students? an empirical assessment during COVID19. Journal of Hospitality, Leisure, Sport and Tourism Education, 30, 1-13. https://doi.org/h9bw

Aziz, S., Mahmood, M., Rehman, Z., & Report, A. (2018). Implementation of CIPP model for quality evaluation at school level: A case study. Journal of Education and Educational Development, 5(1), 189-206.

Baczek, M., Zaganczyk-Baczek, M., Szpringer, M., Jaroszynski, A., & Wozakowska-Kapłon, B. (2021). Student’s perception of online learning during COVID pandemic. Medicine, 100(7), 1–6. https://doi.org/h9bx

Belaya, V. (2018). The use of e-learning in vocational education and training (VET): Systematization of existing theoretical approaches. Journal of Education and Learning, 7(5), 92. https://doi.org/10.5539/jel.v7n5p92

Ben Amara, N., & Atia, L. (2016). E-training and its role inhuman-resources development. Global Journal of Human Resource Management, 4(1), 1–12. https://bit.ly/3OtMEbL

Bernardino, G., & Curado, C. (2020). Training evaluation: A configurational analysis of success and failure of trainers and trainees. European Journal of Training and Development, 44(4–5), 531–546. https://doi.org/gm9rh2

Brauckmann, S., Pashiardis, P., & Ärlestig, H. (2020). Bringing context and educational leadership together: Fostering the professional development of school principals. Professional Development in Education, 1–12. https://doi.org/gjd4kc

Calvo, S., Morales, A., & Wade, J. (2019). The use of MOOCs in social enterprise education: An evaluation of a North–South collaborative FutureLearn program. Journal of Small Business and Entrepreneurship, 31(3), 201–223. https://doi.org/h9bz

Chalikias, M., Raftopoulou, I., Sidiropoulos, G., Kyriakopoulos, G. L., & Zakopoulos, V. (2020). "The school principal's role as a leader in teachers" professional development: The case of public secondary education in Athens". Problems and Perspectives in Management, 18(4), 461–474. https://doi.org/10.21511/ppm.18(4).2020.37

Chang, T. Y., Hsu, M. L., Kwon, J. S., Kusdhany, M. L. S., & Hong, G. (2021). Effect of online learning for dental education in Asia during the pandemic of COVID-19. Journal of Dental Sciences, 16(4), 1095–1101. https://doi.org/h9b2

Cotes, J., & Ugarte, S. M. (2021). A systemic and strategic approach for training needs analysis for the International Bank. Journal of Business Research, 127, 464–473. https://doi.org/10.1016/j.jbusres.2019.05.002

Deodhar, M., & Powdwal, S. (2017). Impact of continuing education programs (CEPs) on LIS professionals in academic libraries in Mumbai, India. Library Management, 38(2/3), 117–130. https://doi.org/h9b3

Dewi, L. R., & Kartowagiran, B. (2018). An evaluation of internship program by using Kirkpatrick evaluation model. Research and Evaluation in Education, 4(2), 155–163. https://doi.org/h9b4

Dhull, I., & Sakshi, M. (2017). Online learning. International Education & Research Journal, 3(8), 32–34. https://bit.ly/3PLwyuA

Divayana, D. G. H., Sanjaya, D. B., Marhaeni, A. A. I. N., & Sudirtha, I. G. (2017). CIPP evaluation model based on mobile phone in evaluating the use of blended learning platforms at vocational schools in Bali. Journal of Theoretical and Applied Information Technology, 95(9), 1983–1995. https://bit.ly/3Ok0E7E

Dorri, S., Akbari, M., & Dorri Sedeh, M. (2016). Kirkpatrick evaluation model for in-service training on cardiopulmonary resuscitation. Iranian Journal of Nursing and Midwifery Research, 21(5), 493–497. https://doi.org/h9b8

Embi, Z., Neo, T.-K., & Neo, M. (2017). Using Kirkpatrick's evaluation model in a multimedia-based blended learning environment. Journal of Multimedia Information System, 4(3), 115–122. https://doi.org/h9bs

Esgaiar, E., & Foster, S. (2019). Implementation of CIPP model for quality evaluation at Zawia University. International Journal of Applied Linguistics and English Literature, 8(5), 106-115. https://doi.org/10.7575/aiac.ijalel.v.8n.5p.106

Finney, T. L. (2019). Confirmative evaluation: New CIPP evaluation model. Journal of Modern Applied Statistical Methods, 18(2), 2–24. https://doi.org/h9b9

Florea, N. V., Duica, A., & Duica, M. (2016). Using models and evaluation planning to improve corporate training activity and trainee performance. Valahian Journal of Economic Studies, 7(1), 45–56. https://bit.ly/3PhjgGM

Foster, C., Rodriguez-Marroyo, J. A., & De Koning, J. J. (2017). Monitoring training loads: The past, the present, and the future. International Journal of Sports Physiology and Performance, 12, 2–8. https://doi.org/gfvxfm

Garg, S., & Sharma, S. (2020). User satisfaction and continuance intention for using e-training: A structural equation model. Vision, 24(4), 441–451. https://doi.org/h9cb

Gokmenoglu, T., Sonmez, E. D., Yavuz, I., & Gok, A. (2021). Turkish ministry of national education school-based disaster education program: A preliminary results of the program evaluation. International Journal of Disaster Risk Reduction, 52, 1-8. https://doi.org/10.1016/j.ijdrr.2020.101943

Gumede, L., & Badriparsad, N. (2021). Online teaching and learning through the students' eyes – Uncertainty through the COVID-19 lockdown: A qualitative case study in Gauteng province, South Africa. Radiography, 28(1), 193–198. https://doi.org/10.1016/j.radi.2021.10.018

Gurmu, T. G. (2020). Primary school principals in Ethiopia: Selection and preparation. Educational Management Administration and Leadership, 48(4), 651–681. https://doi.org/h9cc

Hakan, K., & Seval, F. (2011). CIPP evaluation model scale: Development, reliability and validity. Procedia - Social and Behavioral Sciences, 15, 592–599. https://doi.org/fft545

Hanaysha, J., & Tahir, P. R. (2016). Examining the effects of employee empowerment, teamwork, and employee training on job satisfaction. Procedia - Social and Behavioral Sciences, 219, 272–282. https://doi.org/h9cd

Hassan, A., Hassan, J., & Yen, T. A. (2020). E-Training and development, motivation and employee performance among academicians: Case study of academicians in UniMAP. Journal of Physics: Conference Series, 1529, 1-8. https://doi.org/h9cf

Heydari, M. R., Taghva, F., Amini, M., & Delavari, S. (2019). Using Kirkpatrick's model to measure the effect of a new teaching and learning methods workshop for health care staff. BMC Research Notes, 12(1), 1–5. https://doi.org/h9cg

Hofmeister, C., & Pilz, M. (2020). Using e-learning to deliver in-service teacher training in the vocational education sector: Perception and acceptance in Poland, Italy and Germany. Education Sciences, 10(7), 1–17. https://doi.org/10.3390/educsci10070182

Hussein, E., Daoud, S., Alrabaiah, H., & Badawi, R. (2020). Exploring undergraduate students' attitudes towards emergency online learning during COVID-19: A case from the UAE. Children and Youth Services Review, 119, 1-20. https://doi.org/gmwcj7

Indra, R., Kustati, M., Saregar, A., Warnis, Nelmawarni, & Yusuf, Y. Q. (2020). The effect of principals' leadership towards effective learning at an Indonesian secondary school. European Journal of Educational Research, 9(3), 1063–1074. https://doi.org/10.12973/eu-jer.9.3.1063

Ismail, A., Zaharudin, R., Hashim, N., & Ariffin, J. (2020). The impact of e-NPQEL on the continuance intention of using e-training among aspired school leaders in Malaysia. International Journal of Interactive Mobile Technologies, 14(19), 109–123. https://doi.org/10.3991/ijim.v14i19.15965

Jain, G., Sharma, N., & Shrivastava, A. (2021). Enhancing training effectiveness for organizations through blockchain-enabled training effectiveness measurement (BETEM). Journal of Organizational Change Management, 34(2), 439–461. https://doi.org/10.1108/JOCM-10-2020-0303

Jiang, N., Sumintono, B., Perera, C. J., Harris, A., & Jones, M. S. (2018). Training preparation and the professional development of principals in Henan Province, China: Formal and informal learning. Asia Pacific Education Review, 19(1), 41–51. https://doi.org/10.1007/s12564-017-9513-6

Jones, C., Fraser, J., & Randall, S. (2018). The evaluation of a home-based paediatric nursing service: Concept and design development using the Kirkpatrick model. Journal of Research in Nursing, 23(6), 492–501. https://doi.org/h9ch

Kamal, K., Alaghbari, M., & Atteia, M. (2016). E-training & employees' performance a practical study on the ministry of education in the kingdom of Bahrain. Journal of Resources Development and Management, 18, 1–8. https://bit.ly/3PBx7Y2

Karimnia, A., & Kay, E. (2015). An evaluation of the undergraduate TEFL program in Iran: A multi case study. International Journal of Instruction, 8(2), 83–98. https://doi.org/10.12973/iji.2015.827a

Kirkpatrick, D. L., & Kirkpatrick, J. D. (2007). Implementing the four levels. Berrett-Koehler Publishers. https://bit.ly/3IKUL25

König, J., Jäger-Biela, D. J., & Glutsch, N. (2020). Adapting to online teaching during COVID-19 school closure: Teacher education and teacher competence effects among early career teachers in Germany. European Journal of Teacher Education, 43(4), 608–622. https://doi.org/ghvsw8

Le, L. K., Tran, T., Le, H. T. T., & Le, T. T. T. (2021). A study on factors affecting school principals' competencies in Vietnam's mountainous provinces. Management in Education, 35(4), 174–181. https://doi.org/gg83r4

Lemay, D. J., Bazelais, P., & Doleck, T. (2021). Transition to online learning during the COVID-19 pandemic. Computers in Human Behavior Reports, 4, 1-9. https://doi.org/gm2jb3

Li, J., Qin, C., & Zhu, Y. (2021). Online teaching in universities during the covid-19 epidemic: A study of the situation, effectiveness and countermeasures. Procedia Computer Science, 187, 566–573. https://doi.org/h9cj

Liao, S.-C., & Hsu, S.-Y. (2019). Evaluating a continuing medical education program: New world Kirkpatrick model approach. International Journal of Management, Economics and Social Sciences, 8(4), 266–279. https://doi.org/h9ck

Lippe, M., & Carter, P. (2018). Using the CIPP model to assess nursing education program quality and merit. Teaching and Learning in Nursing, 13(1), 9–13. https://doi.org/h9f6

Ludwikowska, K. (2018). The effectiveness of training needs analysis and its relation to employee efficiency. Zeszyty Naukowe Politechniki Poznańskiej Organizacja i Zarządzanie, 77, 179–193. https://doi.org/h9f7

Maqableh, M., & Alia, M. (2021). Evaluation online learning of undergraduate students under lockdown amidst COVID-19 Pandemic: The online learning experience and students' satisfaction. Children and Youth Services Review, 128, 1-11. https://doi.org/gn3cw2

Maudsley, G., & Taylor, D. (2020). Analysing synthesis of evidence in a systematic review in health professions education: Observations on struggling beyond Kirkpatrick. Medical Education Online, 25(1), 1-10. https://doi.org/h9f8

Mestry, R. (2017). Empowering principals to lead and manage public schools effectively in the 21st century. South African Journal of Education, 37(1), 1–11. https://doi.org/10.15700/saje.v37n1a1334

Miles, M. B., Huberman, A. M., & Saldana, J. (2014). Qualitative data analysis, a methods sourcebook, (4th ed.). Sage Publications. https://bit.ly/3aK8kCz

Mohebbi, N., Akhlaghi, F., Yarmohammadian, M. H., & Khoshgam, M. (2011). Application of CIPP model for evaluating the medical records education course at master of science level at Iranian medical sciences universities. Procedia - Social and Behavioral Sciences, 15, 3286–3290. https://doi.org/10.1016/j.sbspro.2011.04.287

Mohmmed, A. O., Khidhir, B. A., Nazeer, A., & Vijayan, V. J. (2020). Emergency remote teaching during Coronavirus pandemic: The current trend and future directive at Middle East College Oman. Innovative Infrastructure Solutions, 5(3), 1–11. https://doi.org/10.1007/s41062-020-00326-7

Mpofu, M., & Hlatywayo, C. K. (2015). Training and development as a tool for improving basic service delivery; the case of a selected municipality. Journal of Economics, Finance and Administrative Science, 20(39), 133–136. https://doi.org/10.1016/j.jefas.2015.10.004

Nácher, M. J., Badenes-Ribera, L., Torrijos, C., Ballesteros, M. A., & Cebadera, E. (2021). The effectiveness of the gokoan e-learning platform in improving university students' academic performance. Studies in Educational Evaluation, 70, 1-9. https://doi.org/10.1016/j.stueduc.2021.101026

Nasser, M. M., & Ja'ashan, H. (2020). The challenges and prospects of using e-learning among EFL Students in Bisha University. Arab World English Journal, 11(1), 124–137. https://doi.org/10.24093/awej/vol11no1.11

Neyazi, N., Arab, P. M., Farzianpour, F., & Mahmoudi Majdabadi, M. (2016). Evaluation of selected faculties at Tehran University of medical sciences using CIPP model in students and graduates point of view. Evaluation and Program Planning, 59, 88–93. https://doi.org/f87672

Norberg, K. (2019). The Swedish national principal training programme: A programme in constant change. Journal of Educational Administration and History, 51(1), 5–14. https://doi.org/gd8vh4

Park, J., Liu, D., Yi, M. Y., & Santhanam, R. (2019). GAMESIT: A gamified system for information technology training. Computers and Education, 142, 1-43. https://doi.org/10.1016/j.compedu.2019.103643

Park, Y., & Jo, I. H. (2019). Factors that affect the success of learning analytics dashboards. Educational Technology Research and Development, 67(6), 1547–1571. https://doi.org/10.1007/s11423-019-09693-0

Paull, M., Whitsed, C., & Girardi, A. (2016). Applying the Kirkpatrick model: Evaluating an interaction for learning framework curriculum intervention. Issues in Educational Research, 26(3), 490–507.

Purinton, T., & Khalil, D. (2016). Adaptations of international standards on educational leadership preparation in Egypt. Educational Considerations, 43(3), 2–9. https://doi.org/10.4148/0146-9282.1015

Quraishi, U., & Aziz, F. (2016). Selection process for principals in northern Pakistan: Key issues and new directions. Asian Journal of Social Sciences and Humanities, 5(2), 192–206. https://cutt.ly/RX1GfdX

Rachmawati, Y., & Suyatno, S. (2021). The effect of principals' competencies on teachers' job satisfaction and work commitment. Participatory Educational Research, 8(1), 362–378. https://doi.org/10.17275/per.21.21.8.1

Ragsdale, J. W., Berry, A., Gibson, J. W., Herber-Valdez, C. R., Germain, L. J., & Engle, D. L. (2020). Evaluating the effectiveness of undergraduate clinical education programs. Medical Education Online, 25(1), 1-6. https://doi.org/h9f9

Ruskanda, L. (2018). Implementation of the Kirkpatrick model training program evaluation. International Journal of Advanced Research, 6(8), 878–892. https://doi.org/10.21474/ijar01/7590

Sahni, J. (2020). Managerial training effectiveness: An assessment through Kirkpatrick framework. TEM Journal, 9(3), 1227–1233. https://doi.org/10.18421/TEM93-51

Salmah, S., Ahmad, A., & Idkham, A. M. (2020). The effect of principals competencies on teachers' performance of public schools in Makassar city. Universal Journal of Educational Research, 8(12), 7825–7832. https://doi.org/10.13189/ujer.2020.082571

Schukro, C., Emich, M., Fritzer-Szekeres, M., Strametz-Juranek, J., & Sponder, M. (2019). Paper based training diaries for monitoring of performance progress due to long-term physical activity. Polish Archives of Internal Medicine, 129(10), 679–685. https://doi.org/10.20452/pamw.14975

Shi, X. L. (2018). Research on performance evaluation system of college entrepreneurship education level based on CIPP model. Educational Sciences: Theory & Practice, 18(5), 1494–1506. https://doi.org/10.12738/estp.2018.5.046

Shih, Y. C. D., & Yuan, Y. P. (2019). Evaluating an English elite program in Taiwan using the CIPP model. Journal of Asia TEFL, 16(1), 200–219. https://doi.org/gmvdnq

Smith, M. G., & Rains, L. (2020). Evaluation of an accredited training program on implementation of point-of-care testing in community pharmacies. Journal of the American Pharmacists Association, 60(6), 140–144. https://doi.org/10.1016/j.japh.2020.04.015

Sopha, S., & Nanni, A. (2019). The CIPP model: Applications in language program evaluation. Journal of Asia TEFL, 16(4), 1360–1367. https://doi.org/gmvdnp

Sriprasertpap, K. (2015). The development of online training model for Srinakharinwirot University in Thailand. Procedia - Social and Behavioral Sciences, 197, 1913–1917. https://doi.org/h9hm

Steele, L. M., Mulhearn, T. J., Medeiros, K. E., Watts, L. L., Connelly, S., & Mumford, M. D. (2016). How do we know what works? A review and critique of current practices in ethics training evaluation. Accountability in Research, 23(6), 319–350. https://doi.org/gh7hqb

Stufflebeam, D. L., & Zhang, G. (2017). The CIPP evaluation model how to evaluate for improvement and accountability. Guilford Publications. https://bit.ly/3yPIc15

Sunita, D., & Ajeya, J. (2010). Training need assessment: A critical study. Advances in Management, 3(1), 59–65. https://bit.ly/3oW3Yf5

Tuna, H., & Başdal, M. (2021). Curriculum evaluation of tourism undergraduate programs in Turkey: A CIPP model-based framework. Journal of Hospitality, Leisure, Sport and Tourism Education, 29, 1–9. https://doi.org/10.1016/j.jhlste.2021.100324

Ukkonen-Mikkola, T., & Varpanen, J. (2020). Integrated initial and continuing training as a way of developing the professional agency of teachers and student teachers. Teaching and Teacher Education, 96, 103189. https://doi.org/10.1016/j.tate.2020.103189

Welsh, E. T., Wanberg, C. R., Brown, K. G., & Simmering, M. J. (2003). E-learning: Emerging uses, empirical results and future directions. International Journal of Training and Development, 7(4), 245–258. https://doi.org/c64w5j

Winingsih, L. H., & Sulistiono, A. A. (2020). Factors influencing the principal's leadership and its impact on learning quality and learning outcome. Journal of Educational and Social Research, 10(2), 143–156. https://doi.org/10.36941/jesr-2020-0034

Wolor, C. W., Solikhah, S., Fidhyallah, N. F., & Lestari, D. P. (2020). Effectiveness of e-training, e-leadership, and work life balance on employee performance during COVID-19. Journal of Asian Finance, Economics and Business, 7(10), 443–450. https://doi.org/h9gb

Yu-Fong Chang, J., Wang, L. H., Lin, T. C., Cheng, F. C., & Chiang, C. P. (2021). Comparison of learning effectiveness between physical classroom and online learning for dental education during the COVID-19 pandemic. Journal of Dental Sciences, 16(4), 1281–1289. https://doi.org/10.1016/j.jds.2021.07.016

Zahro, S., & Wu, M. C. (2016). Implementing of the employees training evaluation using kirkpatrick's model in tourism industry a case study. International Journal of Innovation and Applied Studies, 17(3), 1042–1049. https://bit.ly/3zchwsE

Zainab, B., Bhatti, M. A., & Pangil, F. (2017). An examination of the factors affecting the adoption of e-training in the Nigerian civil service sector. Global Business and Organizational Excellence, 36(4), 33–42. https://doi.org/10.1002/joe.21788

Zalat, M. M., Hamed, M. S., & Bolbol, S. A. (2021). The experiences, challenges, and acceptance of e-learning as a tool for teaching during the COVID-19 pandemic among university medical staff. PLOS ONE, 16, 1–12. https://doi.org/gmb98b

...