Assessments on Vocational Knowledge and Skills: A Content Validity Analysis
While previous studies on assessments focused on measurement of principles and constructs, existing attention is given to content validation involving.
- Pub. date: July 15, 2021
- Pages: 1529-1540
- 269 Downloads
- 461 Views
- 5 Citations
While previous studies on assessments focused on measurement of principles and constructs, existing attention is given to content validation involving vocational education and skills. The emphasis on content validation has prompted a holistic perspective of teaching and learning to demand alternative research approaches. Using evidence of content validity based on expert judgment and assessment items, this paper argues that vocational knowledge and skills could be determined via constructs and construct functions. Content validity analysis was studied in two major phases, namely, through an assessment of an electrical technology course at a Malaysian higher education institution and expert panels’ examination of items. It was found that to increase confidence in providing reliable instruments for future empirical studies, a careful process in item development and content validity analysis was considered important. Therefore, applying these findings on item analysis and expert panels to reflect content validity can enhance the validity of assessment items.
Keywords: Assessments, content validity analysis, I-CVI, modified kappa, vocational knowledge.
References
Agarwal, P. K. (2019). Retrieval practice & Bloom’s taxonomy: Do students need fact knowledge before higher order learning? Journal of Educational Psychology, 111(2), 189–209. https://doi.org/10.1037/edu0000282
Agency, M. Q. (2013). Garis panduan amalan baik: Penilaian pelajar [Good practice guidelines: Student assessment]. In Agensi Kelayakan Malaysia [Malaysian Qualifications Agency] (MQA). www.mqa.gov.my
Akademik, P. (2017). Silibus: Sistem kawalan [Syllabus: Control system]. BPTV, KPM.
Akbari, R., & Yazdanmehr, E. (2014). A critical analysis of the selection criteria of expert teachers in ELT. Theory and Practice in Language Studies, 4(8), 1653–1658. https://doi.org/10.4304/tpls.4.8.1653-1658
Anderson, L. W., & Krathwohl, D. R. (2001). A Taxonomy for learning, teaching and assessing: A revision of Bloom’s taxonomy of educational objectives. Longman.
Black, P., & Wiliam, D. (2018). Classroom assessment and pedagogy. Assessment in Education: Principles, Policy and Practice, 25(6), 551–575. https://doi.org/10.1080/0969594X.2018.1441807
Carmines, E. G., & Zeller, R. A. (2014). Reliability and validity assessment. Sage Publications.
Cicchetti, D. V., & Sparrow, S. (1981). Developing criteria for establishing interrater reliability of specific items: Application to assessment of adaptive behavior. American Journal of Mental Deficiency, 86(2), 127–137.
Clark, R. W., Threeton, M. D., & Ewing, J. C. (2010). The potential of experiential learning models and practices in career and technical education & career and technical teacher education. Journal of Career and Technical Education, 25(2), 46–62. https://doi.org/10.21061/JCTE.V25I2.479
Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education (6th ed.). Routledge Taylor & Francis. https://doi.org/10.4324/9780203029053
Council, N. Q. (2009). Guide for developing assesment tools (Issue 2). Work-based Education Research Centre of Victoria University. https://cutt.ly/bmddLEh
Daniel, J., & Hultin, G. (2002). Technical and vocational education and training for the twenty-first Century: UNESCO and ILO recommendations. UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000126050
Davis, L. L. (1992). Instrument review: Getting the most from a panel of experts. Applied Nursing Research, 5(4), 194–197. https://doi.org/10.1016/S0897-1897(05)80008-4
Dimopoulos, D. I., & Pantis, J. D. (2003). Knowledge and attitudes regarding sea turtles in elementary students on Zakynthos, Greece. Journal of Environmental Education, 34(3), 30–38. https://doi.org/10.1080/00958960309603491
Education, K. D. (2013). Competency-based education: Helping all Kentucky students succeed. In Kentucky Department of Education. http://www.education.ky.gov
Ekonomi, U. P. (2016). Buku rancangan Malaysia kesebelas 2016-2020 [Malaysia’s eleventh plan book, 2016-2020]. Percetakan Nasional Malaysia Berhad. https://www.kkmm.gov.my/pdf/KPI/PPD KSP.pdf
Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378–382. https://doi.org/10.1037/h0031619
Gay, L. R., Mills, G. E., & Airasian, P. W. (2012). Educational research: Competencies for analysis and applications (10th ed.). Pearson.
Ghafar, M. N. A. (1999). Penyelidikan pendidikan [Educational research] (Vol. 11). Penerbit Universiti Teknologi Malaysia.
Ghafar, M. N. A. (2011). Pembinaan dan analisis ujian bilik darjah [Developing and analysing classroom tests] (2nd ed.). Universiti Teknologi Malaysia.
Gonczi, A. (1996). Reconceptualising competency-based education and training: With particular reference to education for occupations in Australia. University of Technology.
Hamid, R., Baharom, S., Hamzah, N., Wan, W. H., Atiq, R., Rahmat, O. K., & Raihantaha, M. (2012). Assessment of psychomotor domain in materials technology laboratory work. Procedia - Social and Behavioral Sciences, 56, 718–723. https://doi.org/10.1016/j.sbspro.2012.09.708
Jalal, W. N. A. W., Jalal, W. N. W., Sharif, A. M., & Rahman, N. A. (2017). Penggredan semula sekolah menengah vokasional kepada kolej vokasional: Kajian tahap kompetensi kaedah pengajaran dan pembelajaran pengajar [Revaluation of vocational secondary schools as vocational colleges: An investigation into the level of competence]. SkillsMalaysia Journal, 3(1), 9–16.
James, M. (2010). Educational assessment: Overview. In P. Peterson, E. Baker, & B. McGaw (Eds.), International Encyclopedia of Education (pp. 161–171). Elsevier.
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy an overview. Theory Into Practice, 41(4), 212–218. https://doi.org/10.1207/s15430421tip4104_2
Kyobe, E., & Rugumayo, A. (2005). Competency based education and training (CBET): A case study in Uganda. In K. S (Ed.), Proceedings of the 31st WEDC International Conference, Kampala, Uganda, 2005: Maximizing The Benefits From Water And Environmental Sanitation (pp. 189–191). Loughborough University.
Likisa, K. D. (2018). Challenges and prospects of competency-based education: The case of adama science and technology university alumni students and hawas TVET college, Adama, Ethiopia. The Journal of Competency-Based Education, 3(2), 1–7. https://doi.org/10.1002/cbe2.1163
Lynn, M. R. (1986). Determination and quantification of content validity. Nursing Research, 35(6), 382–385. https://doi.org/10.1097/00006199-198611000-00017
Majid, L. A., Abdullah, W. N. W., & Zakhi, N. H. A. (2012). Penerapan nilai murni dan pembentukan jati diri kanak-kanak prasekolah melalui penggunaan multimedia [The application of values and the formation of preschool children’s identity through the use of multimedia]. Hadhari Journal / Jurnal Hadhari, 2012(Special Edition), 51–65.
Makki, M. H., Abd-El-Khalick, F., & Boujaoude, S. (2003). Lebanese secondary school students’ environmental knowledge and attitudes. Environmental Education Research, 9(1), 21–33. https://doi.org/10.1080/1350462032000034340
Malaysian Institute of Teacher Education [Institut Pendidikan Guru Malaysia]. (2019). Buku panduan pentaksiran [Assessment handbook]. Institut Pendidikan Guru Malaysia [Malaysian Institute of Teacher Education]. https://cutt.ly/Lmdgqkp
Malaysian Ministry of Education. (2012). Pelan strategik interim 2011-2020 [Interim strategic plan 2011-2020]. Kememterian Pelajaran Malaysia [Malaysian Ministry of Education]. https://cutt.ly/TmdgnZV
Malaysian Ministry of Education. (2013). Pelan pembangunan pendidikan Malaysia 2013-2025 [Malaysia’s Education Development Plan 2013-2025]. Kememterian Pelajaran Malaysia [Malaysian Ministry of Education]. https://cutt.ly/wmdgAfi
Malaysian Ministry of Education. (2015). Pelan pembangunan pendidikan Malaysia 2015-2025 (Pendidikan Tinggi) [Malaysia’s Education Development Plan 2013-2025 (Higher Education]. Kememterian Pelajaran Malaysia [Malaysian Ministry of Education]. https://cutt.ly/hmdgDK3
Mazin, K. A., Norman, H., Nordin, N., & Ibrahim, R. (2020). Student self-recording videos for TVET competency in MOOCs. Journal of Physics: Conference Series, 1529, 1–8. https://doi.org/10.1088/1742-6596/1529/4/042061
McGrath, S. (2014). Revising the 2001 revised recommendation concerning technical and vocational education. UNESCO-UNEVOC. https://files.eric.ed.gov/fulltext/ED560511.pdf
Meyer-Adams, N., Potts, M. K., Koob, J. J., Dorsey, C. J., & Rosales, A. M. (2011). How to tackle the shift of educational assessment from learning outcomes to competencies : One program’s transition. Journal of Social Work Education, 47(3), 489–507. https://doi.org/10.5175/JSWE.2011.201000017
Mokhtar, M. H. (2016). Pentaksiran pendidikan teori dan praktis ke arah transformasi sistem pendidikan negara [Assessment of theoretical and practical education in the context of national education system transformation]. Kentalmas Jati Sdn. Bhd.
Omar, M. K., Rashid, A. M., & Puad, M. H. M. (2018). Examining job satisfaction factors toward retaining Malaysian TVET instructors in the teaching profession. International Journal of Engineering and Technology, 7(2), 44–49. https://doi.org/10.14419/ijet.v7i2.10.10952
Polit, D. F., Beck, C. T., & Owen, S. V. (2007). Focus on research methods: Is the cvi an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30, 459–467. https://doi.org/10.1002/nur.20199
Russell, M. K., & Airasian, P. W. (2011). Classroom assessment: Concepts and applications (7th ed.). McGraw-Hill Education.
Saha, S. K. (2021). Towards development of a system for automatic assessment of the quality of a question paper. Smart Learning Environments, 8(1), 1–14. https://doi.org/10.1186/s40561-021-00148-9
Simpson, E. J. (1966). The classification of educational objectives, psychomotor domain. ERIC. https://files.eric.ed.gov/fulltext/ED010368.pdf
Ulum, A. R., Hidayah, N., & Yanti, Y. (2021). Development of assessment hots (higher order thinking skills) based on troubleshooting for class v sd/mi. Journal of Madrasah Ibtidaiyah Education/ Jurnal Pendidikan Madrasah Ibtidaiyah, 5(1), 15–29. https://doi.org/10.32934/jmie.v5i1.222