logo logo European Journal of Educational Research

EU-JER is is a, peer reviewed, online academic research journal.

Subscribe to

Receive Email Alerts

for special events, calls for papers, and professional development opportunities.


Publisher (HQ)

Eurasian Society of Educational Research
Eurasian Society of Educational Research
7321 Parkway Drive South, Hanover, MD 21076, USA
Eurasian Society of Educational Research
7321 Parkway Drive South, Hanover, MD 21076, USA
e learning system higher education usability evaluation usability principles preference saudi arabia

Assessing the Relative Importance of an E-learning system’s Usability Design Characteristics Based on Students' Preferences

Ahmed Alshehri , Malcolm Rutter , Sally Smith

This study considers the interaction between an e-learning system, the Blackboard system, and the students who use it in Saudi Arabia. While previous .


This study considers the interaction between an e-learning system, the Blackboard system, and the students who use it in Saudi Arabia. While previous work exists, there is limited consideration of the assessment of the preferences of e-learning system usability variables based on students’ perspectives, especially in developing countries such as Saudi Arabia. This paper attempts to fill the gap by investigating the relative importance of the design criteria developed for e-learning system usability evaluation from the students’ perspective in Saudi tertiary education. Based on reviewed literature, a set of usability principles was developed that have had an influence in the students’ learning process and use of the e-learning system. The list includes system navigation, system learnability, visual design, information quality, instructional assessment and system interactivity. An exploratory study was carried out to identify the most important usability design characteristics from a student’s perspective and then evaluate the overall usability of the current e-learning system, based on this subset. A quantitative approach was adopted to weigh usability design characteristics, based on 181 learners’ perceptions. The sample consists of undergraduates who are users of a web-based e-learning system in a university in Saudi Arabia. The research instrument was tested for construct validity and reliability. The analysed results have shown that information quality is the most important dimension followed by the navigation of the e-learning system. The study has also revealed that the system learnability and visual design came third and fourth in order of importance of e-learning system usability assessment. Finally, the least important design categories that influenced the e-learning system usability assessment were instructional assessment and system interactivity. The empirical results of this study may help to provide insights for designers and evaluators leading to a more effective approach to improve the usability and uptake of the e-learning system.

Keywords: E-learning system, higher education, usability evaluation, usability principles preference, Saudi Arabia.

cloud_download PDF
Article Metrics



Agarwal, R., & Venkatesh, V. (2002). Assessing a firm’s web presence: A heuristic evaluation procedure for the measurement of usability. Information Systems Research, 13(2), 168–186.

Aldiab, A., Chowdhury, H., Kootsookos, A., Alam, F., & Allhibi, H. (2019). Utilization of Learning Management Systems (LMSs) in higher education system: A case review for Saudi Arabia. Energy Procedia, 160, 731–737. http://doi.org/https://doi.org/10.1016/j.egypro.2019.02.186

Alenezi, A. R., Abdul Karim, A. M., & Veloo, A. (2011). Institutional support and e-learning acceptance : An extension of the technology acceptance model. International Journal of Instructional Technology and Distance Learning, 8(2), 3-16.

Asarbakhsh, M., & Sandars, J. (2013). E-learning: the essential usability perspective. Clinical Teacher, 10(1), 47–50. http://doi.org/10.1111/j.1743-498X.2012.00627.x

Bouznif, M. M. (2018). Business students’ continuance intention toward Blackboard usage: An empirical investigation of UTAUT model. International Journal of Business and Management, 13(1), 120. http://doi.org/10.5539/ijbm.v13n1p120

Bringula, R. P., & Basa, R. S. (2011). Factors affecting faculty web portal usability. Educational Technology and Society, 14(4), 253–265.

Bryman, A., & Bell, E. (2015). Business Research Methods. Oxford, UK: Oxford University Press.

Calisir, F., Elvan Bayraktaroğlu, A., Altin Gumussoy, C., İlker Topcu, Y., & Mutlu, T. (2010). The relative importance of usability and functionality factors for online auction and shopping web sites. Online Information Review, 34(3), 420–439. http://doi.org/10.1108/14684521011037025

Cebi, S. (2013). Determining importance degrees of website design parameters based on interactions and types of websites. Decision Support Systems, 54(2), 1030–1043. http://doi.org/10.1016/j.dss.2012.10.036

Cho, V., Cheng, T. C. E., & Lai, W. M. J. (2009). The role of perceived user-interface design in continued usage intention of self-paced e-learning tools. Computers and Education, 53(2), 216–227. http://doi.org/10.1016/j.compedu.2009.01.014

Dix, A., Finlay, J. E., Abowd, G. D., & Beale, R. (2003). Human-Computer Interaction (3rd Edition). Upper Saddle River, NJ: Prentice-Hall, Inc.

Dringus, L. P., & Cohen, M. S. (2005). An adaptable usability heuristic checklist for online courses. Proceedings Frontiers in Education 35th Annual Conference, T2H–6–T2H–11. http://doi.org/10.1109/FIE.2005.1611918

Ellahi, A., & Bokhari, R. H. (2013). Key quality factors affecting users’ perception of social networking websites. Journal of Retailing & Consumer Services, 20(1), 120–129. Retrieved from

Fang, X., & Holsapple, C. W. (2007). An empirical study of web site navigation structures’ impacts on web site usability. Decision Support Systems, 43(2), 476–491. http://doi.org/https://doi.org/10.1016/j.dss.2006.11.004

Gable, G., Sedera, D., & Taizan, C. (2008). Re-conceptualizing information system success: The IS-impact measurement model. Journal of the Association for Information Systems, 9(7), 1–32.

Grossman, T., Fitzmaurice, G., & Attar, R. (2009). A survey of software learnability: metrics, methodologies and guidelines. Proceedings of the 27th International Conference on Human Factors in Computing Systems - CHI 09, 649–658. http://doi.org/10.1145/1518701.1518803

Guo, Y., Wang, J., Moore, J., Liu, M., & Chen, H. L. (2009). A case study of usability testing on an asynchronous e-Learning platform. In 2009 Joint Conferences on Pervasive Computing, JCPC 2009 (pp. 693–697). Red Hook, NY: IEEE eXpress Conference Publishing http://doi.org/10.1109/JCPC.2009.5420093

Hair, Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2010). Multivariate Data Analysis. Faculty Publications. New Jersey: Prentice-Hall. http://doi.org/10.1016/j.ijpharm.2011.02.019

Hasan, L. (2014). Evaluating the usability of educational websites based on students’ preferences of design characteristics. International Arab Journal of E-Technology, 3(3), 179–193.

Holden, H., & Rada, R. (2011). Understanding the influence of perceived usability and technology self-efficacy on teachers’ technology acceptance. Journal of Research on Technology in Education, 43(4), 343–367. http://doi.org/10.1080/15391523.2011.10782576

Horton, W. K. (2000). Designing Web-Based Training : How to Teach Anyone Anything Anywhere Anytime. New York, NY: Wiley.

Hrastinski, S. (2008). Asynchronous and synchronous e-learning. Educause Quarterly, 31(4), 51–55. http://doi.org/citeulike-article-id:8060645

Huang, X., & Hsiao, E. L. (2012). Synchronous and asynchronous communication in an online environment: Faculty experiences and perceptions. Quarterly Review of Distance Education, 13(1), 15–30.

Ivanovic, M., Putnik, Z., Komlenov, Z., Welzer, T., Holbl, M., & Schweighofer, T. (2013). Usability and privacy aspects of moodle: Students’ and teachers’ perspective. Informatica (Slovenia), 37(3), 221–230.

Kiget, N. K., Wanyembi, P. G., & Peters, A. I. (2014). Evaluating usability of e-Learning systems in universities. International Journal of Advanced Computer Science and Applications, 5(8), 97–102.

Kirsh, D. (2014). Metacognition, distributed cognition, and visual design. Cognition, Education, and Communication Technology, 147–180. http://doi.org/10.4324/9781410612892

Kline, R. B. (2016). Principles and practice of structural equation modeling, 4th ed. Principles and practice of structural equation modeling (4th ed). New York, NY: Guilford Press.

Koohang, A., & Du Plessis, J. (2004). Architecting usability properties in the e-learning instructional design process. International Journal on ELearning, 3(3), 38–44.

Lanzilotti, R., Ardito, C., Costabile, M. F., Informatica, D., Bari, U., & Angeli, A. De. (2006). eLSE methodology: a systematic approach to the e-learning systems evaluation. Educational Technology & Society, 9(4), 42–53.

Lin, H., Choong, Y. Y., & Salvendy, G. (1997). A proposed index of usability: A method for comparing the relative usability of different software systems. Behaviour and Information Technology, 16(4–5), 267–277. http://doi.org/10.1080/014492997119833

Lindgaard, G. (1994). Usability testing and system evaluation: A guide for designing useful computer systems. London, UK: Chapman & Hall.

Martin, F. (2013). Blackboard as the learning management system of a computer literacy course. Journal of Online Learning and Teaching, 4(2), 138–145. http://doi.org/10.1080/03043790903038841

Mehlenbacher, B., Bennett, L., Bird, T., Ivey, M., Lucas, J., Morton, J., & Whitman, L. (2005). Usable e-learning: A conceptual model for evaluation and design. In Proceedings of HCI International 2005: 11th International Conference on Human-Computer Interaction- Volume 4, (pp.1–10), Las Vegas, NV: Mira Digital.

Ministry of Education Saudi Arabia. (2019). Higher Education Statistics. Retrieved from https://www.moe.gov.sa/en/Pages/default.aspx

Moreno, V., Cavazotte, F., & Alves, I. (2017). Explaining university students’ effective use of e-learning platforms. British Journal of Educational Technology, 48(4), 995–1009. http://doi.org/10.1111/bjet.12469

Mustafa, S. H., & Al-Zoua’bi, L. F. (2009). Usability of the academic websites of Jordan’s universities an evaluation study. In the Proceedings of the 9th Arab Conference on Information Technology - ACIT 2008 (pp. 1-9). Zarqa, Jordan: ACIT International.

Nielsen, J. (1993). Usability Engineering. San Francisco, CA: Morgan Kaufmann Publishers Inc.

Noorulhasan, Q., Muhammad, A., Sanober, S., Rafik, M., & Shah, A. (2017). A mixed method study for investigating critical success factors (CSFs) of e-learning in Saudi Arabian universities. International Journal of Advanced Computer Science and Applications, 8(5). http://doi.org/10.14569/IJACSA.2017.080522

Orehovački, T., Granic, A., & Kermek, D. (2013). Evaluating the perceived and estimated quality in use of Web 2.0 applications. Journal of Systems and Software, 86(12), 3039–3059. http://doi.org/https://doi.org/10.1016/j.jss.2013.05.071

Oztekin, A., Kong, Z. J., & Uysal, O. (2010). UseLearn: A novel checklist and usability evaluation method for eLearning systems by criticality metric analysis. International Journal of Industrial Ergonomics, 40(4), 455–469. http://doi.org/10.1016/j.ergon.2010.04.001

Oztekin, A., Nikov, A., & Zaim, S. (2009). UWIS: An assessment methodology for usability of web-based information systems. Journal of Systems and Software, 82(12), 2038–2050. http://doi.org/https://doi.org/10.1016/j.jss.2009.06.047

Pallant, J. (2016). SPSS survival manual : a step by step guide to data analysis using IBM SPSS (6th editio). Berkshire, UK : McGraw-Hill Education.

Pearson, J. M., Pearson, A., & Green, D. (2007). Determining the importance of key criteria in web usability. Management Research News, 30(11), 816–828. http://doi.org/10.1108/01409170710832250

Pituch, K. A., & Lee, Y. kuei. (2006). The influence of system characteristics on e-learning use. Computers and Education, 47(2), 222–244. http://doi.org/10.1016/j.compedu.2004.10.007

Preece, J., Rogers, Y., & Sharp, H. (2015). Interaction Design: Beyond Human–Computer Interaction. Interaction Design: Beyond Human–Computer Interaction (4th ed). Chichester, UK: John Wiley & Sons.

Reeves, T. C., Benson, L., Elliott, D., Grant, M., Holschuh, D., Kim, B., … Loh, C. S. (2002). Usability and instructional design heuristics for e-learning evaluation. In P. Barker & S. Rebelsky (Eds.), World Conference on Educational Multimedia, Hypermedia and Telecommunications (pp. 1615–1621). Denver, Colorado, USA: Association for the Advancement of Computing in Education (AACE). Retrieved from http://eric.ed.gov/?id=ED477084

Rubin, B., Fernandes, R., Avgerinou, M. D., & Moore, J. (2010). The effect of learning management systems on student and faculty outcomes. The Internet and Higher Education, 13(1–2), 82–83. http://doi.org/10.1016/J.IHEDUC.2009.10.008

Salloum, S. A., & Shaalan, K. (2019). Factors affecting students’ acceptance of e-learning system in higher education using UTAUT and structural equation modeling approaches. In A. E. Hassnien, K. Shaalan & A.T. Azar (Eds), Proceedings of the International Conference on Advanced Intelligent Systems and Informatics (pp. 469–480). Cham, Switzerland: Springer. http://doi.org/10.1007/978-3-319-99010-1_43

Scholtz, B., Mahmud, I., Mahmud, I., & Ramayah, T. (2016). Does usability matter? An analysis of the impact of usability on technology acceptance in ERP settings. Interdisciplinary Journal of Information, Knowledge, and Management, 11, 309–330.

Selim, H. M. (2007). Critical success factors for e-learning acceptance: Confirmatory factor models. Computers and Education, 49(2), 396–413. http://doi.org/10.1016/j.compedu.2005.09.004

Shee, D. Y., & Wang, Y.-S. (2008). Multi-criteria evaluation of the web-based e-learning system: A methodology based on learner satisfaction and its applications. Computers & Education, 50(3), 894–905. http://doi.org/10.1016/j.compedu.2006.09.005

Shneiderman, B. (1997). Designing the user interface : strategies for effective human-computer-interaction (3rd ed.). Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc.

Smith, L., & Abouammoh, A. (2013). Higher education in Saudi Arabia: Achievements, challenges and opportunities. Springer, 40, 198. http://doi.org/10.1007/978-94-007-6321-0

Squires, D., & Preece, J. (1996). Usability and learning: Evaluating the potential of educational software. Computers and Education, 27(1), 15–22. http://doi.org/10.1016/0360-1315(96)00010-3

Ssemugabi, S., & De Villiers, R. (2007). Usability and learning: A framework for evaluation of web-based e-learning applications. World Conference on Educational Multimedia, Hypermedia and Telecommunications, 2007, 906–913.

Storey, M. A., Phillips, B., Maczewski, M., & Wang, M. (2002). Evaluating the usability of Web-based learning tools. Educational Technology & Society, 5(3), 91–100.

Sun, P., Tsai, R., Finger, G., Chen, Y. Y., & Yeh, D. (2008). What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers and Education, 50(4), 1183–1202. http://doi.org/10.1016/j.compedu.2006.11.007

Tarafdar, M., & Zhang, J. (2005). Analyzing the influence of website design parameters on website usability. Information Resources Management Journal, 18(4), 62–80.

Tee, S. S., Wook, T. S. M. T., & Zainudin, S. (2013). User testing for moodle application. International Journal of Software Engineering and Its Applications, 7(5), 243–252. http://doi.org/10.14257/ijseia.2013.7.5.22

Thowfeek, M. H., & Salam, M. N. A. (2014). Students’ assessment on the usability of E-leaming websites. In Procedia - Social and Behavioral Sciences, 141,  916–922. http://doi.org/10.1016/j.sbspro.2014.05.160

Thuseethan, S., Achchuthan, S., & Kuhanesan, S. (2014). Usability evaluation of learning management systems in Sri Lankan universities. Global Journal of Computer Science and Technology, 15(1), 15-24. Retrieved from http://arxiv.org/abs/1412.0197

Triacca, L., Bolchini, D., Botturi, L., & Inversini, A. (2004). Mile: Systematic usability evaluation for e-Learning web applications. In L. Cantoni & C. McLoughlin (Eds.), Proceedings of EdMedia: World Conference on Educational Media and Technology 2004 (pp. 4398–4405). Lugano, Switzerland: Association for the Advancement of Computing in Education (AACE). Retrieved from https://www.learntechlib.org/p/11709

Usability.gov. (2013). Visual Design Basics. Retrieved from http://www.usability.gov/what-and-why/visual-design.html

Van Nuland, S. E., Eagleson, R., & Rogers, K. A. (2017). Educational software usability: Artifact or Design? Anatomical Sciences Education, 10(2), 190–199. http://doi.org/10.1002/ase.1636

Wu, Y.-L., Tao, Y.-H., & Yang, P.-C. (2009). The discussion on influence of website usability towards user acceptability. In Proceedings - International Conference on Management and Service Science (MASS 2009) (pp. 1–4). Red Hook, NY: IEEE eXpress Conference Publishing. http://doi.org/10.1109/ICMSS.2009.5304443

Zaharias, P. (2009). Usability in the context of e-learning: A framework augmenting “traditional” usability constructs with instructional design and motivation to learn. International Journal of Technology and Human Interaction, 5(4), 37–59. http://doi.org/10.4018/jthi.2009062503

Zaharias, P., & Koutsabasis, P. (2011). Heuristic evaluation of e‐learning courses: a comparative analysis of two e‐learning heuristic sets. Campus-Wide Information Systems, 29(1), 45–60. http://doi.org/10.1108/10650741211192046

Zaharias, P., & Poylymenakou, A. (2009). Developing a usability evaluation method for e-learning applications: Beyond functional usability. International Journal of Human-Computer Interaction, 25(1), 75–98. http://doi.org/10.1080/10447310802546716

Zhang, P., von Dran, G. M., Blake, P., & Pipithsuksunt, V. (2001). Important design features in different web site domains: An empirical study of user perceptions. E-Service Journal, 1(1), 77–91. http://doi.org/10.2979/ESJ.2001.1.1.77