Analysis of the Effectiveness of Different Types of Distance Learning

The purpose of this paper is to determine the relevance of turning the camera on or off during distance learning as an argument for active or passive student participation. Seventy-five (75) students participated in the study and were divided into five groups (1-5) according to teaching method (i.e., synchronous instruction online with camera (1) and without camera (2), synchronous transmission of the recording online with camera (3) and without camera (4) and received the online instruction (5)) only. In the beginning and at the end, all students were tested with the same adapted test to determine general physical and motor status. All groups had the same training program twice a week for 45 minutes for 7 weeks. The first training of the week was dedicated to strength development, the second to endurance. In the end, all participants completed a questionnaire to determine their additional physical activity and how they felt about using a camera. The camera being turned on was identified as a factor that made participants uncomfortable but contributed significantly to the effectiveness of the course. However, 94.6 % of all participants cited non-camera methods as their favourite.


Introduction
The evaluation of new approaches in education is crucial to ensure the quality of educational work (Weaver et al., 2020). Different academic disciplines at different levels of education use different approaches to evaluate educational work (Berk, 2018). Most commonly, student opinions are used in Student Evaluation of Teaching (SET) to determine the success of educational work at the university level (Uttl et al., 2017).
However, a meta-analysis of studies has found that SET has little or no correlation with learning performance (Kreitzer & Sweet-Cushman, 2021). A review of meta-analytic studies reveals at least two aspects of weakness in using student opinions to evaluate teacher's work, namely (1) SETs do not measure teaching effectiveness (Uttl et al., 2017) and (2) males receive higher evaluative scores compared to females (Mengel et al., 2018). Students' opinions are certainly an important indicator, e.g., their well-being, their satisfaction with the implementation of the pedagogical process, etc., but they are not the ones on the basis of which it would be justified to fully evaluate the success of the educator or the pedagogical process.
The evaluation of pedagogical work must accompany the introduction of any new approach. However, it is usual for the new approach to first be tested on small samples and evaluated using a case study approach, followed by quasiexperiments, pedagogical experiments, meta-analytical studies, etc. The gradual introduction of a new online approach by controlling for important perceived influencing factors could not be guaranteed when the coronavirus epidemic broke out as school closures caught most teachers unprepared (Sheikh et al., 2020;United Nations Educational, Scientific and Cultural Organization, 2020). The coronavirus epidemic triggered a global education experiment, as school closures were one of the most common measures used to contain the spread of coronavirus worldwide (Van Lancker & Parolin, 2020;Viner et al., 2020). Although online learning has been around for many decades, most teachers were unprepared, resulting in increased stress for both teachers (Kim & Asbury, 2020) and students (Cachón-Zagalaz et al., 2020).
Online learning is often stigmatized as a weaker option that provides a lower quality of education than face-to-face learning (Hodges et al., 2020). Gierdowski (2019) on a sample of 11.141 faculty members from 131 U.S. institutions found that only 9% of faculty prefer to teach a fully online course, and a sample of 40,000 students from 118 U.S. universities revealed that 70% of the respondents prefer mostly or fully face-to-face learning environments. Even before the epidemic, students' opinions of online study differed greatly from those of their teachers.
The transition from traditional to distance learning could not be well planned because there was no time to plan. However, well-planned online learning is very different from the distance learning we experienced during the crisis, when only the transmission of content via video conferencing systems was used (Hodges et al., 2020).
Early on, teachers indicated that such a rapid transition to distance education was stressful for them (Besser et al., 2020). Many teachers faced teleworking for the first time and therefore faced new challenges in many aspects of teaching (e.g., how to motivate students to participate, how to present learning more effectively, how to record or present practical examples, etc.) (Klapproth et al., 2020). Various trainings on how to use the available distance learning tools (from eclassrooms, video conferencing tools, and various applications to digitize feedback) have been conducted around the world.
Soon the first published studies appeared, again conducted on a sample of students or learners. Numerous studies have indicated that lit cameras in distance education should be avoided due to social norms such as intrusion into students' homes, inequality and discrimination, etc., as well as discomfort in viewing oneself (Castelli & Sarvary, 2021). Researchers (Castelli & Sarvary, 2021;Meister, 2020) have therefore suggested several other active learning strategies, such as using applications that require participation, dividing into smaller groups, participation through involvement in conversations, etc. However, the authors of this article believe that with the same digitized background, we can ensure the same conditions and thus avoid various difficulties. Moreover, research has shown that students often and very often do other things at the same time when the cameras are off (ironing, cleaning up, playing, etc.) (Rostohar, 2020). Thus, the "camera on" is not only about facilitating the pedagogical process for the teacher to manage the lesson, but also about what effect we achieve with the camera on or off.
Most research has understandably focused on students (Lazarevic & Bentz, 2021) and how distance education affects student knowledge and many other psychosocial aspects (Quezada et al., 2020). Numerous studies (Costa, 2020;Huckins et al., 2020;Reich et al., 2020) conclude with suggestions on how teachers should motivate students and what they should look for to avoid causing students additional stress.
One of the possible solutions to reduce stress is sufficient physical activity. Sufficiently frequent, appropriately intense and regular physical activity has an effect on general health and especially on the prevention of diseases such as cardiovascular disease, diabetes, etc. (Lear et al., 2017;Sarma et al., 2015). Numerous studies point to the benefits of physical activity also in the psychosocial domain (Klaperski et al., 2014;von Haaren et al., 2016), namely as a stressregulating effect. At the same time, they effectively influence the health of the immune system (Nieman et al., 2011), which is of particular importance in a coronavirus epidemic.
Numerous studies have shown that physical activity decreased during the closure of schools, gyms, fitness centers, and other measures to contain the spread of the virus (Ammar et al., 2020;de Oliveira Neto et al., 2020;Jurak et al., 2020). Ammar et al. (2020) found that sitting increased by 28% compared to the previous state. Similarly, Jurak et al. (2020) found that closing schools for two months and postponing physical education in Slovenia resulted in a 13% decrease in overall physical performance. Slovenia hosted several international and national conferences on the implementation of the topic of distance physical education (experiences and challenges of teaching during the epidemic, challenges and new opportunities for distance education, teacher to teacher, etc.) and a review of the papers shows that the topic of physical education was implemented in the manner of presenting tasks, performing tasks at any time, and then collecting evidence of completed tasks.
Based on the above facts about the positive effect of physical activity on reducing stress and decreasing physical performance after completing distance learning in physical education, let us explore what we can do to make distance learning in physical education more effective. In this paper, we explore the importance of ongoing monitoring and delivery of physical education in the present and through a video conferencing system.
The main objective of the present article is to determine the correspondence between students' opinions about their active participation during distance learning and more objective indicators of their participation. In accordance with the main objective, we broadly divide the study into two objectives. The first objective is to determine students' opinions about their participation in distance education (a), their comfort level with the camera on or off (b) and their opinions about the importance of having the camera on for more active participation (c). The second objective is to determine the differences in motor performance when students have to turn the camera on or off during the learning process.

Research Design
The present study is based on a descriptive-quantitative investigation. Theoretically, it was designed and dynamized following previous researches that strictly required educators to allow students to turn off the camera during class. They suggested the use of various collaboration tools (conversations, interactive whiteboards, various quizzes, surveys, etc.) instead of the mandatory use of the camera. The consequences of turned off cameras and the effectiveness of such instruction were only observed through assessments, knowledge tests, and opinions, but not through direct effects, which are extremely important in physical education classes. By determining the differences between initial and final general and physical-motor status, we will be able to determine which approach is more effective in physical education. The reference information obtained will be crucial for physical education in all parts of the world.

Scheme of the Research Design
Description of the Sample Figure 1

. Scheme of the Research Design
The population was the undergraduate students of the Faculty of Education who attended the subject of Physical education (University of Ljubljana, Faculty of Education), which took place entirely online and was taught by the same educator. There were 75 such students and we included all of them. They were already divided into 5 groups of 15 participants each. We randomly selected which group would receive which approach of distance learning.
The students were in their 2nd or 3rd year of full-time study and were between 20 and 22 years old. They were all females. The average body mass index (BMI) was 22.6 ± 2.9. We conducted a pre-study to verify the conditions for distance learning. All students were at home or in their home environment at the time of distance learning from March to June 2021. They all had access to the internet (92.7% with good internet connection, 3.7 with worse internet connection) and all had a working camera; 89.9 % had their own workplace (room) and 10.1% had none. We taught them how to digitize the background and everyone was allowed to digitize it.
They all received 7 weeks of training. They had 2 training sessions of 45 minutes per week. The first training in each week was strength development and the second was endurance training. We designed the training for strength development differently for each of the 5 groups. The first group had the camera on and followed synchronous instruction online (SON), the second group followed synchronous instruction online but had the camera off (SOFF), the third group followed synchronous transmission of the recording instruction online and had the camera on (RON), the fourth group followed synchronous transmission of the recording instruction online and had the camera off (ROFF), the fifth group received only the online instruction (OI). All had to do the endurance training at home and outdoors and were monitored via the Polar Beat phone application (Polar Beat app). The Polar Beat app is a tracking application for exercise. It provides detailed tracking of duration, distance and intensity of endurance training. The first and third groups (groups with cameras turned on) were required to submit their results weekly in the e-classroom, while the second, fourth, and fifth groups were not. The final assessment was not dependent on collaboration or progress.

Intervention: Implementation of Study
The sample of variables consisted of three different types:

The Method of Conducting Distance Education
The method differed by the method of observing the class remotely (with the camera on or off) and by the method of presenting the content (synchronous instruction online, synchronous transmission of the recording online, or instruction online only). The content was the same, namely, it lasted 7 weeks, twice a week for 45 minutes. The first training of the week was dedicated to strength development and the second to endurance. The strength training consisted of three parts: the warm-up, the main part of which was based on the principles of progressively increasing the number and repetition series of all major muscle groups, and the stretching at the end. Endurance training consisted of systematic running training, alternating three methods, namely the continuous running method, intervals and fartlek. ("Fartlek" is a term meaning "speed game"). It is a training method for endurance, performed at different speed intervals.) Students wore wearable measuring devices and used the Polar Beat app. In the end, the 1st and 3rd groups posted their results as evidence in the form of a picture in the e-classroom, while the 2nd, 4th and 5th did not.
In Table 1, we present the goals of the learning process and the content as well as the teaching methods and the requirements of the students (participants).

General Physical and Motor Status
For general and physical motor status, we used the adapted "SLOfit" test (Jurak et al., 2020), which we named "FitPeF." It included height, weight, and three domains (thigh, buttock, waist), 4 motor tests for strength (arm, leg, abdomen, and buttock), and an endurance test (beep test). We had to make adjustments to the baseline test to control for performance on the distance test and to ensure that all participants found the same conditions.

Survey Questionnaire
The questionnaire consisted of 4 sets: 3 sets of items about distance education and 1 about additional motor activities during the experiment. The first three sets contained 5 items. Participants were asked to answer on a 5-point Likert scale (1 -does not apply to me at all to 5 -applies to me completely). The first set contained general items about presence during the pedagogical process in distance education (e.g., "I prefer distance learning to face-to-face learning.", "I am more present in distance learning than in traditional classroom learning." etc.). The second set contained items about the feelings when the camera is on (e.g., "When the camera is on, I worry about what others think of me.", "When the camera is on, I am being watched and it bothers me." etc.). The third set contained items about the second, non-camera related activity (e.g., "When I do not need to turn on the camera, I do tasks that I can do at the same time (sweep, iron, etc.)," "When I do not need the camera, I log on and walk away from my computer." etc.). The fourth set was used to identify additional motor activities during the experiment.

Data Collection
The "FiTPeF" test data were obtained by requiring all participants to turn on their cameras during the run. Data were entered into a spreadsheet prepared for this purpose, which was anonymized with codes. An individual password was known only to the owner of the password. For the endurance test, we used the "20 m multistep pendulum run -fitness test" (Beep test). (Beep test is a maximal multistage 20 m shuttle run test. It was designed to determine the maximal aerobic power performance in sports with frequent stops and starts.) The students performed it at home (on an asphalt surface near the house), finally, as evidence, they transmitted a GPS route (determined using the Polar Beat app) to the e-Classroom and recorded their maximum heart rate and level achieved. All data were recorded twice, the first time before the 7-week training started and the second time after it ended.
After 7-week training, all the participants completed a questionnaire consisting of 4 sets of statements (5 for each set). However, the first question was the individual's code. Individuals' responses from the survey were then matched with their test scores.

Data Procedures
We used the statistical programme IBM SPSS Statistics (Version 22) for the statistical processing of our data. Due to the complexity of data processing, we will divide the chapter into 2 parts that are consistent with the objectives of the research. In the first part, we present the method of processing the data obtained with the survey questionnaire and in the second part, we present the method of processing the data obtained with the FitPeF-strength and FitPeF-endurance test.

Survey Questionnaire
The first aim of this article was to determine students' opinions regarding their participation during distance learning (a), their comfort level when the camera is on or off (b), and their opinions about the importance of having the camera on for more active participation (c). To achieve the first objective, we used the data obtained from the first 3 sets of the questionnaire. Achievement of the objective was tied to an individual set of questionnaires. For the first part (a) we used the first set of the questionnaire (set 1), for the second part (b) we used the second set of the questionnaire (set 2), and for the third part (c) we used the third set of the questionnaire (set 3). Before data collection, we checked the validity of the questionnaire using face and sampling validity. Content validity was provided by an external expert in psychology.
For the data obtained, we first checked their reliability using Cronbach's alpha for each of 3 sets of items (Cronbach's alpha for the first set was adequate (.67), for the second moderate (.76), the third adequate (.66)). We then used the Shapiro-Wilk test on set 3 to check whether the variables were normally distributed.
Because of non-normally distributed variables, we used a Kruskal-Wallis H test instead of a one-way analysis to determine differences between groups about the opinions on the importance of having the camera on for more active participation.

FitPeF-Strength and FitPeF-Endurance Test.
The second objective was to determine differences in motor performance when students have the camera on or off during the educational process. To achieve the second objective, we used the data obtained from the FiTPeF-strength and FitPeFendurance test.

FiTPeF-Strength
We calculated the progress in the FitPeF-strength test for each group and plotted it in a figure as percentages. We then calculated the normality of the distribution from the raw progress data by using the Shapiro-Wilk test. According to the non-normal distribution, the Kruskal-Wallis test was used to determine whether there were statistical differences in progress between groups. The Dunn-Bonferroni approach was produced for any dependent variables for which the Kruskal-Wallis test was significant. The Kruskal Wallis epsilon square was used to measure effect size.

FitPeF-Endurance
For endurance, we used a comparison of the FiTPef-endurance test before and after the end of the 7-week training. The Wilcoxon signed-rank test measuring the effect size r was used instead of the paired sample t-test (repeated-measures ttest) as the distribution was non-normal according to the Shapiro-Wilk test.
We verified a statistical significance on the level of 5-percent risk (p<0.05).

Results
The results will be presented in two successive sequences. First, we will present the results of survey of questionnaire responses to gain insight into how students evaluated their online work. Then, we will present the results of the progress in overall physical and motor status (FitPeF in strength and FitPeF in endurance) which we believe will give a more realistic picture of the students' work during distance education.  Table 2 we can see students' opinions about their learning activities during distance education. The results are divided into 3 sets. The first set contains general items about their presence during the pedagogical process in general, the second about their comfort level when they need to have the camera on, and the third about their opinion on the importance of having the camera on for more active participation. Each set contained 5 items related to the content. However, each item could be answered on a 5-point scale (1-does not apply to 5-applies completely to me). We show the means and standard deviations for all items in each set separately by type of distance learning. The means of all items in the first set, which consisted of statements about attendance and participation in general, ranged from 3.00 to 4.20. The second set, which consisted of items about their feelings about appearing on camera and other social norms, indicated that students felt disturbed by the camera and the mean ranged from 4.04 to 4.28. In the third group, the mean score ranged from 2.35 to 2.69, indicating that they generally did not find the camera an important indicator of their participation.
The first set of items concerned their views on presence. Analysis of the results showed that students feel that they were more present online than in classical face-to-face learning. Analysis of the second set of items showed that students felt burdened when teachers require them to have their cameras on. Analysis of the third set revealed that students feel that having their cameras on is not a reason to do other things unrelated to the lesson. The synthesis of all three sets of items shows us that we should not require students to turn on their cameras because they are more relaxed when they can turn off their cameras and they participate in the same way.
To determine the differences between opinions and the actual effects of active participation, we first determined the differences between groups in the opinions about the importance of the camera ON (Set 3) and then analysed the differences between groups in the actual progress. The normality of the data was tested using the Shapiro-Wilk test. Because the data were not normally distributed, we used a Kruskal-Wallis H test instead of a One-way test ANOVA to analyse statistically significant differences between the 5 groups.
Overall, the results show (Table 3) that there were no statistically significant differences between groups in the third set.
Students felt that the camera did not contribute to more activity in class. They felt that the camera was completely unnecessary because they are responsible for themselves and know how important their participation is. In addition to student opinions, it is useful to determine the impact of having the camera on or off. We tested all students at the beginning and end with "FiTPef in strength" and "FitPef in endurance" tests to determine if the camera was irrelevant. Figure 2 shows that the progress on "FitPeF in Strength" is greater for all types of instruction with the camera on than for those with camera off. The results with camera off are very similar to those with instruction only. A Kruskal-Wallis H test showed that there was a statistically significant difference in "FitPeF in Strength" between the different types of distance learning, χ2(4)=53.479, p<.00, with a mean rank of 59.20 for Group 1, 24.00 for Group 2, 60.57 for Group 3, 29.27 for Group 4 and 16.97 for Group 5. Epsilon squared was .716 and shows a large effect. Dunn-Bonferroni approach was created as a post-hoc method to determine which groups differed from the rest. Pairwise Comparisons showed that the differences between cameras on (1st and 3rd groups) and other groups were statistically significant.  Figure 3 shows the differences between the initial and final results of the Beep, which measures endurance. We used Wilcoxon signed-rank test measuring effect size r instead of paired sample t-test because the distribution was nonnormal. Wilcoxon signed-rank test showed that 7 weeks of endurance training produced a statistically significant change in endurance only in the first group (Z=-3.417, p=.001) with a large effect size (r=.623) and in the third group (Z=-3.412, p=.001) with a large effect size (r=.622).

Discussion
The main purpose of this paper was to determine the relevance of turning the camera on or off during distance learning as an argument for active or passive student participation. From the results above, it appears that turning the camera on during the learning process is a suitable argument for more active participation. In the discussion section, we will try to find possible reasons for our findings.
In accordance with the purpose of the study, we formulated two objectives, first, to find out students' opinions about their participation in distance learning (a), their comfort level when the camera is on or off (b) and their opinions about the importance of having the camera on for more active participation (c). The second objective was to determine the differences in motor performance when students have to turn the camera on or off during the learning process.
The first objective was achieved through the opinions of the learners. The educational process must be focused on the learners since it is intended for them. The main goal of any education is to educate the learner. The path to the final success is often assessed indirectly, mainly through the final results (summative), more and more often through intermediate stages (formative), and the educational process is assessed directly with different assessment tools. The results of our study in the first part (Table 2), in which we aimed to assess the educational process only through the opinions of a sample of learners, showed that students do not want to turn on the cameras. They also feel that the camera does not encourage them to actively participate whether they have the camera on or not, that they are uncomfortable being seen by others, and that they are burdened with their appearance and what others will say about them, etc. Also, other researchers (Castelli & Sarvary, 2021;Costa, 2020) have found that turning off the camera and inactivity/inaccuracy, rather than technology and its use, is encouraged by students' consideration of what their classmates will say or think about them.
Student opinion is legitimate, but it should not be the only measure of success in the pedagogical process. Berk (2018) also emphasizes that student opinion about the quality of the pedagogical process has nothing to do with success or achievement.
So, with the second objective of the research, we wanted to determine the effects of the educational process through progress in strength and endurance. We found out, that student engagement was statistically significantly worse when the camera was off. It is evident that the students were not active behind the turned off camera, as it can be observed from Figure 2 in 3. Progress was not statistically significant for all forms of work when the camera was off and it was statistically significant when the camera was on.
Synthesis of students' opinions from the first part of the data (Table 1) and actual progress in motor efficiency from the second part of the data (Figures 2 and 3) clearly shows a discrepancy. Students believe that they are equally effective  Test) BEEP -at the beginning BEEP -at the end whether the camera is on or off, but the results show that this is clearly not true. We see the problem mainly in the sampling of additional stress and also in the need to explain that a camera must be turned on.
It seems that the main reason for not turning the camera on was additional stress and doing other things at the same time (Table 2). Burdening the opinions of others further increases stress (Levkovich & Shinan-Altman, 2020;Rossi et al., 2020). Rossi et al. (2020) in their study (N=18.147) found symptoms of post-traumatic stress disorder and depression in 37% of respondents during the epidemic. Meta-analytic studies have confirmed that coping with stress requires regular physical activity of moderate to high intensity (Mücke et al., 2018).
The research therefore suggests a positive relationship between a closed camera and increased stress (Levkovich & Shinan-Altman, 2020;Rossi et al., 2020) and between exercise and stress reduction (Mücke et al., 2018). Considering that stress increased during the epidemic (Rossi et al., 2020) and that the camera on is an additional contributing factor (Castelli & Sarvary, 2021), it would be reasonable to conclude that exercise should be performed with the camera off, and thus we are contributing doubly to stress reduction.
However, despite this logical conclusion, we found that students were not equally active with the camera on or off. We believe that one of the reasons for this could also be due to students' insufficient responsibility resulting from the traditional notion of "learning for the teacher or grades" rather than for their knowledge and skills.
We believe that future teachers and educators should not be only better equipped with knowledge of stress management techniques, but also need to implement techniques. And one of them could be regular compulsory physical activity.

Conclusion
Most teachers around the world have been confronted with a new way of teaching because of the epidemic. This raised many questions and controversies. First at the level of technology and soon at the level of sociology and psychology. The inequality of conditions has widened the gap between the richer and the poorer not only in terms of possession of technology but also in the field of social inequalities. The epidemic has increased stress in all age groups in terms of health, social security, jobs, etc. This has led to a cautious approach to the demands of distance education. Research has shown that children are disadvantaged and researchers have stressed that teachers should not prescribe cameras on but encourage learners to actively engage in other ways. Our research has shown that there were no significant differences between students in terms of both technology and the provision of space to learn. However, students do not want their cameras on as it puts them under additional stress (they were burdened with looking into the camera at what others will think or say about them). The results of the study, directly and indirectly, showed that students did not participate effectively or actively when the cameras were off, despite the argument of the importance of regular exercise and regular presentation of the exercise program. We found that effective implementation of movement programs, as measured by progress in movement efficiency, was statistically supported by having a camera on, essentially monitoring and correcting students. We wondered about the value of teaching students the importance of physical activity, the value of implementing physical education without a camera, and the value of trusting students to be responsible for their own health.
This got us thinking about the added stress students face when the camera is on. One way to deal with the stress is physical activity, but that only happens when teachers require the use of the camera. All of this leads to a vicious cycle. The main reason students turn off the camera is their low self-esteem and self-acceptance. We think it would be useful to enrich psychology classes with techniques to promote self-esteem and stress management. At the same time, it would be useful to look for simple monitoring techniques (e.g., real-time recording of progress in the exercise diary, through the use of GPS, activity meters, etc.).

Recommendations
These conclusions are particularly addressed to researchers and consultants at the state level in education to emphasize that research on the effectiveness of distance education examines not only knowledge but also skills, especially in the field of psychology and physical education. Considering the findings that "camera on" was statistically more efficient than a presumed "camera off" when it comes to motor skill acquisition, it makes sense to eliminate the reasons why students don't want to have the "camera on". We suggest that school system policymakers introduce system-level »self-esteem building exercises«, »self-acceptance«, and taking responsibility for » turning the camera off« in subjects related to psychology.
The highlights of the study are: the importance of using camera on, especially in subjects related to motor skills (As one of the key factors for active participation).
optional physical activities, although medically supported by solid evidence and well-argued, are not taken seriously by students, it.
Further research is needed to confirm our conclusions. We propose to conduct an experimental study to find out what differences exist in the acquisition of knowledge and skills in different subject areas by "camera on or camera off" between students who had previously turned on their self-esteem skills and those without such skills.

Limitations
This study is limited to aspects of a single subject, physical education, so the results should not be generalized to all subjects. Students' competencies in physical education are not only related to knowledge but also to skill development, which is not typical of all areas/subjects of teaching. The research sample was small. The study lasted only 7 weeks and was bound to a specific schedule. We only observed limited aspects of teaching effectiveness as we also needed to examine effects through a distance learning approach.