A Meta-Analysis of the Effects of Arduino-Based Education in Korean Primary and Secondary Schools in Engineering Education

The Arduino microcontroller enables ordinary people to perform professional tasks that only traditional engineering professionals could perform. Recently, several educational cases have been applied to primary and secondary schools, which is a desirable attempt to popularize engineering education. This study meta-analyzed the effects of Arduino-based education in primary and secondary schools in Korea from the perspective of engineering education. Accordingly, 16 academic journals and dissertations were selected that verified educational effects by Arduino-based education to primary and secondary students in Korea, and 31 effect sizes were confirmed. According to the results of this study, the overall average effect size was 0.656, which confirmed that Arduino-based education had a positive educational effect. Furthermore, this study calculated the effect size as measured by categorical and continuous variables such as school level, the inclusion of curriculum, giftedness, publication status, the programming language used, publication year, number of sessions, and number of students. Implications were suggested from the perspective of engineering education. This study is meaningful because it suggests the application of Arduino to primary and secondary schools in engineering education by confirming the positive educational effect of Arduino-based education.


Introduction
With the advent of the Fourth Industrial Revolution, various methods are being proposed to satisfy human operational needs. In the past, textual documents and pictures were used to obtain information about making something, but today, more realistic and concrete information can be obtained using video media such as YouTube. With the creation of an environment where various digital-based manufacturing tools are universally accessible, an era has emerged where creative output can be readily obtained.
These changes are referred to as the "maker movement," where a "maker" is a person who makes something by hand, and "maker movement" is the phenomenon in which people make what they need-with passion-using new technology and digital equipment (Dougherty, 2012). The maker movement has provided everyone the opportunity to use professional-grade tools that were previously difficult for the general public to use, and open-source sharing has opened a democratic era of technology that allows anyone to explore or modify complex algorithms or systems (Halverson & Sheridan, 2014).
Arduino is one of the major hardware platforms that made this activity possible. Arduino was developed by Massimobench of Italy in 2005 and has been widely used in maker activities worldwide because Arduino is characterized as open-source hardware. The developers did not claim the intellectual property rights on the Arduino board, and because the design of the circuit was disclosed, anyone can freely develop and use the Arduino board (Lee, 2020). Consequently, numerous boards based on Arduino were produced and sold, making it easy to purchase them.
Another popular feature of Arduino is that it facilitates the operation of microcontrollers without specialized knowledge. Traditional microcontrollers required the cumbersome process of creating source code, compiling, and uploading through a separate device. Arduino, in contrast, enables the user to quickly write and compile programs using a free compiler called "Sketch" and upload compiled files via USB in a straightforward process (Al-Haija et al., 2014). Arduino can also run on several computer operating systems, such as Windows, Mac, and Linux. Most importantly, the board is relatively inexpensive (Teikari et al., 2012).
Because of its strengths, Arduino is used in multiple university departments, and, consequently, several engineering education papers reporting on the integration of Arduino and education are published annually (El-Abd, 2017). For example, Arduino was used in the engineering design process of team-based learning in mechanical engineering projects (Istanbullu & Tasci, 2019), in tasks related to the Internet of Things (IoT), automation, and monitoring the control of civil engineering processes (Chacon et al., 2018), and in structural dynamics in the classroom, related experiments, and physical simulations Chacon & Oller, 2017). Arduino was also used in the teaching of programming (Litts et al., 2019;Montironi et al., 2017;Perez & Lopez, 2019), in STEAM education (Lu & Ma, 2019), and as a teaching aid (Carvalho & Hahn, 2016;Kang et al., 2019;Perenc et al., 2019;Rivera-Ortega et al., 2018). Furthermore, Arduino has been used to replace existing expensive laboratory equipment (Arrizabalaga et al., 2017;D'Ausilio, 2012;Goncalves et al., 2019;Hahn et al., 2019;Jin et al., 2018;Mercer & Leech, 2017;Omar, 2018;Pino et al., 2019;Ragazzini et al., 2019;Sarao et al., 2016;Soong et al., 2018;Uyanik & Catalbas, 2018), in automatic data collection and measurement equipment (Galeriu, 2018;Galeriu et al., 2015;Kubinova & Slegr, 2015;Nichols, 2017;Wong et al., 2015), and in prototype implementation (Escobar et al., 2017). Accordingly, attempts have been made recently to apply Arduino to education in primary and secondary schools as a device to help people realize projects that were previously possible only in their imagination (Galeriu et al., 2014), These attempts have become an important opportunity for ordinary students to experience engineering education because microcontrollers, which were only available to minority engineers in computer science or electronics a few years ago in specialized laboratories, are now accessible to ordinary students in ordinary classrooms.
Against this backdrop, studies using Arduino in primary and secondary schools continue to be conducted in Korea, and studies that verify its effectiveness have been conducted by various researchers (Lee, 2020). These individual studies have statistically low verification power due to research constraints such as the number of subjects studied, research conditions, and experimental procedures, and each of the researchers applied Arduino to education under different conditions, so there is a limit to comprehensively and systematically evaluating its effects. The use of meta-analysis can overcome the limitations of individual studies, identify the general level of effectiveness in studies accumulated so far, and derive the factors that affect their effectiveness more systematically and reasonably.
The purpose of this study is to use a meta-analysis to more objectively examine the effectiveness of Arduino-based education in Korean elementary and secondary schools by excluding the biases and opinions of researchers. This process is as an assessment of the recent application of Arduino in education at primary and secondary schools in Korea, and if the results of the study confirm the effects of Arduino education, it will provide incentives and clues for the active use of a specific type of engineering education (e.g., Arduino-based) in primary and secondary schools. It will also provide information related to direction and operation when developing teaching materials related to effective Arduino-based education for primary and secondary school students. Consequently, this study seeks to collect papers (academic papers and dissertations) on the effects of education by the application of Arduino in primary and secondary schools in Korea recently and verify the papers through meta-analysis. Accordingly, three research questions were identified: Research Question 1: What is the overall average effect size of Arduino-based education?
Research Question 2: What is the effect size of Arduino-based education, as measured by five categorical variables (school level, curriculum, giftedness, publication status, and program language)?
Research Question 3: What is the effect size of Arduino-based education, as measured by three continuous variables (year of publication, number of sessions, and number of students)?

Subjects
The purpose of this study was to analyze the effect of Arduino-based education from the perspective of engineering education, and for this purpose, academic papers and dissertations related to Arduino-based education were studied. The search for data was conducted on the Research Information Sharing Service (RISS) site in Korea. The RISS site is the most representative site in Korea on which to search for academic papers and dissertations, used by many researchers in Korea. Papers with the keywords "Arduino" in Korean and "Arduino" in English were searched for on the RISS site on October 25, 2019. The Arduino-related dissertations were first introduced in 2007 and academic papers in 2009.
First, as a result of searching academic papers, 652 were found for Korean "Arduino" and 956 for English "Arudino," for a total of 1608 academic papers. Some papers were published in duplicate, so it was necessary to remove these.
Because the RISS site can save the basic information from these papers in a Microsoft Excel file, duplicate papers were removed using Excel. Consequently, 681 papers were duplicates, and 927 papers remained after deleting the duplicates. Excluding 505 papers because they were academic conference papers, 422 academic papers remained. Of these papers, 72 papers were related to education.
Second, as a result of searching dissertations, 178 were found for Korean "Arduino" and 205 for English "Arudino," for a total of 303 dissertations. Duplicate dissertations were removed using Excel, and dissertations not related to Arduino were removed. Consequently, there were 276 dissertations, 47 of which were related to education.
Of the 72 academic papers and 47 dissertations, papers were selected that could be meta-analyzed (except those unrelated to the verification of effects), such as qualitative research, correlation research, and comparative research. Furthermore, if a dissertation was presented as an academic paper, it was labeled as such. Consequently, 14 academic papers and 2 dissertations were selected, for a final total of 16 papers. Where several research results were presented at different school levels in one paper, each was treated as a separate study and analyzed.

Coding of materials
The 16 studies selected were first reviewed, and the possible categories were coded and extracted. The analysis was divided into categorical variables and continuous variables. Categorical variables include school level, the inclusion of curriculum, giftedness, publication status, and programming language. Continuous variables include publication year, number of sessions, and number of students. Coding was performed by two engineering education experts and one meta-analysis expert, and for disagreements in analysis, the code was reconciled through consultation.

Effect size calculation
Effect size is defined as the average value of the experimental group minus the average value of the comparison group, divided by the integrated standard deviation of the two groups (Hwang, 2014). This value is converted to a common index to statistically analyze the results of several individual studies, suitable for quantifying the degree of relationship between two variables or the difference between two groups (Borenstein et al., 2009). The standardized mean difference (Cohen's d) tends to overestimate the effect size when the sample is small (Hwang, 2014) and requires Hedges' g to correct it (Hedges & Olkin, 1985). In this study, the sample is mixed with small and large studies, so it is appropriate to calculate Hedges' g. The formula for Hedges' g is as follows.
Where is the effect size, ̅ is the mean of the experimental group, ̅̅̅ is the mean of the comparative group, is the integrated standard deviation, is the comparison population, is the experimental population, is the comparison standard deviation, and is the experiment standard deviation. Because the sample size of each study is different, weighting is necessary to accurately calculate the effect size (Hwang, 2014). the larger the sample, the more accurate the weight. In this study, Hedge and Olkin's method was weighted based on the number of cases. The effect size based on the meta-analysis result, presented in Table 1, is an interpretation standard proposed by (Cohen, 1988).

Homogeneity test
A homogeneity test was conducted using the test formula (Borenstein et al., 2009), assuming that the individual study results to be analyzed were from the same population. As depicted in Table 2, the effect sizes extracted from the study subjects were heterogeneous (Q = 141.107, p < .05, I 2 = 79.037). Therefore, in this study, the effect size was compared and analyzed using a random-effects model. Comprehensive Meta-Analysis (CMA) version 3.3, a meta-analysis program, was used to process the data. A meta-ANOVA was used to examine the effect size of each subfactor, and metaregression was used to examine the tendency of variables.

Publication bias inspection
To secure the internal validity of the meta-analysis results, a publication bias test was conducted. First, the distribution of effect sizes was examined using a tunnel plot, and the distribution was not completely symmetrical. To quantitatively evaluate the publication convenience, fail-safe N (Rosenthal, 1979) was calculated. In this case, the coefficient (N) is 1930, which is larger than 5k + 10 (k is the number of studies), and thus the results of this study can be interpreted as reliable. Consequently, it is difficult to assert that there are publication biases among the studies that are the subject of this study.

Results
Overall average effect size Table 3 presents the meta-analysis of the educational effects of Arduino. The total number of papers in the study was 16, and the total number of effect size cases was 31. The overall average effect size was 0.656, and the 95% confidence interval was 0.470 to 0.843. According to Cohen's interpretation of the effect size, used hereinafter when referring to small, medium, or large, the overall average effect size was medium.

Effect size as measured by categorical variables
The results of analyzing the effect size based on school level are depicted in Table 4. The effect sizes were 0.807 (small) for middle school, 0.574 (medium) for elementary school, and 0.150 (large) for high school. From the analysis of variance (ANOVA), the significance level was p > .05, and there was no statistical difference in the effect size based on school level. However, the high school data requires careful interpretation because the number of studies was minimal. The results of analyzing the effect size based on curriculum are depicted in Table 5. The effect sizes were 0.821 (large) for formal curriculum and 0.549 (medium) for non-formal curriculum. From the ANOVA, the significance level was p > .05, and there was no statistical difference in the effect size based on curriculum. The result of analyzing the effect size based on giftedness is depicted in Table 6. The effect sizes were 0.785 (medium) for gifted class and 0.613 (medium) for general class. From the ANOVA, the significance level was p > .05, and the effect sizes based on giftedness were not statistically different. The results of analyzing the effect size based on publication status are depicted in Table 7. The effect sizes were 1.258 (large) for academic papers and 0.567 (medium) for dissertations. From the ANOVA, the significance level was p < .05, and the effect sizes based on publication status were statistically different. The results of analyzing the effect size based on the programming language used are depicted in Table 8. Effect sizes were 0.875 (large) for textual programming language, 0.447 (medium) for both used, and 0.428 (medium) for visual programming language. From the ANOVA, the significance level was p < .05, and the effect sizes based on programming language used were statistically different.

Effect size as measured by continuous variables
Meta-regression analysis was conducted to examine the tendency based on year of publication, number of sessions, and number of students. The results are depicted in Table 9. The slope based on year of publication was -0.0052 (p > .05), and the slope based on number of sessions was -0.0086 (p > .05), which was not statistically significant. The slope based on number of students was 0.0025 (p < .05), and the slope was statistically significant.

Discussion
The purpose of this study was to meta-analyze the effects of recent Arduino-based education. Based on the results of this study, findings are identified as follows.
First, the overall average effect size of Arduino-based education was 0.656 (medium), implying that the application of Arduino in Korea was educationally effective. These findings could provide a basis for the scientific analysis of the assertion that the application of Arduino to education would have some effect. The results of this study are significant because the microcontroller proved effective when used in the engineering field when used educationally in primary and secondary schools.
Second, the effect sizes based on school level were 0.807 (large) for middle school, 0.574 (medium) for elementary school, and 0.15 (small) for high school. The ANOVA of the three groups demonstrated that there was no statistical difference in the effect size based on school level, likely because the number of studies for high school was minimal. Therefore, rather than interpreting the educational effects of applying Arduino between the three groups, it should be interpreted that the educational effects of only two groups, elementary and middle school, do not differ at a statistically significant level.
Meanwhile, there were very few high school cases in this study. The small number of high school cases is similar to other engineering education meta-analysis studies (Kim & Yoo, 2018;Lee et al., 2017). These findings are closely related to the educational background of Korea. Many Korean high schools offer theoretically-oriented lectures rather than experiments and hands-on classes-many high school teachers do not have hands-on activities because the scores for midterm and final exams during the three years of high school in Korea are an essential factor in determining students' universities and departments. High school teachers would have had great difficulty applying devices requiring engineering knowledge, such as Arduino, to their classes. This suggests that there were few cases of applying Arduino in high school, and its educational effect was insignificant.
Third, the effect sizes based on curriculum were 0.821 (large) for formal curriculum and 0.549 (medium) for nonformal curriculum. The ANOVA between the two groups was not statistically significant. The application of Arduino in non-formal curricula-such as after-school activities, club activities, and gifted children's classes-suggests a significant educational effect. These results are the same in the meta-analysis related to the application of engineering education in other primary and secondary education in Korea Kim, Lee, et al., 2016). The reason for this result was that Arduino was relatively easy to apply in the non-formal curriculum. In Korea, the government establishes what should be learned in the formal curriculum. Therefore, it is somewhat challenging for teachers to arbitrarily modify the regular curriculum to conduct classes related to Arduino. Furthermore, for the formal curriculum, the class time is tightly arranged. Arduino requires circuit configuration and programming between the various components, which makes it challenging to do all these activities in the formal curriculum. However, in the non-formal curriculum, these time constraints are small, allowing students to work in a more flexible atmosphere, and the teacher is less burdened with the lessons, implying a large effect size for the non-formal curriculum. Therefore, to understand the effect of Arduino-based education, future researchers should consider studies that apply Arduino to the non-formal curriculum. For reference, maker education has been primarily taught outside the school curriculum, and there have been attempts to incorporate it into formal school education (Bevan, 2017;Godhe et al., 2019). Therefore, future researchers should consider studies applying Arduino-based education to a non-formal curriculum.
Fourth, the effect sizes based on giftedness were 0.785 (medium) for the gifted class and 0.613 (medium) for the general class. The ANOVA between the two groups was not statistically significant. Although both groups' effect sizes were medium, the gifted class effect size was close to large. These results were similar to the meta-analysis studies related to engineering in Korea (Kim et al., 2015), likely because gifted students found Arduino's circuit construction and programming less challenging than did general students. These results suggest the possibility of using Arduino in gifted education. For example, Arduino's IDE is used to teach programming for gifted students (Avcu & Er). In the classroom activities of gifted students, experimental devices or data collection devices are sometimes used. Arduino can be used to replace existing expensive experimental devices (Arrizabalaga et al., 2017;D'Ausilio, 2012;Goncalves et al., 2019;Hahn et al., 2019;Jin et al., 2018;Mercer & Leech, 2017;Omar, 2018;Pino et al., 2019;Ragazzini et al., 2019;Sarao et al., 2016;Soong et al., 2018;Uyanik & Catalbas, 2018) and in automated data collection or measurement devices to process data more efficiently (Galeriu, 2018;Galeriu et al., 2015;Kubinova & Slegr, 2015;Nichols, 2017;Wong et al., 2015). Furthermore, because many gifted students choose STEM majors in college (Vu et al., 2019), a proper Arduino education can be an effective method of attracting talented people in engineering.
Fifth, the effect sizes based on publication status were 1.258 for academic papers and 0.567 for dissertations. From the ANOVA, the effect size based on the publication status was statistically different. These results are consistent with the opinions of (Lipsey & Wilson) , who suggest that academic papers are more effective than dissertations because the academic papers published in journals are usually verified through peer review, and the quality of the academic papers including the program operation are generally higher than those of dissertations.
Sixth, the effect sizes based on the programming language used were 0.875 (large) for textual programming language, 0.428 (medium) for visual programming language, and 0.447 (medium) for both used. From the ANOVA, the effect size based on the programming language was statistically different. The results of this study were inconsistent with the results of several previous studies (Armoni et al., 2015;Chang et al., 2017;M. Mladenovic et al., 2020;S. Mladenovic et al., 2016;Sáez-López et al., 2016;Topalli & Cagiltay, 2018;Zhang et al., 2014) that reported that the application of visual programming language would have a more positive educational effect than the application of textual programming language. The results of this study were affected by the difference in computer teaching ability between teachers who applied the textual programming language and teachers who applied the visual programming language. The textual programming language has a disadvantage because it requires significant time to use the correct syntax and language acquisition, whereas the visual programming language has a low cognitive burden due to easy learning and simple grammar. Many researchers have tried to teach programming using visual programming instead of textual programming and reported that applying visual programming and then textual programming was effective for learning a programming language. The textual programming language used in Arduino's sketch is C-language-based, and teachers who apply the textual programming language should have more knowledge of the basic grammar and syntax of the C language than teachers who apply the visual programming language. Therefore, it could be inferred that the teachers who applied the textual programming language were acquiring more knowledge related to programming than the teachers who applied the visual programming language, and the group that applied the textual programming language had a significant educational effect.
Based on the results of this study, the textual programming language was more effective than the visual programming language, but it is not recommended that the textual programming language be applied unconditionally when teaching Arduino. Many studies have reported that the application of the visual programming language is educationally effective for young students, and the visual programming language is even being developed for engineering students. Therefore, applying the textual or visual programming language based on the students' level is recommended. For example, it may be desirable for elementary school students to use visual programming languages such as Scratch for Arduino (S4A) and Entry and high school students to use textual programming languages such as sketch programs. In comparison, teachers who want to use Arduino in primary and secondary schools are encouraged to experience various maker activities using the textual programming language.
Seventh, the meta-regression analysis of continuous variables reveals that the slope for each year of publication was -0.0028, and the slope for each number of sessions was -0.0088, which were not statistically significant. This indicates that Arduino-based education is not related to the effect size based on publication year or number of sessions. In comparison, the slope based on the number of students was statistically significant at 0.0024, implying that Arduinobased education has a very weak static education effect based on the number of students. These results did not agree with the results of a previous study (Kim, Jo, et al., 2016;Kwon & Lee, 2014) that achieved positive educational effects with fewer students. Therefore, it is necessary to avoid interpreting the results of this study as "the educational effect increases statistically with the number of students taking the class."

Conclusions
Several conclusions can be drawn from this study. First, Arduino-based education had a medium-sized effect in Korea. Second, variables such as publication status and used programming language influenced the effectiveness of Arduinobased education. Third, the effect of Arduino-based education was higher in primary and middle schools than in high school, in a non-formal curriculum than in a formal curriculum, and in gifted class than in general class. Fourth, based on the results of the meta-regression analysis of continuous variables, Arduino-based education had minimal or no positive relationship with variables such as publication year, number of sessions, and number of people.

Limitations and suggestions
The limitations and future directions of this study are as follows.
First, only two previous studies were published in academic journals to support the analysis of this study. This suggests that research related to Arduino-based education in Korea has been reported predominantly in dissertations, rather than in academic papers. Given that Arduino-based education is a practical example of applying engineering education to primary and secondary education, more Korean researchers should present their research on applying Arduino in primary and secondary education.
Second, previous studies analyzing the effects of Arduino-based education in Korean high schools are limited, suggesting few Arduino-based engineering classes exist in Korea. High school is an essential step in preparing for college courses and should apply programs that will motivate their students to pursue engineering. Accordingly, the application of Arduino in high school could provide useful educational material to increase students' interest in engineering. Therefore, studies applying Arduino in high school should be conducted to induce higher motivation among high school students toward engineering.
This study is a systematic analysis of the effects of Arduino-based education-which has been growing in the Korean educational field during the era of the Fourth Industrial Revolution and the maker movement-through meta-analysis from the perspective of engineering education. Therefore, this study contributes many implications to educational researchers and field experts who conduct related studies. Furthermore, this study is meaningful because it suggests the application of Arduino to primary and secondary schools as a direction for engineering education by confirming the positive educational effect of Arduino-based education.