'Assessment instrument' Search Results
Evaluation of Preservice Teachers’ Performance in School through Video Observations during the COVID-19 Pandemic
observation preservice teachers rubrics student evaluation teacher education...
Lesson study, observation and analysis are relevant to professional development and initial teacher education. As a strategy, it helps to bridge the gap between theory and practice. The health conditions brought on by the COVID-19 pandemic forced the restriction of the tutors’ direct observation of preservice teachers at school. This study analyses preservice teachers’ performance through video observations to evaluate their professional activity at school during the COVID-19 pandemic. The Fifteen Items Revised Tsang-Hester Observation Rubric (FIR-THOR) was administered to a sample of 166 preservice teachers in their internship schools and their video recordings each one of 45-minute teaching lessons were analysed. The results show that the FIR-THOR appears as a robust instrument, which allows us to conclude that the instrument works well in the three five-items dimensions that compose it - Instruction, Management, and Assessment - proving to be reliable for assessing teacher intervention in the classroom. Among the three dimensions, the preservice teachers’ performance stands out in the Management of the classroom, as well as in the classroom Instruction. This contribution is relevant considering the potential of lesson analysis in learning and professional development during initial teacher training.
Study Item Parameters of Classical and Modern Theory of Differential Aptitude Test: Is it Comparable?
classical test theory differential aptitude test item parameter modern test theory...
This study aimed to find the Classical Test Theory (CTT) and Modern Test Theory (MTT) item parameters of the Differential Aptitude Test (DAT) and examined their comparability of them. The item parameters being studied are difficulty level and discrimination index. 5.024 data of the result sub-test DAT were documented by the Department of Psychology and Guidance and Counselling bureau. The parameter of classical and modern test items was estimated and correlated by examining the comparability between parameters. The results show that there is a significant correlation between item parameter estimates. The Rasch and IRT 1-PL models have the highest correlation toward CTT regarding the item difficulty level. In contrast, model 2-PL has the highest correlation toward CTT in the item discrimination index. Overall, the study concluded that CTT and MTT were comparable in estimating item parameters of DAT and thus could be used independently or complementary in developing DAT.
Measurement of Students' Chemistry Practicum Skills Using Many Facets Rash Model
chemistry practicum mfrm performance assessment process assessment product assessment...
The accuracy of assessing the capabilities of the process and product in chemical practice activities requires appropriate measurement procedures to be followed. It is crucial to identify the components that can introduce bias while measuring student abilities during the measurement process. This study aims to identify the components or criteria used by teachers to assess student performance in practicum activities and analyze the quality of the rubrics developed. The study was conducted with the participation of three raters, 27 high school students, and nine assessment criteria. A quantitative descriptive approach was employed using the many-facet Rasch model (MFRM) analysis for measurement. The results of the MFRM analysis show no significant measurement bias, with data measurement facets fitting the MFRM model. The reliability of all the facets meets the criteria, and the scale predictor functions appropriately. While all students can easily pass four out of nine items, five items can only be partially passed by students. The assessment criteria that require special attention include communication skills, tools and assembly, interpretation, cleanliness, and accuracy when performing practicums. These criteria provide feedback for teachers and students to ensure successful practicum activities. The Discussion section of this study delves into the findings and their implications.
Do Digital Competency and Self-Leadership Influence Teachers' Innovative Work Behavior?
digital competency innovative work behavior self-leadership...
This study investigates whether digital competency and self-leadership influence teachers’ innovative work behavior in Islamic International schools. The participants in the quantitative research were 108 teachers from the Islamic International School, who were selected using saturated sampling. The researchers used a 41 items questionnaire to collect data on the study variables. Quantitative data were analyzed by applying structural equation modeling partial least squares (SEM-PLS) using Smart-PLS 3. According to the study's findings, digital competency has a positive and significant impact on innovative work behavior, while self-leadership also has a positive impact that significantly influences innovative work behavior. Overall, the impact of digital competency and self-leadership simultaneously was 0.584 or 58.4%, while the other variables not tested in this study influenced the other 41.6%. This research emphasizes schools to improve building their teachers’ innovative work behavior and for teachers themselves through training, self-development programs, building knowledge sharing among teachers and school leaders, and open ideas about developing pedagogical and sustainable programs for schools.
Evaluating the Results of PISA Assessment: Are There Gaps Between the Teaching of Mathematical Literacy at Schools and in PISA Assessment?
education gaps mathematical competence mathematical literacy pisa assessment...
The problems in education in the countries of the Organization for Economic Cooperation and Development (OECD) vary from country to country. The differences between "upper class" and "lower class" countries in PISA assessment results have led to a research gap. The purpose of this study was to (a) test students' mathematical literacy skills on the Program for International Student Assessment (PISA) test and compare the results using the sum of means across OECD countries; (b) examine the relationship between students' mathematical competence, precision, and self-perception of mathematical literacy skills in the PISA test; and (c) analyze the gaps that exist between the implementation of mathematics instruction in school and the mathematical literacy as measured on the PISA test. This study was designed as a mixed method with an explanatory sequential design. The data collection methods included test procedures, questionnaires, and interviews. The result of this study showed that the overall mean score obtained was below the OECD average. In general, the respondents achieved only level 2 mathematics proficiency. A significant relationship was found between mathematical competence, precision, and self-perception in mathematical skills. On the other hand, there was a gap, namely the difference at the implementation level, where mathematical literacy measured by PISA differed from the measurement of mathematical learning achievement by teachers in school. The results showed that teaching that emphasizes only problem-solving procedures affects low mathematical competence and is not useful enough for students to deal with the PISA mathematics test.
Effect of Computer Based Test on Motivation: A Meta-Analysis
computer-based test meta-analysis motivation r studio...
This meta-analysis study investigates the effect of computer-based tests on motivation. The random effect model is the basis for calculating the summary effect, and it is found that the 30 articles obtained through the PRISMA method, which consist of four parts, namely identification, screening, eligibility, and included, can be generalized. Data analysis was performed using R studio and OpenMEE. Based on the research results, the p-value <.05 was obtained, so it was concluded that using a computer-based test significantly affected motivation. In the moderator analysis, it was found that computer-based tests were most effective for intrinsic motivation for the motivation variable, university students for the participants variable, and the country of Iran for the region variable. Evaluation of publication bias using the funnel plot and regression method shows no publication bias, proving that the 30 articles analyzed represent all existing studies on computer-based tests.
Development of Waves Critical Thinking Test: Physics Essay Test for High School Student
assessment in physics essay test physics essay test waves critical thinking test...
This study aims to produce a product to evaluate students' critical thinking skills that departs from physics content where students often have misconceptions. This research is a development research with research stages covering a) research and review literature; (b) planning chapter objectives; (c) developing a preliminary form; (d) field-testing the preliminary form; (e) Revise the preliminary form; (f) conducting a main field-test. The Waves Critical Thinking (WCT) test developed consists of 7 questions with 15 specific domains. Total percentage of content validity test was obtained 87.98% with appropriate criteria and based on the construct validity WCT test, the Goodness of Fit criteria were obtained which were classified as fit. The test instrument being tested consists of 15 objective items. The reliability of WCT test results 0.597 as a Cronbach's alpha score with the medium category and all the components have a good level of composite reliability. The outcome of the study was the WCT test with a valid state for measuring students' CT in a specific domain of physics wave material.
Students’ Learning Independence and Critical Thinking Ability Using Mobile Learning Technology
learning independence critical thinking ability mobile learning technology...
21st-century learning requires teachers and students to integrate literacy skills, scientific literacy, mathematics, reading, writing, and technology in the learning process. Students must have initiative, discipline, responsibility, confidence, motivation for independent learning, and the ability to think critically about the problems presented. This study aims to determine students' autonomous knowledge and critical thinking abilities (CTA) using mobile learning technology (MLT). This research is a quantitative study involving 83 students from four junior high schools in the city of Mataram. The data collection for independent learning and students' CTA was carried out by giving tests and non-tests to students. The test conducted was a written test in the form of a description of 10 questions covering indicators of CTA. The non-test was conducted by giving a student learning independence questionnaire with as many as 15 statements, including five indicators of learning independence. This quantitative research data analysis uses the Rash modeling application with the help of Ministep software. The analysis results show that the learning independence of male and female students in the four junior high schools obtained a percentage of 77.38% in the “good” category. Each indicator of learning independence accepts a percentage above 70%, which is in the excellent category. Meanwhile, the CTA of male and female students from the four junior high schools obtained 75.28% in the “good” category. Each indicator of CTA also gets a percentage of more than 70%, meaning that each indicator is in a good category.
Exploring the Nexus of Digital Leadership and Digital Literacy on Higher Education Performance: The Role of Digital Innovation
digital innovation digital leadership digital literacy higher education performance...
Industry 4.0 has affected various aspects of life, including the organization of higher education. In the current era, higher education is required to transform themselves from using the conventional way of administration to the digitalized one. The said transformation also includes the services provided and management carried out by the organizations. The objective of this study is to measure the understudied mediation of digital innovation in the effect of the nexus of digital leadership and digital literacy on the performance of higher education. This quantitative research was conducted by distributing questionnaires to 234 faculty members of four higher education institutions in Malang City, Indonesia. Partial Least Squares – Structural Equation Modeling was applied to analyze the data. This study finds that digital leadership significantly affects the higher education performance and conclusively predicts digital innovation. As hypothesized, digital literacy has a significant effect on the higher education performance and digital innovation, and digital innovation plays a substantial role in the higher education performance. In addition, digital innovation mediates the influence of digital leadership and digital literacy on the higher education performance.
The Development of Teacher Performance Assessment Instruments in Boarding School
assessment boarding school teacher’s performance...
The objective of this study is to create a tool for evaluating teachers’ effectiveness in boarding schools. Planning and preparation stages, instrument testing, and measurements were used in this study to implement research on the creation of the Mardapi model instrument. In order to generate instrument items, the planning and preparation stage tasks included a literature assessment of teacher performance appraisal manuals. Data were gathered utilizing documentation approaches, and descriptive and qualitative analysis was performed. Six specialists validated the built instruments, which were subsequently put through limited testing at two boarding schools and extensive trials at nine boarding schools in Surakarta residency area. The Aiken formula was used to examine the expert’s evaluation data, and confirmatory factor analysis was used to analyze the test results. The results of this study indicate that the instrument for assessing teacher performance in boarding schools meets the validity criteria. This is indicated by the existence of loading factor values ranging from .51 to .72 (>.4) and t-sign values ranging from 4.75 to 9.25 (>1.96) and meeting the requirements of a fit model since Chi-square = 1307.95 < 2524 (2*df), p-value = .17956 (>.05), and RSMEA value = .014 (<.08). The instrument items are reliable assessment packages, and this is shown by the reliability value of omega .967 > .70. Because there is a guarantee of validity and reliability, the test can be used to further assess teacher performance in boarding schools.
Course Dropout Intention Scale: Development and Validation of a New Brief Measure in Academic College Context
brief measure college student course dropout dropout intention dropout studies...
University students may encounter situations where they perform poorly in a course and contemplate dropping out. This intention to drop out of a course manifests not only in thoughts or ideas but also in a cognitive self-evaluation of their performance and skills, enabling them to reflect on the possibility of dropping out. In this sense, there is a shortage of instruments that evaluate the intention to drop out of a course, so the aim was to develop and validate the Course Dropout Intention Scale (CDIS). Data from two samples (N1 = 198; N2 = 675) were used; the first was for the EFA, and the second was for the CFA, GRM, and SEM. The one-factor model was derived from the EFA and confirmed in the second sample, exhibiting appropriate goodness-of-fit indices. Similarly, the GRM obtained adequate fit indices; all items discriminated adequately, and the difficulty parameter had a monotonic increase. The SEM model of the effect of satisfaction with studies on the CDIS showed a negative and statistically significant effect. Thus, it was demonstrated that the CDIS is a robust instrument in its psychometric properties and empirical evidence with other variables.
Developing Pre-Service Biology Teachers’ Technological Pedagogical Content Knowledge through a TPACK-Based Course
microteaching pre-service biology teachers technology integration tpack...
This study investigated pre-service biology teachers' (PSBTs’) technological pedagogical content knowledge (TPACK) development. A TPACK-based technology integration course (TPACK-BTIC) was implemented. The study employed a convergent parallel mixed-methods approach. A TPACK survey questionnaire and interview schedule were used to collect data from 50 PSBTs. The quantitative data were analysed by computing means, standard deviations, and dependent samples t-tests, while qualitative data were analysed using deductive thematic analysis based on the TPACK domains. Findings indicate that the intervention positively affected PSBTs’ TPACK development with significant improvements in technological knowledge (TK), technological pedagogical knowledge (TPK), technological content knowledge (TCK) and overall technological pedagogical content knowledge domains. In contrast, pedagogical content knowledge (PCK) and pedagogical knowledge domains showed no statistically significant improvements. PSBTs’ engagement in microteaching lesson study, reflection on using technology, and collaboratively designing lesson plans improved PSBTs’ TPACK domains. The study recommends that teacher training institutions consider implementing content-based technology integration courses that engage pre-service teachers in microteaching lesson study, reflecting on technology use and collaborative designing of curriculum materials that involve using technology to support their TPACK development.
The Use of the Teddy Talk Test to Determine Speech and Language Development in Children Aged 4 to 5 Years
screening test speech and language testing teddy talk test...
Speech therapy uses a combination of approaches to assess speech and language, including analysis of spontaneous speech, use of standardized scales and assessment by the child’s parents and/or preschool teachers. In this paper, we present the Teddy Talk Test. Using a sample of sixteen children aged 4 to 5 years, we examined whether the test provided representative data on child language development. The parent questionnaire included information on the parents’ education, the parents’ native language, the child’s gender and the language the parents speak with their child. It was found that the Teddy Talk Test has the characteristics of screening tests for speech and language: it is time-efficient, it covers receptive and expressive language tasks, and it allows for rapid assessment of speech and language in large numbers of children. However, since the Teddy Talk Test has not yet been evaluated, we do not know its value regarding sensitivity, specificity and predictive validity.
Vocational High School Students’ Perception of Assessments in Online Learning: For, As, and Of Learning
assessment online learning vocational high school...
Implementing online learning has led to several issues, especially those related on assessments to measure students’ learning achievement. This study aimed to obtain accurate information on learning assessments for vocational high school (VHS) students during online learning, including assessment for learning (AfL), assessment as learning (AaL), and assessment of learning (AoL). It also investigated students’ perceptions of the assessments based on their grades. This quantitative research employed the survey method, collecting data through questionnaires. The population of this research is VHS students in Indonesia, totaling 86,885 students. The participants of this study were 487 VHS students. The data were analyzed using descriptive and ANOVA techniques. The results show that most VHS students positively perceive the assessment of learning achievements in online learning for AfL, AaL, and AoL aspects. Moreover, there is a significant difference in students' perceptions of the assessment in online learning based on grades. The perception of class XII students is the highest (average value = 3.90) of all classes, while the other classes have an average value of 3.38 for class XI, and the average value of class X is 3.81.
Indonesian Teachers' Acceptance on Online Teaching Technology During the COVID-19 Pandemic
facilitating condition pandemic pls-sem technology acceptance model tpack self-efficacy...
This study investigates the impact of external variables, technological pedagogical and content knowledge (TPACK) self-efficacy, and facilitating conditions on teacher adoption of online teaching technology during the COVID-19 pandemic. It employs explanatory research to characterize the effect of external variables on the variables of the technology acceptance model. 240 high school teachers filled out Google Forms survey questionnaires for six research variables and analyzed by the SmartPLS program. The result indicates that attitude and perceived usefulness significantly and positively influence behavioral intention. Perceived usefulness and ease of use have a strong positive effect on attitude. Furthermore, perceived ease of use has a considerable effect on perceived usefulness. Perceived usefulness and ease of use are not significantly influenced by external variables. Facilitating conditions significantly positively affect behavioral intention, whereas TPACK self-efficacy negatively affects behavioral intention.
Lecture-Based and Project-Based Approaches to Instruction, Classroom Learning Environment, and Deep Learning
approaches to learning classroom learning environment deep learning project-based learning...
Institutions of higher education generally employ both lecture-based and project-based approaches to instruction. This study aims to investigate which aspects of lectures and project-based instructional environments contribute to “deep” and “surface” approaches to student learning. We collected and compared survey data from undergraduate students taking a civil engineering course in which they were assigned to a section taught with lecture-based instruction (n = 181) or with project-based instruction (n = 142). Data analysis was performed after controlling for the effects of the motivational goal orientations of students. A positive correlation can be found between deep learning and higher levels of investigative culture and student involvement in the project-based classroom. Additionally, we found that higher levels of task orientation in the project-based classroom had an inverse correlation with a “surface approach”. We discussed the value of an investigative and participatory learning atmosphere for student approach to learning and its curricular implications for the design of project-based and lecture-based instruction.
A Psychometric Validation of the Sociomathematical Norm Scale for Senior High School Students in Mathematics Learning
developments scale learning mathematics; psychometric validation rasch model sociomathematical norms...
Students in mathematics classes do not understand the importance of sociomathematical norms in learning mathematics. This causes sociomathematical norms not to be teachers' focus when learning mathematics. Besides, there is no standardized instrument for assessing this norm, so developing this instrument is necessary to measure socio-mathematical norms in learning mathematics. This study aims to create and verify the psychometric validity of the sociomathematical norm scale. This research used a survey method with 505 senior high school students from Jakarta and West Java as respondents. The results showed that 25 items had convergent validity, with a loading factor value of > 0.700, meaning they could be declared valid. Concurrent validity indicates that each sociomathematical norms indicator is valid as a whole. Discriminant validity shows that the average variance extracted value on the diagonal is higher than the other values, so each item is declared valid. It was concluded that each item of the sociomathematical norms instrument has accuracy in its measurement function. The reliability test shows that each sociomathematical norms item is declared reliable. The reliability value of the sociomathematical norm item is .99, and the person's reliability is .86. Thus, the instruments developed can measure sociomathematical norms in learning mathematics.
Development and Psychometric Validation of an Information Competency Assessment: The Information Management Brief Scale
information competency information literacy information management inventory inventory...
Today's college students relate directly to the information through which they learn. They are the ones who detect the information they need, search and evaluate it, and also use it to develop academic projects. Information literacy programs have increased in universities to support the development of these skills to students. The aim of this study was to provide an instrument that allows the measurement of information competency in higher level students and generate evidence of its validity and reliability. The Information Management Brief Inventory (IMBS) was developed, which consisted of 16 Likert type items. IMBS was applied to a sample of 1,937 students from a public university in northern Mexico. For the evaluation of the construct validity and identification of the factor grouping of the items, an exploratory factor analysis (EFA) was carried out. Confirmatory factor analysis (CFA) was carried out to explore the goodness of fit of the model. It was found that the 16 items were grouped into three factors: information management, access to information and information ethics. The psychometric properties of IMBS were adequate.
Conceptual Model for the Assessment of Academic Productivity in Research Seedbeds From a Systematic Review
formative research higher education measurement productivity research seedbeds...
Higher education institutions have focused their efforts on promoting research seedbeds as a strategy for formative research. In this regard, the impact of such a strategy remains unknown due to the lack of models that enable its evaluation. Therefore, this study aimed to design an evaluation model for the academic productivity of research seedbeds based on the available evidence in the literature. To achieve this, a systematic review was conducted following the PRISMA model, analyzing 53 documents including articles, book chapters, and conference proceedings from the SCOPUS, ProQuest, Jstor, Scielo, and ScienceDirect databases. The results identified indicators that allowed for the design of a model based on six constructs: research training, institutional capabilities, bibliographic production, innovation and development, social appropriation of knowledge, and human resource training. It was concluded that the indicators evaluating research seedbeds seek greater scientific development involving students and improving the quality of research products, which directly impacts the institutional research mission.
How the Learning Style and Working Memory Capacity of Prospective Mathematics Teachers Affects Their Ability to Solve Higher Order Thinking Problems
higher-order thinking skill problems learning style prospective mathematics teacher working memory capacity...
This study aims to analyze the effects of working memory capacity and learning styles of prospective mathematics teachers on their ability to solve higher-order thinking problems. In the present study, learning style was considered students' tendency to learn visually or verbally. In addition, the types of higher-order thinking skills (HOTS) problems are complex and non-complex. Multiple regression tests were used to analyze the effects of learning style and working memory capacity. An ANOVA test was also conducted to analyze the ability of each group to solve each HOTS problem. In addition, one hundred twenty-six prospective mathematics teachers voluntarily participated in this study. The study found that learning styles only affected visual problems while working memory capacity (WMC) only affected the ability to solve complex problem-solving skills. Furthermore, WMC affected the ability to solve complex HOTS problems, not non-complex ones. The ability of visual students to solve HOTS problems will greatly increase when the problems are presented in visual form. On the other hand, the obstacle for visual students in solving verbal problems was to present the problem appropriately in visual form. The obstacle for students with low WMC in solving complex HOTS problems was to find a solution that met all the requirements set in the problem.