Abstract

This study aimed to designed and developed Dasig NAME, a gamified reviewer on Integral Calculus. The developed reviewer was assessed by 20 Mathematics and IT experts. Descriptive analysis using mean and standard deviation were employed to analyze the expert assessments. Inter-rater reliability using Intraclass Correlation Coefficient (ICC) were also used to assess the level of agreement between experts. The mathematics experts evaluated the gamified reviewer based on two categories: content, and instructional quality. While The IT experts evaluated the gamified reviewer on its usability, based on three categories: system usefulness, information quality, and interface quality. Results shows that mathematics experts rated the DASIG NAME as very highly acceptable for each indicator. ICC results also suggest a moderate agreement between the mathematics experts. IT experts’ evaluation showed that the gamified reviewer demonstrates good system usefulness and information quality, while interface quality emerges as an area for improvement. However, the low level of agreement among IT experts highlights the need for further refinement to enhance overall usability and strengthen expert consensus.

1. Introduction

Mathematics serves as a cornerstone for the Bachelor of Science in Naval Architecture and Marine Engineering (BSNAME) program, and is an essential foundation to understand the complex field of ship architecture and maritime engineering such as structural analysis, fluid dynamics, and ship design. A strong foundation in mathematics like in differential and integral calculus, enables naval architects to model and predict real-world scenarios, ensuring the safety, efficiency, and sustainability of maritime operations ¹.

Despite the importance of mathematics in the BS NAME curriculum, many students face significant challenges in mastering key concepts. These challenges often came from weak foundational skills developed during basic education, where systemic issues, such as insufficient learning materials and inadequate emphasis on core competencies ².

At the University of Science and Technology of Southern Philippines (USTP), the BS NAME program has recently been established. The success of the BS NAME students is critical to the program’s reputation and sustainability. However, academic records indicate that students continue to face difficulties in mathematics, specifically in Integral Calculus. The passing rates for this core subject over the recent academic years are concerning due to their variability and declining trend: 44.19% (19/43) in 2024-2025, 57.14% (19/28) in 2022-2023, and 79.41% (27/34) in 2022-2023. This highlights the need for targeted interventions to improve student performance and consistency in outcomes.

To address these challenges, the BS NAME program in USTP has implemented remedial classes for students. However, traditional approaches often fall short in engaging learnings and maintaining their motivation ³. While some improvements have been observed, the overall impact of these interventions remains limited.

Currently, higher education board programs have developed several gamified reviewers to assist students in preparing for board exams. Recent research highlights the potential of gamified learning tools to improve engagement, motivation, and academic performance. Tools such as ILETyou Review ⁴, Inquiry-based Online GEM Reviewer ⁵ and Make-it-ECE ⁶ have demonstrated success in learning experiences in mathematics and engineering education. However, these tools are often designed for general mathematics or specific disciplines, limiting their scope to specialized fields like BS NAME.

To bridge this gap, this study designed and developed “Dasig NAME”, a gamified reviewer application specifically tailored for BS NAME students. “Dasig” is a Cebuano word commonly used by USTP BSNAME students, which means 'to never give up.' By incorporating game elements such as hints, points, leaderboards, real-time feedback, and badges, Dasig NAME aims to create an interactive learning environment designed to help students review their mathematics subjects effectively, ensuring they are well-prepared for their board exams.

The design and development of Dasig NAME was anchored on the Gamification Education Taxonomy ⁷, Piaget’s Constructivist Learning Theory ⁸, Vygotsky’s Zone of Proximal Development (ZPD) ⁹, Feedback Intervention Theory ¹⁰ and Zimmerman’s Self-Regulated Learning Principles ¹¹. Each theory influenced specific elements of the gamified reviewer to ensure both cognitive development and motivational engagement are addressed.

The Constructivist approach emphasizes that students build knowledge through active exploration. In DASIG NAME, this is done through hints that help students figure out how to solve problems without giving them the answers right away. As students worked on each topic, they are encouraged to make connection based on what they already know. Dasig NAME start with easier questions and work their way up to the more complex problems, following the phases for interactive learning shown in the flowchart (Figure 1).

Dasig NAME have progressive hints and increasing levels of difficulty directly responds to Vygotsky’s ZPD. The format of each chapter helps students move from what they can do on their own to what they can do with help. This scaffolding is built into the flowchart process (Figure 1), which means that students only get help when they are having trouble or making mistakes. This makes sure that help is timely and useful.

The Dasig NAME gives you quick, task-specific feedback after each question attempt. When students get the right answer, they get praise. When they get the wrong answer, they get explanations that help them think about what they did wrong, which helps them learn and make changes. This fits with the Feedback Intervention Theory, which says that tailored feedback makes people do better when it focuses on the task instead of the person.

Zimmerman emphasized metacognition, planning, monitoring, and evaluating learning. Dasig NAME system allows the users to keep track of their own progress and goals, especially with the 70% passing criterion for each chapter. Students must show that they have mastered a skill several times before moving on. This helps them evaluate themselves and improve their strategies. The existence of progress bars, skill points, and several attempts encourages reflection and persistence.

The Gamification Education Taxonomy provides a comprehensive framework for gamified learning, which includes five dimensions that guide the design and development of the "Dasig NAME". The performance/measurement features, such as progress bar, skill points and badges. In Dasig NAME, once a student finished a chapter, the progress meter will fill up. The student will get skill points for every right answer and badge for reaching certain goals. The ecological dimension use storylines to sustain engagement. The students will be given storylines where they are naval architects and marine engineers who have to solve a certain problem in Dasig NAME. The social elements, such as leaderboards fosters healthy competition. In Dasig NAME, the leaderboards will display top-performing students. The personal dimensions, a personalized feedback and connection to real-world applications add connection and meaning to the learning experience. In Dasig NAME, each choices have personalized feedback, and each questions have hints. Fiction, a cohesive narrative that ties all activities together providing a sense of purpose. Dasig NAME has chapters and each chapters have questions. Every completed chapter unlocks new chapters and advances.

The gamified reviewer’s features, from introduction to hint-based scaffolding to feedback mechanism to mastery-based growth, all use the theoretical ideas discussed. The Dasig NAME app covers both cognitive and motivational sides of learning by combining all the theoretical foundations.

The flowchart that outlined how students interact with the gamified reviewer is shown in Figure 1. The overall system architecture of the gamified reviewer is shown in Figure 2.

The main objective of the study is to determine the quality and usefulness of the developed gamified reviewer as evaluated by both mathematics and information technology (IT) experts.

Specifically, the study seeks to address the following questions:

1) What is the mathematics experts’ assessment on the developed gamified reviewer in terms of:

1.1 Content;

1.2 Instructional Quality?

2) What is the Information Technology (IT) experts’ assessment on the developed gamified reviewer in terms of:

2.1 System Usefulness

2.2 Information Quality

2.3 Interface Quality?

2. Methodology

This study employed the descriptive research design focused on the creation and evaluation of educational tools. The design and development phase focused on the content creation, gamified system development, and quality assurance and testing. The evaluation phase utilized a descriptive methodology to analyze data collected from IT and mathematics experts assessing the gamified reviewer.

For the assessment of the mathematics experts, the researcher adopted an instrument based on the "Evaluation of Instructional Technology Materials (EITM)" from the study of Cajandig and Lomibao (2020) ¹². For the assessment of IT experts, the researcher adopted the Version 3 PSSUQ that was published by Sauro & Lewis, (2016) ¹³.

For the preparation and planning, the researcher requested the Table of Specifications (TOS) for the board program, BS Naval Architecture and Marine Engineering (BS NAME), and data from the BS NAME chairperson of University of Science and Technology of Southern Philippines (USTP) Jasaan regarding students’ mathematics performance trends in two calculus subjects: Integral and Differential Calculus. An inconsistent performance trend was observed in Integral Calculus, which led to the identification of this subject as the focus for the development of the gamified reviewer.

After preparation, the researcher proceeded with the design and development of the gamified reviewer. An IT expert was consulted to assist in developing the system of the gamified reviewer, ensuring that it was anchored to the theoretical framework. The design and development phase included content creation, gamified system development, and initial quality assurance and testing.

In the content creation, the researcher developed a comprehensive question bank focused on integral calculus. To ensure that the questions aligned with the BS NAME board exam syllabus, the researcher collaborated with the BS NAME faculty in reviewing the items. The questions where further subjected to item validation in collaboration with the mathematics faculty. After ensuring their validity, the items were uploaded to the Dasig NAME question bank. Topics were then structured into chapters, starting with foundational concepts and gradually advancing to complex applications. A diverse set of multiple-choice questions was crafted with varying levels of difficulty, covering conceptual questions, computational problems, and real-world applications. Gamified elements, such as immediate feedback (e.g., correct/incorrect with explanations), hints, and achievement badges for milestones, were integrated into the content creation.

In the gamified system development, in collaboration with an IT expert, the researcher designed and implemented key system features. These included: progression bar tied to skill points, which reflected how quickly students answered questions and how frequently they engaged with the reviewer; achievement rewards for milestones, such as solving a set number of problems correctly in succession; leaderboard system where students were able to see their rankings compared to peers based on their performance.

For quality assurance and testing, the researcher independently conducted system testing by logging in as both administrator and student users to simulate actual usage and identify possible bugs or system issues. During the testing phase, the researcher logged in as both administrator and student to identify potential bugs and issues, which were immediately communicated to the IT expert. Once no further bugs were observed during testing, the IT expert provided the finalized system for installation in the designated computer laboratories, with the approval of the Campus Director of USTP Jasaan. The system operated through the school’s local Wi-Fi network, requiring users to connect to the local network to access the gamified reviewer. Access to the system was managed by an IT faculty member of USTP Jasaan, who was responsible for opening and closing the system. After securing approval for system installation, the researcher proceeded to the evaluation and assessment phase, wherein research survey questionnaires were distributed to mathematics and IT experts to evaluate the gamified reviewer.

In the evaluation and assessment phase, the researcher distributed Likert-scale instrument to selected mathematics and IT experts to independently evaluate and assess the gamified reviewer. The participants were carefully identified and selected based on their knowledge, experience, and expertise. Experts were determined using the following criteria: (1) a bachelor's degree or higher in Mathematics, IT, or a related field, (2) at least three years of relevant teaching or industry experience, and (3) familiarity with digital tools, learning management systems, or software development in an academic or professional setting. To ensure the reliability of expert input, research suggested that a minimum of 5 to 10 experts was sufficient for expert reviews ¹⁴, for this study, the researcher consulted 10 experts for each field. All experts, were fully informed about the purpose, objectives, and procedures of the study. They were provided with an informed consent form explaining their role in evaluating the gamified reviewer and were assured that participation was entirely voluntary. Experts provided their consent prior to engaging with the system and completing the questionnaire. The experts were provided with system credentials and were instructed to log in freely as administrators. They were also allowed to log in as students and create accounts to further assess the system from a student’s perspective. The system was accessed through the school’s local Wi-Fi network, and the experts were given one week to explore all features, ensuring sufficient time to fully evaluate the reviewer. The feedback gathered from the experts served as the basis for evaluating the functionality, usability, and educational quality of the gamified reviewer. Findings from this phase provided insights into the system’s strengths and areas for improvement and allowed the researcher to craft recommendations and conclusions from the study.

To analyze the experts’ assessments, descriptive analysis was employed using the mean and standard deviation. Inter-rater reliability using the Intraclass Correlation Coefficient (ICC) was also employed to measure the level of agreement between the experts.

3. Results and Discussion

The mathematics experts assessed the gamified reviewer based on two categories: content, and instructional quality, which is shown in Table 1 and Table 2. The content refers to the accuracy, depth, and relevance of the mathematical content. The instructional quality refers to the effectiveness of the instructional design, engagement, and learning impact of the gamified reviewer. Table 3 presents the Inter-rater agreement of the mathematics experts.

The overall mean score for content and instructional quality as shown in Table 1 and Table 2 is 4.94 (SD=0.12), and 4.94 (SD=0.12), respectively. These results indicates that the mathematics experts strongly agrees that the gamified reviewer demonstrates excellent content and instructional quality. The ICC also revealed that Mathematics experts have a moderate level of agreement on their assessment on the content and instructional quality of the gamified reviewer.

The IT experts assessed the gamified reviewer based on these three categories: system usefulness, information quality, and interface quality. System Usefulness (SysUse) refers to the ease of use of the system. Information Quality (InfoQual) refers to how understandable the functions of the system. Interface Quality (InterQual) refers to the quality of the design and features of the gamified reviewer. Table 4, Table 5, and Table 6 present the descriptive analysis of the mean scores and standard deviation of the IT expert assessment. Table 3 presents the Inter-rater agreement of the IT experts.

The overall mean score for SysUse, InfoQual, and InterQual, as shown in Table 4, Table 5, and Table 6 is 2.42 (SD=1.61), 2.62 (SD=1.72), and 3.03 (SD=1.66), respectively. SysUse and InfoQual have an overall mean less than the reference mean, which indicates that the gamified reviewer shows good system usefulness, and information quality. Interface Quality, on the other hand, have an overall mean higher than the reference mean, which indicates that Dasig NAME needs improvement on its interface quality.

However, high SD was observed consistently on the results which may indicates variabilities on the IT experts’ perception. This was further affirmed by the ICC value. All categories have poor agreement. This implies that the IT experts did not share a strong consensus in their evaluation.

4. Conclusion and Recommendation

The experts are satisfied with the gamified reviewer “Dasig NAME” in terms of its content, instructional quality, system usefulness, and information quality, while the interface quality remains an area where the gamified reviewer can still be improved. However, in terms of expert’s agreement, the Mathematics Experts generally agreed on the gamified reviewers’ content and instructional quality, while the IT experts have varied perception on its overall system. It is recommended that future researchers may enhance the gamified reviewer by addressing the identified limitations in interface quality and poor agreement within the IT experts regarding its overall system.

ACKNOWLEDGEMENTS

The researchers are grateful to all the IT and Mathematics experts who extended their time and effort in assessing the gamified reviewer, and to the researchers’ colleagues for their support throughout the implementation of the study.

References