Abstract

Augmented reality (AR) is a medium of transforming K-12 and higher education. The ability to overlay multimedia onto the real world for viewing using web-based environments and technology-enabled devices such as tablets, desktop computers, mackintosh, and phones affords students the ability to access information and resources any place and time they need it. Effective use of technology-based environments and augmented reality reduces cognitive load by scaffolding information and lessons' contents. Augmented reality can readily help in cognitive thinking and enhancing cognitive thinking processes. An increase in patients' cognitive processes using augmented reality in the medical industry where patients use augmented reality tasks to regain cognition, mobility, and ability to perform basic human tasks. The critical literature review will review augmented reality in higher education and its impact on increasing cognitive thinking. Challenges of using augmented reality will be reviewed, including its benefits. There will be a discussion on the gaps in research on augmented reality in higher education and best practices on the use of augmented reality.

1. Introduction

Traditional teaching and learning methods are decreasing. Learning is becoming increasingly digital and driven by technology innovations and multimedia. Augmented reality is increasing in schools ¹ and has been the most spotlighted technology based on advances in computer science ². For example, Google Glasses and HoloLens have been a great attraction ^{2, 3}. By 2023, AR's value in the education technology industry is estimated to surpass $7.9 billion ⁴. Today, corporations and education sectors explore augmented reality solutions for more effective learning because it turns an ordinary classroom or lesson into an engaging learning experience.

Augmented reality (AR) enhances real-world learning environments with sound effects, text, multimedia, animation, and graphics, thus bringing an enriched version of immediate learning surrounding by layering digital content on the graphic representation of the real-world ⁵. A lesson with augmented reality technology provides virtual elements and adds game elements to support the learning content. The instructors can explain a subject using visual representation, and learners can test their understanding of concepts in practice ⁶. As a result, the lessons are more interactive, and learners have a better understanding of the information. The interactivity of the learner with the augmented reality tools could be traced back to the use of the first AR device.

The first AR device was invented in 1968 by Ivan Sutherland, and further developments were made in 1974 by Myron Krueger, who developed a system of cameras and projectors to show projections of computer graphics on a screen ⁷. Early augmented reality systems (ARS) were designed with a processing unit, infrastructure tracker unit, and visual unit ⁸. Reference ⁸ explained that the infrastructure tracker collected real-world information and sent it to the processing unit, which mixed real-world content with virtual content and sent the mixed content to the visual unit using the video out module. The visual unit consisted of video see-through and an optical see-through (Figure 1), which used a Head-Mounted Display (HMD). The HMD was dominantly used; however, they were expensive and low in mobility in size, and had challenges interacting with the real environment. Alternative approaches to developing ARS involved using tablets for direct view and monitors for indirect view.

Various AR applications are used in education. The AR applications include NarratorAR ⁹, used by learners between three and five to learn to write in an engaging and fun way. Mondly ¹⁰, a language learning application, also has an integrated AR-based virtual teacher to help learners practice their language skills as if they are learning with an instructor. Today, the most widely used AR types are markerless AR (Figure 1), which uses a marker or image target that visual trigger to identify the AR content, and marker-based AR ⁷. The AR applications provide a significant value to education which will be reviewed in the research.

This research paper seeks to address how AR affects cognitive thinking by answering the following questions: 1) How is cognitive thinking measured using AR systems, 2) what are the effects of AR on cognitive thinking and learning activities performance, 3) is there a relationship between the effects of cognitive thinking and task performance, and 4) how does AR affect cognitive load. By addressing these questions through a review of past relevant studies, the research paper will contribute to the current knowledge of impact of AR system on development of cognitive thinking and cognitive load. Thus, the researchers and AR developers can design more effective AR tools for increasing learning. Cognitive thinking will be defined in terms of cognitive processes during learning, teaching and instructional design. The classifications of cognitive load; intrinsic, extraneous loads will be evaluated and how they affect learning process. The literature review begins with defining augmented reality, its benefits and challenges, followed by augmented reality and cognitive development, major findings and discussions, gaps in research and best practices in using AR to develop cognitive thinking and reduce cognitive overload.

2. Methodology

This research uses the content analysis-based review method, a well-recognized method of reviewing and synthesizing literature and rationalizing outcomes. The steps followed included: 1) identification of keywords, 2) review of literature generated by the keywords, 3) categorizing information gathered by themes, 4) identifying information relevant to the research study, and 4) analyzing and evaluating the information gathered. The keywords used to gather related research articles for the research study: Augmented Reality, augmented reality in education, augmented reality and cognitive thinking, cognitive thinking processes, cognitive load, metacognition, learning, and teaching. Online databases used to search relevant resources are Science Direct, University of North Texas digital library, and Google Scholar. The articles were reviewed to identify if the content was related to the topic of the research study. The research articles and content were categorized based on themes and the information relevant to the research study.

3. Definition of Augmented Reality

Early definitions of AR were that the real world was augmented only through visual elements. However, with the development of haptic interactions; interfaces that touch-enabled interaction with virtual environments, and audio interactions; audio created to react to user input or changes in an application environment, associated with real-time spatial position, the AR concept has been further defined. AR was defined as a system that allows the user to see the real world composed with the real world, and it combines virtual and real elements, interactive in real-time and registered in a three-dimensional way ^{1, 12}. Further, AR is defined as an interface based on a combination of dynamic and static images, sounds, and sensations with a real user environment provided by technology devices and using natural interaction in the real world ¹³. Other definitions of AR add that the learners' perception of virtual prototyping with real entities is enhanced ³, and then with the continuum of multiple reality, generates a reality to virtuality blended environment, where most of the visual sensation comes from the real world, and the virtual elements contribute less. Augmented reality interactive technology is defined as a more advanced image interactivity technology which learners can use website features to manipulate and create products or environmental images to simulate the users’ actual experience ¹⁴. Lastly, AR is also regarded as a next-generation display technology since it can be used for various applications such as gaming and entertainment applications providing great experiences to interact with the imagery and real-world situations ².

The extraordinary experiences also enhance innovative mobile computing applications for entertainment and educational purposes ^{7, 15}, enhancing interaction between attributes of an object and their potential uses for learning (Table 1). Augmented reality simulated games are creative ways of enhancing learning and teaching Reference ¹⁵. The portable software platform in the augmented reality games allows developers to create cost-effective reality simulation games. Additionally, instructors and instructional designers can create games, content and learning conditions for different locations, custom-tailor games, and learners to also become game designers (Table 1).

The first augmented reality systems (Figure 2) were designed three processing blocks; he infrastructure tracker unit collected data from the real world, processing unit mixed the virtual contend with the real world content, and visual unit received information processed in the processing unit for the user to view using the video out element.

4. Benefits of Augmented Reality in Education

Augmented reality has been viewed to have various benefits in education as it is not limited to particular levels of education or age group as it can be used from kindergarten to university and in workplace environments ^{7, 16}. In learning, augmented reality helps students to process, remember and acquire information. Additionally, AR increases learning engagement. Additionally, the combination of virtual and real learning components fosters learning and reduce extraneous load ⁵. The visual unit could be a head mounted display (HMD) that displayed merged images on a closed view, or optical combiners to merge images withing an open view HMD ⁸.

Unlike other technology, such as virtual reality, AR can use readily available hardware. Learners can use readily available technology devices such as tablets, desktop computers, and smartphones ^{7, 16, 17} by application of spatial contiguity and temporal multimedia principles. According to the 2016 Pew Research Center Survey, 92 percent of young adults own a smartphone, and 30% used it for mobile learning ¹⁸. The affordance of the AR equipment's portability results in the inability to transport the equipment to different sites and move around in a location ¹⁵. However, there is a need to understand the immersive and delivery methods that these technologies offer ¹⁸.

AR can readily replace physical modes, printed material, and textbooks ¹⁶ and enjoy the combination of virtual and real worlds as they interplay between secondary and primary information ¹⁵. The learners can reuse information ³ and gather data unique to their current time, location, and environment, both simulated and real data ¹⁵. Thus, learning materials and resources are portable, resulting in the accessibility of education at any place and time; education becomes mobile.

Gamified and interactive augmented reality learning can have a positive impact on learners. An example is the ARTutor platform, a web-based application that can be used to upload assorted learning objects and learning materials ¹⁹. Additionally, its mobile application allows the interaction between the learning material and the learner ¹⁹. Learners are engaged throughout the learning process, there is increased concentration due to learning interactivity and learning is fun and effortless ^{17, 19}. The learning experiences help in increasing learning motivation such as using visualization structures to teach chemistry, mathematics, or geometry ². Learners also approved the concept and format of learning and the mode of interactions that the augmented reality elicited ¹⁵.

Creating interactive lessons using AR helps educate learners in subjects that would be viewed as uninteresting or complicated ¹⁷. The interactive lessons where all students are involved in the learning also help improve their teamwork and collaboration skills effortless ^{7, 17, 20}. Students learn how to prioritize tasks, share responsibilities and communicate effectively to complete tasks ¹⁹.

Students increase their learning achievement better through complete immersion and visualization of learning material ⁵. Augmented reality can be an educational object, and used in experimental research if it is controlled and supports interference of the user with real objects for the purpose of learning as it gives the learner the realism of the research, provides the cognitive and emotional experience which engages the learner ²¹. Additionally, AR modifies the contextual education, thus boosting the meaningfulness of the information resulting to 80% of information remaining in short term memory than 20% in traditional lectures ²¹. Further, ⁷ stated that learners efficiently process and remember information, and learners help learning coding and science subjects more effortless and more enjoyable. In their research study, ²² had an overall effect size of d = 0.68 in learning. Additionally, a research study that investigated students' hands-on experiences of basic electric circuits in a laboratory study showed that students who used AR showed increased learning gains ²³. Research shows that learning using visual objects helps understand information processing ¹⁷. The learners can also create unique scaffolding customized to the learners learning preferences ¹⁵. AR helps learners apply practical elements to the theoretical concepts learned as they virtually see the practical application of concepts and increases role-playing ¹⁸.

AR can be used to accurately reproduce on-the-job conditions to help master skills required in a particular job ²⁰. For example, how to assemble computer hardware, analyze the contents of an X-Ray, and identify different ailments in the human anatomy. Learners achieve better results by complete immersion and visualization of the subject matter. Instead of reading the subject's theory, they see its practicability in action ⁷.

AR creates opportunities safe learning environments for content or practice of skills that, if done in real-world environments, would cause danger or damages ^{7, 25}. For example, ⁷ stated that medical students could use AR to learn heart surgery techniques. Other examples include using AR to learn to dispose of hazardous material, assemble heavy and expensive machinery. The insertion of digitalized information into the actual workspace using AR can provide the trainees with intuitive tools to implement correct work procedures, improved accuracy and safety, and decreased error production ³.

5. Challenges of Augmented Reality in Education

Educators may have difficulty putting their new AR technologies into practice due to a lack of necessary training and explicit instruction on how to use the AR tools ^{7, 25}. Training would take time, and the educators have no allocation of time for other activities. The instructors need specialized knowledge on how the software and hardware are interdependent (Figure 3) to create learning content that suites the learners’ need and how to customize user-friendly interfaces and models, including effective evaluation methods ³. An example of an evaluation method could be adopted from the construction safety AR/VR system (Figure 4). The learners could be classified as either novice, experienced in the learning content or have prior knowledge of the material and subject experts in the subject. The assignments for AR applications could be classified as either field studies, experimental or case studies and specific evaluation methods for the assignments should be subjective or objective. Aspects that should be evaluated after using AR applications in the assigned tasks could be if information was correctly applied, if there was a change in learner performance and the level of interactivity with learning content and learning environment.

AR requires a technological and electronic base. Therefore, students have to have, for example, a smartphone, tablet, or desktop that can support the AR application ^{7, 25}. Unfortunately, this may not be possible for all students, thus causing a high expense for both the student and education institution to purchase the required hardware.

The AR application designed should work effectively in all devices and platforms to access learning content and resources. However, this may be a challenge in equally providing the same quality content on any device ⁷. For example, the smartphone interface content may look different from a tablet, laptop, or desktop computer. Reference ³ viewed the introduction of 3-dimensional (3D) information of high-definitions to be a challenge to users. The stereoscopic 3D displays such as liquid crystal displays (LCDs) could not successfully attract the general public attention as individuals would not feel comfortable when asked to wear equipment to view images. Transparency displays would be pretty expensive ².

6. Theories of Augmented Reality

Cognitive load, introduced in the 1980s as an instructional design theory ²⁶, is based on human cognitive architecture ^{24, 27}, and the assumption that learners can only process a limited amount of information at a time. The theory focuses on the biologically secondary information that requires explicit instruction and conscious effort ²⁸ and knowledge is domain specific. Biologically primary knowledge, which is acquired effortlessly and naturally as learners evolve in acquiring a category of knowledge ^{28, 29}, influences the acquisition of biologically secondary knowledge. Factors that cause increased cognitive load are inadequate instructional methods and intrinsically complex information. If the cognitive load is too high, it impedes learning ^{26, 29}. The cognitive load theory aims to facilitate the transfer of specific knowledge from the external environment into the working memory and the long-term memory for storage ²⁹. Further, when information is stored in the long-term memory, the information can be transferred to the working memory to govern action to specific learning activities and tasks.

Human memory is defined to have long-term memory and structured memory. The long-term memory has unlimited capacity to store knowledge while working memory is limited ³⁰. The long-term memory is information acquired by learners listening to what other people said, observing what other people have done and reading what other people have written ²⁷ and stores information, chunking multiple information elements into elements for specified functions. The working memory, which is responsible for task-relevant activation, maintenance and processing mental information (Figure 5) during task performance ²⁴, is responsible for cognitive overload, affecting learners’ performance. Similarly, the long-term memory, which was in earlier studies related to rote learning ²⁶, has no duration limits of transfer of information to working memory, and unlimited amounts of information can be held in the working memory ^{26, 29}. The relationship and transformation of knowledge from long-term memory to working memory and vice versa creates cognitive thinking processes, engaging in cognitive activities and active learning ²⁹. Cognitive load is further categories into intrinsic, germane and extraneous. Extraneous load, which can be changed by changing instructional procedures ²⁶, negatively affects learning as is created by poor instructional design and content and learning tasks not related to goals and objectives.

On the contrary, the intrinsic load is created from content learned and processed by the students’ memory simultaneously. The germane load is the working memory resources committed to developing intrinsic load ²⁶, and the memory effort that leads to the acquiring, automating, and creating a schema for the creation of mental knowledge for long-term memory during the learning process. For effective learning, there should be a balance between intrinsic and germane load and minimal extraneous load ^{26, 29}. Reliance on information held in long-term memory transforms the ability of the learner to process the information in the working memory ²⁶, thus reflecting the transformational consequences of education and change in behavior and attitudes.

Embodied learning, defined as the construction of relevant and valid knowledge accumulated by living through and in our bodies, asserts that it is impossible to separate the unity of the mind and body during learning experiences ^{31, 32}, without the body one does not experience the world. Humans use their sensory neural structures to crate multisensory representation of their environment and reuse their active brain structures during perception when mentally imagining an action or object ³³. Observing and influencing other human movements that make movements and mirror systems hypothesis ³¹ are all regarded embodied movements. For example, a learner uses eyes to see information, limbs to interact with different learning environments, and mixed reality tools, and the brain is needed to have all these experiences (Figure 6) ³². Similarly, explained the concept is explained by relying on motor activation when the brain is activated by hearing or reading a story where a particular action is described through imagination or acting out the scene ³¹.

In embodied learning, the human being is viewed as a mind with a body and a being who can get to the truth of learning experiences because the body is embedded in learning and the learning environment ³². Leaners understand things, ideas, and information from their own perspectives by experiencing them as they clearly understand how the information relates to each other and them. Kinect-based and motion-based games provide an example of embodied learning as learners physically engage and interact with learning materials ³⁴ and improve cognitive functioning and academic performance. The increased cognition results from the human sensorimotor system; muscle control and perceptual processing are capable of finding solutions in the physical learning environment and understanding specific learning tasks ³⁵.

Research studies show that physical engagement in lessons improve learners overall academic performance on standardized tests, increase recall of information, improved second-language comprehension, increased learning motivation, self -efficacy, increased positive emotions and influence understanding ³⁴. During cognitive thinking processes, the knowledge flows through the body's pre-reflective sensory routes, and learning is facilitated by cognitive deliberation ³¹. Learners can also ground abstract formal concepts into concrete bodily experiences and understand graphical representations describing dynamic situations. Research shows that there is the generation of knowledge structures in long-term memory compared to teaching methods relying solely on children's verbalization ³³. In augmented and mixed reality learning environments, learners receive immediate feedback on skill acquisition ³⁵. However, a combination of too many embodied cognitive-based interventions can cause extraneous load, and embodiment manipulations using very subtle bodily cues would cause an inability to measure performance for recall tests ³³. Overall incorporating embodied learning in learning and teaching enriches learners' experiences.

7. Augmented reality and Cognitive Development

In the last decades, Augmented Reality (AR) has increased interest in investigating cognitive functions in healthy participants, patients, and education sectors. An extensive set of technologies are utilized in studies related to attention and cognition. A learners cognitive innovativeness on the use of innovative technologies affects their attitude towards augmented reality interactive technologies ¹⁴. Further, high cognitive innovative learners not only enjoy thinking and applying immersive technologies for its own sake, but also have a propensity to devote mental energy to problem solving tasks ^{14, 31}. Combining virtual and real learning components is claimed to foster learning and reduce extraneous cognitive processing ⁵.

When using AR in learning, learners use artifacts to extend their knowledge and reasoning systems to support remembering and processing information and managing cognitive load ³⁶. The artifacts used in the cognitive procession of information are called cognitive artifacts. A cognitive artifact is defined as a software application or physical object that improves reasoning and thinking ⁸. Cognitive artifacts that are effectively used with augmented reality involve memory, attention, motor control multi-sensory perception to allow cognitive exercises. The artifact's interactive user actions must be tangible and accessible to comfort the user and allow force feedback interactions ⁸. Placing the artifacts in three-dimensional spaces demands the user's concentration. Cognitive artifacts are used to develop cognitive tasks that explore concentration and calculation, understanding of social cues, abstraction, judgment, recent and remote memory. Additionally, the cognitive artifacts develop planning and problem-solving skills, immediate recall, multi-sensory processing, language understanding, and speaking, and self-confidence ⁸.

8. Cognitive Load during Learning

There are three categories of cognitive loads that help a learner perform learning tasks: extraneous, germane, and intrinsic cognitive loads ³⁷. Reference ⁵ further explain that intrinsic cognitive load involves all the mental efforts to process complexity of the task, extraneous cognitive load as mental processes that process the presentation of informational material, and the germane cognitive load as the processes involved in processing all information to create new knowledge. The learners working memory is assumed to be limited in terms of the amount of information it can simultaneously process and the time information can be retained ²³. AR applications help in chunking learning tasks to reduce mental effort in processing complex tasks. The instructor can focus developing tasks that help in acquisition of practical skills for real-world experiences.

It is exceeding the limitations of information retention results in cognitive overload. In order to reduce cognitive overload, it is essential to adhere to multimedia principles of learning. The cognitive theory of multimedia learning can help design content for augmented reality as the theory assumes that the working memory processes visual and verbalized pictorial information in two separate channels ³⁸.

9. Findings and Discussions

Few studies have been done to consolidate task performance and mental workload. Further, as ³⁹ stated, there is a need to how AR affects task performance and mental workload. The concept of mental workload has various definitions ⁴⁰, including the interaction between the structure of systems and tasks, motivation and human capabilities, the supply and demand of processing, and attentional resources. Reference ⁴¹ explained that mental workload helps to identify learners' limited cognitive abilities in educational processes, while ³⁹ further defined mental workload as the relation between the mental resources demanded by a task and the resources supplied by the learner. The challenge by educators is to effectively evaluate the amount of mental workload demanded by a task to create tasks with meet the levels of mental workload levels. Mental overload leads to fatigue, stress, accidents, and illnesses and, further, result in poor performance.

AR is generally viewed to reduce mental workload, improve task performance, reduce error rate, and increase task accuracy in learning practical tasks ³⁹ in the medical and construction professions. AR can also reduce cognitive workload by changing the perception of instruction as AR could augment vital information in the real world to enhance understanding of information in instances where the learner would have to rely on textbook instructions ³⁹. The AR systems mitigate the mental activities and interpretation of information, and users would only need to execute a task without physical effort. The 3D models in AR help facilitate mental representation and spatial cognition, making the instructions easy to understand and reducing the mental workload of visualization of tasks and learning activities. However, new challenges can be introduced by the AR systems that could challenge learner performance. Most research studies assess AR systems' effectiveness on performance improvement, mainly on task completion time and error rate, and give little evidence on the effectiveness of AR systems on each stage of learning ³⁹.

10. Gaps in Research

Several research studies indicated that there is a lack of homogeneous findings on how augmented reality systems help in managing cognitive load and fostering cognitive processes ²³. Suggestions for further research included gaining insight into the learning processes and the learner's interaction with visualizations and components and how AR affected attention processes during learning experiences. There are difficulties in assessing the capability differences in hazard forecasting and risk assessment. Challenges assessing the Hawthorne Effect; if learners improved behavior under observation, and Practice Effect; if learners improved behavior under training ³. To avoid these challenges, there is proposal of four-dimensional (4D) environments ³. Additionally, future research should address how to design multi-user environments or collaborative augmented equipment platforms for different subjects and complex projects ³. The learner's cognition and perception have their features and characteristics; thus, it is vital to consider these aspects when designing hardware and learning interfaces. However, the successful design of the AR hardware and learning interfaces is based on mass human experiments' performance ³. Additionally, psychological theories and studies would provide more insight into learner perception and how to improve performance evaluation ³.

11. Best Practices in Use of Augmented Reality in Learning

Using AR in low-Complexity assembly tasks is slower than paper instruction, and when there is low tax complexity, the AR system produces insignificant differences to other teaching methods. Low task complexity may make it challenging to obtain statistically significant results on the effectiveness of AR systems. The instructor should evaluate a task and assess if it is viable to use an AR to develop learning activities.

AR is beneficial in designing tasks that enhance problem-solving. Research study results show no significant difference in performance attributed to low task complexity and lack of critical task-related information ³⁹. The instructor, therefore, needs to know the type of information the learner needs to solve the task effectively.

AR presentations influence mental workload. The instructor should evaluate how the virtual information is presented in the real world and modalities and annotation used, information presentation, and availability. Research on experimenting with high and low levels of simulated physical control and environmental embedding found that high levels of simulated physical control and 3D elements had significant positive effects on cognitive fluency and cognitive load. Graphic-based user interfaces were much preferred than text-based user interfaces, and video instructions induced a higher amount of mental load than AR annotations.

12. Conclusion

From the articles reviewed, augmented reality has a significant benefit in helping learnings develop their cognitive thinking processes ⁵. However, the instructors should use scaffolding to develop tasks that reduce cognitive load and promote long-term memory of the learning content.

References