Abstract

During the COVID-19 pandemic, courses traditionally taught in-person were adapted to an online learning environment, and following the removal of physical distancing restrictions, courses were subsequently transitioned back into a traditional in-person learning environment. During this period of time, concerns were raised regarding student stress levels, engagement, academic performance, and the overall quality of the educational experience provided during pandemic-associated online learning. In this three-year comparative study, we used online surveys to compare three distinct learning environments within the context of the same fourth-year biological science course across three separate years: i) online (winter 2021; n=263), ii) hybrid (combined online and in-person lectures, winter 2022; n=210), and iii) in-person (winter 2023; n=171). The results indicate that there was no difference between learning environments in students’ final grades, perceived stress levels, and learning approach (deep and surface) scores (P>0.05), suggesting that students were adapting to different learning environments and academic outcomes. These results were consistent during the COVID-19 pandemic and in the years transitioning out of COVID-19-associated restrictions. However, in comparison to in-person learning, students associated the online learning environment with more overall stress, specifically stress associated with organizing a schedule and a higher workload (P<0.05). Collectively, these findings demonstrate that a consistent educational experience can be provided in online learning without adverse effects on students’ grades, overall perceived stress levels, and learning approach, which should be considered when making decisions regarding the effectiveness of online learning or the continuation of course delivery within an online environment post-pandemic.

1. Introduction

In response to the COVID-19 pandemic learning environments/lecture formats in post-secondary education transitioned to online learning ¹. Although some courses were already delivered online pre-pandemic, many instructors and students had to quickly adjust to the online learning format and experienced challenges with technology required for online instruction and learning, including individual user technology or digital literacy, internet connectivity/access, and difficulties accessing online course resources (e.g., websites or virtual learning spaces) ^{2, 3}. Some instructors struggled with shifting their pedagogical approach, adapting courses, and their associated workload designed for in-person learning into an online learning environment ^{3, 4}. Similarly, not all students employed the behaviours and learning approaches that would promote success in online learning, such as self-efficacy, self-regulation, motivation, time management, and organizational skills ^{5, 6}. This contributed to variability in students’ satisfaction with the online learning experience ^{6, 7}. Despite the benefit of increased flexibility in online learning ⁸, increased workload has been a reported barrier for some students ⁴. Further, a rise in students’ self-reported mental health challenges including stress, anxiety, and depression levels were reported during pandemic-associated online learning ^{8, 9, 10 11, 12, 13 14, 15 16, 17}; although some studies report lower or similar stress levels in online formats ^{2, 4, 18}. Increased experiences of academic stress in online learning coincided with non-academic and academic sources of uncertainty, social isolation, and limited access to social coping mechanisms to alleviate stress ^{14 19, 20, 21}. In response, instructors were also challenged to develop new teaching approaches and assessments to promote student engagement in an attempt to counter students’ social isolation during the pandemic ^{3, 22}.

Opinions surrounding the optimal learning environment vary (e.g., online versus in-person) ^{23, 24, 25}, despite the fact that there are known benefits associated with any learning format. Online learning benefits include elimination of physical commuting times and associated expenses, schedule and program flexibility, access to recorded lectures to facilitate learning and concept review, and the potential for an enriched learning experience or new skill development as a result of new pedagogical approaches to promote student engagement online ^{3, 7, 8 25, 26, 27 28, 29, 30 21, 32, 33}. Further, some students prefer in-person learning to allow for face-to-face interactions with instructors and fellow students that foster discussion and more engagement, in contrast to online learning, which can be isolating ³⁴.

As pandemic-associated public health restrictions were removed, courses in higher education also transitioned away from online learning and returned to the traditional face-to-face learning environment. Since the educational experience in online and in-person learning environments can differ, and subsequently impact student learning ^{35, 36}, it is important to assess how the transition out of COVID-19-associated online learning impacted critical factors that can influence student learning and academic success. It is particularly relevant to monitor changes in students’ academic stress experience in different learning environments as studies have shown the rise in online learning-associated mental health challenges ^{8, 9, 10 11, 12, 13 14, 15 16, 17}; however, some students may be optimally suited for online learning ⁷⁹ and may wish to continue in an online learning environment post-pandemic. Therefore, the current study was conducted within the same fourth-year biological science course over a three-year period, capturing three distinct learning environments/lecture formats: i) online asynchronous learning, ii) hybrid (students’ choice to attend in-person lectures that were recorded for asynchronous online learning), and iii) in-person learning. This captured academic semesters during both the COVID-19 pandemic and the transition out of COVID-19-associated online learning and the return to traditional in-person learning. This study aimed to determine how critical outcomes, including students’ academic performance, perceived stress, and learning approach differed between these learning environments over a three-year period.

2. Methods

Participants in this study (n=644 total) were undergraduate students enrolled in a fourth-year biological science course from 2021 to 2023. Participation in the study was optional and all students registered in the course were invited to participate in the study. Due to the COVID-19 pandemic, the standard 12-week course format was delivered online asynchronously in the 2021 winter semester (W21; n=263 out of 293 registered students, reflective of 89.8% participation), in a hybrid format (in-person lectures with optional in-person attendance wherein lectures recorded and available online asynchronously) in the winter 2022 semester (W22; n=210 out of 231 registered students, reflective of 90.9% participation), and in-person with no recorded lectures available in the winter 2023 semester (W23; n=171 out of 208 registered students, reflective of 82.2% participation). These three different learning environments during this three-year period provided the opportunity for a comparative study of different learning environments within the same course context. The core course requirements did not change between learning environments during the three-year period; however, final exams were written online in the W21 semester and in-person in the W22 and W23 semesters. The format and questions used on both the midterm and final exams were the same across academic years despite differences in the exam delivery mode (online exams utilized the Respondus lock-down browser remote invigilation and in-person exams with instructor and teaching assistant invigilation). No student demographic information was collected.

Two online surveys were completed in each semester of the project (W21, W22, and W23), at both the start (weeks 1 and 2; Survey 1) and end (weeks 11-12; Survey 2) of the semester. Surveys were conducted using the Qualtrics Insight Platform (Provo, UT, USA) and distributed via a private link to the students’ university email addresses. Each survey assessed students’: i) learning approach using the validated two-factor Revised Study Process Questionnaire (RSPQ-2F) ³⁷, which is a 20-item validated survey that assesses students deep and surface learning approach, and ii) perceived student stress levels using the validated Perceived Stress Scale (PSS) ³⁸. The PSS is a validated 14-item scale that assesses stress experience from all sources, both academic and non-academic. Only students who completed both surveys (Survey 1 and 2) and completed the course were included in the study. Students who completed Survey 1 were awarded a participation bonus of 2% to their midterm exam grade and students who completed Survey 2 were awarded 2% bonus to their final exam grade. Alternative assignments were available to students who did not wish to complete the optional online surveys but still wanted to earn the exam bonus marks. All students provided their informed consent before completing each online survey and this study was approved by the University of Guelph Research Ethics Board (REB#20-10-026).

Statistical analysis was conducted using GraphPad Prism software version 9.3.1 (San Diego, CA, USA). For all data, the predefined upper limit of probability for statistical significance was P<0.05 and data are presented as mean values with the standard error of the mean (SEM). The Shapiro–Wilk test was used to test for normality. Data were analyzed by either one-way ANOVA (main effect: learning environment) or two-way ANOVA (main effects: learning environment and time (i.e., Survey 1 versus Survey 2). This was followed by Tukey's Studentized (for normally distributed data) or Kruskal-Wallis test (for data sets that were not normally distributed). Pearson correlations were conducted to determine the relationships between learning approach, perceived student stress levels, and academic performance (final grade percentage), wherein the correlation coefficient (r) value is shown.

3. Results

Students’ learning approach scores assessing deep and surface learning total scores, motive scores and strategy scores at both the start of the semester and the end of the semester in each learning environment are shown in Figure 1. All deep and surface learning approach scores were consistent across learning environments and did not differ from each other (P>0.05).

At the start of the semester, students’ perceived stress scale (PSS) scores were significantly higher in the in-person learning environment compared to both the online and hybrid learning environments (P<0.05); however, at the end of the semester (Survey 2) PSS scores in the in-person group decreased and did not differ between the online or hybrid learning environments (P>0.05; Figure 2A). PSS scores reflect all sources of stress (both academic and non-academic sources combined) ³⁸; however, 68% of students in the online group, 74% of students in the hybrid group, and 69% of students in the in-person group identified academic sources of stress as the major source of stress experienced. Both academic stress frequency (Figure 2B) and academic stress intensity (Figure 2C), which were assessed at the end of the semester (i.e., Survey 2 only), did not differ between learning environments (P>0.05).

In terms of stress associated with different learning environments, students in the online learning environment identified that online learning was a more stressful learning environment compared to both the hybrid and in-person learning environment cohorts (P<0.05; Figure 3A). Similarly, students in the online learning environment cohort identified their learning environment as having a higher academic workload (P<0.05; Figure 3B) and they experienced more stress associated with organizing a schedule (P<0.05; Figure 3C) compared to students in the hybrid and in-person learning environment cohorts. Students’ perceptions of stress associated with time management did not differ between learning environments (P>0.05; Figure 3D).

There was a significant positive correlation between students’ surface total learning approach score and PSS score when the results from all learning environments were combined (P<0.05; Table 1). Similarly, when data from all learning environments were combined surface motive scores and surface strategy scores exhibited a positive relationship with PSS scores (P<0.05; Table 1). Therefore, independent of learning environment, students experiencing higher stress levels favoured using surface learning approaches. Within individual types of learning environments, PSS scores were positively correlated only with surface strategy scores within the hybrid learning environment (P<0.05; Table 1). There were no significant relationships between students’ PSS scores and any deep learning approach score either within individual types of learning environments or when all learning environments were combined (P>0.05; Table 1).

Academic performance, as assessed by students’ final grade in each learning environment is shown in Figure 4, wherein there was no difference in overall final grades (P>0.05). The relationship between students’ final grade in the course and their PSS scores and learning approach are shown in Table 2. Students’ final grades were negatively correlated with students’ PSS scores independent of learning environment (i.e., all learning environments combined). Within specific learning environments, final grades were negatively correlated with PSS scores in the hybrid and in-person learning environments (P<0.05; Table 2), whereas there was no relationship in the online learning environment.

When all learning environment data were combined there was no relationship between students’ final grade and any surface learning approach score (P>0.05; Table 2). Within the online learning environment, there was a weak negative correlation between students’ final grade in the course and their surface learning total score and surface motive score (P<0.05; Table 2), indicating that students earning lower grades in this specific learning environment were more likely to utilize surface learning approaches. Conversely, surface strategy scores were positively correlated with academic performance only in the hybrid learning environment (P<0.05; Table 2). All deep learning approach scores (total, motive, and strategy) were positively correlated with final grade when all learning environments were combined and in the online and hybrid learning environments (P<0.05; Table 2). There was no relationship between final grade and any deep learning approach score in the in-person learning environment (P>0.05; Table 2). Collectively, these data underscore the value of deep learning approaches to support academic success.

4. Discussion

Within the same biological science course, we examined the influence of different learning environments associated with the COVID-19 pandemic, specifically: i) online (in the W21 semester) when students experienced an emergency shift to remote learning, ii) hybrid (in the W22 semester) when in-person learning was permitted for 8 of the 12 weeks of the semester and students could choose to remain online, return to in-person or attend a combination of lecture formats at their discretion, and iii) in-person (in the W23 semester), which reflects the emergence from COVID-19-associated online learning when all courses had fully returned to in-person learning in the preceding fall semester. These three course offerings had minimal differences in the assessment structure or examination questions but were offered in three distinct learning environments that also reflected distinct periods of time associated with the COVID-19 pandemic: i) in the midst of pandemic-associated lockdowns and social distancing mandates (online, W21), ii) during the early stages of lifting COVID-19 restrictions (hybrid, W22), and iii) during the emergence from the pandemic with no restrictions and a return to pre-COVID academic semester formats (in-person, W23). Importantly, despite different learning environments the adaptation of this course first online and then returning to in-person resulted in no difference in students’ learning approach scores (Figure 1), PSS scores at the end of the semester (Figure 2), and academic performance (i.e., grades) (Figure 4). This consistency across academic years demonstrates the adaptability of both instructors and students to different learning environments.

Final grades in the course were consistent across learning environments (Figure 4), which is similar to other studies that assessed changes within an individual course throughout the COVID-19 pandemic, but it should be noted that the impact of the pandemic on academic performance can vary between courses and subjects ³⁹. Other studies have reported challenges with students’ academic adaptation to a new learning environment ^{45, 46, 47, 48}. For some, the transition to an online learning environment resulted in either no effect or slightly increased grades during online/remote learning ^{40, 41, 42, 43, 44}; however, for others, the transition to online learning resulted in reduced academic performance ^{45, 46, 47, 48}. Furthermore, negative impacts of online learning on students’ perceptions of learning, satisfaction with their learning, and career preparation have also been reported ^{39, 44}. Collectively, this highlights the need for additional academic support for pandemic-impacted students and the cautious interpretation of grades during this period. Not unexpectedly, the transition from online learning and the return to traditional in-person learning required students to adapt to a new learning environment. Lower exam grades, academic performance, and/or academic-related skill competency (e.g., problem-solving skills) and attitudes toward academics (e.g., increased procrastination and declining attendance, time management and interpersonal communication skills) were reported upon returning to in-person learning, thereby indicating that some students experienced challenges adapting to post-pandemic in-person learning environments ^{49, 50, 51}.

Students’ affective/emotional, behavioural, and social engagement was shown to decline during the transition from in-person to online learning, which manifested as declining positive attitudes toward learning about science and participation in class discussions ^{52, 53, 54, 55}. Successful strategies to increase engagement in online learning emphasized maintaining a classroom community ⁵³ and promoting interactions between students via low-stakes peer discussion assignments ⁵⁶ and online learning groups ^{18, 27}. Cognitive engagement in online learning has been reported to increase in some studies ^{52, 55} and to decline in others ⁵⁴. Student engagement is an important indicator for success in online learning, which can be affected by a range of factors including technology challenges in accessing online course materials, enthusiasm for or interest in learning, feelings of belonging in the course environment, self-regulation in learning, and taking a deeper learning approach ^{37 57, 58, 59, 60, 61}. In this connection, students with higher deep learning approach scores have been demonstrated to have higher engagement and satisfaction with course material ³⁷. Conversely, higher surface learning approach scores are associated with lower engagement and self-efficacy, which is a skill that is required for success in online learning ^{37, 60, 62}. Surface learning approaches involve the completion of the minimum amount of work required to meet course expectations, emphasize rote memorization versus understanding and integration of concepts, limited comprehension, and/or engaging with the course content in a comprehensive way ^{37, 63}. Therefore, it is not surprising that higher surface learning scores were associated with lower final grades (Table 2). Further, higher surface learning scores have been positively correlated with experiencing more stress associated with time management and increased difficulty organizing and maintaining a schedule in online learning ¹⁸, outcomes that were also observed in the current study within the online learning environment. Therefore, in the current study students with higher surface learning approaches (independent of individual learning environments) exhibited higher PSS scores (Table 1) and lower final grades in the course (Table 2), in particular in the online and hybrid learning environments. This may reflect students’ adaptation to the learning environment when experiencing challenges associated with online learning that resulted in lower overall engagement in learning and adaptive strategies aimed at memorization with reduced concept comprehension, as seen previously ^{52, 53, 54, 55}.

Attitudes toward learning and stress can impact academic performance, engagement or learning approach. Students experiencing more anxiety, boredom, and anger while learning have been shown to favour a surface learning approach, whereas students experiencing enjoyment while learning were shown to favour deep learning approaches ⁵⁶. There is a strong relationship demonstrating that the more academic stress a student faces, the lower their academic achievement ⁶⁴. In the current study there was no difference between learning environments in any learning approach score (deep or surface) (Figure 1). However, higher deep learning approach scores were positively associated with academic performance, apart from in the in-person learning environment wherein grades exhibited no relationship with either deep or surface learning approach scores (Table 2). This indicates that other factors that were not assessed were more likely to be driving students’ grades during post-pandemic in-person learning and further study is required. Previously, increased levels of stress and anxiety have been reported during COVID-19-associated online learning ^{9, 11, 12, 13, 15, 16, 17, 65}, and stress has been shown to adversely impact students’ engagement and self-efficacy ^{61, 66, 67}. In the current study, students in the in-person learning environment had higher initial perceived stress levels compared to other learning environments; however, at the end of the semester there was no difference in perceived stress between learning environments (Figure 2). This finding is in contrast with other studies reporting higher stress anxiety and/or depression levels during pandemic-associated online learning ^{8, 9, 10, 11 12, 13, 14 15, 16, 17}.

Stress can also impact academic achievement and students’ motivation to learn ^{56, 66, 68, 69, 70, 71, 72, 73}; however, there was no difference in final grades between learning environments (Figure 4). This finding agrees with other studies demonstrating no difference in grade-based academic performance between in-person and online learning environments ^{74, 75}.

A strength of this study was the consistency and focus of the learning experience and academic outcomes (grades, learning approach, and perceived stress) within the context of a single course that did not vary over a three-year period. This permitted the comparison between three distinct learning environments: online, hybrid, and in-person. A consequence of this study design is the limitation that the findings are course context-specific and may not reflect the experiences and outcomes in other courses that also transitioned from in-person to online learning and then back to in-person learning during and emerging from the COVID-19 pandemic. Although it is important to assess changes in students’ academic outcomes (i.e., grades) associated with the pandemic, the variability in findings from many studies combined show variability in academic performance ^{39, 40, 41, 42 43, 44, 45 46, 47, 48}, which highlights the importance of using caution when interpreting transcripts and students grades during this period. Some students struggled with mental health challenges and changes in the learning environment during the pandemic ^{8, 9, 10, 11 12, 13, 14 15, 16, 17}, whereas other students may have been able to adapt more readily. It is important to look beyond grades and consider alternative assessment metrics when assessing the capabilities of students who were in higher education during this period.

This study was conducted over a three-year period, and it is important to note that each year included in this study was associated with different sources of stress attributable to the COVID-19 pandemic. For example, students in the Online Group (Winter 2021) were learning exclusively within an online learning environment in the middle of COVID-19 pandemic-associated lockdowns. Conversely, students in the Hybrid Group (Winter 2022) were in the semester when COVID-19 physical distance restrictions were lifted and in-person classes were permitted to resume mid-semester, providing an opportunity for students to choose between continuing online, resuming in-person learning or both. Lastly, students in the In-Person Group (Winter 2023) were learning exclusively in-person in all courses but had approximately two years of their university program delivered in an online learning format. Therefore, students in each year of study/learning environment would be anticipated to have different experiences with respect to social distancing, isolation and social/personal development. Furthermore, students in each year of the study would have been expected to adapt to a different learning environment (either newly experiencing online learning or transitioning out of online learning and returning to in-person learning). Although we did not see a difference in students’ perceived stress levels, the potential stress experience year-over-year would not be equivalent. Despite the anticipated differences between students in each year of the study/learning environment, the results show that students' perceived stress levels did not differ. In this connection, students’ stress experience for those in the Hybrid Group, who had the option of self-selecting their preferred learning environment once in-person classes were permitted to resume, did not differ between those that remained online and those that elected to return to in-person learning. However, independent of learning environment choice, students experiencing lower stress levels were shown to earn higher final grades ⁷⁹. Before drawing final conclusions regarding an optimal learning environment or the abandonment of online learning in favour of a full return to only in-person learning, it will be important to re-evaluate differences in learning environments without the confounding factor of a global pandemic and ideally within a course that is offered in different learning environments concurrently.

5. Conclusion

This study was conducted during different phases of the COVID-19 pandemic, namely in the middle of the pandemic amid physical distancing mandates when university courses were offered online learning environment (W21; online learning) and during the years when physical distancing mandates were either initially lifting (W22; hybrid learning) or during the first academic year emerging from the pandemic that was fully in-person following the return to a traditional pre-pandemic lecture format (W23; in-person learning). Despite these differences, the current study demonstrates that students’ educational experiences did not differ between learning environments. Within this context of a single course provided through three distinct types of learning environments academic outcomes (i.e., final grades), stress experience, and learning approach (that reflects how students are adapting to the learning environment in either a surface or deep learning manner) were consistent. More importantly, when students from all three learning environments were combined the critical role of learning approach to modulate perceived stress and academic performance (i.e., grades) emerged. Students utilizing a surface learning approach experienced more stress in contrast to students utilizing a deep learning approach earned higher grades. Therefore, a critical recommendation from this work for instructors is to facilitate the development of deep learning attributes and behaviours in their students to attenuate academic stress and support academic success. Collectively these data demonstrate that online learning can be an effective learning environment and could remain a viable educational modality that could be sustained post-pandemic. Based on these findings, if future conditions arise that require in-person courses to be offered in an online learning environment, students may be optimally supported for success in online learning if deep learning approaches and behaviours are fostered by instructors. Institutions drawing conclusions regarding the effectiveness of the online learning environment in contrast to a traditional face-to-face learning environment should utilize an evidence-based approach to inform these instructional decisions.

Statement of Competing Interests

The authors have no conflicts of interest to disclose.

List of Abbreviations

RSPQ-2F, Two-factor Revised Study Process Questionnaire; PSS, Perceived Stress Scale

References