1. Introduction

Scientific literacy is now widely considered by the developed countries as their priorities in response to the challenges arise in the 21st century ^[1]. While scientific literacy has various definitions, there are two important aspects: one is on the understanding of the science/technology enterprise and its relation with the public ^{[2, 3]}; another is to develop generic skills and knowledge, such as critical thinking skills and self-learning skills ^{[4, 5]}.

One way of cultivating scientific literacy could be via reading science classics. There are well established colleges that have long been using the classics reading approach for liberal arts education, such as the St. John’s College ^[6] and Columbia University ^[7]. As being one of the important parts of liberal education, scientific literacy can be attained via reading science classics. According to Gjertsen’s definition of science classics, they are “ones which transform science, or, in more fashionable language, which produce a major intellectual revolution” ^[8].

There are various advantages of the classics reading approach. First, presenting science with its instrumental value may not be enough to sustain non-science students’ interest in it. Science classics, on the other hand, can be used to present science under historical, philosophical, social and cultural contexts, which are much more intimate to the society and oneself. Studies have been reported that under this perspective, students could find science being profoundly relevant to them ^{[9, 10]}. Second, as Muench has pointed out, “The value and appeal of using primary literature in the classroom are rooted in literature’s unique potential to instruct students on the nature of scientific reasoning and communication” ^[11]. Such principle of scientific reasoning could touch upon many aspects in students’ daily life ^[12]. Third, students can retract the train of thought of our predecessors through their writings and be engaged as a dialogue in their own intellectual pursuit ^[13].

In Dialogue with Nature (the course code is abbreviated as UGFN) adopts the classics reading approach to cultivate scientific literacy for the undergraduate students in the Chinese University of Hong Kong (CUHK). UGFN is one of the two compulsory courses in the General Education Foundation (GEF) Programme introduced in 2010 specially designed for the four-year curriculum in CUHK. Beginning from 2012, it became compulsory for all approximately 4000 students per year. Distributed over two regular semesters and summer semester each year, the students are put into approximately 180 classes of maximum 25 students concurrently run by over 15 full-time and part-time lecturers.

UGFN requires students to read one set of excerpts from science-related classics per week and to discuss the core questions brought up by the classics that are enduring in the history of human civilizations. The discussion shall then be put in modern context to stimulate the students to formulate their own view on contemporary issues from a new horizon. The other foundation course, In Dialogue with Humanity (UGFH) used similar approach on humanities-related classics.

There are five Intended Learning Outcomes (ILOs) of the GEF programme as a whole, which are considered as generic skills and knowledge ^[14]:

1. a knowledge of major ideas that shape the world today

2. an intellectual inquisitiveness in addressing issues related to their life and society

3. the willingness and capacity to examine new and different ideas

4. the ability to engage in intensive and close reading

5. the readiness to articulate their own ideas clearly and systematically in writing and oral communication

On top of the Programme ILOs, there are also a separate set of ILOs for each course. The Course ILOs for UGFN includes mostly the understanding of the science/technology enterprise and its relation with the public ^[15]. These are:

1. read and discuss science texts with confidence

2. identify the essential characteristics of various methods of scientific inquiry that have significant impacts on how human beings view life and universe

3. formulate informed personal views on the societal implications of scientific explorations

4. relate the development in natural sciences highlighted in the course to contemporary human conditions

5. evaluate the scopes of application, achievement and limitations of highlighted scientific methods using multiple perspectives

Note that this paper will only focus on UGFN and not on the other foundation course UGFH.A more detailed description of GEF and UGFN has been reported by Kiang, Ng, and Cheung ^[16] and shall not be repeated in this paper.

The effectiveness of UGFN was studied via the entry-exit surveys as reported in ^[16]. The validity of using entry-exit surveys to measure student’s perception can be found in ^[17]. As was reported, the analysed surveys’ results indicated that UGFN is generally successful in changing student’s perceptions towards the ILOs. This is deemed an indicator of successfully cultivating students’ scientific literacy.

Other than the entry-exit surveys, GEF programme as a whole has also been assessed by an external reviewer, Jerry Gaff, Senior Scholar of Association of American Colleges and Universities, in 2014. In his formative evaluation report ^[18], it is noted that “there is considerable evidence from students and teachers alike that students are achieving important learning outcomes; enhanced reading ability and critical analysis, confidence in confronting difficult texts, open-mindedness and appreciation for intellectual diversity, and self-discovery of their own interests, abilities, and tastes.” Association for General and Liberal Studies (AGLS) also recognized the GEF programme with an award for Exemplary International General Education Program Improvement in 2015. The awards assessment criteria can be found in ^[19].

While UGFN has been successful in helping students to attain the ILOs, it is of concern that, students without high school science background were seemingly disadvantaged. Preliminary results in ^[16] have indicated that these non-science students would generally be underperforming - both measured by subjective surveys and objective academic grade performance. As a compulsory course, it is desirable to provide additional help to these students so that the course can truly benefit students of diversified background.

In 2015, an attempt had been made to develop a series of micro-modules to supplement this particular group of students. There are two main reasons that micro-modules were chosen as the means to address this concern. First, it had been widely reported that students who watched video-recorded lectures at their own time could be better engaged in interactive classroom activities ^{[20, 21]}. Second, this coheres with the international trend as micro-module production acquired its momentum in higher education with initiatives of Khan Academy, Coursera and edX in the United States in the new millennium ^[22].

The developed micro-modules consisted of recorded lectures that explain basic scientific concepts in a way that is tailor-made for UGFN. These lectures were recorded live in the fall term of 2014-15 and were provided as micro-modules online in the subsequent terms ^[23]. Each micro-module also includes a set of online exercises to serve as a checkpoint for the students who watched the recorded lectures. More description about the content of the micro-modules can be found in ^[16].

In the subsequent sections, statistical analysis on the effectiveness of the course and of the micro-modules will be reported. The entry-exit surveys’ result and academic grade analysis are of UGFN students in the fall term of 2015-16. It should be noted that, while the results of this investigation have specific importance to the teaching of UGFN, the analysis will also have general relevance to the teaching of classics-reading courses.

2. The Overall Effectiveness of the Micro-modules as Analysed by Surveys

This section will report the overall effectiveness of the micro-modules as measured by the entry and exit surveys. This is to evaluate whether the micro-modules can provide any positive impact to the class as a whole.

The design of the entry-exit surveys consisted of two separate surveys conducted at two time points of the course. Generally, the two surveys were conducted during the first and the last lesson of the participating classes. For practical reasons, some classes conducted the exit survey during the second last lesson due to the different class arrangement by different lecturers. The surveys were given by the lecturers to the students who were voluntarily participating. The surveys requested the students to fill in the last five digits of their student ID numbers to uniquely identify each student so that additional information related to the students could be retrieved. This information was acquired with the consent of the student volunteers with a guarantee of having no consequences to their academic grade.

The collected data from the entry survey consist of two parts. Part 1 is a list of 17 items measured in a 6-point Likert scale and is designed to be directly related to both the programme and course ILOs. The 17 items can be further divided into three sub-parts, corresponding to the Programme ILOs (Part 1a, Q1-Q5), course ILOs for UGFN (Part 1b, Q6-Q12) and the course ILOs for the other foundation course not covered in the discussion of this paper (Part 1c, Q13-Q17). Part 2 requested background information which contains basic demographic data as well as data of their high school academic background. For the exit survey, Part 1 is the identical 17 ILO-related items of the entry survey. Part 2 collected information about the effort spent on this course. Part 3 collected information about the learning methods of this course. Part 4 collected information about the use of the micro-modules, which only appeared to the students in the classes provided with the access to the micro-modules. In this paper, only Part 1a, 1b (common to both entry and exit surveys), Part 2 of the entry survey and Part 4 of the exit survey are relevant and are listed out in Table 1.

Table 1. A partial list of the survey items

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View next table

In the fall term 2015-16, a total of 38 classes of UGFN taught by 10 different lecturers provided the students with the opportunity to participate in the entry-exit survey. The total number of enrolled students in these 38 classes was 853. Among them, 565 were successfully tagged; a rate of 66.23%. Most of the remaining students were not tagged due to late add/drop from the classes or absence from the first or last lesson.

Among the 565 tagged students, a total of 414 students (73.3%) from the classes provided with the access of the micro-modules served as the experimental group. The remaining 151 students (26.7%) from classes with no access to the micro-modules served as the control group for comparison.

Table 2 listed the analysed results of Q1-Q12 ILO-related items from Part 1 of the entry and exit surveys. The results showed the comparison between the students with and without access to the micro-modules. For each of the two groups of students, “with” and “without” access to the micro-modules, their mean score from entry and exit surveys of each item are listed. △1 and △2 each indicates the exit minus the entry mean score of the students from the two groups correspondingly. △3 indicates △1 minus △2.

Table 2. Change of students’ perception towards ILO-related items “With” and “Without” micro-modules (measured by mean score difference of Entry and Exit in a 6-point Likert scale)

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

It should be first noted that, for all ILO items, △1 and △2 are all positive numbers and most of them are statistically significant (indicated by the asterisks next to the delta value). This shows that the course as a whole has a positive effect on students regardless of the provision of the micro-modules. △3 in here represents how much more effective on changing students’ perception on the ILOs with the provision of the micro-modules. As can be seen from the table, with the exception of Q5 (-0.03), all of the items are positive ranging from 0.10 to 0.47 with an overall mean of 0.28. All △3 were tested for their statistical significance (unpaired t-test, p≤ .05) and 8 out of the 12 items indicated statistical significance denoted by the asterisks. If the 12 items are further categorized into the two sub parts reflecting the programme ILOs and the course ILOs respectively, it becomes clearer on where were the effects of the micro-modules. Only 1 out of the 5 items in Part 1a has statistically significant △3, with an average of 0.13 in Part 1a. On the other hand, all 7 items in Part 1b are statistically significant with an average delta of 0.38, which is almost 3 times as large as in Part 1a. This phenomenon is best explained by the nature of the micro-modules. The micro-modules focus on basic scientific concepts, which are clearly more related to the course ILOs regarding the characteristics and method of science. Items in Part 1b understandably experienced greater effects. Items in Part 1a that are related to the programme ILOs, which are mainly generic skills, would in comparison experience only lesser effects.

Overall, the results show that the micro-modules have generally boosted the students’ perceptions further on their attainments of the programme ILOs and the course ILOs in comparison to the classes without it.

To provide another perspective on the data acquired, the survey data was also analyzed by treating it as dichotomous categorical data as in ^[24]. This can eliminate some reported concerns of the use of mean score on Likert-scale survey data ^{[25, 26]}. Moreover, treating the survey data as scores would be more influenced by students having larger effects, whereas treating it as simply two categories, would measure the percentage of students that attains the intended outcome, regardless of the magnitude of the individual’s effect. It should be noted that, however, such dichotomization of data would lead to another drawback which is the loss of information about the magnitude of effects per student ^[27]. The two ways of analysis are hence complementary to each other to provide a more comprehensive view on the effects of the micro-modules. In another words, the former method measures the average effect per student and hence could be described as measuring the depth of effects, whereas the latter method measures the percentage of students having effects and hence could be described as measuring the width of effects.

In the following analysis, a rating of 1-4 is considered to be a low rating while a rating of 5-6 is considered as a high rating given by a student. The decision of choosing the border line to be between 4 and 5 is first data driven – the collected data were skewed towards the left with the median usually located at 4 or 5. Hence, the line between 4 and 5 would be a more important distinction than the symmetrical choice of between 3 and 4. Second, in the standard 6 point Likert scale, a rating of 5 or 6 are usually interpreted as “agree” and “strongly agree”. It is deemed that changing students’ perception from “disagree” (1-3) or only “slightly agree” (4) to “agree” (5) and “strongly agree” (6) on those survey items matches the ILOs of the course and the programme.

The result is shown in Table 3. The two groups of students labelled “With” and “Without” are now further divided into 3 groups. “Up” represents the percentage of students who have initially in the entry survey gave a rating of 1-4 in the 6-point Likert scale and then a rating of 5-6 in the exit survey. “Down” represents the opposite. “Flat” represents the percentage of students who have remained in the same category. The column labelled “△ (up)” indicates the difference between the “Up” from the two groups of students, which represents what percentage of how many more students were converted from low to high ratings on the perception on the ILOs with the provision of the micro-modules. With the exception of Q2 (-1.49%) and Q5 (-0.89%), all these deltas are positive, ranging from 6.07% to 16.85%. The items were statistically tested (Chi-squared test, p≤ .05) and 8 out of the 12 deltas were statistically significant denoted by the asterisks.

The △(up) of the Part means also reflect a similar pattern as in Table 2. The micro-modules were more effective on Part 1b (13.13%) which represents the course ILOs, and a lesser effect on Part 1a (4.97%), which represents the programme ILOs.

Table 3. Change of students’ perception towards ILO-related items “With” and “Without” micro-modules (measured by percentage of students converted from low (1-4) to high (5-6) rating in a 6-point Likert scale)

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

Viewing the data from both the two representation, it is reasonable to conclude that there are significant effects of the micro-modules class-wide regardless whether it is measured by the depth or by the width.

3. The Effectiveness of the Micro-modules on the Users as Analysed by Surveys

This section shall investigate the effectiveness of the micro-modules by separating the actual users and the non-users. As mentioned in Section 2.1, Part 4 of the exit survey collected information about the usage of the micro-modules. This part only has 2 items, which asks the student about their usage of the two forms of the supplementary material provided in the fall term 2015-16 - either attending the live supplementary lecture or using the micro-module website. The following analysis treats the students who reported to have used either of the two options as the students who have “Used” the micro-modules. Two other groups of students are to be compared to this group. Those students from the same classes that provided the micro-modules but reported not to have used the micro-modules are the “Not-used” group. Those students from the classes without the provision of the micro-modules are the “Without” group.

Among the 414 students provided with the access of the micro-modules in fall 2015-16, 166 students (40.1%) reported to have used the micro-modules in Part 4 of the exit survey. 237 students (57.2%) reported to have not used the micro-modules. The remaining 11 students (2.7%) have not reported their usage on the micro-modules. These students will be considered as invalid cases and were excluded in the following analysis.

Table 4 used a similar layout of Table 2 but listing 3 categories of students and two deltas in one table. △1 is the difference between students who used and not-used the micro-modules. △2 is the difference between students who have not used the micro-modules from the classes with the provision of the micro-modules and the students from the classes without the provision of the micro-modules. Here in this section, it is argued that △1 represents the improvement directly from using the micro-modules by the individuals, while △2 represents the improvement indirectly from an improved class dynamics.

Firstly in △1, with an exception of one item (Q2), which is almost zero, all the other deltas are positive, range from 0.05 to 0.36 with an overall mean of 0.17. 3 out of 12 ILO items have statistically significant deltas.

On the other hand, △2 reflects a more significant improvement than △1. As shown in the table, with the exception of one negative number in Q5, all △2 are positives range from 0.09 to 0.41, with 7 out of the 12 items statistically significant. More importantly, items in Part 1b are all statistically significant. Part 1b mean for △2 (0.30) is even higher than △1 (0.21).

Table 4. Change of students’ perception towards ILO-related items “Used”, “Not-used” and “Without” micro-modules (measured by mean score difference of Entry and Exit in a 6-point Likert scale)

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

Similar to the analysis in Section 2, the comparison is conducted again by treating it as dichotomous categorical data. The result is shown in Table 5. As in Table 4, this table compares the same three groups of students and the same two deltas. Δ1 ranges from -7.42 to 10.41%, with 2 items having statistical significant deltas. Similar to Table 4, △2 is generally larger than △1 and ranges from -4.43% to 12.68%.

It is now clear that measuring from either the depth or the width, △2 is generally larger than △1. This is deemed evidence that students do not only benefit directly from using the micro-modules, but also from some other indirect benefit.

Since neither the “Not-used” and “Without” group of students have used the micro-modules directly but simply being located in the two different classes, the difference could be a result of the change of the entire class environment. This phenomenon could be explained by the fact that when more student in the class are equipped with sufficient basic scientific understandings, discussions among the students can be more in-depth and fruitful, and hence generate a better atmosphere for all students to reflect on the core questions and contemporary issues. It also allows the instructor to focus less on explanation on basic concepts in the tutorials and hence provide more time for in-depth discussion. Students who have not used the micro-modules could hence be able to have more chance of reflections on the core questions than if they were in the classes without micro-modules. Seeing from this perspective, the micro-modules become not just a tool that benefits a small group of students but could be as a means to boost the entire class dynamics. This is deemed an important environmental factor in order to cultivate students’ scientific literacy.

It should be noted that, result from Section 2 only showed that there was a significant average improvement of the classes by providing the micro-modules. That result did not differentiate the actual user of the micro-modules from the non-users. There could be two scenarios, either the overall improvement is purely dominated by the large improvement of the actual users, or that it could be generated by both the students used and not used the micro-modules. This section confirms that the second scenario is more likely to be the case.

Table 5. Change of students’ perception towards ILO-related items “Used”, “Not-used” and “Without” micro-modules (measured by percentage of students converted from low (1-4) to high (5-6) rating in a 6-point Likert scale)

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

4. The Effectiveness of the Micro-modules for the Targeted Users as Analysed by Surveys

This section will further analyse the effectiveness of the micro-modules by taking consideration of student backgrounds. The micro-modules, covering Physics, Biology and Chemistry, are designed to assist students who had not studied any of the three subjects in high school (the “Targeted students”). Students who took one or two of these three subjects in high school could still benefit from the micro-modules by supplementing his/her understanding in the missing subject(s) and were therefore considered as part of the Targeted students in this research. Only those students who had taken all three subjects in high school were considered as the “Non-targeted students”

This section of the study counted only the 414 students from the classes provided with the micro-modules. Among the 166 students who reported to have used the micro-modules in fall 2015-16, 133 students were considered as the targeted students. 31 students were considered as the Non-targeted students. Among the 237 students who did not use the micro-modules, 185 students were considered as the Targeted students. 51 students were considered as the Non-targeted students. A total of 3 students (1 used and 2 not-used the micro-modules) did not correctly report their science background in the Entry survey and were considered invalid cases. They were not included in the analysis of this section.

Table 6 put in parallel the analysis for the Targeted group and the Non-targeted group. Each of the deltas is the difference between the subgroups, Used and Not-used, within the group. For the Targeted group, the deltas range from -0.09 to 0.38 with an overall mean of 0.18. 4 out of the 12 items are statistically significant. This result provides some evidence that Targeted students who used micro-modules could generally achieve better in the ILOs.

In comparison, for the Non-targeted students, the deltas range from -0.20 to 0.40 with an overall mean of 0.13. While the deltas are smaller than the deltas in the Targeted group but still mostly positive, none of the 12 items are statistically significant.

Similar to the analysis in Sections 2 and 3, the comparison is conducted again by treating it as dichotomous categorical data. The result is shown in Table 7. For the Targeted students, the deltas range from -9.82 to 12.96%, with an overall mean of 5.29%. For the Non-targeted students, the deltas range from -10.44% to 16.32%, with an overall mean of 2.85%. The result shows similar pattern to Table 6.

Table 6. Change of students’ perception towards ILO-related items “Used” and “Not-used” micro-modules in Targeted and Non-targeted groups (measured by mean score difference of Entry and Exit in a 6-point Likert scale)

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

Table 7. Change of students’ perception towards ILO-related items “Used” and “Not-used” micro-modules in Targeted and Non-targeted groups (measured by percentage of students converted from low (1-4) to high (5-6) rating in a 6-point Likert scale)

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

This result could well be explained by the following reason. Since the micro-modules are providing merely foundational science concepts to the three core science subjects, students whose background omitted any of the three could be benefited from the micro-modules. On the other hand, for the Non-targeted students, while they may have used the micro-modules to help them to refresh their memory on what they should have known, the effects are less than that of the Targeted students.

5. The Effectiveness of the Micro-modules as Analysed by Academic Performance

This and the next section will focus on analysing another important indicator of the effectiveness of the micro-modules, namely, academic performance. Compared to the surveys that reflect students’ self-reported perception, academic performance could be deemed as an objective way to measure the effectiveness of the micro-modules. But readers should also bear in mind the limitations of how much academic performance could truly reflect student’s attainments in the ILOs., ILOs are sometimes immeasurable by assessments and the weighting of the assessment components may not always correspond to that of the ILOs. There is no simple conversion of the academic grades into the achievements of the ILOs. However, it is the achievements in the ILOs, not academic performance that reflect the intention of what should be taught by the teachers and how students have performed in the course. Moreover, due to the universities’ regulations, grade distribution of most classes shall follow certain guidelines. In CUHK, the guidelines include fitting all the students’ grades into predefined ranges of percentages, regardless of the differences of the overall performance between classes. This means any class-wide improvement can hardly be reflected in the grade distribution.

On the other hand, performance of any sub-class groupings relative to the whole class, such as students “With” and “Without” science background or “Used” and “Not-used” the micro-modules, could be compared. This is the reason why in this paper, academic performances are not compared class-wide, but compared between the sub-class groupings of different classes.

Differences between teachers’ assessment scheme could also be another concern for analysing the effectiveness via academic performance. It should be noted that in UGFN, these differences are already largely under control. First, all teachers agreed to use the same basic assessment scheme. That includes 50% from two written papers, 24% from 5-6 quizzes, and 26% from in-class and online discussion performance. Second, most teachers agreed to use the same eleven sets of classics with only some teachers may omit or add no more than one set. Third, the classes are all composed of no more than 25 students per class with mixed academic background. The use of student data originally from 38 classes taught by 10 different lecturers is considered appropriate to diminish any other uncontrolled differences between classes in accordance with existing literature ^{[28, 29, 30, 31]}.

A comparison of the academic performance between students who “Used”, “Not-used” and “Without” the provision of micro-modules was conducted. As mentioned in Section 2, among the classes that participated in the entry-exit surveys, 565 students were successfully tagged in the surveys. However, only 564 students received a letter grade at the end of the course with 1 person’s grade still pending. Among the 564 students, 414 students were from the classes provided with micro-modules. Within these students, 166 students used the micro-modules. 236 students did not use the micro-modules and 11 students did not give response to Part 4 of the exit survey on their usage of the micro-modules. Moreover, 151 students came from classes without micro-modules.

This section uses the concept of Grade Point Average (GPA) to make comparisons. Similar to many universities, CUHK adopted a standard grade conversion method. A letter grade indicating a student’s performance in a course (A, A-, B+, and etc.) is converted to a numerical value. For example, A is equivalent to 4, A- to 3.7 and B+ to 3.3. Refer to ^[32] for more details on the conversion method. Note that GPA usually refers to the average of all the courses studied by a single student within a certain period. In this paper, this calculation method is used for calculating the average grade of a group of students in a single course.

Table 8 shows that the GPA of those students who used the micro-modules (3.32) is higher than the GPA of the students not using the micro-modules (3.18) by △1 (0.15) and is statistically significant (unpaired t-test at p ≤ .05). This result objectively supports that students could improve in academic performance after using the micro-modules, alongside with the self-perceived improvements in ILOs.

Interestingly, the GPA difference between Not-used and Without micro-modules (△2=-0.02) is negligible. When this result is compared with the survey analysis reported in Section 3, an interesting difference would be apparent. In Table 4, △2 is generally greater or at least comparable to that of △1; in Table 8, △2 is unnoticeable. This result shall be interpreted with the following causes. First, due to the limitation of grading guidelines as mentioned, any class-wide improvement in academic performance between the classes “With” and “Without” micro-modules would not be reflected in the GPA. Second, academic assessments are more related to individual’s hardworking and are possibly less benefited as a result of improved class dynamics. Since the use of the micro-modules is voluntary, the Used group might well be more overlapping with the hardworking students in percentage. Hence, the higher GPA in the Used group than the two other groups is reasonable and conforms to our understanding of the limitation of making comparisons in academic performance.

Table 8. Students’ academic performance “Used”, “Not-used” and “Without” micro-modules (measured by GPA)

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

Similar to the previous sections, two perspectives that correspond to the depth and width of the effects shall be reported. GPA measures the depth of the effects by the micro-modules. Another measure shall be used to measure the width of the effects. In Table 9, an analysis is made to measure the percentage of students receiving a grade of B+ or above inside each group. It is common that in UGFN classes and possibly in most courses within CUHK that the median grade falls on B. In between B+ and B is therefore a good division line to separate a good grade and a bad grade.

As shown in Table 9, 61.45% of students who used the micro-modules receives a B+ or above grade, while only 49.58% of students who did not use the micro-modules receives a B+ or above grade. The difference, indicates by △1 is 11.87% and is statistically significant (Chi-Squared test at p ≤ .05). Similar to the result in Table 8, △2 (0.57%) is also negligible.

Table 9. Students’ academic performance “Used”, “Not-used” and “Without” micro-modules (measured by percentage of students who have B+ or above)

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

6. The Effectiveness of the Micro-modules for the Targeted Users as Analysed by Academic Performance

Similar to the groupings in Section 4, academic performance comparison is made on the Targeted and Non-targeted group of students from the classes provided with the micro-modules. Excluding the invalid cases mentioned earlier in the previous sections, a total of 399 students remained valid in the analysis in this section. Among them, 164 used the micro-modules and can be labelled as Targeted (133) or Non-targeted (31). Similarly, there are 237 students who have not used the micro-modules and can be labelled as Targeted (184) or Non-targeted (51).

Table 10 shows the GPA comparison of two pairs of groups of students – Targeted students who Used and Not-used the micro-modules, and Non-targeted students on the same split. The delta between Used (3.33) and Not-used (3.19) of the Targeted group is 0.13 and is statistically significant (unpaired t-test at p ≤ .05). Similarly, the delta between Used (3.33) and Not-used (3.13) of the Non-targeted group is 0.20 but is statistically insignificant.

Table 10. Student’s academic performance “Used” and “Not-used” micro-modules in Targeted and Non-targeted groups (measured by GPA)

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

Similar to Table 9, Table 11 measures the percentage of students received grades of B+ or above. The delta between Used (64.66%) and Not-used (48.37%) of the Targeted group is 16.29% and is statistically significant (Chi-Squared test at p ≤ .05). Conversely, the delta between Used (51.61%) and Not-used (54.90%) of the Non-targeted group is -3.29% but is statistically insignificant.

Given the results of Table 10 and Table 11, it seems to support that for the Targeted students, using the micro-modules would be a factor of receiving better grades at the end. But for the Non-targeted students, this relationship is not statistically significant.

Table 11. Students’ academic performance “Used” and “Not-used” micro-modules in Targeted and Non-targeted groups (measured by percentage of students who have B+ or above)

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

7. Conclusion

Analysis has been conducted to measure the effectiveness of the micro-modules developed for the course In Dialogue with Nature. This effectiveness was mainly measured using student surveys and student academic performance. Both measures were analysed by treating the data in two ways that corresponds to the depth and width of the effect, if any. In either way, the effects were significantly positive. This indicates that the micro-modules were effectively helping the students to perform better in the intended learning outcomes and the academic grades. It is argued that as a course that requires reading science-related classics involving certain level of scientific concepts, supplementing the students without this knowledge has shown to be beneficial to the individual users as well as the class as a whole. The micro-modules were therefore not just a tool that benefits a small group of students but could be used as a means to boost the entire class dynamics. This is deemed an important environmental factor to cultivate students’ scientific literacy.

There is immediate future work to do in this research. To further investigate the causes of the improvements, it is anticipated that evidence shall be collected via qualitative methodologies such as focus groups. Moreover, the effectiveness of this classics reading approach as compared to the other approaches in cultivating scientific literacy can be studied. This may require setting objective tests on students’ understanding of the nature of science, the scientific method, etc. In the long run, it is expected that this research can provide some insights to the educators in the field of science education on the benefits of the classics reading approach.

Acknowledgement

This research is funded by the Micro-Module Courseware Development Grant at the Chinese University of Hong Kong with project title “Micro-modules for UGFN1000 classroom flipping” and “Effects and Risks on Micro-module Implementation in UGFN1000”.

In regards to the development of the micro-modules, we need to thank Dr. WU Jun Vivian from GEF Programme and Ms. Judy Lo from Information Technology Services Centre for providing important help in preparation of the supplementary material as well as technical support.

For conducting the entry-exit surveys and data analysis, we are in-debt to our colleagues in the Office of University General Education in the Chinese University of Hong Kong including Prof. LEUNG Mei-Yee, Dr. WONG Wing-Hung, Ms. WONG Carina, and Miss LAI Ann.

References

[1]	OECD, PISA 2015Draft Science Framework, 2013. Available: https://www.oecd.org/pisa/pisaproducts/ Draft%20PISA%202015%20Science%20Framework%20.pdf. [Accessed 15 June 2016].
	In article
[2]	DeBoer, G. E., “Scientific Literacy: Another Look at Its Historical and Contemporary Meanings and Its Relationship to Science Education Reform”, Journal of Research in Science Teaching. Vol. 37, No. 6, 582-601, 2000.
	In article		View Article
[3]	Roberts, D. A., “Scientific literacy/science literacy. In S. K. Abell & N. G. Lederman (Eds.)”, Handbook of research in science education, pp. 729-779, 2007.
	In article		PubMed
[4]	Norris, S. P., & Phillips, L. M., “How literacy in its fundamental sense is central to scientific literacy”, Science Education, 37, 224-240, 2003.
	In article		View Article
[5]	Hurd, P., “Science education for the 21st century”, School Science and Mathematics, 100(6), 282, 2000.
	In article		View Article
[6]	St. John’s College, “Colleges That Change Lives”. Available: https://www.sjc.edu/about/why-sjc/colleges-that-change-lives/. [Accessed 15 June 2016].
	In article
[7]	Columbia University, “Core Curriculum”. Available: https://undergrad.admissions.columbia.edu/learn/academiclife/college/core. [Accessed 15 June 2016].
	In article
[8]	Gjertsen, D., The Classics of Science, Lilia Barber Press, New York, 1984.
	In article
[9]	Carson, R.N., “Science and the Ideals of Liberal Education”, Science and Education, 6, 225-238, 1997.
	In article		View Article
[10]	Goodney, D.E. and Long, C.S., “The Collective Classic: A Case for the Reading of Science”, Science and Education, 12, 167-184, 2003.
	In article		View Article
[11]	Muench, S.B., “Choosing Primary Literature in Biology to Achieve Specific Educational Goals”, Journal of College Science Teaching, 29, 255-260, 2000.
	In article
[12]	Movahedzadeh F., “Improving Students’ Attitude Toward Science Through Blended Learning”, Science Education and Civic Engagement, 3(2), 2011.
	In article
[13]	Chan, C.W., Szeto, W.M. and Wong, W.H., In Dialogue with Nature, Office of University General Education, The Chinese University of Hong Kong, 2nd ed., 2012.
	In article
[14]	Office of University General Education, “Mission & Vision”, Available: https://www5.cuhk.edu.hk/oge/index.php/en/gef-overview/mission-a-vision [Accessed Feb. 11, 2016].
	In article
[15]	Office of University General Education, “Why Nature?”, Available: https://www5.cuhk.edu.hk/oge/index.php/en/2011-06-22-08-12-11/why-nature [Accessed Feb. 11, 2016].
	In article
[16]	Kiang, K. M., Ng, A. K. L., Cheung, D. H. C., “Teaching Science to Non-Science Students with Science Classics”, American Journal of Educational Research, 2015, Vol. 3, No. 10, 1291-1297.
	In article
[17]	Ng, A. K. L., Kiang, K. M., Cheung, D. H. C., “Assessing Students’ Attainment in Learning Outcomes: A Comparison of Course-End Evaluation and Entry-Exit Surveys”, World Journal of Education, Vol. 6, No. 3, 56-65, 2016.
	In article		View Article
[18]	Gaff, J.G., The Chinese University of Hong Kong General Education Program: An Evaluation, unpublished internal report, Office of University General Education, the Chinese University of Hong Kong, 2014.
	In article
[19]	Nichols, J., Mauldin R, and Gaff, J., Improving Learning in General Education: An AGLS Guide to Assessment and Program Review, Association for General and Liberal Studies, 2015.
	In article
[20]	Lage, M. J., Platt, G. J., and Treglia, M., “Inverting the classroom: A gateway to creating an inclusive learning environment”, The Journal of Economic Education, 31:30-43, 2000.
	In article		View Article
[21]	Mason, G. S., Shuman, T. R., and Cook, K. E., “Comparing the Effectiveness of an Inverted Classroom to a Traditional Classroom in an Upper-Division Engineering Course”, IEEE Transactions on Education, 56:430-435, 2013.
	In article		View Article
[22]	Bishop, J. L., and Verleger, M. A., “The flipped classroom: A survey of the research,” Paper presented at the 120th American Society for Engineering Education Annual Conference and Exposition, Atlanta, GA, 2013.
	In article
[23]	The Chinese University of Hong Kong, “Supplementary courseware for non-science students studying UGFN1000”. Available: https://www.cuhk.edu.hk/eLearning/c_tnl/mmcd/showcase/2015_23.html. [Accessed 23 Jul 2015].
	In article
[24]	Astin, A., Assessment for Excellence: The Philosophy and Practice of Assessment and Evaluation in Higher Education, Rowman & Littlefield Publishers, Lanham, 2nd ed., 2012.
	In article
[25]	Jamieson, S., “Likert scales: how to (ab) use them.” Medical education, 38(12), 1217-1218, 2004.
	In article		View Article  PubMed
[26]	Allen, I. E., & Seaman, C. A., Likert scales and data analyses. Quality Progress, 40(7), 64, 2007.
	In article
[27]	MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D., “On the practice of dichotomization of quantitative variables”, Psychological methods, 7(1), 19, 2002.
	In article		View Article  PubMed
[28]	Field, A., Discovering statistics using IBM SPSS statistics, Sage, 2013.
	In article
[29]	Cohen, L., Manion, L., & Morrison K., Research methods in education, Routledge, 2013.
	In article
[30]	Delice, A., “The Sampling Issues in Quantitative Research”, Educational Sciences: Theory and Practice, 10(4), 2001-2018, 2010.
	In article
[31]	Rhiel, S. and Chaffin, W.W., “An investigation of the large‐sample/small‐sample approach to the one‐sample test for a mean (Sigma Unknown)”, Journal of Statistics Education, 4(3), 1996. Available: www.amstat.org/publications/jse/v4n3/rhiel.html. [Accessed July 22, 2016].
	In article
[32]	Registration and Examinations Section, Registry, the Chinese University of Hong Kong, “Guide to Grading System”. Available: https://rgsntl.rgs.cuhk.edu.hk/rws_prd_life/re_menu/gn_115063720.aas [Accessed 15 June 2016].
	In article