Development and Evaluation of a Practice Model Based on the QUILT Framework in Case of the State Change: Analysis of Utterance Protocol Using Text Mining

Tables index

Veiw figure View Table

These conclusions were based on the described contents of the questionnaire survey and worksheet, analyzed quantitatively. This study adopts the techniques of qualitative analysis. It includes an analysis of the classroom dialogue transcribed from IC recorder data. All recorded data transcribed as text were analyzed. From now on, “all recorded data transcribed as text” will be referred to simply as the “utterance protocol.”

2. Process of Analysis

2.1. Subjects of Analysis

The purpose of this study was to investigate the science-specific Questioning framework based on the QUILT framework that organized a variety of teaching strategies. In this study, two strategies were applied to the Questioning framework: One was the “Puzzling picture,” which aims to induce cognitive conflict, and the other was “Think-Pair-Share,” which aims to build discussion.

Science classes based on the QUILT framework were conducted from January to March 2014 in the 7th grade of Ehime Prefecture public A junior high school. The experimental group used the Questioning framework teaching strategy; the control group did not. A lesson on “change of state in a substance” was tested in 71 junior high school students in the 7th grade to examine the effects of the Questioning framework. The breakdown was as follows: from a total of 71 students (31 males and 40 females), the experimental group contained 34 students, and the control group contained 37 students. Evaluation of the Questioning framework was conducted by qualitative analysis of the utterance protocol using text mining.

2.2. Objective of Learning

This study was implemented during the “change of state in a substance” unit of the junior high school’s 7th grade. Learning activities primarily focused on the following two aspects:

(1) The particles cannot increase or decrease.

(2) The volume of the water when it turns from liquid to steam expands about 1700 times.

The learning objectives of this study were for students to consider the concept of particles when understanding change of state in a substance.

2.3. Teaching Strategy
2.3.1. Puzzling Picture

Even if science teachers teach lessons about the same natural phenomena, differences in their teaching methods have a strong influence on the students' understanding. For example, teaching science using graphs produces completely different results from teaching it using calculations. It's safe to say that the questions teachers ask their students play a very important role in determining the depth or extent of the students' ability to think.

Table 2. Comparison points of the utterance protocol

Download as

Tables index

Veiw figure View Table

Styre & Sound (1975) describes the importance of questioning through picture riddles effectively that is the “Puzzling picture” ^[10]. Then, the experimental group attempted to inspire various opinions from the students before actually performing the experiment by utilizing the mysterious picture shown in Figure 1 for the “Puzzling picture” activity, inducing cognitive conflict.

Download as

Veiw figureFigures index

NEW
View larger figure in new window

View next figure

Figure 1. The mystery picture “Balloon in the flask”

The picture was distributed to all students, and the teacher then asked the Divergent Question “What is this picture?” After allowing students to express their opinions, the teacher asked the Convergent Question “How do you create the phenomenon that is reflected in the picture?” Of course, students were not intended to provide the correct answer at this step. Priority was given to describing various related concepts, leading to understanding of the experiments conducted following this activity.

2.4. Qualitative Analysis

By using a single IC recorder for every two students, the utterances of pair of students to each other during class was recorded. From the back of the classroom, the scene of the class was taken in a single continuous video.

In this study, qualitative analysis was mainly performed using the utterances recorded by the IC recorder. The video recording was to be used as supplementary reference material. The 35 IC recorders (experimental group: 17 units; control group: 18 units) surveyed and recorded the utterances of 71 students (experimental group: N = 34; control group: N = 37), and all of the recorded audio was transcribed as text data. In this study, the object of analysis was this utterance protocol.

Actual analysis was conducted separately both before and after the experiment. As shown in Table 2, before the experiment, the first analysis concerned the “Puzzling picture’s” ability to induce to cognitive conflict in the experimental group. The second analysis, performed after the experiment, compared the experimental group using the “Think-Pair-Share” strategy and the control group, which did not use the teaching strategy.

In addition, the “wait time” teaching strategy provided a brief pause after both the teachers’ Questioning and the students’ reply. This teaching strategy was utilized on the premise that it is effective, based on the studies of Rowe (1973) ^[11] and Walsh, et al. (2011) ^[12]. In this study, the “wait time” strategy was attempted to implement practice with the assumption that is effective in encouraging students’ thinking activities. Thus, the analysis of “wait time” was not related to analysis of the other teaching strategy, and it was decided to treat them separately.

2.5. The Analysis Protocol Using Text Mining

The utterance protocol was analyzed using the text mining technique. The effects of the Questioning framework were investigated by comparing students’ understanding in the control group and the experimental group. Text mining techniques were also used to elucidate the themes found in every utterance protocol. This study was performed using the text analysis system KH Coder Ver. 2.Beta.32 (hereafter, “KH Coder”), developed by Higuchi(2014) ^[13]. KH Coder analyzes a variety of qualitative data, such as text, audio, and video. This method incorporates aggressively specific methods of content analysis, and can perform quantitative analysis of the data as well. By using KH Coder, the entire text can be divided into the smallest possible units; as a result, it is possible to extract more term patterns. In this study, term patterns appearing in the utterance protocols were extracted using KH Coder, and utterance protocols were developed from these results.

2.6. The Network of Collocation in KH Coder

KH Coder can divide data into units called morphemes, the smallest possible units of text, and can thereby extract more apparent term patterns. The extracted terms are drawn in a circle; circles appearing pattern similar can draw a network of collocation connected by a line. The greater the number of occurrences of an extracted term, the larger the circle becomes, and the appearance pattern resembles closely, the line becomes thick that connects the circle each other. In this study, the KH Coder network of collocation was drawn in order to investigate the effects of the Questioning framework.

3. Results and Discussion

3.1. Test of Homogeneity of Both Groups

To investigate the effects of the Questioning framework, the test subjects were divided into two groups, the experimental group and the control group. The average point scores of each group’s 7th grade science second semester examination results, and the results of obtaining the standard deviation, are shown in Table 3.

Table 3. 7th grade science second semester examination results (100 points total)

Download as

Tables index

Veiw figure View Table

No significant difference was observed between the groups (F (1,69)=1.06, p>.05). Therefore, both groups are homogeneous. It was determined that there was no hindrance to investigating the effects of the Questioning framework.

Table 4. Opinions observed during cognitive conflict (in part)

Download as

Tables index

Veiw figure View Table

Table 5. Appearance pattern a lot of terms of extraction results (in part)

Download as

Tables index

Veiw figure View Table

3.2. Utterance Analysis of Induction of Cognitive Conflict

In the experimental group, cognitive conflict was induced by showing a photo with mysterious elements before experiments were carried out (the “Puzzling picture” activity). As a result, while trial and error about something that can occur as much as possible, was able to express their opinions along with the next to the seat of students and entire class of students. Specific opinions expressed during the activity are shown in Table 4.

In Table 4, one category of opinions involved “workmanship,” or holes drilled in the flask. The other involves the use of “other substances,” such as those that are heavier than air.

3.3. First Analysis (Before Experiment)
3.3.1. Extraction Results of the Appearance Pattern

In the utterances protocol of teaching strategy "Puzzling picture" that was used in the experimental group, it was extracted around the term appearance pattern of terms that have been used in many cases. Using KH Coder, the utterance protocol was examined to extract common term patterns. The resulting extracted terms, frequencies of use, and actual examples are summarized in Table 5.

As a result, it was possible to extract characteristic ideas common among the students’ dialogue. Among the extracted terms in Table 5, those discussing the “workmanship” and “other substances” categories in Table 4 had been extracted terms considered to have been performed.

3.3.2. The Network of Collocation for the “Puzzling picture”

A network of collocation was drawn to tie similar terms from the utterance protocol together, showing the emergence of patterns with lines. The minimum number of occurrences was extracted five times more terms. The resulting network of collocation for the “Puzzling picture” activity’s utterance protocol is shown in Figure 2.

Download as

Veiw figureFigures index

View next figure

Figure 2. The network of collocation for the “Puzzling picture”

The extracted terms were divided into four groups. Paying attention to the extracted terms and collocation relationships allows for greater understanding about what was discussed from all perspectives.

The first group of related terms, shown in the upper left of Figure 2, included those such as “air,” “CO,” “oxygen,” “hydrogen,” and “ammonia.” The terms “CO,” “oxygen,” “hydrogen,” and “ammonia” were used mainly in connection with “air.” Furthermore, all were found to have collocation relations with the terms “light” and “heavy.” Therefore, it was found that students discussed the nature of substances, such as carbon dioxide or oxygen, which are heavier or lighter than air.

The second group, shown in the middle of Figure 2, included “balloon,” “ethanol,” “water,” and “Add.” A strong link was found between the three terms “water,” “put,” and “below.” These also had collocation relations to “ethanol,” “hot water,” and “balloon.” In other words, it was found that one perspective tried to explain the phenomenon in terms of changes of state, such as balloon swelling and the reaction of warm ethanol in hot water.

The third group, shown in the upper right of Figure 2, included terms such as “flask,” “set,” and “swell.” A strong link was revealed between the two terms “set” and “inflatable.” Based on the fact that the picture showed swollen balloons in a flask, one perspective considered the “setting” of an experimental tool significant.

The fourth group, shown in the lower portion of Figure 2, included terms such as “gas,” “out,” “medium,” “fall,” and “view.” This group revealed a perspective focused on invisible gas. In the “Puzzling picture” activity, it was found that discussions concerned comparison of the weight of air and other substances, or changes in state.

3.4. Second Analysis (After Experiment)
3.4.1. Extraction Result of the Appearance Pattern

After the “Puzzling picture” activity, the utterance protocol for the “Think-Pair-Share” teaching strategy, used in the experimental group, was extracted to centered things appearance pattern of the term that was used often. The “Think-Pair-Share” activity was conducted mainly through the following two Questioning categories presented by the teachers.

Question group 1: What happens when you warm the flask containing the balloon again? Can you explain using the concept of particles?

Question group 2: After warming it with a balloon inside from the beginning, what happens when the flask cools? Can you explain using the concept of particles?

Extraction results are summarized in Table 6. As shown, the average number of occurrences of terms recorded for the experimental group was 21.1, while it was 14.5 for the control group was 14.5. The tendency to use terms frequently was therefore observed in the utterance protocol of the experimental group.

Table 6. Appearance pattern a lot of terms of extraction results (in part)

Download as

Tables index

Veiw figure View Table