Development and Evaluation of a Practice Model Based on the QUILT Framework in Case of the State Cha...

Takekuni Yamaoka, Shinji Matsumoto

American Journal of Educational Research

Development and Evaluation of a Practice Model Based on the QUILT Framework in Case of the State Change: Analysis of Utterance Protocol Using Text Mining

Takekuni Yamaoka1, 2,, Shinji Matsumoto3

1Kitauwa Upper Secondary School, Japan

2The Joint Graduate School in Science of School Education, Hyogo University of Teacher Education, Japan

3Hyogo University of Teacher Education, Japan

Abstract

The purpose of this study was to investigate the science-specific Questioning framework, based on the QUILT framework, used to organize a variety of teaching strategies. In this study, two strategies were applied under the Questioning framework: One was the “Puzzling picture,” which aims to induce cognitive conflict, and the other was “Think-Pair-Share,” which aims to foster discussion. A lesson on “change of state in a substance” was developed and tested on 71 junior high school students in the 7th grade to examine the effects of the Questioning framework. Evaluation of the Questioning framework was conducted by qualitative analysis of the utterance protocol using text mining. Two important results were gathered: 1) The “Puzzling picture” was effective in encouraging perspective and wide-ranging discussion; and 2) the Questioning framework was effective in promoting students’ science-related utterances in the science class.

Cite this article:

  • Takekuni Yamaoka, Shinji Matsumoto. Development and Evaluation of a Practice Model Based on the QUILT Framework in Case of the State Change: Analysis of Utterance Protocol Using Text Mining. American Journal of Educational Research. Vol. 4, No. 4, 2016, pp 360-365. http://pubs.sciepub.com/education/4/4/10
  • Yamaoka, Takekuni, and Shinji Matsumoto. "Development and Evaluation of a Practice Model Based on the QUILT Framework in Case of the State Change: Analysis of Utterance Protocol Using Text Mining." American Journal of Educational Research 4.4 (2016): 360-365.
  • Yamaoka, T. , & Matsumoto, S. (2016). Development and Evaluation of a Practice Model Based on the QUILT Framework in Case of the State Change: Analysis of Utterance Protocol Using Text Mining. American Journal of Educational Research, 4(4), 360-365.
  • Yamaoka, Takekuni, and Shinji Matsumoto. "Development and Evaluation of a Practice Model Based on the QUILT Framework in Case of the State Change: Analysis of Utterance Protocol Using Text Mining." American Journal of Educational Research 4, no. 4 (2016): 360-365.

Import into BibTeX Import into EndNote Import into RefMan Import into RefWorks

At a glance: Figures

1. Introduction

The role of the teacher in mentoring learners is to teach educational contents in class and to confirm the status of preparation and review. In addition, teachers should make every effort to research educational materials and to analyze and improve their classes. Gagnè, Wager, Golas, and Keller (2007) said that the importance of assuming the conceptual change of learners to effectively design comprehensive instruction intended to support the learning process [1]. Saito (1969) said that systematic and constructive learning is necessary to stack learning among students in order to continue to develop [2]. Practically speaking, questioning methods and teachers’ statements in class present the opportunity to create a dialogue that encourages thinking activities and interaction among learners. For example, it’s no exaggeration to say that whether the student-centered is constructed or not depends on the handling of learners’ utterances. Miyazaki (2009) said that discourse in the classroom is an optimal learning environment, and that the teacher’s technical organization in class influences learning [3]. In other words, it is crucial that the teacher’s technique organize the content of students’ utterances in order to realize mutual exchange of learning between teacher and students. Thus, study of the utterance content of both teachers and students, which triggers this organizing, is highly significant.

Walsh and States (2005) built an organized program by focusing on the questioning based on the research [4]. The effective Questioning program they developed, the “Questioning and Understanding to Improve Learning and Thinking” (QUILT) framework, organized a variety of teaching strategies. However, the QUILT framework is intended for education in general; it is not a science-specific Questioning framework.

As a result, Yamaoka and Matsumoto (2014) developed a science-specific Questioning framework based on the QUILT framework that organized a variety of teaching strategies [5]. Actual examples are shown in Table 1. Specifically, these strategies include “wait time 1,” which aims to encourage thinking activities by a slight latency in questioning; “wait time 2,” which aims to encourage thinking activities by a slight latency in answering; “Think-Pair-Share,” which aims to foster discussion; and the “Puzzling picture,” which raises cognitive conflict.

Yamaoka and Matsumoto (2015) implemented science lessons relying on this Questioning framework [6]. In this study, “the experimental group” referred to the group using the Questioning framework teaching strategy, and “the control group” referred to the group not using it. The described contents of the questionnaire survey and worksheet were analyzed quantitatively. Three important results were gathered: 1) Many positive opinions suggested that both “wait time 1” and “wait time 2” are necessary; 2) “Think-Pair-Share” was effective in encouraging perspective, understanding, thinking, and cooperation with others; and 3) the Questioning framework was effective in developing students’ retention of science concepts, effects of which can be seen one month later.

Table 1. The science-specific Questioning framework

These conclusions were based on the described contents of the questionnaire survey and worksheet, analyzed quantitatively. This study adopts the techniques of qualitative analysis. It includes an analysis of the classroom dialogue transcribed from IC recorder data. All recorded data transcribed as text were analyzed. From now on, “all recorded data transcribed as text” will be referred to simply as the “utterance protocol.”

2. Process of Analysis

2.1. Subjects of Analysis

The purpose of this study was to investigate the science-specific Questioning framework based on the QUILT framework that organized a variety of teaching strategies. In this study, two strategies were applied to the Questioning framework: One was the “Puzzling picture,” which aims to induce cognitive conflict, and the other was “Think-Pair-Share,” which aims to build discussion.

Science classes based on the QUILT framework were conducted from January to March 2014 in the 7th grade of Ehime Prefecture public A junior high school. The experimental group used the Questioning framework teaching strategy; the control group did not. A lesson on “change of state in a substance” was tested in 71 junior high school students in the 7th grade to examine the effects of the Questioning framework. The breakdown was as follows: from a total of 71 students (31 males and 40 females), the experimental group contained 34 students, and the control group contained 37 students. Evaluation of the Questioning framework was conducted by qualitative analysis of the utterance protocol using text mining.

2.2. Objective of Learning

This study was implemented during the “change of state in a substance” unit of the junior high school’s 7th grade. Learning activities primarily focused on the following two aspects:

(1) The particles cannot increase or decrease.

(2) The volume of the water when it turns from liquid to steam expands about 1700 times.

The learning objectives of this study were for students to consider the concept of particles when understanding change of state in a substance.

2.3. Teaching Strategy
2.3.1. Puzzling Picture

Even if science teachers teach lessons about the same natural phenomena, differences in their teaching methods have a strong influence on the students' understanding. For example, teaching science using graphs produces completely different results from teaching it using calculations. It's safe to say that the questions teachers ask their students play a very important role in determining the depth or extent of the students' ability to think.

Table 2. Comparison points of the utterance protocol

Styre & Sound (1975) describes the importance of questioning through picture riddles effectively that is the “Puzzling picture” [10]. Then, the experimental group attempted to inspire various opinions from the students before actually performing the experiment by utilizing the mysterious picture shown in Figure 1 for the “Puzzling picture” activity, inducing cognitive conflict.

Figure 1. The mystery picture “Balloon in the flask”

The picture was distributed to all students, and the teacher then asked the Divergent Question “What is this picture?” After allowing students to express their opinions, the teacher asked the Convergent Question “How do you create the phenomenon that is reflected in the picture?” Of course, students were not intended to provide the correct answer at this step. Priority was given to describing various related concepts, leading to understanding of the experiments conducted following this activity.

2.4. Qualitative Analysis

By using a single IC recorder for every two students, the utterances of pair of students to each other during class was recorded. From the back of the classroom, the scene of the class was taken in a single continuous video.

In this study, qualitative analysis was mainly performed using the utterances recorded by the IC recorder. The video recording was to be used as supplementary reference material. The 35 IC recorders (experimental group: 17 units; control group: 18 units) surveyed and recorded the utterances of 71 students (experimental group: N = 34; control group: N = 37), and all of the recorded audio was transcribed as text data. In this study, the object of analysis was this utterance protocol.

Actual analysis was conducted separately both before and after the experiment. As shown in Table 2, before the experiment, the first analysis concerned the “Puzzling picture’s” ability to induce to cognitive conflict in the experimental group. The second analysis, performed after the experiment, compared the experimental group using the “Think-Pair-Share” strategy and the control group, which did not use the teaching strategy.

In addition, the “wait time” teaching strategy provided a brief pause after both the teachers’ Questioning and the students’ reply. This teaching strategy was utilized on the premise that it is effective, based on the studies of Rowe (1973) [11] and Walsh, et al. (2011) [12]. In this study, the “wait time” strategy was attempted to implement practice with the assumption that is effective in encouraging students’ thinking activities. Thus, the analysis of “wait time” was not related to analysis of the other teaching strategy, and it was decided to treat them separately.

2.5. The Analysis Protocol Using Text Mining

The utterance protocol was analyzed using the text mining technique. The effects of the Questioning framework were investigated by comparing students’ understanding in the control group and the experimental group. Text mining techniques were also used to elucidate the themes found in every utterance protocol. This study was performed using the text analysis system KH Coder Ver. 2.Beta.32 (hereafter, “KH Coder”), developed by Higuchi(2014) [13]. KH Coder analyzes a variety of qualitative data, such as text, audio, and video. This method incorporates aggressively specific methods of content analysis, and can perform quantitative analysis of the data as well. By using KH Coder, the entire text can be divided into the smallest possible units; as a result, it is possible to extract more term patterns. In this study, term patterns appearing in the utterance protocols were extracted using KH Coder, and utterance protocols were developed from these results.

2.6. The Network of Collocation in KH Coder

KH Coder can divide data into units called morphemes, the smallest possible units of text, and can thereby extract more apparent term patterns. The extracted terms are drawn in a circle; circles appearing pattern similar can draw a network of collocation connected by a line. The greater the number of occurrences of an extracted term, the larger the circle becomes, and the appearance pattern resembles closely, the line becomes thick that connects the circle each other. In this study, the KH Coder network of collocation was drawn in order to investigate the effects of the Questioning framework.

3. Results and Discussion

3.1. Test of Homogeneity of Both Groups

To investigate the effects of the Questioning framework, the test subjects were divided into two groups, the experimental group and the control group. The average point scores of each group’s 7th grade science second semester examination results, and the results of obtaining the standard deviation, are shown in Table 3.

Table 3. 7th grade science second semester examination results (100 points total)

No significant difference was observed between the groups (F (1,69)=1.06, p>.05). Therefore, both groups are homogeneous. It was determined that there was no hindrance to investigating the effects of the Questioning framework.

Table 4. Opinions observed during cognitive conflict (in part)

Table 5. Appearance pattern a lot of terms of extraction results (in part)

3.2. Utterance Analysis of Induction of Cognitive Conflict

In the experimental group, cognitive conflict was induced by showing a photo with mysterious elements before experiments were carried out (the “Puzzling picture” activity). As a result, while trial and error about something that can occur as much as possible, was able to express their opinions along with the next to the seat of students and entire class of students. Specific opinions expressed during the activity are shown in Table 4.

In Table 4, one category of opinions involved “workmanship,” or holes drilled in the flask. The other involves the use of “other substances,” such as those that are heavier than air.

3.3. First Analysis (Before Experiment)
3.3.1. Extraction Results of the Appearance Pattern

In the utterances protocol of teaching strategy "Puzzling picture" that was used in the experimental group, it was extracted around the term appearance pattern of terms that have been used in many cases. Using KH Coder, the utterance protocol was examined to extract common term patterns. The resulting extracted terms, frequencies of use, and actual examples are summarized in Table 5.

As a result, it was possible to extract characteristic ideas common among the students’ dialogue. Among the extracted terms in Table 5, those discussing the “workmanship” and “other substances” categories in Table 4 had been extracted terms considered to have been performed.


3.3.2. The Network of Collocation for the “Puzzling picture”

A network of collocation was drawn to tie similar terms from the utterance protocol together, showing the emergence of patterns with lines. The minimum number of occurrences was extracted five times more terms. The resulting network of collocation for the “Puzzling picture” activity’s utterance protocol is shown in Figure 2.

Figure 2. The network of collocation for the “Puzzling picture”

The extracted terms were divided into four groups. Paying attention to the extracted terms and collocation relationships allows for greater understanding about what was discussed from all perspectives.

The first group of related terms, shown in the upper left of Figure 2, included those such as “air,” “CO,” “oxygen,” “hydrogen,” and “ammonia.” The terms “CO,” “oxygen,” “hydrogen,” and “ammonia” were used mainly in connection with “air.” Furthermore, all were found to have collocation relations with the terms “light” and “heavy.” Therefore, it was found that students discussed the nature of substances, such as carbon dioxide or oxygen, which are heavier or lighter than air.

The second group, shown in the middle of Figure 2, included “balloon,” “ethanol,” “water,” and “Add.” A strong link was found between the three terms “water,” “put,” and “below.” These also had collocation relations to “ethanol,” “hot water,” and “balloon.” In other words, it was found that one perspective tried to explain the phenomenon in terms of changes of state, such as balloon swelling and the reaction of warm ethanol in hot water.

The third group, shown in the upper right of Figure 2, included terms such as “flask,” “set,” and “swell.” A strong link was revealed between the two terms “set” and “inflatable.” Based on the fact that the picture showed swollen balloons in a flask, one perspective considered the “setting” of an experimental tool significant.

The fourth group, shown in the lower portion of Figure 2, included terms such as “gas,” “out,” “medium,” “fall,” and “view.” This group revealed a perspective focused on invisible gas. In the “Puzzling picture” activity, it was found that discussions concerned comparison of the weight of air and other substances, or changes in state.

3.4. Second Analysis (After Experiment)
3.4.1. Extraction Result of the Appearance Pattern

After the “Puzzling picture” activity, the utterance protocol for the “Think-Pair-Share” teaching strategy, used in the experimental group, was extracted to centered things appearance pattern of the term that was used often. The “Think-Pair-Share” activity was conducted mainly through the following two Questioning categories presented by the teachers.

Question group 1: What happens when you warm the flask containing the balloon again? Can you explain using the concept of particles?

Question group 2: After warming it with a balloon inside from the beginning, what happens when the flask cools? Can you explain using the concept of particles?

Extraction results are summarized in Table 6. As shown, the average number of occurrences of terms recorded for the experimental group was 21.1, while it was 14.5 for the control group was 14.5. The tendency to use terms frequently was therefore observed in the utterance protocol of the experimental group.

Table 6. Appearance pattern a lot of terms of extraction results (in part)

Terms such as “balloon” and “flask” were observed in response to both Questioning category 1 and 2. This was the result of brainstorming opinions based on both groups of questions. However, the term “particles,” observed in response to both Questioning categories, was found to have a frequency of 23 in the experimental group, and of 6 in the control group. This study aimed to consider students’ use of the concept of particles to explain the scientific phenomenon. It was found that the experimental group made many references to particles.

The number of occurrences of the terms “balloon,” “cool,” “boiling,” and “swell,” in the control group exceeded that of the experimental group. In response to the teachers’ Questioning, the control group was seen to make many visual observations, such as that the balloon was inflated.

Thus, it became clear that the experimental group frequently referred to particles, and used many of the terms spoken.

Figure 3. Networks of collocation for the “Think-Pair-Share” activity

3.4.2. The Network of Collocation for the “Think-Pair-Share” Activity

Networks of collocation were drawn based on the utterance protocol of the experimental group, using the teaching strategy “Think-Pair-Share,” and the utterance protocol of the control group, not using the strategy. The minimum number of occurrences was extracted five times more terms. The resulting networks of collocation concerning the “Think-Pair-Share” activity’s utterance protocols are shown in Figure 3.

Both the experimental group’s and control group’s networks of collocation displayed a group consisting mainly of the terms “enter,” “balloon,” “flask,” “cool,” and “think.” However, in the experimental group, this pattern group was associated with terms concerning changes of state or particle concepts. On the other hand, in the control group, although there were strong ties between the term “balloon” and the terms “out,” “go,” and so on, no association was made with the term “particle.” Furthermore, in the control group, the term “swell” was not associated with other terms and groups. From the extracted terms “medium” and “fall,” it was found that the students discussed visual perspectives of the phenomenon occurring in the flask, such as “balloons go up” or “go down.”

In addition, terms were extracted from the experimental group such as “boiling,” “air,” “push,” “concept,” and “write.” It was inferred from this that the experimental group discussed invisible phenomena such as “the air is pushed out by boiling.”

Therefore, both experimental group and control group tried to explain the phenomenon. However, it was revealed that the experimental group tended to use the concept of particles in their explanations.

4. Conclusion

In this study, science classes were conducted relying on the science-specific Questioning framework developed by Yamaoka and Matsumoto (2015). Using the utterance protocol of students in practice, two analyses were conducted, both before and after the experiment. Before the experiment, the “Puzzling picture” teaching strategy was used in the experimental group. During this activity, students were encouraged to brainstorm based on a mysterious photo. After the experiment, qualitative analysis was carried out to clarify the effects of the “Think-Pair-Share” teaching strategy.

Results revealed a tendency among the students to use scientific concepts to describe the phenomena in question, confirming the effect of the Questioning framework.

In the future, also in the field of non-state change, we wish to conduct a detailed study on the observed effects of this Questioning framework.

Acknowledgement

This work was supported by JSPS KAKENHI Grant Number 15H00188 and 11th Hakuho Research Grant for Child Education.

References

[1]  Gagnè,R., Wager,W., Golas,K., & Keller,J: The principle of Instructional Design, 150-152, Kitaooji Shobo, 2007.
In article      
 
[2]  Saito Kihaku: Saito KihakuComplete Works Volume 4 Introduction of Class; Future birth, 49-55, Kokudosya, 1969.
In article      
 
[3]  Miyazaki Kiyotaka: Learning of children and Learning of teacher- Saito Kihaku and Vygotsky school education, 161-164, Jijitutosouzou, 2009.
In article      
 
[4]  Walsh, J., & Sattes, B.: Quality Questioning, Research-Based Practice to Engage Every Learner, V-Viii, Corwin Press, 2005.
In article      
 
[5]  Yamaoka,T. , Matsumoto,S.: Development of a practice model in science lessons based on the QUILT framework- A case of the State Change: Research report Shikoku Branch of Japan Society for Science Education, Vol.28 No.6, 1-6, 2014.
In article      
 
[6]  Yamaoka,T. , Matsumoto,S.: Development of a practice model in science lessons based on the QUILT framework- A case of the State Change: An Anual Report of Japan Society for Science Education, Vol.39 No.3, 252-263, 2015.
In article      
 
[7]  Yamaoka, T.: Trends and Patterns Within the Test Content of Ehime Prefecture's Upper Secondary School Entrance Examinations: Journal of Research in Science Education, Vol.50 No.3, 145-154, 2010.
In article      
 
[8]  Yamaoka,T., Sumida,M., Nakayama,H., Matsumoto,S.: Comparative Study of Trends and Patterns in the questions between Lower Secondary School Science Textbooks and Ehime Prefecture's Upper Secondary School Entrance Examinations, Journal of Research in Science Education, 55(4), 415-423, 2015.
In article      View Article
 
[9]  Yamaoka, T, op. cit. 145-154, 2010.
In article      
 
[10]  Styre, S., & Sound, R.: All Science Stimulating Creative Thought Through Picture Riddles, The Science Teacher, 46-47, 1975.
In article      
 
[11]  Rowe, M.: TEACHING SCIENCE AS CONTINUOUS INQUIRY:A Basic, 273-285, McGrow-Hill Book Company, 1973.
In article      
 
[12]  Walsh, J., & Sattes, B.: Thinking Through Quality Questioning; Deeping Student Engagement, 52-58, Corwin Press, 2011.
In article      
 
[13]  Higuchi Koichi: Quantitative text analysis for social research - Toward the inheritance and development of content analysis-, 1-16, Nakanishiya publication, 2014.
In article      
 
  • CiteULikeCiteULike
  • MendeleyMendeley
  • StumbleUponStumbleUpon
  • Add to DeliciousDelicious
  • FacebookFacebook
  • TwitterTwitter
  • LinkedInLinkedIn