sciepub.com
Quick Submission
A Model for Assessing the Development of HOT Skills in Students
1Computer Science Department Al Qasemi Academic College of Education, Baqa El-Gharbieh, Israel
Abstract
In this paper, we propose a dynamic model for building a framework for the adaptive, complex assessment of developing the higher-order thinking (HOT) skills in students. Adaptability is provided by the dynamics of instructor assessment taking into account the development of HOT skills and providing a flexible choice of instructional problems for students. The complexity of the assessment is provided by initial, formative, adaptive and summative assessments of HOT skills. The proposed coefficients for HOT skill development serve as a constructive means of evaluating developing HOT skills in students. The creation of the model includes the elaboration and integration of interconnected model components. The dynamics of the model are provided by changes in its parameters, which express the dynamic process of assessment. The model involves the following assessment components: initial, formative, adaptive, and summative (SIFA). It fosters the development of HOT skills by adapting the assessment of HOT skills to the dynamics of the problem solving process.
Keywords: assessment model, higher-order thinking (HOT) skills
Copyright © 2017 Science and Education Publishing. All Rights Reserved.Cite this article:
- Jamal Raiyn, Oleg Tilchin. A Model for Assessing the Development of HOT Skills in Students. American Journal of Educational Research. Vol. 5, No. 2, 2017, pp 184-188. https://pubs.sciepub.com/education/5/2/12
- Raiyn, Jamal, and Oleg Tilchin. "A Model for Assessing the Development of HOT Skills in Students." American Journal of Educational Research 5.2 (2017): 184-188.
- Raiyn, J. , & Tilchin, O. (2017). A Model for Assessing the Development of HOT Skills in Students. American Journal of Educational Research, 5(2), 184-188.
- Raiyn, Jamal, and Oleg Tilchin. "A Model for Assessing the Development of HOT Skills in Students." American Journal of Educational Research 5, no. 2 (2017): 184-188.
| Import into BibTeX | Import into EndNote | Import into RefMan | Import into RefWorks |
At a glance: Figures
1. Introduction
The traditional learning process focuses on the acquisition of knowledge and on the lower-order cognitive skills of students [7]. However, a contemporary knowledge-based society requires specialists with higher-order thinking (HOT) skills. A more suitable strategy aimed at effective HOT skill development in students is problem-based learning (PBL) [1, 3, 4].
We considered two groups of HOT skills: analytical thinking skills and creative thinking skills. The group of analytical thinking skills comprises ordering, comparing, contrasting, evaluating, and selecting. The group of creative thinking skills includes problem finding (identifying the problem), efficiency (producing many ideas to solve problem), flexibility (producing a broad range of ideas), originality (producing uncommon ideas), and elaboration (developing ideas) [5, 8, 13].
PBL involves a three-stage process. The primary development of the analytical thinking skills is realized during the first stage. The extended development of analytical skills and the primary development of creative skills are realized during the second stage. The extended development of creative thinking skills is realized during the third stage.
Problem-based learning can be significantly improved by assessment. The proposed SIFA model provides a framework for the adaptive, complex assessment of HOT skills development during the performance of the three-stage PBL process. The model comprises four components: initial, formative, adaptive, and summative. The initial component is aimed at inducing students to develop HOT skills. The formative component consists of the first level assessment of HOT skill development in students. The adaptive component motivates students to continue developing their HOT skills, owing to the flexible assigning of instructional problems to students and the setting up of adaptive assessments. The summative component consists of an evaluation of HOT skill development upon the completion of the PBL activities.
The rest of the paper is organized as follows: Section 2 gives an overview of the various assessment approaches to HOT skill development. Section3 describes the SIFA model and discusses the HOT skill development that results from use of the model. Section 4 concludes the paper and focuses on future works.
2. Related Research
The PBL strategy prompts students to become active participants in the learning process. It is based on work done in small groups, in which students analyze practical problems to gain knowledge and skill [16]. Moreover, it teaches students to engage in the self-regulation of learning (SRL), [10, 11].
Assessment plays an important role in the learning process. It influences students’ distribution of effort, their approach to learning, and their application to studying. In a recent review, Raiyn and Tilchin [18] describe various types of assessment, and other reviews show the connection between assessment and student motivation [9, 12, 15, 20].
According to Bloom [6], there are different levels of assessment, and bloom’s taxonomy of cognitive domains, shown in Figure 1, is a well-established framework for categorizing assessment items into six levels, based on the thinking patterns.
Download as
Traditional learning focuses on the first two levels of Bloom’s taxonomy, which correspond to remembering and understanding, and is directed towards developing lower-order cognitive skills [7]. The third level of the taxonomy is an intermediate level [7]. The three higher levels of the taxonomy (analyze, synthesize, and evaluate) require higher-order cognitive skills.
Traditional exams are written at the “remember” level of Bloom’s taxonomy, requiring no more than the memorization of material to perform well, whereas unit exams are written at high levels and are designed to require higher-order cognitive skills. In both parts students then received an identical cumulative final exam, composed of lower-level questions focused on memory for factual knowledge and higher-level questions focused on application, analysis, and evaluation.
A more complex account hinges on the idea that in order to reach the higher levels of Bloom’s taxonomy, students must have mastered the lower levels (i.e., remembering and understanding basic terminology specific to the subject) [6].
In this paper, we propose a use of the SIFA model aimed at promoting the development of HOT skills in students through an adaptive complex assessment process.
3. The SIFA Model for Assessing the Development of HOT Skills in Students
The purpose of creating a dynamic model is to build a framework for an adaptive, complex assessment of the development of higher-order thinking (HOT) skills in students. The dynamics of the model are provided by changing its parameters to reflect the dynamic process of assessment. The adaptability of the assessment is provided by the dynamics of instructor assessment, taking into account the development of HOT skills and providing a flexible choice of instructional problems for students to solve. The complexity of the assessment is provided by initial, formative, adaptive, and summative assessments of HOT skills. The assessment process is conducted by an instructor.
PBL involve a three-stage process. The primary development of analytical thinking skills is realized during the first stage. The extended development of analytical skills and the primary development of creative skills are realized during the second stage. The extended development of the creative thinking skills is realized during the third stage.
The SIFA model involves the following assessment components: Initial, formative, and adaptive, and summative (see Figure 2).
Download as
The “initial” component
The objective of the “initial” component is to conduct initial assessments of HOT skills to induce students to develop these skills. It serves as the starting point of the assessment process.
The procedures for this component are:
1. Setting the total initial assessment of HOT skills for the analytical and creative groups. The total initial assessment for both groups of analytical and creative HOT skills is set at 100%.
2. Setting the initial assessments for analytical HOT skills.
Analytical skills are basic HOT skills since they are needed for developing creative skills. Consequently, the primary development of these skills should be performed during the first stage of PBL.
An order for developing analytical skills should be established. It should be marked by consistency in the development of students’ analytical problem- solving abilities; thus, it should be as follows: comparing, contracting, ordering, evaluating, and selecting. The development of each skill in the ordering sequence is required before the development of the next one.
The initial assessments of analytical skills should match the ordering of skills, and the assessments should induce students to consistently develop their skills according to the determined order. Therefore, the initial assessments proceed in descending order. The highest assessment is set for the first skill in sequence of analytical skills. The initial assessments for the analytical skills are presented in Table 1.
3. Setting the initial assessments for creative HOT skills.
To start, an ordering of the sequence for creative skills should be established.
The requirements for ordering them are dictated by their consistent interdependence.
This means each skill must be developed before the next one in the sequence. The order of the sequence is problem finding, elaboration, originality, efficiency, and flexibility.
The assessments should induce students to consistently develop their creative skills in keeping with the determined sequence. Therefore, the initial assessments are set up in order, with the highest assessment set for first skill in the ordered sequence of the creative skills. The initial assessments for creative skills are presented in Table 1.
The “formative” component
The objective of the formative component is to realize the first level assessments of the HOT skill development of the study group students.
The procedures for the formative component are:
1. Determining the formative assessments of students’ HOT skills
The formative assessments are determined by the instructor, based on the solving of instructional problems by the study group students. As an example, the results for the formative assessments of the students’ HOT skills are presented in Table 2.
The corresponding initial assessments are marked alongside of the skill names. The last column in Table 2 contains the average formative assessments of the students’ HOT skills.
2. Evaluating the development of HOT skills in students
The development of HOT skills in students is assessed using a coefficient for HOT skill development.
This coefficient constitutes an adaptive assessments of students’ HOT skills. The values for the coefficient are calculated by the formula:
 | (1) |
where
is a coefficient of developing skill 
,
is the average formative assessment of skill 
of a study group student,
is a basic (initial) assessment of skill 
as illustrated in Table 1.
The values of the coefficients for HOT skill development for the students are presented in Table 3 and Table 4.
The “adaptive” component
The objectives of the adaptive component are to flexibly assign instructional problems to students and to set up adaptive assessments to induce students to continue developing their HOT skills.
The procedures for the adaptive component are:
1. Assigning instructional problems to students
Instructional problems are assigned to students on the basis of the values of the coefficients for HOT skill development. If the value of the development coefficient for a particular skill is low, then instructional problems addressing the needed skill are presented to the student, which induces the student to develop that skill.
2. Determining adaptive assessments for HOT skills
The adaptive assessments for the HOT skills of students are determined as a result of analysis of the skill development. Intention to induce the students to further skill development causes determining adaptive assessments on the basis of the principle: if the value of the development coefficient regarding some skill is lower, then its adaptive assessment should be more. It entails determination of the adaptive assessments pro rata the values of the development coefficient.
For example, the adaptive assessments determined pro rate the values of the development coefficient (Table 3 and Table 4) are presented in Table 5 and Table 6.
The “summative” component
The objectives of the “summative” component are to evaluating students’ HOT skill development and to conduct their summative assessments after they have completed the PBL activities [19].
The procedures for the “summative” component are:
1. Assessing students’ HOT skills after their completion of PBL.
The results for the assessments of HOT skills are obtained from the solving of problems by the study group students. They are determined by the instructor. The assessment results are presented in Table 7. The corresponding adaptive assessments are marked alongside of the skill names.
2. Evaluating students’ HOT skill development after the completion of PBL.
At first, the values for the coefficients of HOT skill development in the students are calculated with Formula 1, which uses the average resulting assessments of the students and the adaptive assessments as basic assessments. The adaptive assessments for creative and analytical skills, and the average resulting assessments of students are presented in Table 5, Table 6 and Table 7, respectively. The values for the coefficients of HOT skill development in students after completion of the PBL activities are presented in Table 8 and Table 9.
The values of the coefficients of HOT skill development determined after the intermediate stage (Table 3 and Table 4) and the final stage of PBL (Table 8 and Table 9) are then compared. The results of this comparison show the degree of development of HOT skills.
3. Conducting summative assessments of students’ HOT skills
First, the total assessments of the analytical and creative skills of students are determined by summing the resulting assessments of student’s HOT skills. For example, the resulting assessments of student’s HOT skills from Table 7 are used to determine the total assessments. Second, the total assessments of HOT skills are calculated as the sums of the total assessments of the analytical and the creative skills. Lastly, the summative assessments of students are determined as average values of the total assessments of the HOT skills. The total and summative assessments are presented in Table 10.
4. Conclusion
The SIFA model is proposed here for shaping the assessment procedure for HOT skill development in students during PBL. The model includes initial, formative, adaptive and summative components. The initial component induces the students to develop their HOT skills. The formative component realizes the first level assessment of HOT skills development in the students. The adaptive component motivates the students to continue developing their HOT skills, owing to the flexible assigning of instructional problems, and set up adaptive assessments for HOT skills. The summative component evaluates HOT skill development in students as a result of their completion of PBL activities.
The model promotes the development of HOT skills in students. It figures PBL as the three-stage process involving: (1) the complexity and adaptability of assessment, (2) the sequencing of model components, and (3) an established order in which HOT skills are required. The efficiency of the model was confirmed by comparing the values of the proposed coefficients of HOT skill development calculated at different stages of the PBL.
Future work will be devoted to developing a computer supported assessment for the development of HOT skills in students.
References
| [1] | Amador, J. A., Miles, L., & Peters, C. B. (2006). The practice of problem-based learning: A guide to Implementing PBL in the college classroom, Jossey-Bass. | ||
 In article | |||
| [2] | Angelo, T & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers. San Francisco, CA: Jossey-Bass. pp 427. | ||
| In article | |||
| [3] | Barell J. F. (2006). Problem-Based Learning: An inquiry approach, Corwin. | ||
| In article | |||
| [4] | Barret, T. & Moore, S. (2010). New approaches to problem-based learning: Revitalising Your practice in higher education, Routledge. | ||
| In article | |||
| [5] | Bednarz, T. F. (2011). Developing critical thinking skills: Pinpoint Leadership Skill Development Training Series, Majorium Business Press. | ||
| In article | |||
| [6] | Bloom, B. S. (1984). Taxonomy of educational objectives. Boston: Allyn and Bacon. | ||
| In article | |||
| [7] | Crowe, A., Dirks, C., & Wenderoth, M. P. (2008). Biology in bloom: Implementing Bloom’s taxonomy to enhance student learning in biology. CBE - Life Sciences Education, 7, 368-381. | ||
| In article | View Article PubMed PubMed | ||
| [8] | Cottrell, S. (2013). The study skills handbook, (4th ed.). Palgrave Macmil. | ||
| In article | |||
| [9] | Dickie, L. O. (2003). Approach to learning, the cognitive demands of assessment, and achievement in physics. [Reports - Research]. Canadian Journal of Higher Education, 33(1), 87-111. | ||
| In article | |||
| [10] | Boekaerts, M. (1997). Self-regulated learning: A new concept embraced by researchers, policy makers, educators, teachers, and students. Learning and Instruction, 7(2), 161-186. | ||
| In article | View Article | ||
| [11] | Entwistle, N., McCune, V., & Hounsell, J. (2002). Approaches to studying and perceptions of university teaching-learning environments: Concepts, measures and preliminary findings. Edinburgh: School of Education, University of Edinburgh. | ||
| In article | |||
| [12] | Harlen, W., & Deakin Crick, R. (2002). A systematic review of the impact of summative assessment and tests on students’ motivation for learning, Research Evidence in Education Library, 1. London: EPPI-Centre, Social Science Research Unit, Institute of Education. | ||
| In article | |||
| [13] | Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn?, Educational Psychology Review, 16, 3. | ||
| In article | |||
| [14] | Jensen, J,L., McDaniel, M.A., Woodard, S.M. & Kummer, T.A.. (2014). Teaching to the test…or testing to teach: Exams requiring higher order thinking skills encourage greater conceptual understanding, Educ Psychol Rev. 26:307-329. | ||
| In article | |||
| [15] | Joughin, G. (2010). The hidden curriculum revisited: A critical review of research into the influence of summative assessment on learning. Assessment and Evaluation in Higher Education, 35(3), 335-345. | ||
| In article | View Article | ||
| [16] | Päivi, H., (2009). Designing and implementing a PBL course on educational digital video production: Lessons learned from a design-based research, Education Tech Research Dev 57: 211-228. | ||
| In article | |||
| [17] | Tracey, P. (2009). Are deep strategic learners better suited to PBL? A preliminary study, Adv in Health Sci Educ, 14, 337-353. | ||
| In article | |||
| [18] | Raiyn, J. & Tilchin, O. (2016). The self-formation of collaborative groups in a problem based learning environment, Journal of Education and Practice, 7, 25, 120-126. | ||
| In article | |||
| [19] | Raiyn, J. (2016). The Role of Visual Learning in Improving Students’ High Order Thinking Skills, Journal of Education and Practice, 7, 24, 115-121. | ||
| In article | |||
| [20] | Struyven, K., Dochy, F., & Janssens, S. (2005). Students’ perceptions about evaluation and assessment in higher education: A review. Assessment and Evaluation in Higher Education, 30(4), 325-341. | ||
| In article | View Article | ||
CiteULike
Mendeley
StumbleUpon
Delicious
Facebook
Twitter
LinkedIn
