Test Tool Creation and Examination of Selected Cognitive Skills of Students at the Slovak University of Technology
Mária Bednáriková1,, Karol Kováč1
1Department of Human Sciences, Faculty of Materials Science and Technology, Slovak University of Technology, Trnava, Slovakia
Abstract
In the article we analyze the possibility of testing critical thinking using a self-compiled questionnaire: A test of selected critical thinking skills. There are four main dimensions of critical thinking studied in the test. In the first dimension, we tested the ability to identify relations between two terms. In the second dimension, we tested the ability of the formal - logical analysis combined with logical operations such as implication, conjunction, negation, and equivalence. In the third dimension we tested the ability of determining relations or contradictions in more extensive text units. The importance lies in the ability to differentiate own interpretation of statements from statements that have been shown and proven in the text. The fourth dimension focuses on the ability to interpret the text in order to identify causality and formulate logically consistent arguments. It was very important to understand the text and identify the tacit explicit information (such as the main idea of the text). The research questions and hypotheses were drawn to validate the parameters of a specific instrument. Results of the analyses showed the existence of statistically significant links between the first and second dimensions of the test.
At a glance: Figures
Keywords: critical thinking, cognitive operations, dimensions of critical thinking, a tool to measure critical thinking, correlation research
American Journal of Educational Research, 2014 2 (12),
pp 1182-1189.
DOI: 10.12691/education-2-12-8
Received October 10, 2014; Revised November 20, 2014; Accepted December 07, 2014
Copyright © 2013 Science and Education Publishing. All Rights Reserved.Cite this article:
- Bednáriková, Mária, and Karol Kováč. "Test Tool Creation and Examination of Selected Cognitive Skills of Students at the Slovak University of Technology." American Journal of Educational Research 2.12 (2014): 1182-1189.
- Bednáriková, M. , & Kováč, K. (2014). Test Tool Creation and Examination of Selected Cognitive Skills of Students at the Slovak University of Technology. American Journal of Educational Research, 2(12), 1182-1189.
- Bednáriková, Mária, and Karol Kováč. "Test Tool Creation and Examination of Selected Cognitive Skills of Students at the Slovak University of Technology." American Journal of Educational Research 2, no. 12 (2014): 1182-1189.
Import into BibTeX | Import into EndNote | Import into RefMan | Import into RefWorks |
1. Introduction
“Critical thinking is a process of evaluation and categorization in terms of some preciously accepted standards. It is a logical examination of data which avoids fantasies and judgments on an emotional basis only.” (Bertrand Russell)
The ability to analytically process pieces of information and make own conclusions is often referred as the top ability of the 21st century. Amidst all sorts of information and changing circumstances, the ability to think critically and solve problems effectively is becoming one of the most important skills.
Today, there are many theories about critical thinking, which are schematically captured in the form of various models (e.g. [1, 2, 3, 4] and others). System analysis of these models can generally allocate two main components of critical thinking: personal disposition and cognitive skills. Specific calculation and capabilities available from different authors vary.
When testing the level of critical thinking skills of university students, we proceeded as follows:
1. Based on the analyses of different types of CT models, we’ve set up a precise CT model which we want to test.
2. We have created a testing tool to verify the CT skills consistent with our self-compiled model.
As a result of this process, we have come up with a new testing tool named “Test of selected critical thinking skills”. Here we’ve tried to focus primarily on the detection of so-called general intelligence ([1]), in other words universal cognitive abilities of the first-year students studying engineering at the Slovak University of Technology, Faculty of Materials Science and Technology in Trnava (MTF STUBA). We evaluated the levels of deduction, induction, interpretation, evaluation and argumentation. The basic evaluation material was to work with the given text.
1.1. Critical ThinkingCritical thinking in our understanding is based mainly on scientific rationality and methodology of science. Scientific method is generally described in several basic steps:
1 observation of phenomenon or groups of phenomena; 2 formulation of hypothesis which has to explain the found phenomenon on the basis of causality principle; 3 design of tests on the basis of hypothesis predictions which have either to confirm or disconfirm the hypothesis, 4 in the last step the multiply confirmed hypothesis is incorporated in the system of other confirmed assumptions and laws of scientific theories.
The process of critical thinking analogically develops in the following stages:
Identifying the subject matter – ability to judge a certain situation,
Defining the subject matter – ability to differentiate between real and fictive troubles,
Purposeful observation – ability to gather relevant information; look for new facts and key circumstances,
Looking for sources – ability to prepare a specific information database,
Evaluating the facts – ability to assign the different information, events and situations various rate of validity and reliability,
Deductive and inductive judging,
Identifying the logical mistakes,
Eliciting own reasoned conclusions.
In the “Test of selected critical thinking skills” we focused primarily on monitoring basic thought processes that have a significant impact on critical thinking abilities. Here are the following cognitive operations: analysis, synthesis, induction, deduction, abstraction, idealization, generalization, comparison, analogy, classification and modeling. We tried to define the aforementioned thought processes for testing [5]:
Analysis means the thought decomposition of the complex whole into individual parts. There are several types of analyses – system (analysis of events, things, processes from the point of their system character), relationship (investigation of relations of basic elements and formulation of their functions within the system), classification (analysis of system on the basis of classification features), etc.
Synthesis means joining of individual parts into system wholes, unification, creating complex files. It is subject to comparison, classification, generalization and inductive thought operations.
Induction is the process of deriving the general principles of individual facts and cases. It is the way of argumentation proceeding from the empirical premises to empirical conclusions while the conclusions are not deductively derivable of these premises. The inductive arguments are therefore a kind of enhancing argument in which it is derived on the basis of probability more than it is comprised in its premises. The induction is the key though procedure in empirical sciences.
Deduction is the thought process in which the conclusion necessarily results from the premises, and therefore, it cannot be false, if the premises are true. The deductive argument (syllogism) proceeds from general laws to individual cases. The deductive thought procedure is characteristic for formal sciences.
Abstraction is the process of systematic looking away from certain properties, relations, and features which results in building a certain abstract object. This is separated from the specific sense concept /picture/visualization and is considered only of its own. The abstract term can be represented also by naming the relation (quantitative, qualitative), interaction, activity, etc.
Idealization represents the simplification of complex entities aimed at building theoretical objects and models of them. By the idealization we try to simplify and accommodate the investigated phenomena so that they do not lose their empirical potential. Some authors describe it as the higher kind of abstraction (in contrast to the elementary abstraction which particularly looks away from individual properties).
Generalization is the process of eliciting the common features or properties in different situations, events, relations. The inductive procedure is applied here. The opposite case is represented by concretization (representation) coming out from the deductive cognitive operations and directs to the individual cases identification of general regularities, phenomena or processes (from genus to species, etc.).
Comparison represents the thought process of searching common (consistent) or different (inconsistent) properties of diverse group of objects, phenomena, etc. It is closely related to analogy, generalization and classification thought procedures.
The Greek origin of the word analogy denotes its primary meaning – proportion. The basic scheme of analogy is as follows: A is in relation to B as C is in relation to D. The analogical thinking utilizes the existence of the proportional relations of the initial and target systems and regarding this it derives the probable relations of the new target system. The analogy is the form of inductive inference.
Classification means classification of terms, phenomena, processes into specific groups on the basis of established criteria (class marks) representing the „boundaries” among these groups.
Modeling – forming structured files of theoretical assumptions about the X objective, which presents the starting point of the X objective. A model must represent an initial objective and must simplify its description and explanation.
In the process of interpretation and evaluation of the text, we can identify cognitive processes in the following sequence: after setting the basic topic of an article, it is necessary to analyze the problem area, identify the key concepts (system analysis). It is followed by a comparison and finding analogy relations between the analytes (relational analysis). The next step is a specification of common characteristics (classification) of the fundamental components towards gradual systematization and synthetization of the studied situation (synthesis).
Afterwards, we proceed to form an abstract model of a complex problem situation (abstraction, idealization and modeling). Deduction (and concretization) refers to an analytical phase of thought processes, proceeding from the general to the more specific. Inductive thought processes, on the other hand, work from the specific to the general (induction and generalization). Thus established process of critical thinking presents a model which is going to be the subject of our testing.
1.2. Objectives, Research Questions and HypothesesThe research questions and hypotheses of our pilot research focus on verification of the testing tool, which consists of 4 dimensions examining:
Analogy relationships of the concepts (tasks 1-3).
Logical reasoning at the level of relationships between statements (tasks 4-6).
The ability of logical judgment at the level of partial texts, which represent a coherent idea (tasks 7-9).
Certain type of argumentation at the level of text units (tasks 9-12).
We’ve set up the following partial aims:
• to verify the functionality of the testing tool
• to find out the level of internal consistency of the test
• to find out any relationships between 4 dimensions in our questionnaire
• to find out frequency of correct answers at the level of items within the file
• to find out frequency of correct answers at the level of each dimension.
The research questions and hypotheses are set up to validate parameters of the testing tool:
Research question 1: Which tasks were participants the most successful in?
Research question 2: In which dimensions did participants rank highest?
Hypothesis 1: We assume there are relationships between single dimensions of the testing tool.
2. Methods
2.1. SampleResearch data were collected in October and November 2013. Participants are first-year students of Faculty of Materials Science and Technology, Slovak University of Technology, Trnava, Slovakia studying presently or in a combined form of study. A selection of students is subjected to KEGA project (no.052STU-4/2013) Implementation of critical thinking standards within the university subject Introduction to scientific work.
Students filled in the questionnaire during lessons in the charge of their lecturers. The test consisted of exact instructions and technical terms (mainly logic terms such as “negation”, “implication”, and “equivalence”). Time was limited – 40 minutes. 100 out of 369 participants formed a validating file (Table 1). The participants were selected randomly.
Table 1. Distribution of Participants According to the Gender, Living, Parents’ Highest Education and a Form of Study
The research study is a questionnaire, a so called correlation research. In this type of research, we do not identify the independent and dependent variables since there’s no intervention or measurement before and after the act. [9]
2.2.1. Test of Selected Critical Thinking Skills
To test some selected skills and cognitive processes, we’ve compiled a “Test of selected critical thinking skills”. It is divided into four parts depending on the complexity of cognitive operations which determine the proper solutions of test tasks. Each of the parts consists of three test tasks (a total of 13 tasks). Participants always have the choice of five possible answers, one of which is the correct one.
The first part (tasks 1-3) focuses on identifying relationships between two terms. To solve this task successfully, it is important to think analogically and comparatively, i.e. finding and identifying common and discrepant characteristics of the given terms and analyzing their logical nature is the key concept here. It is an inevitable skill to think in the language (verbal intelligence) and clearly understand the meaning of each term.
Task:
Choose one of the following pairs in which the relationship between the terms is the most similar to the terms assigned (the order of terms matters)
1. book: paper
A) sentence: word
B) cabinet: wood
C) book: page
D) body: space
E) set: element
When solving the task, a student must clearly identify the relationship between the terms “book” and “paper”. However, if he took into consideration only this relationship, regardless the other options, he could identify a relatively large number of relations between the book and the paper. By analyzing relationships between other pairs of words and by their interactive comparison, the student must find common characteristics of identified relationships. Comparison takes place both in the A-E options and also referring to the given sample of words (book-paper). If the student is able to cope with the processes and sequences, he is also capable of identifying the correct analogy between the terms book – paper: cabinet – wood.
2. vehicle: motorcycle
A) car: vehicle
B) apple tree: apple
C) furniture: table
D) chair: back of a chair
E) wheel: motorcycle
When searching for analogy relationships between the sample and given options (A-E), a student has to identify the relationship nature between the terms “vehicle” and “motorcycle”. The first term represents a class of things; the second term is one of the elements belonging to this class. Thus is the analogy relationship between “furniture” and “table”.
3. sun: day
A) star: dark
B) fire: heat
C) lamp: light
D) moon: night
E) water: life
Again, it is important to correctly identify the relationship between the terms “sun” and “day”. The period during the sun shines is called a day. There’s nothing more than a time definition. Therefore, the analogy is found between the terms “moon” and “night”
The second part of the test (tasks 4-6) focuses on identification of two types of logical relations between two (or three) statements: logical entailment and logical contradictions. While in the first part of the test students have to work with single terms, the second part is more difficult because they have to work with statements. Implication, conjunction, negation and equivalence are some of the basic logical procedures relating to tasks’ solutions. Logical intelligence is primarily examined here.
There are three given statements:
A) If Peter is a student, he has a bus ticket discount.
B) f Peter has a discount, he is a student.
C) Peter has a discount, but he is not a student.
Which of the following statements are mutually contradictory? (Two of the statements are mutually contradictory, if the first statement is true, the second false and vice versa.)
A) only A and B statements are mutually contradictory
B) only A and C statements are mutually contradictory
C) only B and C statements are mutually contradictory
D) only two pairs of statements A and B, B and C are mutually contradictory
E) none of the statements are mutually contradictory
To solve this task successfully, it is important to understand the nature of contradictory relationship between two statements: if one statement is true, the other one is false. In this case, the statements B and C cannot be both true. The B statement defines a chance to save on a bus ticket. The C statement denies this condition.
5. It is given a statement T: If it rains, I will stay at home and clean.
Which of the following statements results from the statement T? (The statement is true only if the statement T is true)
A) It will rain.
B) If it does not rain, I will not stay at home.
C) If it rains, I will clean.
D) I will stay at home and will clean.
E) If I stay at home, I will clean.
For this part of the test, understanding the term entailment, each student has to apply his deductive skills to derive a true deduction from a true statement.
6. It is given a statement T: All people in the room are students. What statement is the logical opposite of the statement T? (As the logical opposite of the statement T we consider such a statement which if holds, then T does not hold, and if it does not hold, then T holds).
A) All people in the room are teachers.
B) No man in the room is a student.
C) At least one man in the room is not a student.
D) At least one man in the room is a student.
E) Some people in the room are students, some are not.
If the students want to answer correctly – find a logical opposite of the statement – they have to proceed through indirect evidence. First, it is necessary to make logical contradictions of the statements A-E. Comparing these alternatives with the given statement and by defining relationships of their possible contradictions, the students come up with correct answers.
The third part of the test (task 7-9) deals with the ability to analyze more comprehensive text units. Attention is paid to the ability to process new pieces of information by determining if the result follows previous statements or it’s contradictory. To manage such tasks successfully, it’s important to identify analogies, equivalences, negations, implications not only within single statements, but also in a wider context of coherent thought operations. In these tasks, a verbal and logical intelligence is examined.
The following text is prior to tasks 7-9.
“Theory speaking about “the killer instinct” taken from teachings of Sigmund Freud and Conrad Lorenz does not clarify much. A term does not explain anything. It’s more helpful to understand aggression as a solution to a large amount of different problems. First of them is gaining other people’s resources, such as farmlands, food, tools, gold and jewelry, weapons, fertile women, access to drinking water. The second one is a defense against attack. The third problem represents rivals of the same gender competing for the same source valuable for members of opposite sex. The fourth presents a rise of the power within a group. The fifth, sixth and seventh focus on discouraging sexual partners from infidelity, on gaining lost love, gaining lost sexual partners who are not easily reachable”. [10]
7. Which of the following is clear from the previous text?
A) It is necessary to understand aggression as a solution to certain problems.
B) Aggression as a solution to several types of problems is more useful than it’s explained in the theory of “killer instinct” taken from teachings of Sigmund Freud and Conrad Lorenz
C) The theory speaking about “killer instinct” taken from teachings of Sigmund Freud and Conrad Lorenz does not claim that people solve problems related to sexuality by using aggression.
D) Solving problems by using aggression is sometimes justifiable.
E) Aggression is not a proper reaction no matter what the problem is.
The aforementioned test task examines students’ ability to process and derive information from the text. It is important to separate own interpretations from provable interpretations in the text. Therefore, to answer questions correctly, it is essential to understand the context and main ideas precisely.
8. Which of the following statements is contradictory to what is stated in the text? (A contradictory statement to the given statement means that a statement cannot be true if it is true everything that is said in the text)
A) Giving a name to the studied phenomenon, does not explain anything.
B) There are several causes of aggression.
C) Solving a problem using aggression is sometimes justifiable.
D) If we want to explain aggression theoretically, it is necessary to understand it as a solution to certain problems.
E) Name the cause of aggression as a “killer instinct” is one of the possible types of aggression explanations.
When solving this task, students have to analyze the text precisely and by using a comparison of possible answers (A-E), they need to identify contradictory bonds.
9. According to the text, theory speaking about the “killer instinct” taken from the teachings of Sigmund Freud and Conrad Lorenz does not clarify much because:
A) There are several causes of aggression.
B) Aggression does not always mean killing, so naming aggression as the “killer instinct” is not appropriate.
C) Giving a name to the studied phenomenon, does not explain anything.
D) It is named inappropriately.
E) It is useful to understand aggression differently than it’s understood in this theory.
In the task 9, it is important to identify the causality relationship between two events. This relationship is explicitly mentioned in the text (as opposed to tasks 10-12).
The fourth part of the test (tasks 10-12) examines eligibility of students to formulate arguments. The key skill becomes the ability to analyze causality relationships between thought units expressed in the text. Students have to interpret the text by identifying contradictory or implicit relationships between main streams. A deeper analysis aiming to an explanation (determining casual relationships) and evaluation (determining the basic idea) is an inevitable step. This part of the test focuses on verbal and logical intelligence.
Tasks 7-9 are related to the following text:
To answer the following questions, use only information mentioned in the text.
“Intelligence if often described in slightly narrow terms as if it were a certain type of number when more is better, and could be assigned to a person same as the average of shooting in cricket. It has always been measured by diverse understanding of spatial skills, verbal fluency, numerical skills, inductive reasoning, perceptual speed, deductive reasoning, mechanical memory and so on. Recent decades have brought a perception to speak about different subtest as “multiple intelligences”. Honestly, why do we unite these skills in order to reduce intelligence into a single type? The short answer to this is that only one number tells us something more – it can be arbitrary, but it still represents interesting pieces of information. Here’s the reason why: an excellent result in one type of intelligence tests does not predict any results in other tests; it is not sure that one skill is above the other ones. On the other hand, a person who succeeds in such a test often achieves better results in other tests than the average. Is there an existence of a common factor, something like the ability to do tests? The so called “general g-factor” expresses this interesting correlation of tests. IQ is one of the fascinating aspects of intelligence, but it does not include other aspects: we should not make a mistake by reducing the subject of intelligence into one number on a rating scale. That would be similar as rating football matches by one statistics only, for example, giving a percentage of successful penetrations. If we take a football league as a whole, the number of victories correlates with this statistics, but there’s something else in football that also counts. Some teams win without a single penetration, by using other skills. In many environments IQ correlate with the “number of wins”, but intelligence is about something more as same as the number of successful penetrations does not always counts!” [[11],19-21].
10. Choose one of the following statements which support the claim that it’s advantageous to reduce intelligence to a single type:
A) Excellent results in one type of intelligence tests predict a good result in a different test.
B) Evaluate intelligence by IQ is the same as judge football match by a number of successful penetrations.
C) There is a certain correlation in solving various types of intelligence tests successfully.
D) Recent decades have brought the idea of dividing intelligence into different types.
E) Intelligence tests resulting in a single number as the value of IQ, are often used.
The ability to identify arguments is prior to our ability to build up a model of arguments’ structure, particularly from its three main components: a claim, proof and reasoning. A claim proclaims author’s opinion and attitudes which are held and justified in the text. A proof presents objectively verifiable factual material demonstrating the validity of author’s claim. Reasoning documents a connection between two previous components (claim and proof) and draws a conclusion which results from subjective opinion and its proved validity [12].
11. Choose one from the following arguments saying that IQ is only one aspect of intelligence:
A) A number of successful penetrations is not important in a football league.
B) There is a so-called “general g factor” which represent a correlation among results of different types of intelligence.
C) IQ cannot be accurately measured and therefore it is not appropriate to correlate it with intelligence.
D) The existence of correlation among results of different types of intelligence is not a sufficient reason for complete identification of IQ with intelligence.
E) There is no reason to reduce tests of different types of intelligence to one type of intelligence.
In this task, we focus primarily on the ability to identify relationships between reasoning and so called rebuttal, which specifies the conditions of weakened validity (so it presents certain counterarguments). The students are asked to answer the following questions: What is a contradictory argument to the given argument (T) in task 10? On what basis it is possible to identify that statements in tasks 10 and 11 are contradictory?
12. The main idea of the text is:
A) There is no significant relationship between intelligence and IQ.
B) Relationship between intelligence and IQ is same as relation between length and a meter.
C) Intelligence represents an indivisible unit.
D) Intelligence cannot be measured and therefore, IQ speaks about it only partially.
E) Intelligence is a multiaspectual phenomenon, whereby IQ testifies it only partially.
Identifying the main idea is the process of interpretation and text evaluation as a whole. It is important to avoid specifying the main idea of individual sections and identify explicit, unstated information mediating through the whole text. Therefore, in this case, students need a certain degree of abstraction and idealization.
From overall findings we can conclude that based on four types of tasks, we examined the following abilities:
Tasks 1-3 focus on verbal intelligence and thought processes – analyses, synthesis, comparison, analogy and classification to some extent. Tasks 4-6 examine logical intelligence (implication, conjunction, negation and equivalence) and mainly deductive thought processes. Tasks 7-9 reveal verbal and logical intelligence (logical entailment, forming logical judgments) in a critical analysis of text. It is concerned with a wide range of thought processes – analysis (classification and correlation), synthesis, deduction, induction, comparison and others. Tasks 10-12 test verbal and logical intelligence (forming judgments) in the text interpretation and evaluation. It is important to identify causality. Thought processes used in argument making are especially analysis, comparison, generalization, classification, modeling, abstraction, deduction, induction and synthesis.
2.3. Methods of Data AnalysisFrom the primary data, we calculated averages. Summation indices were set as indicators of each dimension. We examined consistency of the items by a Cronbach reliability index (α). When analyzing the data, we used factor and correlation analyses.
Our aim was to verify the suitability and accuracy of the selected tool, which we analyzed through testing the internal consistency of overall critical thinking index, correlations between dimensions and validation of gained results. Subsequently, we will test entire population and thus verify reliability of the test. The obtained data under validation file were interpreted using the statistical SPSS program.
3. Results
We’ve verified the validity of participants’ answers at several levels. Questionnaires were taken while maintaining standard conditions. There was no need to exclude any of the tests due to improper or inadequate filling. Students’ responses underwent formal and logical control. We wanted to avoid the chance of skipping or accidental choosing answers and therefore we compared distribution of successful answers of all 12 tasks with the calculation of statistical probability of success in our test by accidental filling. Distribution of answers in both cases is significantly different (Table 2).
A certain validity of the testing tool can be illustrated by a distribution curve of correct answers in single items (Figure 1).
Research question 1: Which tasks are the participants of our sample the most successful in?
To test the validity and reliability of individual items and the overall framework of our test, we chose testing through a performance index (Facility index). Each of the items was positively tested within the range of acceptability. Analysis of distribution of correct answers revealed a different percentage of success in single tasks. Participants did the best in tasks no. 1, 2, 7 and 12. The most difficult tasks were 6, 8, 9, 10 (Table 3).
Research question 2: In which dimensions do participant score the highest?
At the level of basic sorting, we’ve found differences in the distribution of correct answers in each dimension. Students achieved the highest score in the first dimension – verbal intelligence; 30 of them answered all the three questions correctly. However, such a model (3 correct answers) in other dimensions was less significant. The weakest part and the least correct answers students achieved in dimension 2. When comparing results of each dimension, the lowest average score was achieved in the second dimension (Table 4). There’s an interesting point – a relatively stable number of respondents with zero success in each dimension. First-level sorting of correct answers according to the dimensions indicates the fact that the level of critical thinking does not have to be consistent for each dimension. It means if a student is successful in one dimension, he does not have to succeed in other dimensions.
H1: We assume there are relations between dimensions of the testing tool.
Relationships between the dimensions were examined by Pearson correlation coefficient. We’ve verified statistically significant links only between the first and third dimensions. There’s a positive moderate correlation with significance less than 0,001 (Table 5). The more successful the participant was in the first dimension, the higher score he achieved in the third dimension. Correlation analysis indicates a multidimensionality of critical thinking, because out of 6 possible correlation relations between 4 dimensions, 5 are statistically insignificant. The success rate in one dimension does not necessarily mean a higher or a lower score in other dimensions.
We used a factor analysis to find out whether the 4 dimensions measured in our test do form one component. Our assumption was confirmed, however, the reduction of 4 dimensions to one summation index explains only 37% of the variability of variables.
Reliability analysis of the summation index of critical thinking demonstrates similar results when including all 4 dimensions to one index showed Cronbach´s alpha 0,425, low value, and after removal of all 4 dimensions, it has not risen significantly.
Reliability of the test results are often influenced by the composition of participants. In our case, gender and age play role as significant variables. Aforesaid findings have inspired us to deeper analyses of our aim – to verify the suitability and accuracy of our test tool to measure critical thinking. Summation index for all 4 dimensions would not be appropriate according to previous correlation calculations, factor analysis and reliability analysis. The dimensions are not internally consistent, it does not mean the tool is inappropriate, but it’s important to find a different way of data analysis.
For a sample of 100 participants we’ve created a typology (cluster analysis) which divided the participants into 3 even groups. The first group had consistent successful results in all 4 dimensions and the other 2 groups achieved highest score only in the first dimension.
Based on these findings, we can logically allow creating summation index which will not be primarily analyzed for critical thinking testing with our questionnaire, but it will be a supplementary indicator of analyses according to each dimension. Cluster analysis confirmed a suitability and contribution for testing critical thinking by means of our tool.
4. Discussion
Our goal was to create a tool to test selected critical thinking skills and its verification in practice. The questionnaire consists of 4 dimensions measuring critical thinking, selected demographic and other identifiers.
We consider the questionnaire measuring levels of critical thinking to be an appropriate and successfully tested instrument to achieve the goal. Testing on a sample of 100 respondents revealed new findings that contribute to effective measuring levels of students’ critical thinking and analysis. In the next step, we’d like to test all 1st year students. Subsequently, we’ll compare the results with preliminary results and will carry a deeper analysis of second-level sorting in comparison with demographic and other identification signs.
In our self-compiled questionnaire of critical thinking, we assume existence of 4 hierarchical dimensions ordered by the level of cognitive operations complexity which are related to correct solutions of test tasks. All tasks examining critical thinking skills are based on verbal assignment (terms, statements and text units).
Results of the analysis showed existence of statistically significant links between the first and second dimension of the test. There seems to be a link between the ability to understand the meaning of terms and meaning of the text. In the case of identifying analogical relationships between statements it’s important to clearly understand the scope (extension) and the content (intension) of each concept.
A similar mechanism is probably applied when analyzing texts and deriving statements which are obvious from the text or are contradictory. On the other hand, there are no statistically significant relations between the first and fourth dimensions. It is probably caused by a higher level of abstraction where there’s a necessity to identify casual relationships in forming an argument. Identification of the main idea of the whole text required a proper abstract thinking ability because the main idea is not literally mentioned. Surprisingly, our findings have shown no correlation between correct answers to tasks in the second and third dimensions. However, both dimensions are about identifying entailment or contradiction. Tasks in the second dimensions are formulated in the language of formal logic that’s what participants seem to have no experience with (although all the technical terms are explained in the assignment). The number of correct answers in the second dimensions was the lowest despite the fact that in tasks 4-6 students could help themselves with formal notes and basic rules for operations validity within a propositional logic (i.e. implication is invalid only if the true statement implies a false argument and so on).
On the basis of statistical analyses of our test tool, we can summarize the following statements:
Critical thinking is a very diverse phenomenon whereby the dimensions are shown to be independent from each other (with the exception of the relationship between the first and the third dimension). Thus successful solution in one dimension does not have any impact on successful solutions in other dimensions. When comparing the dimensions, evenly distributed correct answers were mainly shown within one dimension (see Table 3).
Complexity of the tests and accuracy of answered questions is subjected to comprehensiveness of the analyzed text on one hand and the level of formal expression and abstraction on the other hand. It can be illustrated by the lowest frequency of correct answers in task 6 (requiring high level of formalization and abstraction) and in task 10 (requiring analyses of very comprehensive and abstract text; and was also subjected to understanding the formal structure of statements).
References
[1] | Ennis, R., H. “A Logical Basis for Measuring Critical Thinking Skills“, In: Educational Leadership, 40 (10). 44-48. October 1985. | ||
In article | |||
[2] | Facione P.A, Giancarlo C.A, Facione N.C, Gainen J. The disposition toward critical thinking. In: Journal of General Education. 1995, vol 44, 1-25. | ||
In article | |||
[3] | Sadler, G. “Reconciling Four Models of Critical Thinking: FSU QEP, Paul-Elder, CLA, and APA Delphi.” [Online]. Available: . [Accessed March. 5, 2014]. | ||
In article | |||
[4] | Niewoehner, J., R. “Applied Disciplines: A Critical Thinking Model for Engineering.” [Online]. Available: . [Accessed April. 26, 2014]. | ||
In article | |||
[5] | Garrison, D. R., Anderson, T. and Archer, W. "Critical thinking, cognitive presence, and computer conferencing in distance education."American Journal of distance education 15 (1). 7-23. 2001. | ||
In article | CrossRef | ||
[6] | Odlerová, E., Bednáriková, M. The problem of evaluating approaches of university students in the study of ecophilosophy and in guiding towards critical thinking. In: Proceedings of the Joint International IGIP-SEFI: Annual Conference 2010. Diversity unifies - Diversity in Engineering Education, 19th - 22 th September 2010, Trnava, Slovakia. Brussel: SEFI, 2010. | ||
In article | |||
[7] | Ennis, R., H. “Critical Thinking Assessment.” In: Theory Into Practice, Volume 32, Issue 3, 1993. | ||
In article | |||
[8] | Bednáriková, M. Introduction to Cognitive Science. Filozofická fakulta Trnavskej univerzity v Trnave, Towarzystwo Słowaków w Polsce, Kraków 2013, 47-52. | ||
In article | |||
[9] | Kováč, K., Ritomský, A. Advancement in the Scientific Reasoning of online consumer behavior. In: Proceedings of the Joint International IGIP-SEFI: Annual Conference 2010. Diversity unifies - Diversity in Engineering Education, 19th - 22 th September 2010, Trnava, Slovakia. Brussel: SEFI, 2010. | ||
In article | |||
[10] | Koukolik, F. Zvíře politické. Eseje o lidské nátuře. Galen, Praha 2012, 25-26. | ||
In article | |||
[11] | Calvin, W., H. Ako myslí mozog. Kaligram, Bratislava 2000, 19-21. | ||
In article | |||
[12] | Toulmin Model of Argumentation. [Online]. Available: . [Accessed May. 30, 2014]. | ||
In article | |||