1. Introduction

„With all of these multiplying interpretations and perspectives, and after more than four decades of dedicated scholary work, critical thinking remains more elusive than ever.“ (^[3], 3)

The way of testing and evaluating the critical thinking (CT) is conditioned by its fundamental defining frame and model too. It is not easy to specify one generally acceptable model which would represent reliable platform for creation of the tasks that are suitable for evaluation of CT. First tendencies leading to defining and modelling the CT can be found at the beginning of 1990´s in works of American Philosophical Association ^[4]. In the frame of this model the board of experts defined 5 categories that are closely connected with cognitive abilities (interpretation, analysis, evaluation, inference and explanation) and one category dealing with metacognitive strategies (self-regulation – self-reflection and self-control). This way of creating a CT model - as a connection of cognitive abilities and metacognitive strategies or personal predispositions – is generally reflected in majority of existing CT models.

The abovementioned tendency of CT modelling from the end of previous century was conditioned by development in the area of comprehension and explanation of CT at the beginning of 1970´s. Richard Paul ^[7] describes this phenomenon in three developmental phases. Firstly, CT was understood in a way that was defined by the ability of argumentation that was concretized in theories of informal logic. Identification of particular logical (or rational) structures in human thinking that can be formalized or schematized played an important role. Understanding and consequent deployment of this phenomenon could become a straightforward educational aim. A good example for the tendency of explaining CT is the model of argumentation created by Stephen Tuolmin, whose most important advantage lies in its simple schematic display of the process of argumentation. It consists of three fundamental parts (claim, evidence, a warrant), that are extended by three other categories – backing (as the support for reasoning), rebuttal (as the conditions of weaken validity of a statement) and qualifier (defines the level of probable validity of a statement) ^[8]. Schematic formulation of argumentative abilities was specified into more details in course of time, and so more and more argumentative schemes were created, for example, argument from popular opinion, argument from analogy, argument from correlation to cause, argument from consequences and slippery slope, argument from sign, argument from commitment or argument “ad hominem”) ^[9].

In this first phase, CT was explained as a way of thinking that is from the point of view being filled by information formulated in natural language, but it processes the information based on the principles of formal logic. Although critical thinking does not work with formal symbols, as it is in case of sentential or predicate logic, still it is ruled by the same principles and criteria for true inferences that are valid in formal languages. CT is based on the semantics and syntax of natural language that are derived from the pattern of correctness which is valid in tautology of formal logic. To think critically means to think logically. Processes and schemes of critical thinking can be formalized and so operationalized and measured. All ways of testing the CT that are derived from this way of explication of critical rationality are orientated on testing of cognitive abilities. These can be formulated from more specific point of view, for example, as analysis, interpretation, inference, evaluation, explanation, induction and deduction (CCTST), or from more general point of view as analytical thinking, problem solving, ability to draw up texts (CLA), or the ability to identify predispositions, deduce consequences, interpret and derive, create logical arguments (Watson-Glaser Critical Thinking Appraisal), or the ability to explain statements, reveal their context and preconditions and finally to define their implications and consequences (AASCU) (compare ^{[1, 5, 6]}).

In the frame of second wave of conceptualisation of CT in 1980´s, a wider (more general) interpretation regarding CT abilities was prevailing. The emphasis was put especially on personal development of students and CT was analysed also in relation to civilian attitudes, emotions, creativity etc. Critical pedagogy was on the move and the CT was applied in the frame of discipline-specific approaches. From now on, not only thinker´s abilities themselves, but also their attributes and personality started to play an important role in the process of critical thinking.

In the third wave that is taking place nowadays, according to Paul, the period of reconceptualization of CT should arrive what means unification of the two previous tendencies. This new complex theory should explain abilities regarding critical analysis and logical argumentation as well as dispositions and personal “settings” which instigate natural curiosity and inspire us to search for the truth.

This contribution is a content continuance to our contribution published in Americal Journal of Educational Research 2 (12) (look at ^[2]). In the mentioned scientific output we tested CT abilities of students using the Test in Selected Abilities of Critical Thinking that was made by ourselves. Here, CT is understood as a set of abilities that are based on the implementation of logical structures into cognitive process and lead to individual creation of rational models of reality that is being cognized. We propose following way of classification of fundamental thought operations which constitute the ability of critical thinking:

•  Thought processes

a) Cognitive operations (CO): analysis, comparison, abstraction, idealization, generalization, concretization, reduction, classification and others

b) Complex operations of critical thinking (COCT): argumentation, evaluation, interpretation, inference (deduction, induction, abduction), analogy and others.

•  Formal-logical operations (FLO): implication, equivalence, conjunction, disjunction, negation, syllogism and others

This system of classification is ordered in hierarchical way (from fundamental cognitive operations through complex operations of critical thinking up to formalized logical rules). The first category (CO) conditions the two other categories: so that we are able to understand FLO and COCT we must fully master cognitive operations of abstraction, idealization, synthesis, analysis, reduction and so on. Difference between CO and FLO lies in their formal inscription. Formal-logical operations use exactly defined conditions of what is considered to be true (validity), however, in case of cognitive operations there is no exact formal procedure that can be used as the criterion for specification of their validity (truth). In case of COCT we deal with broad information wholes and that is the reason why it is extremely difficult to identify validity conditions of these cognitive operations (for example: correctness of a judgement, correctness of evaluative criteria, reliability of facts needed for argumentation and the like). COCT are applied when we need to: determine the validity of arguments (from the point of view of their logical coherence and content relevance), evaluate conditions and level of probability of statements, evaluate information (from the point of view of the reliability and importance). COCT are also used in the process of inductive generalization and creation of general rules (for example scientific law) and when inferring from consequences of phenomena and predicting their probable development etc.

Based on the abovementioned categorization we created the test tool that consists of 4 dimensions orientated on:

1. Identification of analogical relations at the level of terms (OT 1-3)

2. Formal-logical reasoning at the level of relations between statements (OT 4-6)

3. Ability to reason (searching for consequences and contradictions) at the level of text passages that represent compact thoughts (OT 7-9)

4. Ability of argumentation at the level of complex text wholes (OT 9-12).

Altogether, we tested 369 students of Faculty of Materials Science and Technology at Slovak University of Technology in Bratislava. In this contribution we deal with findings regarding possible correlations between overall fruitfulness in the test and independent variables that are age, gender, form of study (daily form vs. external form), place of residence (town vs. village) and previous education (general vs. vocational)

Research questions were orientated in a way to find out differences in particular dimensions according to the selected socio-demographic characteristics and to create a typology of participants in our research.

Research question n.1: Are there any differences in the overall fruitfulness of the participants in particular dimensions selected according to the specific socio-demographic characteristics?

Research question n.2: Are there any differences in the degree of fruitfulness of participants in particular dimensions when taking into consideration their form of study?

Research question n.3: Is it possible to create the typology of participants which is based on their overall fruitfulness in particular dimensions?

We have set particular partial goals:

•  To verify the functionality of our tool for the whole group of respondents

•  To find out distribution of the correct answers at the level of particular items

•  To find out distribution of the correct answers at the level of particular dimensions

•  To define relationships between particular dimensions in relation to the socio-demographic features

•  To recognize mutual relationships between dimensions according to the typology of participants.

2. Methods

All the data were collected in October and November 2013. The target group was represented by the students of the 1st year of master´s study at the faculty of Materials Science and Technology of Slovak University of Technology in Trnava both in daily and external forms of study. We have administered 369 participants all together. Selection of the participants was done through the precise choice, what means that the size of population equals the size of sample. The target group was defined in project KEGA 1750 – Implementation of the Standards of Critical Thinking in the Frame of the School Subject Introduction to the Scientific Work.

Description of the sample in absolute numbers and percents according to the particular variables – gender, age, inhabitation, previous education and education of parents, is stated below in Table 1.

Table 1. Distribution of the participants according to their gender, age, inhabitation, previous education and education of their parents

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View next table

In our research we used the test of selected competencies regarding critical thinking. Its validity and reliability was tested in previous period of time and its suitability and overall assets were confirmed in the consequent analyses of critical thinking (see ^[2]).

The test itself consists of 12 particular questions whose difficultness was tested via Facility Index. Questions 6 and 9 were evaluated as the most difficult ones.

Question n.6: There is a statement S: All people in the room are students. Which of the following is the logical opposite to the statement S? Possible answers:

A) All people in the room are teachers.

B) No man in the room is a student.

C) At least one man in the room is not a student.

D) At least one man in the room is a student.

E) Some people in the room are students, but some are not.

Question n.9 deals with the text analysis:

Theory about “killer instinct” that is derived from the teaching of Sigmund Freud and Konrad Lorenz does not illuminate much. The title does not offer the explanation.

It is more fertile to understand the aggression as a solution for bigger number of various problems.

The first problem is to get the sources from other people; for example, agricultural soil, food, tools, gold and jewellery, weapons, fertile women, or access to drinking water.

The second one is the defence against possible attacks.

The third problem is represented by the rivals of the same sex competing for the same source (valuable members of the opposite sex).

Next problem is the possible advance in power hierarchy of the society.

The fifth, sixth and seventh ones are as they follow – discouraging sexual partner from infidelity, attempt to regain the lost sexual partner and finally it is the attempt to gain a sexual partner who is practically unobtainable under common circumstances.

(F. Koukolík: Zvířepolitické, Galén 2012)

Question 9. According to the text above, thetheory about killer instinct that is derived from the teaching of Sigmund Freud and Konrad Lorenz does not illuminate much, because:

A) There are more causes of aggression.

B) Aggression is not always aimed to kill someone, so to call the cause of aggression as the killer instinct is not appropriate.

C) If we just call the examined phenomenon by any name, it does not explain anything.

D) It is called unsuitably.

E) It is fertile to understand the aggression differently from its conception in this theory.

Questions n.6 and n.9 were correctly answered by 20.1% (q6) and 32.2% (q9) of participants. These two questions reach the level of critical values of the test index. The questions 12, 1 and 7 were evaluated and can be considered as the easiest ones. For more details see Table 2.

Table 2. Number of correct answers of the participants with reference to particular questions

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

All the questions measuring the level of critical thinking were constructed according to the degree of complexity of given tasks and applied logical procedures, so that they create closed subgroups consisting of three questions. Thanks to this division we could identify 4 dimensions of critical thinking.

Based on the evaluation of the correct answers in particular subgroups we can state that the easiest subgroup or dimension is the first one (95 participants) in which all 3 questions were answered correctly. In case of the first dimension (compared to the 3 other subgroups) we measured the highest average score (1.69). Dimension n.2 seems to be the most difficult one, not only by the average value (1.19), but also by the least number of participants (75) who did not answer correctly any of the three questions in this subgroup. Similarly, about one fifth of all participants could not correctly answer any of the three questions in dimension n.3. For more detailed information see Table 3 below.

Table 3. Number of correct answers in the frame of particular dimensions

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

In the overall analysis of all correct answers to all questions we compared this distribution to the statistical probability of students` success so that we can either confirm or eliminate accidental or so called hit-and-miss way of guessing the correct answers. Since there were obvious differences between the distribution of real answers and their statistical probability confirmed, this fact increased both validity and reliability of our research tool.

Table 4. Distribution of the fruitfulness of participants` answers to all the questions in comparism to the calculation of statistical probability

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

The cases of extreme values (0 or 12 correct answers) were observed only twice – one participant per one extreme state. By the mode comparison (searching for value occurring most frequently in a distribution), the majority of all participants correctly answered 4, 5, 6 or 7 questions. This makes approximately 60 % of all the students. (Table 4).

Average fruitfulness of the whole population of students equals 5.45 with the standard deviation 2.24. Visual representation of distribution according to the number of correct respondents` answers is displayed below.

Download as

• PPT
  PowerPoint Slide
• PNG
  Larger image(png format)

Veiw figureFigures index

•  NEW
  View larger figure in new window

Figure 1. Distribution of correct answers in the frame of the set of participants

In the pre-research phase we faced the problem regarding creation of summing index of critical thinking by adding all the correct answers in all four dimensions. Unfortunately, the low value of Cronbach`s alpha (0.452) is valid for whole population what does not confirm reliability of the abovementioned summing index. Factor analysis evaluated and confirmed the fact that all four dimensions create one integrated component, but one summing index would describe only 38% of variables.

This fact has been proved also by the obtained values of mutual correlations between particular dimensions. The strongest relation can be found in dimensions n.1 and n.3 (this means that if students succeeded in dimension n.1, they could have reached better score also in dimension n.3). In a similar way, the low values of correlation coefficients were measured also in other comparisons of particular couples. There was no statistically significant relation confirmed in case of relation between dimensions n.2 and n.4.

Table 5. Relationships between dimensions of the measuring tool

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

3. Results

Research question n.1: Are there any differences in the overall fruitfulness of the participants in particular dimensions selected according to the specific socio-demographic characteristics?

We have been dealing with the analysis of measure of fruitfulness at various levels - not only paying attention towards particular dimensions, but also from the point of view of overall index of critical thinking. In the first case we were detecting differences in the average score by demographic signs. We did not find out any statistically significant differences in the group of our participants when taking into consideration their gender, inhabitation, age categories and the form of study. (Table 6).

Table 6. Differences in average score of the participants in particular dimensions according to the selected socio-demographic data

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

The previous education is a variable that makes the difference in the fruitfulness in dimensions 1, 2, 3 and in the index of critical thinking. Statistically important differences were measured for the benefit of those students who attended secondary grammar schools. The most obvious differences are in dimension n.2 and also in the index of critical thinking (Table 7).

Table 7. Average score in particular dimensions according to the previous education of participants

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

The correlation analysis was used in testing the measure of fruitfulness in particular dimensions and also in the overall index of critical thinking compared to the age of participants, frequency of reading and the education of their parents. In all the three mentioned variables only weak or none relations with dimensions and the index of critical thinking were observed.

Still some facts appeared in the analysis – the younger our participants were, the higher score they achieved in the first dimension. The higher education of their parents was, the higher score the students reached not only in dimensions n.1, n.2 and n.4 but also in the overall index. And finally, the more frequently they read, the higher score they reached in dimensions n.1, n.3, n.4 and also the overall index. Once again we must emphasize the fact that all the measured correlations were extremely weak.

Table 8. Relations between particular dimensions showing degree of fruitfulness according to the frequency of reading, education of parents and age

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

In correlation analyses we found out some interesting differences in dependences between success in particular dimensions when dividing the file according to gender, previous education and form of study.

There is a weak relation between success in dimensions n.1 and n.2 found only in the group of men. Similarly, in case of relation between success in dimensions n.1 and n.4, the correlation coefficient of significant value 0.213 was measured, still there was absolutely no relation confirmed between these two dimensions in the group of women. In contrast, if the women were successful in dimension n.2, they also succeeded in dimension n.3 (r=0.323), in case of men the value 0.152 of correlation coefficient was measured. See Table 9 below.

Table 9. Differences in mutual dependencies between particular dimensions according to the age of participants

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

When comparing differences in correlations between particular dimensions from the point of view of the previous study of our participants, we found out that there is a connection between success obtained both in the first and fourth dimension and the second and fourth one in the group of participants who attended secondary grammar schools. Only weak or no relation was measured in the group of students who came from secondary vocational schools.

Table 10. Differences in mutual dependencies between particular dimensions according to the previous education of participants

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

Research question n.2: Are there any differences in the degree of fruitfulness of participants in particular dimensions when taking into consideration their form of study?

Measure of good results in the group of students attending the university in the daily form of study (DFS) was observed in the results in dimensions n.1 and n.2, meanwhile this relation is not valid for the group of students from the external form of study (EFS). Even bigger difference in correlation coefficient was measured in results from dimensions n.3 and n.4, where the value of correlation index is statistically insignificant (0.100) in the group of EFS students, meanwhile the DSF students reached statistically significant value 0.215.

Table 11. Differences in mutual dependencies between particular dimensions according to the form of study of participants

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

Research question n.3: Is it possible to create the typology of participants which is based on their overall fruitfulness in particular dimensions?

Since the Cronbach`s alpha could not reach the level sufficient for the creation of summing index of critical thinking, we used the cluster analysis that helped us reveal differences in distribution of correct answers according to particular types of the participants. We have created 4 types with an emphasis on acceptable proportional division of the population. See Table 12.

Table 12. Distribution of participants into particular types according to the overall number of correct answers

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

Participants in particular groups can be described by the average value of fruitfulness in all four dimensions. (see Table 13) The first type is characterized by the highest level of fruitfulness in all four categories with 2 correct answers (on average) in every dimension.

Table 13. Average values of fruitfulness in particular dimensions according to the defined types

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

The second type has got low values in dimensions n.1, n.3 and n.4. They obtain the highest score in the second dimension, with 1 correct answer on average.

The third type reaches extremely high score in the first dimension (2.41) and one correct answer on average in the three other groups.

The fourth type has got the lowest score in the first dimension, and on the contrary, the second highest score (1.66) in the third dimension.

By typological comparing we have found out that there are statistically significant differences between participants only from the point of view of their previous education, education of their parents and the frequency of their reading. (see Table 14, Table 15).

Table 14. Typological differences of participants according to their previous education

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

View next table

Table 15. Typological differences of participants according to the eduction of their parents

Download as

PowerPoint Slide

Larger image(png format)

• Tables index

Veiw figure

View current table in a new window

View previous table

4. Discussion

Based on our findings we can state that we did not identify any statistically significant correlations between test fruitfulness in four dimensions of CT and gender, place of residence, age or form of study. We also can state that there is no existence of correlation between index of critical thinking and the abovementioned observed variables (see Table 6). On the contrary, statistically significant difference in students´ fruitfulness in the test can be identified when considering their previous type of secondary school. Postgraduates from secondary grammar schools obtained significantly higher average score in our test. The difference was most visible in the second dimension that was orientated on solution of formal-logical tasks and this fact also reflects in higher overall index of CT (see Table 7). This finding can be explained by more frequent need of using the abstract thinking that is required in the frame of generally orientated education at the level of secondary grammar schools. Secondary vocational schools are primarily orientated on practical education and training that is predominantly orientated on obtaining of practical skills. On the other side, secondary grammar schools prefer the concept of widely orientated curriculum in the areas of human and natural disciplines. Substantial emphasis is put on the mathematics that is primarily based on the abstract quantitative thinking. Average obtained score in CT index also supports the affirmation regarding higher level of abstract thinking and better ability of creating rational schemes among students of secondary grammar schools. Our efforts to examine possible connection between particular dimensions of CT and selected variables (age, gender, place of residence, form of study, previous education, frequency of reading) led us to creation of typology of participants (based on cluster analysis) according to the average value of effectiveness in particular dimensions of CT. Here, we confirmed again, the statistically significant dependence between students´ overall effectiveness in the test and their previous education (see Table 14).

Based on results of our test we assume that CT represents extremely heterogeneous and complex ability and that is why it is not easy to define basic determining factors (if we can mention just one particular ability at all). On the other hand, the ability of abstract theoretical thinking has, according to our research, quite obvious positive impact on the task solutions that are connected to analogies, logical relations, inference or argumentation.

Although nowadays the emphasis is put especially on the practical application of transferred knowledge, also theoretical education that supports production of abstract mental models and identification of rational structures should not be underestimated. In the process of their creation we learn how to as if ignore or eliminate conditions of concrete situation and examine problems from the overview position. Hand in hand with increasing rate of abstraction the mental model loses its factual content what enables more distinct perception of its rational structure (to identify substantial constitutive parts and relations between them). There is no doubt that in the process of creation of these abstract models all the abovementioned CT abilities are applied (see Introduction). When slightly simplifying, we can presume that the ability to abstract a rational structure from problem situations is the basic condition for successful deployment of critical analysis, evaluation and argumentation. In close connection to these enunciations, theoretical and abstract thinking seems to be the key factor for potential development of CT abilities.

References

[1]	AASCU. 2006. Value-added Assessment. Perspectives. Washington, DC: American Association of State Colleges and Universities.
	In article
[2]	Bednáriková, M., &Ková, K. (2014). Test Tool Creation and Examination of Selected Cognitive Skills of Students at the Slovak University of Technology. American Journal of Educational Research, 2(12), 1182-1189.
	In article		View Article
[3]	Davies, M., Barnett, R. (Eds.) (2015). The Palgrave Handbook of Critical Thinking in Higher Education. London: Palgrave Macmillan.
	In article		View Article
[4]	Facione, P., A. (1990). Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Asesment and Instruction. Research Findings and Recomendations. Retrieved from https://files.eric.ed.gov/fulltext/ED315423.pdf.
	In article
[5]	Facione P.A, Giancarlo C.A, Facione N.C, Gainen J. (1995). The disposition toward critical thinking. Journal of General Education., 44, 1-25.
	In article
[6]	Klein, S., Benjamin, R., Shavelson, R., Bolus, R. (2007).The Collegiate Learning Assessment Facts and Fantasies. Evaluation Review October 31 (5), 415-439.
	In article		View Article  PubMed
[7]	Paul, R. (2011). Critical Thinking Movement: 3 Waves. Retrieved from: https://www.criticalthinking.org/pages/critical-thinking-movement-3-waves/856.
	In article
[8]	Toulmin, S. E., Rieke, R., Janik, A. (1979). An introduction to reasoning. New York: Macmillan.
	In article		PubMed
[9]	Walton, D. (2006). Fundamentals of Critical Argumentation. Cambridge : Cambridge University Press.
	In article