The Bottom up Evaluation at Universities

Veiw figure View Table

In total 1 114 items of students´ evaluation of lectures and exercises were collected.

3. Results

The idea of students´ feedback is very good, but somewhere is a problem and students do not use this tool. Therefore data of these surveys are not representative. The system permits to know only total numbers of respondents (students). It is sufficient if there is only one teacher per group and it is no problem if one teacher leads more groups (e.g. exercises). In that case is in the system feedback from all students it means all received feedbacks and an information how many students should evaluate. It is only impossible to know what group students attended.

Example presented in Table 1 shows two problems. The first one is very low number of feedbacks and the second one is impossibility to find the size of taught groups. The knowledge of number of students in the group is necessary to judge an importance of evaluation. For example ten evaluators from seventeen enrolled students is much more important than 16 evaluators from 176 (see Table 1).

In some cases it would be possible to increase the number of feedbacks a bit, because a lot of students forgot to put teachers´ name to the feedback. If there is subject with only one teacher it is possible to add these anonymous feedbacks to his ones.

The students in Table 1 have only one lecturer but they can choose one of seven groups per exercises. Theoretically it is possible to find the real number of students they should evaluate each subject in another item of Educational info system. It is Schedule for a semester with taught subjects divided into learning groups and numbers of enrolled students for each group. But students can as change their group so attend lectures or exercises for another group. And they always can evaluate all teachers in list of teacher for a subject. Changing a group is often if students prefer another time or “style of teaching” or if they are not satisfied with the teacher. The exemplary subject - Chemical Engineering is quite small subject, e.g. Mathematics had 1 450 and General and Inorganic Chemistry I 1 272 enrolled students.

Similar situation is in case of participation two or more lecturers. If lectures are e.g. taught by two teachers (the first one lead 10 weeks and the second one 4 remaining ones) then students can evaluate both two lecturers. The system gives only general results; e.g. the first lecturer was evaluated by 7 students and the second one by 8 students from a group of 17 enrolled students. It is impossible to find if some students evaluated both teachers. The situation is much complicated in case of labs because here is common a cooperation of 5 or more employees (teachers). Each of them leads only some lab works and students are not obliged to realize all works. Therefore some students do not meet all teachers and it is impossible indirectly find the size of evaluators´ group.

Next problematic group are different projects and seminars, where is one guarantor but a lot of possible people who really lead students. Students can evaluate all teachers who are in a group of lecturers for the subject. Sometimes are therefore evaluated also teachers who did not lead any lecture or exercise.

For illustration the problem with small number of respondents it is used an example of Faculty of chemical Engineering, one of four Faculties at ICT Prague. In the winter semester 2012/2013 there was taught 141 subjects in this Faculty in 350 groups, excluding laboratories, seminars and semestral projects, because in these events it is impossible to find how many teachers student should evaluate. Table 2 shows the numbers of enrolled students in all Faculty departments (signed by numbers 401-407).

Table 2. Enrolled students at Faculty of Chemical Engineering

Download as

Veiw figure View Table

Table 3 shows the numbers of received feedback in the semester.

Table 3. Nr. of feedbacks at Faculty of Chemical Engineering

Download as

Veiw figure View Table

If we compare these Tables we can see e.g. that at the department 402 was taught 15 groups (11+2+1+1) with more than 20 enrolled students but only in 2 cases there were received feedbacks from more than 20 students per taught subject. The majority of evaluation comes from 1 up to 5 respondents. There is no evaluation of more than 50 students from the same taught group although at the Faculty there were 6 groups with more than 200 enrolled students.

Another example can give the view to history of evaluation. In Table 4 there are numbers of feedbacks for haphazardly chosen subject from the ones with higher number of enrolled students. It is Management of Enterprise Processes (MEP) taught at Faculty of Chemical Engineering. During the years there were changed teachers and amounts of taught group of students. Some teachers were leading the exercises only in one or two semesters, so in the Table 4 will be used initials only for the teacher, who taught the subject in more years. Under the symbols 1-5 are other teachers, always one symbol for one teacher.

Table 4. History of MEP evaluation

Download as

Veiw figure View Table

It is possible to see that after two, maximally three years the willingness of students to give feedbacks decrease a lot.

Apart from evaluation of education realized directly by universities, there are some informal evaluations. In the Czech Republic is probably the most popular source for informal evaluation of universities webpages “Primat”. Here it is possible to find not only evaluation of teachers and subjects from all Czech universities but also buy textbooks or notes from courses, find roommates or flats for rent etc. For each activity, for example subject evaluation, user receives a benefit – “bananas”. This is the currency for these webpages and it can be used for buy notes or preparations to exams, essays etc.

Also in this source the reliability of evaluation is very poor. The “Primat” statistics does not distinguish evaluation of lectures and exercises and use only general results for subject. If there are more teachers participated in a course, it is possible to receive only total number of evaluators and general evaluation for subject. For each teacher it exist information about general evaluation for the whole history of webpages. If someone would like to find teachers´ evaluation, he can receive only average grade for all taught subjects in the current year or general evaluation for each taught subject per whole history of webpages.

The positive but also problematic point of evaluation is the fact, that evaluator has to fulfill not only grades for 5 factors (clearness, approach, interest, usefulness and demandingness) but also write at least 50 symbols note.

4. Discussion

The problem with small number of students evaluated lecturers is possible to solve using mandatory evaluations. But there are some disadvantages, here. At one private Prague University the students were obliged to evaluate each lecture. After the lecture they were automatically asked for feedback in each login into University info system until they fulfill the feedback form. They had to give the feedback because without it they cannot continue use the system. The number of evaluations was close to 100 percent but validity was very low. Feedbacks gave also students they didn’t attend the lecture. Some of them chose for all items the best or the average rating but not all students. Author of this article had two times zero attendance in group consists of 10 enrolled students, but in feedbacks he found, for instance high level of “student´s interest of lecture” and sometimes two or three high ratings of items in set of average scored feedbacks (See Table 5). Therefore significant were particularly seldom added notes from students.

Table 5. Bottom up evaluation (no student was attended)

Download as

Veiw figure View Table