Abstract

This article aims to compare the perceptions of Brazilian and Portuguese teachers about the evaluation of teaching mediated by information and communication devices. It is a qualitative, explanatory study. The analysis of these responses considered six categories belonging to three dimensions created in the WebQDA qualitative analysis software and concludes that teachers still face institutonal and personal difficulties to insert the information and communication devices in teaching. It is also concluded that there is a strong need to research and discuss teacher assessment with teachers.

1. Introduction

The cultural production of humanity advances to a state of full mediation of the digital technologies of communication. In this context, these technologies have contributed to an increasingly large migration from everyday activities to virtual environments ¹, which makes the domain of technical knowledge and critical-reflexive understanding of the use of digital technologies of communication a relevant factor so that the subjects that integrate society can be fully inserted. Forming citizens who are capable of interpersonal and professional living in a knowledge-based society ² presupposes that technologies should be part of the educational process not only as a support or platform but as content and language.

These technologies expand educational possibilities and, in conjunction with skilled professionals and curriculum projects that integrate digital resources into the educational curriculum, can culminate in an educational process that addresses the needs of today's world in particular by enabling the production and access to contents and knowledge in an almost unrestricted way. By this characteristic they not only help the teaching pratice, but shape and disseminate ways of being, acting as devices ³.

The inclusion of the digital technologies of communication in teaching presupposes "accepting the challenge of change" ( ⁴: 51), which demands interest on the part of the teacher to develop technical-operational skills related to equipment and software. Knowing the deficiencies and potentialities of the teaching/technology relationship is a fundamental premise to ensure that the investment of time and money in training equipment and actions is not in vain, which is possible through the evaluation of teaching.

For more than a decade these evaluations have been carried out through indicators. They are developed by governments and institutions, but always based on general indicators, which are not focused on specific aspects of teaching. This article presents a qualitative research report that aimed to compare the perceptions of Brazilian and Portuguese teachers about the evaluation of teaching mediated by information and communication devices. It contains a comparative analysis of the answers given by teachers of these two countries and obtained through online questionnaires answered voluntarily. The data analysis was developed with the support of the WebQDA qualitative analysis software ⁵.

2. Materials and Methods

The present study aims to compare the perceptions of Brazilian and Portuguese teachers about the evaluation of teaching mediated by information and communication devices. The research was constructed from a qualitative approach and has explanatory nature, since it is focused on the analysis of factors that influence the occurrence of social phenomena ^{6, 7}. Discussions take into account responses provided by two groups of teachers. The first one was formed by 17 teachers from the city of Aveiro and environs, central region of Portugal and the second by 266 teachers from the state of Sergipe, northeast region of Brazil.

These teachers teach in public elementary school classes and have specialization for the teaching work mediated by information and communication devices. All of them have participated in projects involving the use of technologies in the classroom, the E-school Project in Portugal and Proinfo in Brazil. These teachers collaborated with an investigation carried out in 2011 in which they voluntarily answered a questionnaire pointing out indicators they considered as more qualified for the assessment of teaching mediated by digital technologies of communication. This questionnaire had a subjective question, which was not mandatory and allowed not only the exposition of positions and opinions, but also the suggestion of other indicators. Of all 17 participants, five answered this question.

The second group consisted of 266 teachers from the State Public Network of the Sergipe state, in the northeastern region of Brazil, who underwent continuous training for the pedagogical uses of information and communication devices and work in network units with three technological inclusion projects in operation. This second group collaborated with a survey conducted in 2016, which applied the indicators obtained from the research that was developed in Portugal and described previously. In this second case the questionnaire also had a subjective question for opinions and indications, not mandatory. In this research, 16 teachers answered the subjective question.

The participation of the teachers who collaborated with the research follows the pattern set out in the following table

The comparison of the answers was developed in the WebQDA qualitative analysis software ⁵. All of them were typed in .doc format, and inserted in the software, which advocates categorical analysis. To meet this need, three dimensions were created: a) observations on the questionnaire; b) comments on indicators; and c) observations about teaching mediated by information and communication devices. Each of these dimensions was divided into two categories of analysis. The first two dimensions received the critical categories or suggestions for the questionnaire and the indicators, respectively. In the dimension that gathers answers about teaching, the categories group answers that deal with the facilities or difficulties of teaching mediated by information and communication devices.

The option to construct a comparative analysis was based on the relation that this methodological proposal has with the investigations of the social field ^{8, 9}. For Burke ⁸ the adoption of this strategy has always been considered in studies about phenomena of society, especially because through comparison "we can see what is not there; [...] understand the importance of a specific absence" ( ⁸: 40). It is also necessary to emphasize that the comparative analysis presupposes the use of a "theoretical-methodological reference” ¹⁰ to guarantee the scientific rigor to the inferences and conclusions.

The choice of this reference for the discussion proposed here is another point that deserves to be highlighted, given that the present article is constructed from the perspective of multireferentiality, which proposes an analysis "under different points of view, implying both specific visions and languages appropriate to the descriptions Required, depending on different reference systems that is, heterogeneous" ( ¹¹: 24). This methodological option considers that the reflections should take into account aspects of the practice and that the studied phenomena are all inserted in complex contexts ^{12, 13}.

Based on this premise, the answers given by teachers in Brazil and Portugal were described and compared at two levels: between the two groups (Brazilian teachers and Portuguese teachers) so that similarities and divergences could be verified; And with the theoretical references that deal with evaluation of teaching and teaching mediated by information and communication devices.

3. Discussion

The constant relationship between individuals and technologies culminates in the emergence of needs that interfere with the ways of being and/or thinking and influence the subjects' training strategies, which are born and develop in specific historical moments and, according to Foucault ¹⁴, constitute from the self-care, a set of pedagogical and political techniques aimed at the knowledge about oneself and the other.

In spite of advocating full and free subjects, these techniques were incorporated by institutions and transformed into mechanisms of subjectivation, established by those of the utterances, discursive practices producing meaning ¹⁵ which are to interdict and exclude everything that opposes the idea of normality. In the relationship between the statements and the subject is that we find the device, by which the statements reach and act with the subjects.

Deleuze ^{16, 17} argues that the device should be understood as a multilinear set of lines of visibility, force and enunciation that intersect, overlap and contradict from three dimensions: a) production of knowledge; b) power strategy; and c) construction of subjectivity ^{16, 17}. The characteristics described here can be attributed to the digital technologies of communication, which justifies the assignment of the term information device to these technologies.

The constancy of information and communication devices in subjects' daily lives, as well as their impact on cultural production, is already perceived by governments of different countries. The issue to be considered is that these programs, while ensuring access to information and communication devices in schools, have not translated into learning outcomes considered to be satisfactory in international examinations and assessments.

There are several reasons for this phenomenon: school management, availability of initial and continuing education, access to technologies outside the school environment ^{18, 19}. However, one of the main factors that can contribute to the configuration of the current scenario regarding the insertion of the information and communication devices in the educational process is the impact that the technologies produces in the teaching.

Even with the advent and consolidation of the information and communication devices and all the resources of production and access to knowledge from them, the teacher continues to be a basic figure in the personal and, mainly, professional training of the subjects. Understanding how they have assimilated the information and communication devices in teaching is an action of strategic importance and one of the consolidated ways to verify the impacts of technologies in teaching is through the evaluation by indicators.

In the last decade the evaluation of teaching based on the use of indicators has come to be considered as standard, especially after the publication of the Wisis Outcome Document ²⁰. The methodology adopted by Unesco ²⁰, which used indicators, is justified by the possibility of using such indicators to develop diagnoses that consider a large number of teachers.

Indicators allow the establishment of a relationship between data and contribute to objective evaluations ²¹. They are inductive and through statistical samples are able to characterize a large universe of elements ²². They are essentially quantitative, but they make up qualitative research instruments, because they are "a value, a magnitude or a criterion that has meaning" ( ²³: 1).

With the adoption of indicators in the evaluation of teaching mediated by digital technologies of communication, there was an increase in the number of studies on this object. Some of them, such as the studies developed by ^{24, 25} and ²⁶, are aimed at the elaboration of indicators based on the analysis of the composition, nature and application of indicators created, tested and already validated by Institutions.

The studies considered as references in the evaluation of the impacts of information and communication devices in teaching are those developed by institutions such as the ^{27, 28, 29, 30} and made public through dissemination of reports in virtual environments.

The criticism here is that specific indicators on teaching are lacking at all stages. In other words, the teaching mediated by the information and communication devices has been evaluated from aspects that only touch on the teaching activities. In order to collaborate with this discussion, two convergent researches were carried out, one with Portuguese professors and the other with Brazilian professors, in which these teachers were able to discuss the subject, to give an opinion about teaching mediated by information and communication devices and to suggest indicators that, in the vision Of them, better evaluate the pedagogical use of the teacher. These perceptions are used for the comparative analysis of this article.

4. Results

Considering the categorization constructed in the WebQDA ⁵ it is noticeable that the study of teacher evaluation is still a matter to be discussed and disseminated among the teachers themselves ³¹. In all 22 responses there were 26 occurrences of the composite analysis categories for this research, of which only six were specifically about indicators or evaluation. Most of the occurrences of the categories are in the dimension on teaching, totaling 16. Only four teachers spoke on the research questionnaire.

In general, teachers used the subjective question to express opinions and report difficulties of teaching mediated by information and communication devices. This approach was most evident in the responses of Brazilian teachers. The Portuguese responses bring more contributions to teacher evaluation indicators, although these contributions are pointed out by only one teacher.

Of the 17 Portuguese teachers who participated in the research, five answered the subjective question. Of these, four effectively contributed to the discussion, since teacher 4 did not use the space of the question for answers in accordance with the requested one. He stated that he had no interest in collaborating, as stated in the reply:

Teacher 4 "Sorry for this text, but there are few people who talk about it in schools. Votes from a good job"

The dimension of the questionnaire had as its first category that of suggestions. In Portugal she is present in the teacher's response 5.

Professor 5 "I think the questionnaire will be able to evaluate six main questions: Profile of gender and age of the sample, professional situation of the sample, use of ICT at a personal level, use of ICT at the professional level, ICT training, Use of ICT in teaching/learning"

This answer makes clear the position that teacher assessment needs to be focused on the teacher figure. For Professor 5 to be able to evaluate the pedagogical use of information and communication devices, it is necessary to know the social categorization of teachers who have developed experiences of this use, as well as the ways of using information and communication devices at different levels. This perception is confirmed when analyzing the categories of the size of indicators. In the Portuguese case only suggestions were registered, since teachers 1 and 5 did not criticize the indicators.

Professor 1 "Modalities for the use of ICTs, regardless of the general status and practices of colleagues (whether or not they use ICT) or the" trendy "currents, countercurrents may sometimes be" good "ICT users for Pedagogical? "

Professor 5 "If the purpose of the survey is to analyze the level of penetration of teaching / learning technologies, to understand what types of training are needed in the area. [...] About the use of ICT at a personal level, if educators and teachers have computer equipment and have an Internet connection at home? [...] As regards the use of the Internet: for reading and sending e-mail? [...] Use of ICT at a professional level rectangular use of ICT. Class preparation In carrying out administrative tasks? During class Extracurricular activities? "

The considerations on the indicators show three aspects of the perception of Portuguese teachers about the evaluation of teaching: for them, the analysis of the pedagogical uses of information communication devices is directly related to personal uses, which should be considered in the evaluation; It is also necessary to think about the supply and the processes of continuous training; And the internet is a fundamental element of technology-mediated teaching, which brings the view of teachers closer to that of ¹.

The last dimension of analysis, which gathers the answers about teaching in specific, is found in two answers. Teachers 2 and 3 pointed out difficulties in the teaching work that takes into account the use of information and communication devices.

Teacher 2 "I've never been enthused... a lot of preparation work for potential results"

Professor 3 "Personally, it seems to me that I can never really use ICT in a meaningful way to use its true potential"

The answer from Professor 2 discusses a relevant issue when discussing teaching: the distribution of time between the various stages of this professional activity. Teaching is a process that involves not only the class, but the study, research, planning, practice and evaluation. When Professor 2 speaks in "a lot of work" he calls attention to a characteristic already verified by ³². Using the information and communication devices implies technical preparation through courses and personal uses that are usually carried out at times opposed to the school, which could be intended for leisure and rest of the teachers.

This is one of the reasons for resistance presented by many teachers, especially considering the content of Professor 3's response, which highlights a certain lack of knowledge about the impact that technology has on education in the eyes of teachers.

In the Brazilian case the configuration of the responses was different both in terms of quantity and content. In total, 16 teachers answered the subjective question and the categories appeared 17 times in these answers. The most recurrent dimension was that of considerations about teaching, present in 15 answers and no teacher suggested or criticized the indicators.

The categories about the questionnaire appear in only one answer, that of Professor 16, who criticized the questionnaire and suggested some changes.

Teacher 16 "There is a need to include a previous diagnosis and evaluation about the conditions of the availability of ICT in the school environment. I suggest that the basics of the legend be re-evaluated. They induce a dissonant response to reality "

When suggesting changes in the legend of the questionnaire, Professor 16 makes a reference to the indicators that composed the closed stage of the questionnaire, answered by Likert Scale. The contribution is relevant, especially if we consider that the questionnaire was answered by 266 teachers, but only one has critically positioned itself in relation to the format and content.

The answer also shows that for this teacher it is necessary to consider the provision of equipment in schools when evaluating teaching mediated by information and communication devices, a recurring aspect in the other teachers' responses.

The availability of equipment and internet access were the themes that most appeared in the responses of the teaching dimension, more specifically in the category that gathers answers about the difficulties of teaching mediated by information and communication devices. This category appeared in 10 moments, in responses from Teachers 1, 2, 3, 8, 9, 10, 11, 14 and 15.

Teacher 1 "There are two barriers to ICT in public education: when there is equipment, there is a lack of skill; When there is skill, the "

Teacher 2 "What is missing in schools is a computer lab with active internet"

Professor 3 "Technologies have much to offer pedagogical practices, but there is a problem that needs to be urgently solved: The internet in schools"

Teacher 8 "for teaching in a public school, with limited resources and all problems that involve public education in that country, unfortunately, I can not use them with my students"

Teacher 9 "order to insert the technology the teacher has to bring in particular to use"

Professor 10 "Unfortunately the use of ICTs in public schools is not always possible. There are flaws in internet connections and access to students is not allowed because of the fear of the system "

Teacher 11 "I still feel resistant to insertion of some IT resources. Possibly because of the lack of resources and time to develop them. "

Teacher 14 "Unfortunately, access to computers and the internet at school is limited even in classrooms"

Teacher 15 "I agree and make my plans using ICT, but it is not feasible to put it into practice, because the school does not constantly have a maintenance on the computers."

According to these answers, the main obstacle to the consolidation of information and communication devices in teaching is the lack of provision in schools. There are reports of difficulties in access to computer labs, insufficient maintenance and training and time issues. Although they are directly related to the evaluation, these aspects should be considered when evaluating teaching, especially considering the evaluation as a practice of analyzing specific phenomena and contexts.

The second category of this dimension, addressed to the considerations on the facilities of teaching mediated by information and communication devices, is present in the answers of Professors 4, 5, 7, 12 and 13.

Teacher 4 "I use a lot to improve my classes, I search a lot"

Professor 5 "The use of ICT as a support in the development of classes, content and, above all, learning is undeniable"

Teacher 7 "The use of ICT makes the teaching / learning process less cumbersome and more enjoyable for both teachers and students"

Teacher 12 "Working with ICT facilitates teacher work and promotes student learning"

Professor 13 "I consider the use of audiovisual resources in teaching learning, especially in this century in which the Internet and social networks seem more attractive to our students."

Teachers who spoke about the facilities of teaching mediated by information and communication devices report that these elements serve as facilitators of teaching, helping in research and planning. They are also important because, in the view of teachers, they help students to learn, especially by enabling them to make contact with the multiple languages common to the Knowledge Society ² and, thus, providing access to useful knowledge and languages for Insertion of the subjects.

The comparison between all the answers shows that there is little inference about the evaluation of teaching, which confirms the positioning that it is necessary to study, research and discuss this practice with teachers. In general, they only cite how they think they should be evaluated, but they do not comment or critique the evaluation formats or the already consolidated experiences.

Indicators are also not of interest to these teachers. Discussions on the use and models of available indicators are superficial, which may indicate little knowledge or even little interest in the subject. Regarding the teaching mediated by information and communication devices, the answers indicate that there are still significant obstacles to their consolidation. This problem is institutional, since the governments of both countries can invest in more availability of equipment, maintenance and training, but it is also personal, since teachers may have an interest in using technologies in the personal and professional spheres, always seeking a positioning critical and reflective.

What is clear with qualitative research that deals with the insertion of information and communication devices in teaching is that there is a discrepancy between practice and discourse. In speeches, teachers demonstrate understanding of the importance of technologies to today's society, but are still resilient and facing problems at a time when they could use those resources in teaching.

5. Results

From the methodological point of view, the realization of this comparative analysis from the perspective of multireferentiality allowed the deepening of a discussion that had already matured during two investigations carried out in the last five years and which deals with a complex social phenomenon, which is the insertion of the informational and communicational systems in educational processes.

The possibility of comparing results obtained in Brazil and Portugal was important for a description that considers two countries with different socioeconomic realities and contexts. Still in the field of procedures, the use of WebQDA was configured as important, especially since the software served as a facilitator for the arrangement, organization and categorization of the data collected with the questionnaires.

The responses of the two groups of teachers show that there are still gaps in the insertion of technologies in school and teaching. There is a shortage of equipment, a lack of critical understanding on the part of those involved in the educational process (teachers, managers and students) and there is little training. In relation specifically to the teachers, a strong resistance to the use of information and communication devices was verified, as a facilitator, if considered as support, but a problem if considered the aspects of content and language.

The problem is even greater in the evaluation of teaching mediated by information and communication devices. While this is already a well-established practice, especially with the use of indicators, it is still unknown by teachers, who have no contact with the subject at the training level and do not show interest in discussing the practices and indicators of evaluation.

The study makes it possible to conclude that there are differences in Portuguese approaches, which are more punctual, objective and in collaboration with the study and the Brazilian approaches, of personal reports. It is therefore evident that it is necessary to disseminate the importance of discussing not only the insertion of technologies in school and teaching, but also to assess this insertion in a timely manner.

Acknowledgements

To all the professors from Brazil and Portugual who collaborated with the research, as well as to the research supervisors described here. We also thank the National Council for Scientific and Technological Development and the Coordination for the Improvement of Higher Education Personnel.

References