Abstract

This study utilized the concerns-based adoption model (CBAM) to understand faculty’s responses to the implementation of edTPA as a requirement for initial teacher certification. Two instruments were used to examine faculty’s behavioral and motivational response to the policy change. Faculty representing private and public institutions across the state (N = 56) responded to both instruments. Questionnaire items were analyzed by converting raw scores to percentiles, while open-ended responses were coded by stage of concern. Positive correlations between stages of concern and levels of integration indicated that engaging in a collaborative process of analyzing candidates’ scores possibly leads to a deeper understanding of edTPA as a construct and may help faculty make informed decisions about their emphasis on different aspects of edTPA in their courses. Faculty who were unconcerned or focused on personal issues were less likely to be involved in activities such as analyzing and making informed decisions utilizing student data from local and national scores. Faculty who integrated edTPA-like content into their courses were more focused on finding ways to manage time and resources related to edTPA and were more involved in collaboration with other faculty members. Opportunities for collective data analysis within institutions may have played an important role in faculty’s involvement in assessing personal strengths and areas of improvement in preparing candidates to pass edTPA in the consequential year.

1. Introduction

Many states have moved toward the use of edTPA as a certification and/or program completion requirement for teacher candidates. The Georgia Professional Standards Commission (GPSC) required all teacher-preparation programs to implement edTPA in programs in the year directly preceding edTPA becoming consequential for certification. In our study, we were interested in understanding the nature of changes that faculty members made in their courses in response to the mandated edTPA adoption and any concerns faculty members may have had related to this significant policy change.

The concerns-based adoption model (CBAM) provides a theoretical lens for understanding how concerns of policy implementers can influence the success or failure of an educational innovation ¹. According to the CBAM model, the construct of Level of Use (LoU) describes individuals’ actual behavioral response to change, and the construct of Stages of Concern (SoC) describes implementers’ attitudinal responses to change. The researchers in this study utilized an instrument developed from the LoU of an innovation protocol ² to understand faculty members’ innovations specifically related to edTPA. The edTPA Levels of Integration (LoI) survey was validated as a measure for determining the extent of edTPA-related initiatives used in the practice of faculty ³. The questionnaire for assessing faculty SoC ⁴ helped to identify the peak concerns faculty were experiencing while implementing edTPA along the sevenfold SoC continuum ^{5, 6}. Specifically, we addressed the following research questions:

1) During the year preceding edTPA becoming consequential, what were faculty’s LoI in relation to their personal practice?

2) During the year preceding edTPA becoming consequential, what were faculty’s SoC?

3) How did faculty’s SoC relate with the edTPA LoI during the implementation year?

2. Literature Review

The CBAM provides a theoretical lens for examining how concerns of participants can influence the success or failure of an educational innovation ^{1, 7, 8}. From Hall et al.’s ⁸ perspective, to be concerned is “. . . to be in a mentally aroused state about something. The intensity of the arousal will depend on the person’s past experiences and associations with the subject of the arousal, as well as how close to the persona and how immediate the stimulus is perceived to be” (p. 3). CBAM was initially developed in the 1970s as a model to analyze implementation of an innovation and the change process ⁹. Concerns about an innovation were proposed in the CBAM as one of the key diagnostic dimensions that change facilitators should consider in designing interventions. CBAM has been widely applied to study the process of implementing educational change ^{7, 10}. CBAM posits that the single-most important factor in any change process is the people involved in the change; therefore, facilitating change means understanding the existing attitudes and perceptions of those involved in the process ⁵. The underlying premise behind the CBAM is that “change is a process, not an event” ( ⁷, p. 234) that occurs in stages.

For researchers to determine whether a new educational policy initiative is successful, it is important to know how changes are being integrated by the key players in their practice. In our case, the faculty teaching methods and/or practicum courses for supervising student teaching in initial teacher education programs were the key implementers of initiatives in response to the new edTPA policy in our state. We utilized the edTPA Levels of Integration (LoI) instrument, which was adapted from the CBAM’s two constructs: (a) levels of use and (b) innovation configuration ⁷. The LoU construct complicates the use/non-use dichotomy often used to describe attempts at implementing innovations and provides varying levels and degrees of behaviors exhibited by the users of an innovation ⁵. The construct of innovation configuration (or innovation mapping) focuses specifically on the nature of content or activities undertaken during implementation of an innovation. Items for the edTPA LoI related to the extent to which faculty had been involved in the integration of edTPA (LoU) and included a listing of potential components of what such integration might entail (innovation configuration).

The edTPA LoI model follows a developmental trajectory. As faculty learn about edTPA, the interaction of the information they have and the degree to which they integrate change in their practice determines their movement along these developmental states. The LoI consists of items focusing on the integration of initiatives: (a) at the program level and (b) at the faculty level in courses. Given the national trends in adopting the edTPA as a high-stakes assessment, these components allow us to understand how edTPA is being integrated within the programs and within faculty’s personal practices. A LoI at the program level and at the faculty level within programs could be different. Institutions or programs might recommend implementation of edTPA initiatives, but personal philosophies, concerns, or other circumstances may obstruct integration of edTPA components in faculty practice. Conversely, an individual faculty member may be motivated to implement innovations in response to edTPA, but the overall acceptance for edTPA may be lesser at the program level.

Change advocates often seek to understand the progress of new innovations or policies by measuring how well the innovations are being executed. This focus on the innovations themselves tends to ignore a critical factor, the group of people charged with implementing the change. The SoC process spotlights the implementers of change (in our case, the faculty of methods and student teaching courses) and helps with identification of worries/concerns, attitudes, and perceptions of staff as they deal with the challenges of integrating the change in their personal practice ⁴.

According to CBAM, the SoC educators implementing changes experience occur across a continuum, peaking in a linear fashion ⁸. Individuals not experiencing concerns about implementing an innovation (Unconcerned Stage) may not be close enough to the implementation process or may be so involved in other activities that their concerns about those other duties take precedence. Crossing the continuum, educators progress through concerns related to obtaining information about the innovation (Informational Stage), dealing with personal concerns related to their roles and abilities (Personal Stage), managing technical issues (Management Stage), dealing with the consequences of the innovation (Consequences Stage), and with collaborating effectively (Collaboration Stage). The far end of the stages of concern continuum is the Refocusing Stage where individuals focus on how an innovation may be adapted, improved, or replaced to contribute to overall success.

Understanding teacher educators’ responses to policy changes, such as adoption of edTPA as a high-stakes assessment, can help teacher educators understand contextual factors that may impact educational reform efforts. Literature indicates that when teacher educators respond to policy/innovations as learning experiences and explore the teacher performance assessments as instructional tools, they can become leaders in the process and develop a “culture of inquiry” ^{11, 12, 13}. Drawing on the work of Dewey ¹⁴, Whittaker and Nelson ¹³ explain that inquiry is not a means to an end but rather the means leading to purposeful, on-going events which are meaningful for program continuity and which move teacher educators beyond compliance. Using an inquiry approach, faculty can develop common understandings about assessment, teaching, and learning; use data for curriculum mapping and program improvement; view their analyses of student work samples as authentic ways for revising course assignments; and become knowledgeable and responsible for program coursework ^{12, 15, 16, 17, 18, 19}.

While some educators espouse the benefits resulting from an inquiry approach, others have wrestled with tensions arising from edTPA implementation. One challenge faced by teacher educators has involved preparing candidates for rigorous requirements of edTPA without teaching to the test ^{17, 20}. Finding the balance between teaching components of the test to ensure candidate understanding of the tasks and edTPA language, while maintaining what has been seen as strong instruction and content in methods courses, can be difficult ^{17, 20, 21}. Others question the educative intent of the assessment and believe that when teacher performance assessments (TPAs) are high stakes, changes that occur in curriculum may take candidates’ attention away from learning pedagogy centered on problems that arise during practice ¹¹. Aligning beliefs and practices creates additional tensions for teacher educators when adjusting curriculum for the summative assessment with other valued perspectives on teaching and learning, including a critical pedagogy stance ^{20, 21, 22}.

To resolve some of these tensions, teacher educators have begun to recognize the need for an organizational structure within institutions to provide needed supports for teacher candidates in understanding the instructional tasks as well as how to manage technical procedures during the edTPA implementation process ¹⁶. They have proposed recommendations for “distributed leadership” between and across stakeholders (e.g., teacher educators, administrators, mentor teachers, edTPA coordinators/liaisons, etc.) to create a system of collaboration ( ¹⁹, p. 29).

This study, therefore, examines the process of edTPA policy rollout within institutions across the state, utilizing the edTPA LoI as well as CBAM’s survey instrument for implementers’ SoC. We sought to better understand the experiences and concerns of faculty as well as the extent to which they perceived edTPA to be integrated within their personal practices prior to the year when edTPA would be used for initial teacher licensure in the state. We were particularly interested in how faculty members’ SoC might relate to the extent of their LoI of edTPA activities. Such information could help educational leaders and administrators at state, institutional, and program levels to better support faculty in the change process, considering the high-stakes nature of this policy.

3. Methodology

This study utilized the concerns-based adoption model (CBAM; ^{1, 5}) as its theoretical framework in understanding the faculty’s response to a major policy change-the implementation of edTPA as a requirement for initial teacher certification. Two instruments, the edTPA Levels of Integration (LoI; ³) and the faculty Stages of Concern Questionnaire (SoCQ; ⁴), were used to examine faculty’s behavioral and motivational response to the policy change. The complete edTPA Levels of Integration (LoI) survey consists of 19 items, where one set of items focused on LoI at the program level and a second set focused on faculty’s personal LoI. For this study, we selected and used the five items from faculty’s personal LoI because we were specifically interested in identifying the relationship between edTPA faculty’s LoI in their personal practice. These items were:

LoI_1. Have you analyzed scores from Local Evaluation of portfolios to identify what you need to address in your course(s)?

LoI_2. Have you Integrated edTPA related content in your course lectures/seminars, discussions and/or activities?

LoI_3. Have you integrated technical knowledge and skills needed for edTPA portfolio construction in course lectures/seminars, discussions and/or activities?

LoI_4. Have you participated in professional development to understand the technical knowledge and skills needed to submit an edTPA portfolio?

LoI_5. Have you analyzed scores from National Scoring of portfolios to identify what you need to address in your course(s)?

The responses to these edTPA LoI items were rated on a 6-point Likert scale (1 – nonuse, 2 – orientation, 3 – preparation, 4 – mechanical use, 5 – refinement, and 6 – integration). The progression on the 6-level scale is developmental and reflects the stages individuals go through when faced with an innovation or change. This scale is modeled on Hall ⁷’s descriptions of faculty’s level of use of an innovation where levels 1–3 are within the realm of non-use but include different levels of awareness/preparation, while levels 4–6 represent varying levels of use.

The Stages of Concern Questionnaire (SoCQ; ⁴) measures concerns along seven stages (Awareness, Informational, Personal, Management, Consequence, Collaboration, and Refocusing). The survey consists of 35 items related to their concerns in relation to the implementation of this innovation, the requirement to prepare candidates for taking edTPA as a high-stakes assessment. As part of the survey, faculty were also asked to respond to two open-ended questions regarding any additional information they would like to share with respect to edTPA implementation.

In spring 2015, an edTPA LoI survey was distributed to 453 edTPA faculty teaching methods and student teaching courses across the state, and 145 responded. This sample was used for establishing the validity and internal consistency of the LoI instrument ³. For this purpose of this study, we sent the 145 respondents of the LoI survey the Stages of Concern Questionnaire (SoCQ) to understand their concerns related to edTPA implementation. We received 56 responses for SoCQ from instructors and university supervisors who were teaching methods or student teaching courses, representing private and public institutions across the state, as well was incorporating 17 different handbooks in their courses. To analyze the SoCQ data, first the survey responses for the 35 SoCQ items were analyzed by converting raw scores to percentiles for each set of times associated with particular stages. Graphing of percentiles for each stage was used to identify the peak-stage for each facilitator, in accordance with the user manual ⁴. Additionally, the responses to the open-ended questions were coded into one of the stages of concern. If stages had a substantial number of qualitative responses, these were further analyzed using a constant-comparative method ²³ to identify patterns within the stage. Next, we ran the frequency distribution for the five items on the edTPA LoI and the faculty members’ peak SoC. Subsequently, we ran a Pearson’s moment correlation coefficient for the peak SoC of the respondents and the selected 5 items at the personal level from the edTPA LoI survey.

4. Results

Below we provide (a) descriptive findings of the respondents’ personal edTPA LoI during the implementation year, (b) the SoC of respondents as indicated by our analysis of their peak SoC identified by the 35-item questionnaire and our analysis of their open-ended responses, and (c) an examination of the relationship of faculty members’ LoI and their SoC to determine potential associations between faculty concerns about edTPA with their utilization of edTPA in practice.

Of the 56 responses for the edTPA LoI survey, about 30% were at the refinement level across all five questions, indicating that the faculty had begun implementing edTPA and were adjusting based on the results of implementation (see Table 1). The second most frequent level of use for the integration into personal practice was at the mechanical level, which meant that the faculty were primarily focused on compliance with the requirements, followed by the level of nonuse, which indicated that they were not involved with edTPA, or did not know about it. Questions 1, 4, and 5 focused on analyzing local and national scores and participating in professional development in order to integrate edTPA into their courses, and these questions received the highest frequency of responses at the Nonuse level. Interestingly, question 2, which asked about the integration of edTPA content into lectures, discussions, and seminars, received the highest frequency of use by the faculty, with 27% rating it at the integration level, showing that they had collaborated with others and were drawing on professional resources to integrate edTPA into their practice. Thus, it appeared that integrating edTPA related content into their courses had been unavoidable for the faculty, even when they had not availed themselves of all the resources available for them to get a deeper understanding of edTPA and the policies surrounding it.

The SoCQ showed that the majority of the faculty were at the unconcerned stage (46%), showing little concern about edTPA policy or its implementation. For detailed descriptive statistics for faculty’s stages of concern, see Table 2. Participants were invited to respond to a concluding open-ended question asking if they had additional information they would like to share regarding the integration of edTPA in their courses or programs. While the SoCQ showed that most of the faculty were at the unconcerned stage (45%), showing little concern about edTPA policy or its implementation (see Table 2), most of the qualitative responses were within the personal stage, voicing philosophical disagreement with the mandate.

Unconcerned stage. The first SoC is unconcerned in which faculty are introduced to new information but show indifference to this new information. Close to half of the respondents on the SoCQ (45%) showed little or no concern toward edTPA implementation, and there were no qualitative responses to the open-ended question in this category. Faculty were aware of edTPA but were either focused elsewhere, had limited or no involvement with edTPA, or their concern was minimal.

Informational stage. In the informational stage, faculty are aware of the change and take a personal interest in learning about the innovation ⁷. SoCQ analysis indicated only 2 (4%) of the participants were at the informational stage of concern. Qualitative responses from four respondents indicated that they showed interest in the implementation of edTPA. For example, one faculty member shared: “I’m concerned with inconsistent information from around the state. GADA meetings reveal that one university allows students to complete edTPA during the semester before student teaching and the handbook for edTPA states it occurs during student teaching.” Another respondent wrote: “Information about how edTPA will improve teacher quality and evaluation would be appreciated.”

Personal stage. Faculty at the personal stage of concern note uncertainty about demands, their own personal ability to meet these demands, and their own role in relation to the innovation. While only 13% of the SoCQ responses were in the personal stage, most responses to the open-ended question were in the personal SoC. Two main themes of disagreement concerning edTPA implementation were identified: a) Pearson, Inc.’s involvement and b) incursion on academic freedom and professional autonomy, which includes teaching to the test.

Pearson, Inc.’s involvement. Participants viewed Pearson, Inc.’s involvement as another initiative in teacher preparation that will last for a couple of years and then be replaced by another. One participant commented: “It’s really kind of a joke, like many fads in education.” They also viewed edTPA as another financially lucrative venture for Pearson, Inc. that does not necessarily have students’ best interests in mind. For example, one faculty member wrote, “This is just another way for Pearson to monetarily exploit our pre-service teachers and waste everybody’s time. The sooner we get rid of this crap, the better off public education will be.” Another stated, “This one [edTPA] is time consuming and expensive and in the end, not particularly helpful to faculty or students.” Another stated: “edTPA is a Pearson product, and how this company has been able to lead states blindly into its adoption is particularly disquieting.”

Incursion on academic freedom. Faculty also voiced their concerns about the possibility of having to work longer hours to focus on guiding students through the edTPA process instead of being able to teach to the needs of the students. One faculty member remarked: “It is an indictment of Teacher Education and I am embarrassed to be a part of it.” Another participant voiced concerns about losing valued clinical supervisors because of the edTPA mandate, stating: “Clinical supervisors are caught in the middle between district curriculum mandates and edTPA requirements. This is forcing our students and university supervisors to conduct negotiations that could hamper or damage school partnerships.” Faculty were also concerned that curriculum would shift from a global perspective to a more state-centric paradigm. For example, one faculty member expressed it this way: “We are too Georgia-centric in our approach and are losing sight of a broader picture.”

Faculty autonomy was another concern: “As a faculty member, autonomy in my classroom is affected by Pearson.” And another faculty member stated, “I do not have enough time to fully concentrate on secondary methods or curriculum and assessment because now (under the prospect of negative employee evaluations and potential student lawsuits) I am required to tie everything in my college classes to edTPA.”

Management stage. During the management stage, faculty are concerned about issues such as time and logistics with their focus on edTPA instead of curriculum development. Eight (14%) of respondents indicated being at this SoC. For example, one faculty member noted how “demands of time taken to prepare students and monitor edTPA has increased since implementation.” Another voiced time-related concerns regarding students: “The major issue for my students during their Clinical Experience Semester was that completion of edTPA took much time away from the preparation time for planning and teaching the curriculum in the school where they were assigned.” In addition, faculty members thought that the top-down decision to implement edTPA in Georgia was too hasty. For example, one respondent wrote, “I am concerned that this process has gone too fast. I would like GA to adopt what TN is doing and score in house-at least for a few years until we are grounded in the process. We are expecting too much too fast.”

Consequence stage. Faculty center their attention on the impact an innovation is going to have on improving practices in the consequence stage of concern. Only one faculty member (2%) responding to the SoCQ indicated being at the consequence SoC. The qualitative responses indicated faculty were concerned “with the high stakes impact the edTPA has on future candidates’ licensure.” Faculty were also concerned about the added pressure edTPA places on teacher candidates and questioned if this additional pressure is “worth it.”

Collaboration stage. At the collaboration SoC, faculty focus on coordinating and cooperating with others to improve the implementation of innovations ⁷. Ten faculty who responded to the SoCQ (18%) were at the collaboration SoC. Qualitative data indicated that some faculty felt the collaboration component was in place and that they were ready to move forward: “We feel strongly we’ve worked out the ‘bugs’ to ensure successful implementation and pass rate.” For example, another respondent wrote, “I am interested in the types of university collaboration across disciplines at my university that is necessary for teacher candidates to best integrate the necessary skills for teaching that are reflected in the edTPA tasks.” Another reiterated this point by reflecting, “We need to coordinate more among various faculty, teaching similar courses to make sure students have a chance to do similar tasks to edTPA before they get to student teaching.”

Refocusing stage. Faculty at the refocusing SOC explore ideas about alternatives to an innovation, including changes to policy or initiatives (Hall, 2010). Implementers at this stage also look for ways to maximize the effectiveness of the innovation. Three faculty members (5%) responding to the SoCQ were at the refocusing stage; however, none of the responses on the open-ended question could be coded at this stage.

Correlation analysis found significant positive correlations between faculty SoC and LoI questions on analyzing local (r = .46, p < .01**) and national (r = .49, p < .01**) edTPA scores. These positive correlations indicated that faculty’s stage of concern increased as they integrated changes in courses which were informed by an analysis of the scores from local or national evaluation of portfolios. This finding is revealing because about 40% of faculty were within the nonuse realm (the first three levels within the LoI) for participating in professional development related to edTPA. Additionally, 37% of faculty had not taken advantage of analyzing local edTPA scores, and 47% had not examined national scores for their candidates (Table 1). In contrast, 65% of faculty were within the use realm of the LoI (the last three levels of LoI), mentioning that they implemented edTPA in their courses and assignments. There was also a significant positive relationship between faculty SoC and integrating edTPA-related content in the courses (r = 0.27, p < .01*), indicating that faculty progressed to a higher SoC as they integrated edTPA-related content into their courses (see Table 3).

5. Discussion

CBAM posits that faculty concerns may directly influence the extent to which an innovation is successfully implemented. In our study, faculty who were unconcerned or focused on personal issues were less likely to be involved in activities such as analyzing and making informed decisions utilizing student data from local and national scores. Those who were involved in those activities, and who integrated edTPA-like content into their courses, were more focused on finding ways to manage time and resources related to edTPA and to be involved in collaboration with other faculty members. Opportunities for collective data analysis within institutions may have played an important role in faculty’s involvement in assessing personal strengths and areas of improvement in preparing candidates to pass edTPA in the consequential year. Although attitudes about the policy influenced personal practice, institutional supports in turn influenced attitudes and helped faculty address developmental concerns. What follows is a discussion of significant concerns that existed among faculty regarding the purpose and validity of edTPA-primary among these the corporatization and standardization of teacher education and impediments to academic freedom-and our suggestions to ease faculty’s concerns.

Teacher educators are concerned that edTPA is another politicized reform that promotes the interests of large for-profit corporations (i.e., Pearson, Inc.) and seeks to standardize the teaching profession. School reformers in the mid-1990s moved toward a business model of innovation ²⁴, yet this top-down decision-making business model has made little improvement in student performance or teacher morale in K–12 schools. This business-oriented and politicized approach to school governance and reform have how creeped into the field of teacher education. Teaching is an art, yet edTPA reduces it to a standardized, procedural task ²⁵.

Conversations in teaching methods courses are shifting away from critical pedagogy and the politics of social justice to the testing procedures for success on edTPA ²⁶. Teacher educators have taught their students to avoid the pressure to teach to the test, but now these pedagogical experts are finding themselves pressured to teach to a test because their students’ certification depends on it. Teaching in higher education should be about expanding students’ horizons; however, in the age of edTPA, it has devolved into test preparation. Faculty in this study were vocal regarding their concerns and the need to rally against this push away from their academic freedom and underscored the importance of retaining their ability to make curricular and pedagogical choices in their own courses.

While faculty concerns highlight their focus on pushing back on the implementation of edTPA, our data also indicate how faculty engagement in specific initiatives has been found to be associated with moving them along the implementation spectrum to other stages of concern. A high-stakes policy implementation often brings with it a need for making major adjustments, personally and professionally. Where the edTPA policy was being implemented statewide, the responses from the participants indicated that such feelings were experienced across all institutions in the state irrespective of the edTPA handbook used for implementation.

Our finding that 45% of the faculty were at the unconcerned stage about edTPA during the implementation year of the policy was surprising. Positive correlations between SoC and LoI for analyzing local and national edTPA scores indicated that when faculty engage in a collaborative process of analyzing candidates’ scores, it possibly leads to a deeper understanding of edTPA as a construct, and it also may help them make informed decisions about their emphasis on different aspects of edTPA in their courses.

The second highest SoC for the participants in this study was collaboration, showing that 18% of the respondents were collaborating with others to improve edTPA implementation in their institution and beyond. Therefore, activities leading to collaborative engagement, such as analyzing candidate scores, possibly move faculty beyond the early SoC (Unconcern, Informational, and Personal) and provide collegial support in the period of transition to the new policy’s implementation. The combined understanding of the two constructs-SoC and LoI-gives insight into the faculty’s emotional response to change as well as the behavioral response to change, to see what faculty are doing in response to the edTPA policy within their courses and assignments, and what might be the underlying emotions for their actions. This information provides helpful feedback to policymakers at the state level as well as administrators at various institutions to purposefully provide support structures and opportunities for faculty engagement and professional development to alleviate concerns.

References