Abstract

Socio-emotional skills (SES) are a solid indicator of student success in the future (CASEL, 2016). However, in the area of evaluating the developmental level of SES in students hasn’t been quantified in research. The SEStimate Checklist works to provide teachers with a quantifiable number in this area, so they can directly affect SES development in their classroom with interventions. This reliability and validity study of the SEStimate Checklist was conducted on 250 prekindergarten age (3-5 years of age) students. The checklist is filled out by the teacher and scored. Then the score is graphed against an aim line to determine level of SES needs. The teacher can then implement interventions to improve low areas within the classroom, small group, or on an individual need.

1. Introduction

Traditional preschool curricula have always valued the development of socio-emotional skills (SES). This is not in deference to cognitive development, but as a necessary and vital building block in what is a hierarchy of skill development ¹. Over the last decade preschool curricula has shifted in focus to more cognitive in scope. The change in educational philosophy to higher stakes testing has created a demand on students getting more academic training in preschool while sacrificing SES development. The Collaborative for Academic, Social, and Emotional Learning (CASEL) highlights five “core competencies” associated with SES, self-awareness, self-management, social awareness, relationship skills, and responsible decision-making ². Social-emotional skills are the knowledge, attitudes and skills necessary to understand and manage emotions, set and achieve positive goals, feel and show empathy for others, establish and maintain positive relationships, and make responsible decisions.

Many researchers have studied the long-term effect of positive early childhood experiences. The Perry Preschool Project and the Abecedarian Project concluded that preschool provides positive outcomes towards the reduction in violence, crime, teen pregnancy, substance abuse, and poverty ³ Early Childhood Education (ECE) researchers have delineated that these positive outcomes are attributed to children’s competence with SES ⁴. Existing research highlights the significance of ECE experiences in the childcare context in the realm of SES development. However, ways to establish SES needs is an area that is far less researched. The present study contributes to this understanding, through studying the reliability and validity of a new checklist to be used by teachers to estimate children’s SES development and needs. This study uses the SEStimate Checklist (Appendix 1). The SEStimate Checklist, created by System 1-2-3, is a quick estimate of a young child’s social and emotional competence. Each child’s score is then plotted on a classroom graph by age. The SEStimate and its corresponding graph (Table 1) has three major advantages: 1) efficient, 2) effective, and 3) tracks everyone.

2. Development of SEStimate

System 1-2-3, a Pennsylvania based organization, developed the SEStimate Checklist to help classroom teachers quantify the need for classroom, small group, or individual SES development. The 20-item checklist was written against the Pennsylvania State Early Learning Standards (PAKeys.org, 2016) ⁵ (Table 1).

In a local search the items can also be closely related to New York, Ohio, West Virginia, and Tennessee state early learning standards allowing SEStimate to be used in multiple states. In a broader search, it is expected that the items would fit or could be adjusted slightly to approach all 50 states early learning standards.

3. Validity of SEStimate

The SEStimate being designed on state early childhood standards sets the validity first by being underpinned with state standard needs. Second, the SEStimate was shown to five prekindergarten teachers, not involved in the reliability study, five school administrators, and five university professors of Early Childhood Education teacher preparation programs to look over the items to determine if they are scoring and reaching the needs of early childhood students. In reviewing the feedback given by all, there was a unanimous agreement that the 20 items the SEStimate rates are key to early childhood students’ development.

“I like the items and the get to the root of early childhood developmental needs very clearly”, Ohio State University professor.

Prekindergarten teacher, Pittsburgh Public Schools, “The items are right on point as to the skills needed by students to be successful.”

4. Efficiency

The SEStimate was designed to be completed in less than 3 minutes per child. Each child’s SEStimate is a single page with 20 prompts based upon early childhood standards for social and emotional development. The prompts are scored on a Lickert scale ranging from 1-10 (0, 1-3, 4-6, 7-9, 10), for the estimator to simply check. The SEStimate is an estimate based upon what the scorer has observed the child doing over time in context of the daily routine. Therefore, it can be completed at any time-before school, during rest time, and after school. For a class of 20 children, one hour of work will result in a graph like the one below (Table 2). This graph can be used for a remarkable amount of planning instruction and intervention as well as serve as a baseline prior to those efforts.

5. Effectiveness

At the classroom level, the SEStimate and its resulting graph were designed to support the planning and use of tiered approaches in addressing social and emotional developmental needs. Its format includes a dashed aim line and concern line to guide planning, while also indicating potential movement between levels of need. The classroom question is, “What can we do to help all children progress at least at the rate indicated by the slope of the aim line?” On the individual level, it estimates a child’s successful responses to naturally occurring opportunities. How many times is the child likely to demonstrate a skill given 10 chances to do so? This approach takes the skill out of isolation and into the context of the daily routine. Situating assessment in a classroom’s routine leads to practical instruction and intervention within that routine. The question for the teaching team in relation to an individual child of concern is, “How can we help the child progress at a rate that beats the slope of the aim line?”

6. Everyone

The most powerful part of this type of assessment is its inclusiveness. It provides a snap shot of the classroom. Traditionally, there is a tendency to isolate our focus on children with individual or Tier III needs. Ironically, seeing this big picture has been helpful to planning for children with the highest level of need. This occurs because we are looking at Johnny’s needs in relation to several other children we know, rather than a nationally normed sample. The question can be asked, “What is it that the other children can do, that Johnny is not doing, and how can we help him learn to do it?” Another benefit of this perspective of the big picture, while planning strategies for individual children, the responsibility of the classroom teaching team to meet everyone’s needs challenges support personnel to make recommendations. These recommendations can now practically be implemented in the context of a full room.

The participants for this study are 16 ECE teachers and 250 ECE children in a Midwestern city. The children were rated by their classroom teachers using the SEStimate Checklist to determine the reliability of the instrument. To establish reliability of the SEStimate Checklist teachers observed SES within the classroom, and rated each child based on a 1 to 5 scale over 20 items. The first step in the process was to calculate The Kaiser-Meyer-Olkin (KMO) Measure of Sampling Adequacy, as well as Bartlett's test of sphericity. A KMO above .7 is considered statistically acceptable. The SEStimate’s KMO value (Table 3) was .943 revealing the sample size of our study as acceptable. Bartlett’s test of Sphericity tests the hypothesis that the correlation matrix is an identity matrix. If the significance value is less than the alpha level, the data rejects the null hypothesis that the population matrix is an identity matrix. The Bartlett test on the SEStimate checklist data indicates a significance value of .000 (Table 3). This value is less than the alpha level determined, and requires the rejection of the null hypothesis. In the rejection of the null hypothesis, the conclusion to run a principal component analysis to establish uni-dimensionality is in order.

Using SPSS dimensionality was established first as uni-dimensional based upon a Principal Component Analysis (PCA). A PCA is used in qualitative research as a statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components. The PCA (Table 4) revealed that the eigenvalue for the first factor was very large (13.261) while other eigenvalues were less than 1.3. Additionally, the first factor accounted for 66% of the total variance. The values extracted indicate the data set being examined was uni-dimensional.

Scree Plot graphs of the data display the eigenvalues associated with a component or factor in descending order versus the number of the component or factor. Once uni-dimensionality was established SPSS was used to compute Cronbach’s Alpha for the entire checklist. Cronbach’s Alpha is a measure of internal consistency, meaning how closely related a set of items are as a group. An alpha value over .7 is considered statistically significant. The Cronbach’s Alpha value for the SEStimate Checklist is .958 (Table 5). Based on the values computed, the SEStimate Checklist is a reliable instrument to determine a child’s SES level for age 36 months through 72 months.

7. Item Validation

Having established a statistically significant reliability for the SEStimate Checklist the study also looked at item validation. To do this participating classroom teachers and classroom assistants were asked to evaluate using the SEStimate Checklist the same child at the same time without comparing results. Table 6 displays the values for all 20 items on the SEStimate Checklist scored on the same child by three separate teachers. The entire document should be in times New Roman font size 10. Paper title must be centered, bold, regular font size 20 and all with upper case. Author names must be centered, bold, regular font size 10 . Author affiliation must be regular font size 9. Email address must be centered, italic, font size 9. Recommended font sizes are shown in Table 1. No more than 3 levels of headings should be used. Level 1 heading must be left-justified, bold, regular font size 14 and numbered using Arabic numerals. Level 2 headings must be left-justified, bold, regular font size 12 and numbered as sub-heading (i.e 1.1). Level 3 heading must be left-justified, bold, italic font size 10 and numbered as sub-sub heading (i.e 1.1.1) and the first letter of each word capitalized.

Table 6 displays data supporting the 20 items over three different teachers for the same student are almost identical in scoring. Seven out of 20 items were scored differently across all three evaluators. Those scores were off by one, and all 7 had two of the three evaluators rate the same score. This scoring shows the SEStimate Checklist contains valid items, based on state standards of SES development, that are scored objectively instead of subjectively.

8. Conclusion

Based on the data collected and the analysis performed a strong, statistical significance for reliable usage of the SEStimate Checklist is revealed. The .958 Cronbach’s Alpha value speaks to the development of the checklist in a way that all teachers can use it to determine the SES development needs within the classroom. Looking at item validation, it appears the items can be scored across multiple evaluators in an objective manner. Future research on the SEStimate Checklist will be focused on a pretest-posttest analysis to determine growth over a school year, as well as determine effectiveness of SES interventions between control and experimental classrooms.

References