Abstract

This study examines the relationship between physical activity (PA) levels and academic achievement and student behavior in schools that completed a three-year initiative to increase and improve PA for students. Using difference-in-difference and discontinuity analyses, we find schools that participated in a PA intervention saw greater academic achievement during the three years of PA programming as compared to non-participating schools. The significant effects of the intervention were realized during the first year of programming; subsequent years saw stable achievement but not significant growth. There did not, however, appear to be differences in maladaptive student behavior between schools based on participation in the PA intervention.

1. Introduction

This study examines the relationship between physical activity (PA) levels and academic achievement and student behavior in U.S. schools that completed a three-year initiative to increase and improve PA opportunities for students. In recent years, school leaders and policymakers in the United States have paid increasing interest to children’s levels of PA in school. Although much of the interest is in the relationship between PA and health, the link between PA and other outcomes has also seen growing attention. This recent interest is in contrast to a trend in less school-based PA that began almost 20 years ago.

With the passage of No Child Left Behind (NCLB) legislation in 2002 (P.L. No. 107-110, Section 115) by the U.S. Congress, schools increasingly focused on academic achievement and standardized test scores and placed less value on, among other things, PA through physical education or recess ^{1, 2, 3}. Yet, evidence suggests devaluing physical education may be counter-productive with respect to increasing academic achievement, as an increasing number of studies find PA can contribute to improved academic performance. In 2010, the Centers for Disease Control and Prevention (CDC) reviewed the related literature and found half of the associations between PA and academic achievement to be positive and almost half of the other associations reporting null relationships ⁴. Only a small portion of the studies found a negative association between PA and academic achievement. Moreover, a meta-analysis published in 2017 concluded PA improves mathematics, reading, and general composite scores in youth ⁵.

While evidence suggests providing PA breaks during the school day can improve academic performance, there is less research to show PA decreases maladaptive behavior of students (i.e., behaviors that lead to detentions, suspensions, expulsions, etc.). Several studies ^{6, 7} have found PA breaks during the school day improve on-task classroom behavior (e.g., working, fidgeting). However, there is no research of which we are aware that has assessed the impact of PA during the school day on the rate of more extreme behavior problems. This is a striking omission given the clear need for interventions to reduce aggression, bullying, violence, and other behaviors that disrupt the learning environment and lead to serious disciplinary actions such as suspensions and expulsions. This illustrates yet another potential negative implication from a move away from opportunities for PA in schools.

The post-NCLB trend away from school-based PA is also in contrast to state-level policies about physical education. All states in the United States have policies that require schools to provide some form of physical education, although policies vary by state in the number of minutes and the type of activity that must be provided. In Colorado-the state in which the present study occurred-House Bill 1069 established a minimum requirement for the amount of time each elementary school must provide students in the form of opportunities to be physically active during the school day (150 minutes a week). Schools were required to comply with this mandate starting in the 2011-12 academic year.

To help schools achieve this, Kaiser Permanente Colorado-a large nonprofit health insurance provider in the United States-funded school districts through their Thriving Schools (TS) initiative to increase opportunities for PA in schools. Spanning school years 2015-2017, the goal of this initiative was for schools to provide at least 30 minutes of PA a day (or 150 minutes a week) through before, during, and after school programming. Funded districts selected priority schools and used the funding to provide professional development for physical education teachers, recess monitors, and classroom teachers and staff about how to increase opportunities for students and staff to be active during school hours (e.g., quality physical education, in-class activity breaks, or movement-based brain breaks) and to be active before or after school (e.g., walking school buses, intramural, or PA club sports). For more information about the intervention, see ⁸. The purpose of this study was to assess if schools in the TS initiative demonstrated improved academic performance and reduced maladaptive student behavior compared to schools not participating in the initiative.

The unique contribution of our study is four-fold. First, we examine the results of a PA initiative that was widely implemented across many schools and districts in one state. Many prior studies examine the effects of PA in more limited settings. Second, the analysis considered results from a PA initiative that was implemented over a three-year period; many prior studies considered short program implementations. Third, we examine a unique dependent measure-maladaptive student behavior-that appears to be essentially unexamined. Finally, our design and analyses are unique-a pre-post design with three years of pre-program data and three years of in-program data comparing intervention to non-intervention schools using difference-in-difference and discontinuity analyses.

2. Methods

The primary question guiding our research is: Is there a significant difference in student achievement and behavior between schools that implemented a structured PA program and those that did not? Or more specifically, did schools in the Thriving Schools initiative demonstrate improved academic performance and fewer disciplinary actions compared to schools not participating in the PA initiative?

The unit of analysis for this study is pre K-12 public schools. The sample includes all non-alternative public schools in Colorado for the school years 2012 through 2017. Colorado is a mountain-west state with a population of approximately 5.6 million residents, about 850,000 of which are school-aged children. The state educates its pre Kindergarten through 12^th grade children through 178 public school districts operating approximately 1,800 schools.

The treatment schools are those that participated in the aforementioned TS program, while control schools did not. We excluded from the sample any alternative public schools, such as correctional or psychiatric facilities, etc. The final sample included 1,726 schools for the achievement data and 1,699 for the discipline data-100 TS schools and 1,626 non-TS schools in achievement and 1,599 non-TS schools in discipline. According to the achievement data, the total number of students served by these schools ranged from 50,316 in 2012 to 52,370 in 2017 for TS schools and 737,305 in 2012 to 818,969 in 2017 for non-TS schools.

Table 1 includes descriptive statistics for the sample using the achievement data, disaggregated by treatment and control schools. Means and standard deviations are derived across years for each group. For example, the mean size of TS schools for the years 2012 through 2017 was almost 509 students. As indicated, the mean enrollments for all schools across all years were similar between groups-approximately 500 students. The percentage of students qualifying for the free and reduced lunch program was even more similar at about 23%. The percentage of minority students served was somewhat greater for non-TS schools, at 43%, although the difference on this indicator was significant (p=.03). Thus, at least on the demographic variables used in the analyses below, the schools were quite similar.

On one of the study’s most important variables-mean minutes per day spent in PA-we do not know definitively how much the TS schools differed from non-TS schools. We know TS schools offered a mean of 55.5 minutes of PA per day (SD = 3.0), but because non-TS schools were not part of the study, data collection related to PA was not possible in those schools. Yet, because of the ubiquity of recess and PE classes, we believe it is safe to assume non-TS schools offered some amount of PA to students. And given Colorado law requiring an average of at least 30 minutes per day, a reasonable assumption could be that during the initiative period (2015-2017), TS schools offered, on average, about 25 minutes more PA per day than non-TS schools.

Almost all of the study data were drawn from statistics reported by the Colorado Department of Education (CDE) on its website. The school discipline data were provided by CDE upon request. The variables used in the analyses are listed in Table 2, with respective scales of measurement. Details for each are self-evident, save for ELA, Math, and the discipline variables. Like all other states, Colorado tests students annually in English Language Arts (ELA) and Math, and mean scale scores are reported by the state for every school. These scale scores could not be used in their original forms, however, because Colorado began using a new assessment system in 2015, and scores were not scaled for comparisons between the old and new assessment. Therefore, we transformed school scale scores into standard scores (z-scores) for analyses. As for the discipline variables, these represent the percentage of the student body receiving said disciplinary actions.

We included enrollment, percent free and reduced lunch (FRL)-a measure of family poverty-and percent racial/ethnic minority because of previous research demonstrating significant relationships between these variables and student/school achievement or student behavior ^{9, 10, 11, 12, 13, 14}. Therefore, these variables may confound the relationship between TS participation and achievement or behavior. Moreover, as discussed above, the TS intervention was not randomly assigned, making our analyses quasi-experimental. Even though we use strong analytical methods that help overcome omitted variable bias from time invariant variables, these three variables are time varying and can be included in the models below to control for their effects and improve model estimates.

Our study used two primary analyses to answer the research question-difference-in-difference analysis and discontinuity analysis.

Using three years of data pre TS and three years during TS enabled us to examine if the TS schools reported a significant difference in pre to post achievement and disciplinary actions between periods as compared to the non-TS schools. As such the general model took the form:

where δ = year fixed effects, Y = ELA, Math, Removed from class, In school suspension, Out of school suspension, Expulsion, Law enforcement referral, Other actions, or Unduplicated count and all other variables are defined as above.

We ran this model with and without school fixed effects; results from both iterations are reported below. Note that this model estimates average effects when comparing the pre to post periods. That is, the outcome measures are averaged in the pre and post periods separately and then compared. This model does not measure change within the pre or post periods.

Unlike the difference-in-difference analysis, the discontinuity analysis enables us to measure if the slopes in the pre and post periods differ between TS and non-TS schools. This model takes the form:

Given the coding described in Table 2, and consistent with Singer and Willett ¹⁵, this model examines the effect of the intervention at both the time of its introduction (the TSschool*intercept term) and also the effect of the intervention in the years after the intervention (the TSschool*slope term). If, for the TS schools compared to the non-TS schools, the intervention results in an immediate change in test scores or disciplinary actions or a change over time after the introduction of the PA program, the respective regression terms-β₉ or β₁₀-will be significant. As with the difference-in-difference model, we ran this model with and without school fixed effects; results from both iterations are reported below.

3. Results

We begin by presenting the achievement trends disaggregated by TS and non-TS schools. As illustrated in Figure 1 and Figure 2, trends were quite similar in ELA and math-TS schools showed achievement levels less than those of non-TS school in the pre-intervention period. In the first year of implementation TS schools saw sharp increases in achievement, followed by stable achievement levels in subsequent years. The trends of non-TS schools showed generally stable achievement levels across all years of the study, with no sharp increases or decreases in achievement. Achievement gaps between TS and non-TS schools did not close entirely during the PA implementation period, but they did significantly narrow during that time.

Statistical analyses largely confirm what is displayed in Figure 1 and Figure 2. First, we report results from the difference-in-difference analysis. Table 3 includes findings from ELA and math analyses. For each test type, Model 1 reports results without school fixed effects, and Model 2 includes school fixed effects. As indicated, the differences between models are trivial.

For both ELA and math, the variable of primary interest is the TSXprepost interaction. This indicates if the pre-post differences evident in Figure 1 and Figure 2 were greater for TS schools as compared to non-TS schools. Results for both ELA and math show there are significant differences between TS and non-TS schools. Moreover, the positive coefficients indicate the pre-post difference for TS schools were greater than those in non-TS schools, confirming what is plainly evident in the figures. Thus, TS schools saw, on average, significantly greater growth in ELA and math scores between the three years before the implementation of the TS PA programs and the three years of TS implementation.

Turning to the discontinuity analysis, Table 4 includes results for ELA and math. As in Table 3, two models are reported in Table 4-Model 1 without school fixed effects and Model 2 with school fixed effects. And as in Table 3, the differences in results based in fixed effects are trivial.

In this analysis, there are two variables of interest-TSXintercept and TSXslope. For both ELA and math, results for TSXintercept are statistically significant, and the coefficients are positive. This means at the end of the first year of the PA intervention (2015), TS schools saw an immediate and significant increase in ELA and math achievement compared to non-TS schools, which is demonstrated in the figures. The TS schools did not, however, see a comparatively greater increase in achievement in the years following the implementation of the PA programs. In addition to the trend lines in the figures, this is also indicated by the non-significant TSXslope interaction variables for both ELA and math. It is also important to note that while TS schools did not see comparatively greater achievement growth after the implementation of PA, they also did not see a decrease in achievement. Thus, PA programs appeared to produce immediate effects, which then remained stable over subsequent years.

Unlike with student achievement, the relationship between PA and disciplinary actions was not significant in either model. As with achievement, we begin with graphical presentations of the trends for the seven discipline types. Figure 3, Figure 4, Figure 5, Figure 6, Figure 7, Figure 8, and Figure 9 illustrate that with the exception of Removal from Class, the trend lines for TS and non-TS schools generally follow similar directions. More specifically, TS schools do not show divergence from non-TS schools following the implementation of PA programs in 2015. As for Removal from Class, although the TS trend differs from non-TS, there does not appear to be a divergence in the trends that could reasonably be associated with the implementation of PA programs from 2015 through 2017.

When subjected to difference-in-difference analysis, results for the disciplinary actions pre and post do not appear to differ between TS and non-TS schools. Table 5 includes primary results for all seven discipline measures (covariates are omitted for space but are available from the authors), and the interaction variable for all outcome variables indicates non-significance for all. This was true whether or not models included school fixed effects.

Results from the discontinuity analyses tell much the same story. The variables of specific interest include the interactions of TSpartXnewinterc and TSpartXslope. Across all models, these two variables are consistently not significant, meaning trends before and after the implementation of PA programs did not differ between TS and non-TS schools. Again, this was true with and without school fixed effects.

4. Discussion and Conclusion

This study examined the relationship between PA levels, academic achievement, and student behavior, as represented by disciplinary actions, in schools that completed a three-year initiative to increase and improve physical activities for students. Results from difference-in-difference and discontinuity analyses indicated schools that implemented the PA programs saw greater achievement in the years of PA implementation compared to schools that did not participate in the initiative. There were no differences, however, in any of the behavioral measures.

Such results contribute to the growing body of evidence suggesting a positive relationship between PA and achievement. What made these results particularly important in their contribution to the related literature were the causal estimates generated through difference-in-difference and discontinuity analyses. Because random assignment is rare in studies of PA programs, most studies can report only correlational findings. But the use of more rigorous econometrics with panel data enabled us to discuss effects that are more akin to causal estimates.

What made the results for TS schools even more striking was the sharp increase in achievement in the first year of PA program implementation (2015) and that it occurred consistently in both reading and math. Recall that 2015 was also the year when Colorado implemented a new achievement test for its public schools. While it is conceivable a notable difference in test scores could have simply been an artifact of new tests, we would expect to see this effect present in all schools, not just TS schools. Yet, the fact that TS schools saw an increase in test scores in the first year of new tests and PA implementation, while non-TS schools did not, strongly suggests a real effect from the PA programs.

But what is to be made of the significant effects in year one of the PA programming but the lack of achievement growth in subsequent years? The effects of any intervention can, of course, take several forms-an immediate effect with no subsequent increase or decrease in an outcome, an immediate effect with a subsequent increase or decrease, or a gradual increase or decrease over time but no immediate effect. The prior research to date in this space has been dominated by examinations of short-term outcomes, so there is little evidence or theory to suggest which of the three types of effects would be expected. Our finding of immediate effect and then no subsequent increase or decrease may reflect a ceiling on the effects of PA on achievement. Or, like many interventions, the introduction of a new PA program may have a ‘novelty effect’ that creates a new baseline but does not result in continued growth in the outcome. Whatever the reason, it is a unique finding in this literature and worthy of additional research.

By way of implications, these findings suggest schools that reduce or eliminate PA opportunities in their schools to increase academic ‘seat time,’ may be undermining their efforts to increase student achievement. Instead, maintaining or even increasing PA may assist in focusing student learning and increasing achievement levels. Moreover, we note that the TS schools in this study reported offering PA at a frequency close to the levels recommended by the Physical Activity Guidelines for America (i.e., 60 minutes per day). TS schools also offered a variety of PA opportunities, including PE and recess, before and after school programs, and in-class activities. This was consistent with the CDCs recommendation of a Comprehensive School Physical Activity Program (CSPAP) that includes PA before, during, and after school ¹⁶. For schools considering the implementation of PA programming for the purpose, in part, of helping to increase student achievement, these results suggest the Physical Activity Guidelines for America and the CDC’s CSPAP recommendations are a fine place to start.

Turning to the findings concerning disciplinary actions, it may be tempting to conclude PA is not an efficacious intervention to address negative student behavior. We caution, however, against drawing strong conclusions based on this single study. As mentioned above, the literature linking PA to student behavior is essentially non-existent, and among the few related articles, the outcomes of interest are on-task classroom behavior ^{6, 7}. Whether students fidget in their seats is hardly the same thing as the type of behavior that leads to suspension, expulsion, or referral to law enforcement. Additionally, increased PA alone does not address the multitude of psychological and emotional needs that are often precursors to students’ externalizing behaviors. Our analysis makes an important, unique empirical contribution to understanding the relationship between PA and dysfunctional student behavior. Like any topic, a consensus of research findings is important before discussing implications and drawing firm conclusions. We hope others will pursue studies of their own to consider this important question.

Funding

This work was supported by the Kaiser Foundation Health Plan of Colorado under grant number RNG200313.

Conflict of Interest

The authors declare that they have no conflict of interest.

References