The Effects of Class Size on Student Achievement: New Evidence from Population Variation

I. Introduction 

Class size reduction among schools received attention – and budget – in the United States. It became popular as parents favored small classes because it enables teachers to use the teaching method they prefer. Teachers also find them favorable since they call for less effort in delivering instruction. For teachers’ unions, small classes are beneficial because they augment the demand for teachers. Meanwhile, administrators find reduced class size well-disposed because they expand the size of their domain.

Class size, however, is an example of the education production function erroneous belief, which suggests that educational inputs systemically transform into achievement. This analogy does not hold for the reason that the firms’ production functions are not merely a result of their ability to turn inputs into outputs; it is a consequence of maximizing an objective given a production possibilities set. Class size is also difficult to analyze since variation in class size stems from the various choices that are made by parents, schooling providers, or courts and legislatures. Class size is correlated to other variables of student performance making the results biased; analysts, however, hardly ever explain where the variation really comes from.

Experiments are rarely explicit. The majority of them take place in developing countries and most of the time, primary participants are aware that there is an undergoing experiment. Some players of the study are cognizant that if the experiment becomes unsuccessful, the education policy will not be passed. Hence, some schools append incentives that will not be included when the policy is actually passed. Some individuals also enhance their productivity when they are being evaluated (Hawthorne effect). In addition, a few experiment participants attempt to undo the randomness of the experiment by assigning the best students or teachers in the test classes.

These empirical and economic concerns are addressed in this study using two identification strategies, both of which use variation in class size that comes from population variation. The first strategy uses natural randomness in the population and straightforward logic – making use of variation from the random timing of births, which corresponds directly into differences in class size between cohorts. The said identification strategy is employed by separating the random component of population variation using long panels of data on enrollment and kindergarten cohorts in Connecticut school districts. The long panels are capable of eliminating all smooth changes in population. Residuals that remain after fitting a quartic function of time are then used separately for each grade in each school.

The second strategy makes use of the fact that class size abruptly shoots when a class has to be added to or subtracted from a grade when enrollment triggers a maximum or minimum class size rule. This identification strategy acknowledges that there is a discontinuous connection between class size and enrollment at certain known levels of enrollment while there is a smooth relationship between achievement and the determinants of enrollment. The panels of data are utilized to detect small changes in enrollment that are associated with changes in the number of classes in each grade in each school. Meanwhile, each district’s class size rules are used to ascertain if adjustments in the number of classes are just the result of small changes in enrollment triggering a rule. This strategy is accomplished through comparing the class size and achievement of adjacent cohorts that go before and after each event.

When using population variation, the range of class size is conveniently the range that is relevant for policy. Also, schools are examined under the normal conditions that they usually experience.

II. Sources of Variation in School Inputs and the Potential for Bias

The major source of variation stems from school choice as a result of residence selection. Between-district school input differences that result from parent choices yield upward biased estimates of the input efficiency. Systematic variation within a district over time faces a similar effect. Simple comparisons of schools using cross-section data or time-series data may likely generate bias in favor of class size reductions since such reductions will seem to be more efficient than they really are if parents who are more involved to their children’s learning also choose school districts that offer smaller class sizes.

Empirical results suffer from different biases depending on the source of variation of the employed school inputs. Bias is ambiguous because parent involvement can either be compensatory or reinforcing. The bias generated by administrators and teachers can also be vague because it depends on whether they lean towards meeting parents’ demand for higher learning or attending to children with learning problems. Inputs are also determined by state judges and legislators who have the tendency to favor compensatory policies than reinforcing policies.

However, this paper attempts to explicitly formulate a source of exogenous variation, not just eliminate sources of bias.

III. Empirical Strategy

A general education function uses the natural log of class size, a vector of cohort indicator variables, a vector of school indicator variables, and a vector of observed student, parent, and community attributes to determine student achievement. Test scores, the common measure for student achievement, are divided by the standard deviation of students’ scores on the test to help understand unfamiliar test scores and make comparisons across different studies. Meanwhile, the natural log of class size is used to incorporate the fact that a one-student reduction is proportionately larger from a base of students. The vector of cohort indicator variables is taken into account to allow for tests that vary annually and to allow for teachers who adjust their teaching to a test’s content. The vector of school indicator variables is added to control for schools populations’ attributes that are constant across time. Lastly, the vector of observed student, parent, and community characteristics normally includes variables that describe variables such as the racial composition, educational attainment, and income of students and households.

1. The First Identification Method

Actual enrollment has a deterministic component – enrollment if the timing and number of births were a deterministic function; and a random component – variation in enrollment resulting from the fact that biology causes random variation in the timing and number of births. It is expected that the random component affects regular enrollment proportionally. The first identification strategy is obtained by doing the following:

a. Get estimates of the random part of enrollment variation;

b. Use the random variation in enrollment to identify random variation in class size;

c. See how achievement is affected by random variation in class size.

This method makes use of the fact that the observable student, parent, and community characteristics and the random component change much more uninterruptedly than enrollment in a specific grade-school-time does. However, since parents can respond to the class size of their child, the method allows for little bias. They can decide to transfer their children to another school, making the observable characteristics endogenous to the random component.

At the district level, transfers among schools within the district will cancel out. This will not completely eliminate the possibility of bias due to the fact that parents can still take their children and transfer them. The said bias, fortunately, can be removed by using the potential kindergarten cohort at the district level (‘‘K5’’) as the source of random variation in class size. They will, however, be stronger instruments for class size in early elementary grades than in later elementary grades because student mobility weakens the correlation between kindergarten cohort size and later grades’ cohort sizes.

The costs and benefits of adding another class rely not simply on how much local parents care about schooling but also on actual enrollment. If the procedure for the first identification method is carried out and the changes in the number of classes discounted, an increase in enrollment will reduce class size if it activates an increase in the number of classes. This can be remedied by having an indicator variable for each combination of a school and expected number of classes instead of having school indicator variables in the first- and second-stage equations.

2. The Second Identification Method

Unlike the first identification method, the second identification method takes advantages on changes in the number of classes and the fact that changes in the number of classes in a grade can produce sudden changes in class size. The simplest way to use these discontinuities is the cross-section method of exploiting maximum class size thresholds.

The second method is independent of the identification that comes from using the log of residuals as an instrument for class size; this makes the two methods as checks for each other. Also, predicted class size is also not a valid instrument except when the rule causes a change in the number of classes; estimates will be valid if identification depends only in discontinuities. Finally, identification develops only when the rule binds, so if one uses a rule that binds only in some schools, the effects of class size will only be applicable for those schools.

Because there are issues about identification based on discontinuities, changes in the number of classes that are created by small within-school changes in enrollment that cause a district’s maximum or minimum class size rule are used. The said method is more accurate and less prone to bias than the cross-section method because one can follow enrollment in a grade in a school over time and observe small changes in enrollment. It also compares adjacent cohorts within a school and uses variation from different districts.

The second identification method involves the following procedure:

a. Identify all of the events in which a school increased or decreased the number of classes in one of its grades;

b. Given just the change in enrollment and the district’s maximum and minimum class size rules, identify all the events in which the change in the number of classes was predictable;

c. Within the subset, maintain the events in which the change in enrollment that induced the change in class size was smaller than 20 percent;

d. Estimate a first-differenced version of the achievement equation using just the cohorts immediately before and after each event.

IV. Data

Data on elementary grades are the ones looked-for owing to the fact that the integer nature of teachers and classrooms is useful to relate natural population variation into variation in class size and composition. Elementary classes are less divisible than secondary school classes since elementary school instruction usually involves one teacher spending hours of each school day with a regular group of students in one classroom. Furthermore, class size is also necessary in elementary schools but poorly defined in middle and high schools, where pupils attend different class sizes in different subjects. Moreover, elementary schools are not large in comparison to higher school levels. Lastly, since school cohorts are defined by birth date, data on population-by-age at the school entry cut-off date is needed.

The Connecticut school data that was explored consists of 649 elementary schools from 146 elementary districts. Overall, 25 percent of the schools have typical cohort sizes smaller than 46 students; 50 percent have smaller than 63 students; and 75 percent have smaller than 92 students. Since 1986, Connecticut has administered state-wide tests in the fourth, sixth, and eighth grades annually. All data are available publicly and were obtained from the Connecticut Department of Education or its publications.

V. Some Illustrations

Graphs of the graduating class versus enrollment of schools that have different classrooms per grade show the variation that is useful for the first identification method. Meanwhile, the graphs of two of the three schools illustrate the variation that is useful for the second identification method. However, these graphs cannot be used officially to confirm whether there is a significant relationship between achievement and class size. Regression analysis must still be conducted.

VI. Results

1. Results from Commonly Used Methods of Identification

Conventional methods are inclined to yield estimates that are biased by correlation between class size and unobserved parent and community attributes. Parents with unobserved good characteristics are more likely to select schools with small class sizes and communities with unobserved good characteristics. If the estimates are to be interpreted, it can be concluded that the a 10 percent reduction in class size in grades 1 through 3 improves fourth grade math scores by 0.1468 (about 15 percent) of a standard deviation and a 10 percent reduction in class size in grades 1 through 5 appears to improve sixth grade math scores by about 13 percent of a standard deviation.

Examining district-level variables controls the observed parent and community characteristics. They reduce the estimated effect of class size on test scores, but the estimates are still all negative in sign, and two of the six estimates are statistically significant at the 10 percent level. Results are of mixed or minimal statistical significance, and the impacts are small. Observed demographics are controlled but it is uncertain whether the variation stems from exogenous sources.

Value-added specifications are thought to control for all the effects of family background and neighborhood. Such claims are invalid as value-added estimates may be biased either negatively or positively. However, it is likely that the preponderance of the bias favors class size appearing to be effective.

Conventional methods try to eliminate one source of suspect variation at the expense of having more ambiguous sources. When considering a policy variable like class size, it is better to start with sources of variation that are known to be exogenous and work from there.

2. Results from the First Identification Method

Results suggest that a random rise in enrollment increases class size. It suggests that enrollment residuals are strong instruments for class size. However, it does not affirm that smaller class sizes produce achievement gains. The estimates are mixed in sign, and none is statistically significant at the 5 percent level. Moreover, the estimates do not accept the hypothesis that class size reductions are more efficacious in districts that contain low income or African-American students.

3. Results from the Cross-Section Regression Discontinuity Method

The initial result shows that decline in class size improve achievement significantly. More specifically, a 10 percent reduction in third grade class size augments fourth grade math scores by about 12 percent of a standard deviation. As discontinuities are narrowed in, however, the variables become insignificant and some reverse their sign. Therefore, the initial statistically significant results are generated not by the discontinuities in the predicted class size function, but by the suspect parts of the function.

4. Results from the Second Identification Method: (The Within-School Regression Discontinuity Method)

The within-school regression discontinuity method is quite powerful despite the fact that it depends purely on discontinuous changes in class size driven by changes in the number of classes because it compares adjacent cohorts in the same school, who have little reason to be different apart from their different class size experiences. It also produces standard errors so small that if a 10 percent reduction in class size were to change test scores by just 2 to 4 percent of a standard deviation, the change would be statistically significant at the 5 percent level. Despite the small standard errors, none of the estimates is statistically significantly different from zero at the 5 percent level. This suggests that the estimated effects of class size reductions are rather precisely estimated zeros.

VII. Interpretation

The two identification methods show that class size reductions have little or no impact on achievement. They are independent and they serve as checks for each other. Their results are also robust to specification changes.

The findings are based on class sizes that range to between ten to thirty students per class. This is the appropriate range for American schools. To extend the results to classes with more than thirty students – the usual class size of most developing countries – might yield incorrect interpretation. Similarly, it will also be a mistake to extrapolate these results to classes with less than ten students.

In contrast to the results of this paper, Kruger (1999) observed that a ten percent reduction in class size for three years improves scores by about 13 percent of a standard deviation and a 10 percent reduction in class size for five years improves scores by about 17.5 percent of standard deviation. The difference results may be due to the fact that the natural experiment changed class size but did not altered incentives, while the policy experiment changed class size and contained implicit incentives for teachers and administrators to make good use of smaller class sizes. If this assumption is correct, then class size reduction policies should contain built-in evaluation and incentives.

Since Connecticut school personnel were unaware of the natural experiment, they did not have the chance to react to or alter the evaluation. Other policy experiments may be different because of Hawthorne effects or other reactive participant behavior.

Some of the difference in results might be attributed to the fact that teachers experience small class sizes repeatedly, but not every year. They may not modify their primary classroom style significantly when they have the opportunities presented by a smaller class. Class size reduction should be combined with instruction for teachers that helps them modify their teaching techniques. However, even if she does not lecture differently to a smaller class, a teacher can devote more effort to each student during every teaching activity that has an individual element.

VIII. Conclusions

This study uses natural variation to identify the effects of class size on student achievement. This method gives three benefits:

a. The variation in class size is realistically exogenous;

b. The participants in the natural experiment were not aware of being evaluated or mindful of rewards being contingent upon the outcome; and

c. Natural population variation generates fluctuations in class size that are in the range relevant to current policy.

The study used two methods, which both show how population variation can be used to consistently estimate the effect of class size on student achievement. The first method is based on segregating the random component of the natural variation in population for a grade in a school. Meanwhile, the second method is based on using the discontinuous changes in class size that occur when a small change in enrollment induces a maximum or minimum class size rule. Both methods produce results that are appropriate for considering class size changes in the range of 10 to 30 students.

Under the said methods, it was found that reductions in class size have no effect on student achievement. The estimates are so precise that a 10 percent reduction in class size improved achievement by just 2 to 4 percent of a standard deviation. Moreover, there is no evidence that class size reductions are more efficient in schools that contain high concentrations of low income students or African-American students.

The results do not suffer from bias due to omitted variables and endogeneity than are typical estimates that depend on variation in class size that is generated by parents, teachers, administrators, or policy-makers’ decisions. Also, unlike other studies, participants included in the study are not aware of being evaluated. Incentives did not alter the results of the experiment. The experiments represent actual class size reduction policies, which hardly include evaluations or incentives for schools to make good use of the opportunities provided by smaller class sizes.

Source:

Caroline M. Hoxby, “The Effects of Class Size on Student Achievement: New Evidence from Population Variation” Quarterly Journal of Economics, Vol. 115, No. 4 (November 2000), pp. 1239-1285.