Conceptual and Empirical Issues in the Estimation of Educational Production Functions

Although confusion is stemming from their administration and interpretation, much research has been done to study educational production relationships and to determine pupil performance. These studies are crucial since they demonstrate the intricacy of empirical studies. Their findings also have implications to topics such as salary determination, status achievement, the school financing, and the impacts of education quality on urban location and housing choice. They also have continually influenced judicial proceedings, legislative debate, and executive branch policy deliberations. However, these kinds of research have been difficult to follow because they cross disciplinary boundaries.

I. Background

The Equality of Educational Opportunity or the Coleman Report mandated by the Civil Rights Act of 1964 remains to be the most influential study. It contains data on the pupil attributes from 3,000 schools,       as well as their performance. It also focused on the significance of the relationship between school inputs and pupil achievement and introduced a collection of technical and cryptic concerns such as statistical significance, analysis of covariance, production efficiency, multicollinearity, residual variation, estimation bias, and simultaneous equations into the public policy arena.

Input-output studies such as those found in the Coleman Report have led to a fast growth in the number of analyses and a strong effort to interpret the varying and contradictory outcomes. As more studies were conducted, the estimated relationships were coined as "educational production functions".

II. Textbook Analyses

The difference between a production function and other descriptions of input and output relationships is the concept that it denotes the maximum possible output for given inputs. The production function provides a source of describing efficient production, the appropriate response of firms to changes in technology or input costs. Its basic analytical concepts are applicable to a wide variety of applications. They are, however, quite too simple to use in practice; they require extensive modification for them to be applicable to certain situations. The function is not generally known theoretically and must be estimated, which raises further statistical and conceptual concerns.

The greatest difference between production functions to education and to other industries comes from the potential uses of the analysis. While most firms will not alter their behaviour based on the results of their production function, the findings and interpretations of the educational production functions are discussed more extensively. Indeed, Congress even holds hearings on the size of estimated coefficients; commissions include results in supporting their policies; and courts receive testimony with regards regression equations.

III. Conceptual Production Functions and Educational Realities

Studies for educational production functions comprise of statistical analyses using observed pupil outputs. This information is obtained from standardized test scores, attitudes, college continuation, and attendance patterns. Variations are observed in terms of the actual measured inputs, the level of aggregation of the variables, and the statistical methods employed. Due to this, the studies generate different outcomes – some of them even contradicting each other.

A. Measurement of Output

Most production function studies use standardized achievement test scores, while others have used variables such as pupil attitude, attendance rate, and dropout rates. On the other side, although most production functions concentrates on varying quantities of a homogenous output, this may not be suitable for education production functions as education transforms inputs into pupils with varying attributes. Standardized tests lack external validation, making discrimination and the division of the population into different groups more likely.

Some studies attempt to deal with the influence of education in the latter lives of citizens. For instance, economists have looked into the impact of education to earnings and labor market performance. However, the measure of how much education individuals attain is often insufficient. Analysts frequently use schooling years to estimate education. The quality of education, a more vital aspect, is seldom measured. Although some have tried to include qualitative variables, these studies have been limited by data availability and stringent assumptions with regards to school operations. The findings become inconclusive and inadequate in giving guidance.

The relationship between schooling and labor market performance is the chief concern of many policy studies. Many analysts have also explored other areas like education’s part to job satisfaction, personal health, the productivity of mothers occupied in household production, and the impacts of the mother's education on the young children education. Furthermore, scientists have also studied how education influences political socialization, voting behavior, and even criminality.

In spite of these developments, they have failed to address how the mentioned outcomes are impacted by varying school programs and operations. The examination of nonlabor market areas has the potential of providing a balanced perspective on educational productivity. But so far, the existing studies yield inconclusive results.

The notions of how education affects skill formation and later experiences are still superficial. This is because cognitive skills may not be the most significant outcome in distinguishing an individual’s success. Although intuitively speaking, a more educated person will get a job done more rapidly, less education may be better for tasks that require manual labor. The hypothesis that educated individuals are more capable of performing complicated tasks has important implications in studying the productivity and outputs of schools. This can take place by considering the mechanisms by which school interacts with the work place. This could offer insight on how to properly assess the schooling outputs and how these consequences can impact the economy.

The school-earnings relationship becomes blurred because of the fact that schools are capable of either yielding qualified individuals or determining the competent ones. Economists and sociologists have paid more attention to screening models as the social value of schooling may be considerably less than the private value if it happens that schools are just identifying the more able instead of actually sharpening their skills. Moreover, the screening model proposes both potential reinterpretation of the historical contribution of education to economic growth and revisions of expectations about future returns to schooling.

The model also has direct implications for the measurement of educational outputs and the analysis of educational production relationships. Under it, the school outputs are information regarding the relative pupil abilities. This indicates that more attention should be given to the distribution of observed educational outcomes and their relationship to the distribution of underlying abilities. In addition, the interpretation of some studies may be significantly transformed since schools with a higher dropout rate might actually be supplying greater information than those having lower rates. However, no persuasive test has been made to differentiate between a screening model and the standard production model.

Test scores seem to be an inappropriate measure for outcome since empirical evidence is inconclusive about the relationship of test scores and achievement. Despite of this, test scores are still being widely used because they are easily available and since educators and parents still value them important. Test scores also seem to efficiently pick individuals for further schooling, which may relate to the real outputs of the selection mechanism.

If one intends to use test scores as an output measurement, an appropriate scaling of scores must be considered, one that indicates how different individuals are rather than one that merely ranks them. Also, there is currently some movement to use criterion-reference tests. These tests relate to some set of educational objectives. Although it is more intuitive to adopt goals that relate to performance outside school, present studies are not inclined toward the use of these objectives.

B. Multiple Outputs

Ordinary Least Squares (OLS) regression is unsuitable to use if there are more multiple outcomes that are simultaneously produced. In a two-outcome assumption, where there are intermediate and a final outcomes and an underlying production relationships are such that attitudes affect investment and vice versa, the two structural equations can be estimated (but not with OLS). It is possible, however, to estimate the reduced form of the equation through OLS. This form explains both the direct and indirect impacts of the explanatory variables. A number of single-equation analyses can be taken as estimates for reduced-form relationships. Multiple-outcome methods are more complicated.

It is feasible to construct models where joint production appears important. However, there are circumstances where these issues are insignificant. Without sufficient information about the range of outputs and the potential decision rules, there is little that can be discussed.

Production functions for total output are quite appealing as this is also what is done for production function estimates for other sectors. Market prices are used to gather outputs. Such prices, however, are not available for the education sector. Besides, they are also inappropriate because the weights in the decision function for outputs may be different from the market prices. Production functions estimated with test-score measures might be more appropriate in earlier grades, where more focus is given to reading and arithmetic skills than in later grades. These outputs are more heavily weighted than others at earlier grades, lessening the potential problems of multiple outputs in later grades.

C. Inputs to the Production Process

Learning theorists in practice have little guidance to the development of production functions; they do not participate with the specification of inputs. Despite of this, these set of inputs verify the pedagogical models.

The relatively fixed input of labor and capital could only explain little. The fact that education studies attempt to provide much detail about input differences makes it open to critique regarding the specification of inputs. This is partly because input specification has not    much attention in past analyses. There was little conceptual clarity and the choice of input was deviation guided by availability rather than conceptual desirability.

A somewhat acceptable model for pupil achievement comprises of variables including family background, peer influences, school inputs, and innate abilities. The chosen inputs are those considered relevant to the individual pupil. The education production relationship is also cumulative in a sense that past inputs still have effects on output although the value in explaining output diminishes with more distant outputs. This, in turn, requires the amount of data to be large. An alternative version of the model answers this huge data requirement by taking only the value-added inputs. This, however, lessens the data requirements at the expense of some assumptions.

Many educational inputs are not measured directly but are substituted by other variables. This results to considerable measurement error as effects of different inputs accumulate. The resulting deviation of the empirical models to the conceptual models calls for some implicit assumptions.

One deviation from the conceptual models comes from the measurement of inherent abilities. When this variable is not included, the estimated impact of family background on achievement is biased upward but biases in other parts of the model are smaller. If the “value-added” model is used, the importance of the variables that are taken out will be even less as the level effect will still be present and only the growth effect of the omitted variable, innate abilities, will be taken out.

Another problem with the empirical evidences is the accuracy of variable measurement. Only contemporaneous measures of explanatory variables are usually available, which bloats the measurement error of cumulative variables and makes the coefficients biased. The original model has more severe measurement problems than the “value-added” model.

The measurement concern is also more severe for school inputs. Data that are usually provided only assess the schools or the teachers, but they are not linked to the individual students. This dilemma is aggravated at later grade levels where mean attributes may yield misleading indications of the actual inputs to any given student.

In education production functions, there are two types of concerns: the organizational and process attributes of the school and of the individual teachers. The former can be accommodated in the conceptual framework, but the latter creates serious problems. This is because educational decisions that are made by the teachers are difficult to observe and measure. They, however, cannot be ignored since variations in the teacher attributes are vital.

Recognizing skill differences is important in discussing the efficiency in production. In addition to that, it also modifies the interpretation of teacher and school inputs. It is still rational to evaluate the influence of the attributes of teachers because a lot of school decisions are based upon these features. However, the estimated impact of these attributes shows the ability either to predict or to develop more skilled teachers.

More effort should be given to understand and measure both the macro and micro organization and process school attributes. This will be a potential breakthrough in production function analysis. Since there is no assumption that schools systematically choose the best process for given inputs, estimates of education technology should be conditional upon the chosen macro organization and process characteristics. At the individual teacher level, the impact of teacher characteristics can be thought of as reduced form coefficients.

D. Efficiency in Production

The efficiency of schools in production has important policy implications since inefficiency opens the possibility of increasing school outputs with no additional inputs. However, because estimation is based upon the observed behavior of schools, the estimated relationships may not yield the production frontier if schools are not producing the maximum output for given inputs. The relationships will then portray average behavior, and this may not be useful in predicting how changes in inputs affect outputs.

Two concepts are considered: economic and technical efficiency. Economic efficiency refers to the correct choice of input mix given the prices of inputs (and the production function). Technical efficiency refers to operating on the production frontier.

Two arguments support the argument that schools are technically inefficient. One asserts that educational decision-makers are not guided by incentives to maximize profits or to minimize costs. The other one argues that they might not understand the production process that’s why they cannot be expected to be on the production frontier. The first argument does not necessarily imply being off the production frontier, while the focus of the second argument is related to the importance of macro organizational and process choices.

The standard framework suggests that if two production processes have the same input but have different outputs, inefficiencies are present. But with skill differences, individuals with the same attributes make productions decisions that are difficult to detect, measure, and model. Hence, variations in outputs may not essentially reflect efficacy differences. This point, however, does not eliminate the significance of the production framework.

E. Miscellaneous Issues and Nonissues

Criticisms of estimated educational production functions include the following:

1. Functional form. There is little guidance about functional form. This issue seems to be a second order problem because differentiating among varying functional forms is impossible. Different functional forms may produce different results, which mean that predictions based upon changes that are different from current observations may be dangerous.

2. Level of Aggregation. The conceptual model is at the individual student level, but much analysis is conducted at a more aggregate level. The effects on the estimates of aggregation are dependent upon the nature of educational relationships. The most serious observed aggregation issue is one of errors of measurement. With individual data about students but only aggregate data about schools, aggregation becomes helpful because the errors in measurement for a model of mean achievement and mean attributes are less than with individual achievement and mean school qualities.

3. Selection Effects and Causation. Although deemed important, information about causal relationships between school factors and achievement cannot be directly estimated from the observed data and correlations. They must be introduced from information about the structure of the overall model. The most important issue in the production function setting is the effects of teacher selection and assignment mechanisms.

4. Multicollinearity. In most observed production functions, multicollinearity has been observed. This stems from the issue of not being able to unravel the separate effects of highly intercorrelated exogenous variables. It often further yields to the wrong sign of coefficients. This issue has been overrated. What is important is to determine the right statistical method to use.

5. Statistical Methods. The analysis of covariance is usually useless for answering the questions under consideration. It just raises more questions without yielding any added value.

IV. An Assessment of What We Know and What We Should Do

Empirical analyses of production functions often have suffered 1) from concentrating upon a given attribute of schools or learning process, or 2) from excluding other attributes that simultaneously affect outcomes. These studies become a bit useless in modifying school policies. Production function must produce some policy relevance by investigating the effects of different factors on the performance of the schooling system.

It is observed that differences in family socioeconomic background lead to significant achievement differences. Also, variations among schools and teachers are detected to be important in achievement. On the other hand, there is some indication that schools are economically inefficient as they do not use the best mixes of inputs given their input prices and effectiveness. Lastly, there are significant differences in production functions by race and family background: the interaction of school resources with the background characteristics of individuals is crucial.

There is much to be studied regarding the relationship between school quality and performance. The earlier results on the operations of the school system relate exclusively to test-score achievement, even though validity of this measure is uncertain. This line of inquiry is important and must be pursued. In addition, the influences of peer compositions as well as the dynamics of the educational process are still vague. Moreover, severe data problems call for the collection of new data for a variety of school situations, which avoids the measurement issues and allows the observation of longitudinal information. On top of these concerns, the following also needs further attention: measures of alternative outputs, investigation of decision processes with regard to alternative output mixes, identification and measurement of process and organizational variables for schools and classrooms, and expansion of models for more complicated realities such as high schools.

V. Concluding Remarks

Aside from influencing education policy-making, educational production relationships also have connections to other research and policies like the distribution of per student spending under alternative financing mechanisms, the quality of governmental services, and the racial composition on achievement. The importance of understanding the education sector has important consequences for understanding other areas. Further research studies are yet to be done as the current actions are quite superficial.

Source:

Eric A. Hanushek, “Conceptual and Empirical Issues in the Estimation of Educational Production Functions”, Journal of Human Resources, Vol. 14, No. 3 (Summer, 1979), pp. 351-388.