More Real-time Education Research

Very interesting and relevant! This may shed some light on the situation in California. CA’s education system is consistently ranked near the bottom yet we spend more than average. This research leads me to conclude there are other negative factors to our poor performance, not just spending.

I also agree that we economists need to focus some of our research on areas which are more relevant. Helping companies, countries, and citizens in everyday endeavors would be very beneficial.

Grouping the data would be a much more efficient way to display the statistics.

Maybe I’m missing something here, but shouldn’t these analyses simply covary for (partial out) the effects the FARMS then look at the relationship between spending per pupil and pass rate?

I made these same observations not long ago on a comment I made here on this blog by simply looking at spending rates per pupil per state and comparing it to test scores in those states. No correlation. Actually, there appeared to be a subtle bias in favor of low-spending schools.

Presumably the key issue with all of this sort of thing is to try to find exogenous variation in spending, such that you can rule out the possibility that schools that do worse receive more money because they’re doing worse rather than the other way around.

I would assume Hanushek, Hoxby and co have done stuff on this?

Very interesting and relevant! This may shed some light on the situation in California. CA’s education system is consistently ranked near the bottom yet we spend more than average. This research leads me to conclude there are other negative factors to our poor performance, not just spending.

Indeed, but keep in mind correlation does not equal causation. CA may spend more because they have more problems. If they reduced spending to match a lower spending state they may find even worse performance.

spencer
But I would like to see this analysis done for three groups.

I suspect the issue isn’t urban so much as cultural barriers. Language is probably the most important issue. If you have to worry about improving English skills first that leaves less time for math and other subjects.

Suppose you took out these two factors:
1. Students who don’t speak English well.
2. Students whose native language is neither English or Spanish. (There are plenty of people who are fluent in English and Spanish, if a school is confronted by many kids who speak a diverse array of other languages it is harder to address the problem by just hiring some bi-lingual teachers).

I suspect you would then notice a positive correlation between spending and results.

I understand why conservatives like the idea that school spending doesn’t produce better results but why this is the case is pretty troubling. It can’t be about competition. School spending usually happens at the most local level in the US. Real estate agents will tell you the quality of a school district is one of the drivers of home prices. Also, unlike local police and fire most people will come into direct contact with the local public school system in some format. A poorly run public school is a lot more noticeable to a lot more voters than, say, a poorly run zoning dept. Likewise better private schools also tend to cost more than lower quality ones (don’t jump to bring up Catholic schools here, the question is how much do they cost not how much do parents pay. Catholic schools tend to charge parents less than their cost using donations from their congregation as well as indirect gov’t subsidy to make up the difference).

Let me turn around and play economics professor to the economics professor. You’ve earned a C by showing there’s no simple relationship between spending more money and getting better performing schools. Now to get a B+ or better tell us why?

It is almost unheard of for something with better quality not to cost more. Assuming that high quality teachers are less common than average ones, wouldn’t the market model insist that the price of a high quality teacher be more than a low quality one? Isn’t there something that smells like a free lunch in this conservative meme against public schools? “We can get more by spending less (or at least not more)!” Why doesn’t that sound a bit like the mythical liberal social program that ‘pays for itself’ by boosting income, lowering unemployment and crime?

We come back to how do you measure a positive result. I think a math score is a bit like measuring the quality of food by calories. If you’re talking about a survival situation, that’s a good metric. If you’re the food critic for the NY Times that’s a horrible metric.

In the last thread I suggested that residents measure schools by a similiar set of relative metrics. If the district is doing horribly, they may focus on the crude but easy to track metrics like what % of students pass the state math test. Once you get to a certain level (say 80% passing), though, the focus shifts to more subtle metrics like the quality of extra-curricular activities.

This seems perfectly rational to me. If the school is already doing a good job of the most basic of basics (reading, writing and arithmatic) wouldn’t consumers then demand the next level of service…namley making one more ‘well rounded’. How do you become more well rounded by cutting back on activities?

Many people would say a school that had no extra-curricular activities, no sports teams, no debate teams, no clubs…is less of an education than one that has those things even if the math scores are exactly the same. The market seems to as colleges (private and public) and employers seem to give such things credit.

In a free market economy, things of better quality do in fact cost more. But when you have a true monopoly, price goes up as quality goes down. This is basically what we have. The poor go to public schools more than other groups. The government thus has a monopoly over the education of the poor

Even monopolies must face competition. Microsoft may have 90% of the desktop operating systems but that doesn’t mean there are no alternatives.

First of all, public schools grab most students because they are free. No they aren’t free to the taxpayer but they are free to the parent. Simply because a good is free, though, doesn’t mean it has an automatic lock on the market. If that was the case we’d all be working in offices that run OpenOffice.

Second, public schools are probably the most intensely local of all gov’t services. It’s a lot easier to ‘shop’ for a good school by moving around than it is to ‘shop’ for a good state tax system or regulatory regime. People do make the quality of the schools one of their prime considerations in choosing where to live and the fact that schools are locally different means you can often choose while keeping the same job and still living in the same general area.

Third, local schools are the one feature of local gov’t that many voters come into direct contact with. IMO it’s a lot easier for a local gov’t to get away with having a totally inept building inspector or zoning board than a local school. What % of voters interact with those two things in a year? 2%? The local school, though, impacts a majority of voters at any given time directly or indirectly. On the local level there’s simply a lot more scrutiny of the schools than anything else. I’m not saying that’s a perfect substitute for competition but it usually should be a good thing.

Not surprisingly, I suspect most of the time when you have a really bad local school you also have a really bad local gov’t which leads me to suspect the problem is a dysfunctional system rather than just the schools. Washington DC, for example, has a rep for bad schools but also has a rep for lots of other bad things from police who can’t solve most of the murders to even snow plowing and garbage pickup.

With middle-income and wealthy people, the schools have to compete with home schooling and various private schools. Thus, they have to provide better quality

True, having more money means more options which means more competition. Someone who can spend up to $50K a year on vacation will have more travel agents beating on his door with a greater array of options than someone who needs to stretch to spend $2K. For this to become a major factor in public schools, though, you need to have a lot of segregation between rich and poor.

For the most part, I don’t think you often see that. Yes there are districts of deep poverty as well as districts with great wealth but in many parts of the US you’ll find mixes of both.

It just so happens that you can in fact provide high quality education for not too much money, so those public schools don’t have to spend as much as the schools in poorer areas, where the administrators use the poor quality of education they are giving to argue for more money for themselves. And then there is the issue of corruption in the poorer districts. My wife used to work for a poor Dallas school, and the AC was down all the time. She said people would come in, look around, do nothing at all, and presumably get paid for it.

I’m not at all sure “it just so happens” that you can provide high quality education for “not too much money” all the time. This is the problem that often erupts in these types of discussions, we fall into ancedotes. Yes I believe there are schools that are money pits who could see dramatic improvements paid for by simply going after waste and corruption. Likewise I’m sure there’s some middle class family who could double their consumption by simply shopping more wisely and clipping coupons. On average, though, for a middle class family to consume twice as much will require twice as much income. Is that true for schools?

Well look at what Michael Rizzo found:

Schools that perform better than expected spend $13,843 per student.

Schools that perform worse than expected spend $14,195 per student.

Let’s say poor schools are run by corrupt people who rack up the spending without improving results. What would the results look like? Schools that perform worse than expected would be spending a lot more per student (after all, the money would be going to kickbacks for the administrator’s ‘cousin’ who owns the AC repair company etc.).

But on average that doesn’t seem to be the case. Yes worse than expected schools do seem to spend slightly more (and maybe the very corrupt schools are the outliers driving the average up) but the relationship doesn’t seem to be there.

My hypothesis is that some schools just happen to benefit from minor random variations that allow them to do slightly better than expected. The example I gave previously was to imagine two poor districts. One district just happens to have a Church that sets up a volunteer tutoring program while the other doesn’t. Both have the same style of school administration. The school with the nearby Church, though, will indirectly benefit. If the school without it wants to match its sister school, it would have to spend money to create its own in-house tutoring program. The numbers, though, would make it seem like the first school is being run by smarter people who get better results with less money. The reality, though, is that both schools are being run equally well….one is just being benefitted by statistical ‘noise’ that you don’t see by looking at a crude figure like a poverty rate.

I think a math score is a bit like measuring the quality of food by calories. If you’re talking about a survival situation, that’s a good metric. If you’re the food critic for the NY Times that’s a horrible metric.

Too many calories are harmful. How is too much maths harmful?

If the district is doing horribly, they may focus on the crude but easy to track metrics like what % of students pass the state math test. Once you get to a certain level (say 80% passing), though, the focus shifts to more subtle metrics like the quality of extra-curricular activities.

Quite possibly. The point of the exercise though was to show that increased funding did not improve test passing.

Many people would say a school that had no extra-curricular activities, no sports teams, no debate teams, no clubs…is less of an education than one that has those things even if the math scores are exactly the same.

Perhaps. That was not the question the post was seeking to answer though.

Another point about the metric….if people stop seeing higher pass rates as a metric for school quality they will fund other things…..

In other words, imagine a middle class school gets from 50% to 85% pass rate by funding a tutoring program. Now that the parents and voters see the 85% rate they decide additional funding will go into music and art programs rather than trying to make the pass rate 95%.

What’s going to happen to the data is you’re going to get schools that are spending more money but have little or no improvement in the math score. It would be an error, though, to conclude that higher math scores don’t require more money. On the contrary they do, but because some schools spend money on things other than math scores the regression’s results show no relationship between money.

To do this right….and it may not be possible to do it right….you should try to measure how much money is spent on math education and compare that to test scores.

I didn’t say the data proved poor schools don’t suffer from more corruption than rich schools, I said that if poor schools were all corrupt (or on average corrupt) then the data would show that poor schools performing worse than expected would be spending a lot more than poor schools performing better than expected. The difference between them, though, is not statistically important. The bad poor schools spend $352 more than the good ones, or about 2.5% more.

Spending wise the difference between a poor school getting good results and one getting bad results is less than a hair.

Also, it’s been proven that learning music improves math scores,

Keep in mind marginal cost here. Music very well might help math but it probably takes a lot of spending on music education to boost math scores by 10 points as opposed to direct tutoring in math.

The point is that ‘quality’ is a very broad term. In a deeply troubled school it is easy to measure improvements in quality by very basic and crude metrics like pass rates on a standardized math test. As a school undertakes an effort to improve, it probably would target those easy metrics first because the payoff is quick and easy to see. At a certain point, marginal costs increase. Going from a 60% to a 85% pass rate is probably something that can be accomplished with some concentrated effort. Once you’re at 85%, though, the remaining 15% who fail probably reflect individual problems or issues rather than issues with the whole school. Going from just 85% to 90% might take as much effort or even more than the original improvement of 60%-85%.

That doesn’t mean all the ‘easy fixes’ have been exhausted, though. At this point the effort to improve quality might turn to other things like adding to music and art programs or sports or school clubs. IF the school gets additional funds it might put it into those things rather than pouring all resourses into a quest to get the pass rate to 100%.

What this means for the data analysis we are trying to do here is that the relationship between spending and math scores will get muffled. Some spending is being focused on things other than better math scores. This doesn’t mean that you can get better math scores without spending, although it may appear so at first glance.

Remember regression analysis is probably the most dangerous thing social science has ever developed. It’s really easy to set Excel to churning on data but anything it tells you has to be mulled over for ages. That’s why statistics text books maybe give you one chapter on how to do regression (if that, sometimes they just say let Excel do it) and then half a dozen chapter of hurdles you have to make your results jump through before youc an begin to say anything useful.

Troy,

Assume a corrupt school will have bad results. If that is the case then the schools with bad results would show up in the regression as spending a lot more than the schools with good results. The real results were that the schools with bad results spent only a statistically tiny amount more than the schools with good results.

This means either that a corrupt school that wastes money rather than spending it wisely will do just as good on test scores as a pure one or the number of corrupt schools is not statistically significant at any particular income level.

If I read you correctly, you’re saying that the corrupt school is wasting its money so it performs worse than expected. It does, however, limit its corruption so its per pupil spending is not too far above average….perhaps to avoid drawing too much attention.

I suppose that could be a possibility. It does essentially support my main assertion that the data does not disprove a relationship between education and money (i.e. to get more of one you need more of the other). It shows that the regression might hide this truth if money is spent on things other than the dependent variable you are testing for.

One way to address this might be to come up with several measures of educational quality. Perhaps math scores, reading scores, a metric of culture like how many books were read in a year and so on. Then take a random sample of schools and try to divide their budgets….’math centered spending’, ‘reading spending’ and so on lumping everything else into ‘other’ or ‘admin’. Then test to see if there’s a relationship between spending and the metrics. Does a poorer school that puts more $ in the library produce more books read per year while another poor school that does supplemental math tutoring scores better on math?