Bryan Caplan  

The Taboo Trade-Offs of Tracking

PRINT
Consider the Substance... Uber reacts to the Spanish ban...
Most Americans are okay with educational "tracking" - measuring potential, then tailoring each student's education to his measured potential.  But if you advocate extending or expanding the role of tracking, most Americans resist.  Suppose you propose, for example, that the bottom third of high school students get vocational education instead of college prep.  Americans suddenly rally behind feel-good egalitarian slogans like, "We have to make sure that every student has the opportunity to live up to his full potential."

Taken literally, such slogans damn not only the extra tracking we could have, but the tracking we've already got.  All tracking is a trade-off between two evils.  Classicists call them Scylla and Charybdis, statisticians call them Type 1 and Type 2 error.  But let's just call them Overlooked Potential and Wasted Resources. 

The evil of Overlooked Potential: The tougher your tracking, the more qualified students you fail to teach.  The evil of Wasted Resources: The laxer your tracking, the more unqualified students you teach to fail.  Accept no one, and you won't waste a penny, but you'll also miss every opportunity to do good.  Accept everyone, and you'll miss no one - but you'll burn a fortune of time and money on Hail Mary passes. 

Every system - the status quo included - strikes a balance between Overlooked Potential and Wasted Resources.  But almost no one explicitly argues that what we currently do strikes the optimal balance.  Why not?  Probably because accepting Overlooked Potential for the greater good is, in Philip Tetlock's phrase, a taboo trade-off.  Saying, "Sure, I don't like overlooking potential; but I'm even more opposed to wasting resources" sounds terrible - no matter how trivial the Overlooked Potential and how massive the Wasted Resources. 

How do Americans cope with their silly scruples?  They salve their consciences by pretending that the problem of Overlooked Potential only emerges if tracking extends or expands.  This preserves a modicum of common sense; at least we won't abandon tracking altogether.  But if tracking is currently underused, Americans' taboo trade-off blockades any further progress.  Is it possible that more robust tracking might deprive someone somewhere of a valuable opportunity?  Uh... yes; it is a big world.  Then robust tracking gets vetoed, regardless of its upside.

Is there any evidence that tracking is currently underused?  Sure.  Partly under the influence of the No Child Left Behind Act, high schools today teach as if every student is a future college graduate.  But most aren't; indeed, over 20% of high school freshmen don't even earn a normal high school diploma.  Furthermore, many college grads don't get college-type jobs.  I submit that these disparities between aspiration and results are, in themselves,  strong signs that tracking is underused.  Stricter tracking wouldn't magically turn more students into successful college graduates.  But it would prepare the majority of students who won't get college-type jobs for the careers they're actually going to have.  

Perhaps I'm wrong about this; maybe the status quo is wise beyond my ken.  But I'm not wrong to think that the trade-off between Overlooked Potential and Wasted Resources is socially taboo.  And as long as this trade-off is socially taboo, we should assume that the case for stricter tracking is intellectually stronger than it looks




COMMENTS (17 to date)
austrartsua writes:

My immediate reaction to this is the liberty instinct. Students should decide for themselves what classes they'd like to take. If they think they are smart enough to take the highest level of math, they should feel free to attempt it. If they fail, that's on them. This will solve the "tracking" problem: students will self-select into the area they want to be in. Of course, classes could have an entrance level exam to rule out potential students who cannot survive, or students could be weeded out as they go along, thus raising tracking issues. However students will take care of this themselves: as much as they want to get the highest level of math, the pain will be too much for some.

Admittedly I think you are probably referring to educating children at an age where they can't make up their own minds. In which case tracking will pose a problem. But its not a big one. My school in Australia had three levels of math in grades 7-12. In Grades 7-10 kids were assigned to a level based on their performance. In Grades 11-12, kids chose for themselves. I think this is an appropriate mix of tracking and liberty.

I'll see if I can beat everyone else to pointing out that you've forgotten all about race. Our society is still largely untroubled by tracking in homogenous groups. Schools that are predominantly white or Asian often track and remorselessly. Diverse schools that track will end up with unacceptable racial balances--and end up facing lawsuits.

In fact, it's very common these days to hear people talk about "the time when we deliberately tracked by race"--the bad old days. I actually went and looked for evidence that districts ever deliberately tracked by race instead of test scores, and can find almost no evidence. All the case law blames the schools and districts for the fact that test scores result in sorting by race.

https://educationrealist.wordpress.com/2014/04/23/finding-the-bad-old-days/

For the life of me, I can't see how any reasonably well-informed person can write about tracking without mentioning race and not feel like an idiot about the omission. Like yo, Mrs. Lincoln, how was the play?

Andrew_FL writes:

I agree that there is probably some optimum balance/tradeoff. And I agree we're almost certainly not there, for the same reasons you would.

The question I have to ask though, is what sort of institutional changes would be necessary for the educational system to find the optimum automatically, whatever it is? It seems like as long as people are opposed to the idea of more tracking as a culture, any educational system will under-utilize it.

Yaakov writes:

If the question is of choosing proper government policy, I believe the debate is a waste of time. Government cannot do anything properly.

If the question is what parents should do, that is a very serious question. Should we encourage our kids to try for high goals or direct them to lower and more realistic goals. I directed my kids to high standards and it is not easy. My daughter (age: 18.5) is completing her bachelor degree in computer science this year, but is having a hard time at the moment, saying she hates math and computers. My son tried for the same program, dropped out of school altogether and is now homeschooling with a limited curriculum.

I would say the most important thing in tracking is flexibility - always keep an open mind to changing tracks. I think that is where the government has its worst performance.

Harold Cockerill writes:

Yaakov, wouldn't you agree that government is good at spending money, breaking things and killing people?

Doug Holton writes:

There is research on tracking. I don't really have an opinion one way or the other since this is an area of educational research that I'm not familiar with, but here is a summary/meta-analysis:

"John Hattie conducted a meta-analysis of more than 300 studies of ability grouping and tracking that included all grade levels and areas of curriculum. He found that overall there was no effect for ability grouping in reading and that the effect in mathematics was slight. Hattie concluded that "tracking has minimal effects on learning outcomes and profound negative effects on equity outcomes." Hattie also examined the effects on subgroups of students and concluded that "no one profits," including high achievers, from ability grouping."

From http://blogs.edweek.org/teachers/classroom_qa_with_larry_ferlazzo/2013/05/response_ability_grouping_in_schools_--_part_two.html

Yaakov writes:

Harold, I believe that a "something" that you are good at is something that you claim you want to do. Governments usually do not claim they want to spend money, break things and kill people. Maybe Nazi Germany was good at killing people. Maybe ISIS is good at breaking things.

Granite26 writes:

@Doug

I don't see anything in there about cost. Am I missing it? The 'no one profits' seems to mean none of the students profit, which seems a trivial result. The goal would be that tracking has minimal effects on learning outcomes, but high effects on costs.

(The equity gets back to some third rail or other)

Stella writes:

"My immediate reaction to this is the liberty instinct. Students should decide for themselves what classes they'd like to take."

That's how my university did it. Anyone who wanted could sign up for Honors classes, and even my program (engineering) didn't require an entry exam or special application. Instead, after the first two years, you were required to talk to an adviser who would look at your grades in the earlier classes and either tell you that you were probably going to do fine in the rest of the program or gently advise you to switch majors.

Zeke writes:

It would seem the liberty approach espoused in the comments makes sense, provided the grading curve is steep. If half of the students get As and the other half Bs, well... that isn't very good at discouraging kids from classes beyond their capabilities.

Thomas Sewell writes:

Agreeing with Austrartsua and Zeke, the key issue/objection is that of "Who decides?"

If the student/parent decides based on the available information, then more information to inform their decision isn't threatening.

If the government/school district decides based on the rules of the elite bureaucrats, people have had lots of time to get used to seeing that fail miserably, mostly in corner cases and unintended consequences.

A writes:

Perhaps lower intensity tracking is a way to hedge against uncertain factors in educational/welfare outcomes. Other commentors mention the problem of assigning value to outcome metrics. Lotteries seem pretty strange to me, but that opinion begs the question of strange compared to what, and to whom. There is also the issue of not knowing the deep factors behind performance differentials. Some evidence shows that students recursively update their performance based upon feedback, even when the feedback is independent of performance. Enhanced tracking could actually reinforce the status quo by marking the present as a start date, rather than another period in a long process of "injustice".

Granite26 writes:

'Who decides' should be the same as 'Who pays' (IMHO)

I'm A-OK with a subsidized 'equality' boosting scheme where tracking is used to fund education in a very leaking (strict, with lots of potential lost) way, while people who want to disagree can pay their own way.

The trick is finding a way to have social/government funding without pushing up the price to the point people can't afford to pay their own way. That's clearly not where we're at now.

Floccina writes:

Tracking is how we get people like my son's boss. He has a reputation of being the best plumber in town and BTW he owns the plumbing company and is rich. Also that is how you get great auto mechanics and he owners of those business can make a lot of money if they are smart. BTW there is another plumber in town who graduated form Brown University, his father owns the business but he actually does some hands on plumbing.

Floccina writes:

With more tracking you might get more people like my son's boss. He has a reputation of being the best plumber in town, he owns the plumbing company and is rich. You can also get some great auto mechanics and if they are smart enough they can be owners of those business and make a lot of money. BTW there is another plumber in town who graduated form Brown University, his father owns the business but he actually does some hands on plumbing.

Future Educator writes:

Why must vocational education mean tracking? Can’t a student take AP classes and shop or nursing or pharmacy preparation courses? Are trade skills just for the educationally disinclined, or lets call it like it is: poor people’s children? Our current tracking practices and attitudes about appropriate educational goals are contributing to the devaluing of the trades, and the over-valuing of 4 year college programs. This encourages our students to go into debt for schooling that may not serve them or help them get work that will support and satisfy them, whatever their academic abilities.

I would argue with the point that most Americans are ok with our current level of tracking or think that it is a good enough balance. The level of Wasted Resources is greater than it would be if tracking were carried out in a completely rational, unbiased way. Students placed in non-college bound tracks are not necessarily there because of a lack of academic potential but because our school system is failing to teach them the skills they need to function in mainstream educational situations and ultimately mainstream society. Add to this the unconscious bias based on class indicators and race on the part of those doing the tracking, and the situation looks even less appealing.

I do not disagree that greatly increased access to vocational education and a more nuanced valuing of college education would better serve our students and our society. I disagree with the idea that stricter tracking, imposed by those other than the individual being tracked, is the most effective or socially just way to accomplish this.

I think that a middle ground that achieves some (if not all) of the benefits of tracking, while still being acceptable to American moral sensibilities, is to a) actually have distinct (but not mutually exclusive) tracks starting in the last two to four years of high school, b) make it possible to combine or switch tracks, and c) use tracking purely for informational purposes, and to make both the raw and distilled (statistical, not personal) information available to students when they have to make a choice.

Such a database can track (in a suitably anonymised fashion, of course) the performance of students in great detail, and keep detailed information about them which is useful for making predictions, such as: IQ, parental wealth, parental education background, performance on standardised tests, personality factors (at least the ones that are stable during childhood, if there are any such), geographic region, school funding, teacher quality/qualifications, and whatever else.

So when student has to choose, he can perform a query on the tracking database that tells him the odds of his getting various outcomes. He can make the query as specific as he likes. He isn't bound by the odds that the database gives him to accept or reject a particular choice, but unless he has convincing information about why he already is, or is likely to be an outlier, the query is quite likely to give him accurate information about the consequences of his choices.

Comments for this entry have been closed
Return to top