The Allocation Problem
Congratulations if you’ve made it to Part V (the beginning is here). The allocation problem is easily stated: in America today there are a relatively small number of well-paid career tracks that are also highly satisfying and personally rewarding. I’ve named them as doctor, lawyer, professor, engineer, MBA, scientist. Demand for these careers exceeds supply. How should that limited capacity be allocated?
I’d be the first to admit that these careers do not provide the only path to a fulfilling work life; many other jobs can be personally rewarding. And it’s almost too obvious to state that you can make much more money as a Silicon Valley entrepreneur than along any of these paths. In fact, you probably won’t get wealthy in these jobs; rather, you will become securely well-off. What makes these paths unique is the combination of financial and personal rewards, security and comfort, challenge and esteem.
Potential solutions
First, let’s rule out the simple solution: let anyone who wants to be a doctor, practice as one.
Okay, bad example. Try again: let anyone who wants to practice as an engineer, do so. The bridge won’t fall down just because of a silly little math error, will it?
My point: by definition, a profession allows us to presume that practitioners are well-qualified, which presumes demanding standards, which presume a hurdle that has to be surmounted, a hurdle high enough that not just anyone can overcome it.
*There’s a vast literature in sociology, dating back decades, on how the professions as we know them were forged decades ago. On Google Scholar, enter the two words sociology and professions.
Once you have a society with professions, you have an allocation problem. Not everybody who thinks it would be a cool job gets to be a doctor.
In 21st century America, we solve the problem of allocating entry into the professions by laying out a curriculum that must be followed, and sometimes, by capping that curriculum with an exam, as with the bar exam. Or not; there’s no equivalent of the bar exam on the MBA or engineering paths; instead, the examinations are spread throughout the curriculum, without a final culmination, or even the comprehensive examation typical of the Ph.D. path.
In a word, in American society we solve the allocation problem via schooling. You have to complete medical school to practice as a doctor. But of course, that doesn’t solve the problem so much as kick it back one step: who should be allowed to enter medical school? Well, only people who have graduated from college. Still not a solution—there are far too many interested college graduates.
An option not taken, in contemporary American graduate schools, but seen in other contexts and other societies, would be to admit every interested college graduate into the first year of medical school, and then flunk most of them out in the course of the first year, and continue to flunk more out in subsequent years, until the allocation problem was solved. It’s worth asking why we, as a society, haven’t taken this option. First, it would be enormously expensive, for us collectively, in terms of classrooms and faculty funding, and for students individually (imagine flunking out after the first hundred thousand dollars in loans …). The cost would not only be financial; imagine the suicide rate. Second, the quality of education, for those who do not flunk out, is likely to be impaired (enormous lectures and no mentorship; studying to the test rather than for learning; etc.).
Third, the way we would flunk students is by … giving them tests! So why not save endless amounts of money and grief by giving one big test upfront, and solve the allocation problem before medical school begins?
And now you have the MCAT.
To anticipate an objection: although it’s easy to reject the option of solving the allocation problem within medical school by flunking people out, there remain alternatives to administering an MCAT-style test up front. We could alternatively decree that only the best college students will be granted entry. Let’s explore that option.
Either:
- “Best” means best performing students, as measured by grades, which are generally based on … tests!
Or
- “Best” means best colleges; so, allow any interested graduate from Harvard et al.—e.g., the top-ranked colleges—into medical school, but also, only admit applicants from Harvard et al.
Aside from its obvious absurdity, the second option likewise does no more than kick the problem back a step. Either the question becomes, Who gets into Harvard (answer: high school graduates with the best … test scores!); or the question becomes, Who determines which colleges are “best” (answer: the schools whose entering classes have the best … test scores!)
To school is to test.
To test is to distinguish better from worse performance—all tests (outside Lake Woebegone) have score distributions. All school-based allocation problems can be solved by means of drawing a line at some point on that score distribution. And every schoolman or woman, after drawing that line, can look in a mirror and say, “I made that decision based on merit. I admitted / gave A grades to / passed the better students.”
And yet every one of those tests—in school, before school, and after school—almost certainly betrays socio-economic bias.
Next time you hear that criticism of the SAT, remember Churchill: “standardized testing is the worst possible way of solving the allocation problem for schools—excepting all the other approaches that have been tried.”
But you might not yet be ready to accept that claim. Let’s next attempt to imagine a solution to the allocation problem that does not rest on tests.
For that purpose, it helps to step back from the professions, and from the allocation problem in contemporary America, to sketch a broader canvas.
The allocation problem is fundamental, but neither universal nor necessary
By definition, there can be no allocation problem in a pure caste system: you retain the social position with which you were endowed at birth.
By analysis, the allocation problem cannot be pressing in systems where there is not much inequality. Who cares what position their children get, if up and down are close neighbors?
By contrast whenever inequality is substantial, demand for the up positions must exceed the supply, and the allocation problem rears its head.
However, neither schooling nor tests provide the only possible solution.
In many traditional societies, the allocation problem is solved by what the Chinese call guanxi: greased social connections. Wealthy or socially prominent individuals would have their children accepted into medical school; no one else would enter.
The idea of letting guanxi determine entrance to medical school will be repugnant to many Americans. Scarce resources are to be allocated by merit, however defined. Merit-free systems strike us as inherently corrupt.
To find a satisfactory alternative to tests, we need a procedure that still promises to be keyed to merit, without suffering from the socio-economic bias with which tests have been charged. That commitment to merit is bred deep into Americans. Hence, acceptable solutions to contemporary American problems of allocation must be limited to those based on merit.
The next logical alternative: human judgment. Let people, mature human beings in full possession of all their faculties, decide who should be accepted into medical school, and who turned away. Let these judges be good judges, not corrupt or venal, but public servants charged with fairly allocating access to the good career paths, not only medicine, but law, science and the rest. Let these judgments be fair because based strictly on merit, true merit, global merit, and never false, narrow or partial views of merit, such as the SAT and MCAT give us.
So far, so good: we have completely dispensed with tests, while keeping merit central.
Whoa, Dude—we already have that system in place. It’s called an Admissions Committee, man. Not quite public servants, true, yet these are dedicated staff committed to the careful selection of applicants based on a holistic appraisal.
There’s just one problem: under present conditions these Adcoms require MCAT tests from applicants, and appear to consider test results, along with college grades–which mostly consist of the precipitate of tests–in deciding who to admit.
Biased tests must lead to biased Adcoms.
The solution, it would appear, would be for medical schools to stop requiring the MCAT, and to stop requesting grades. Then the existing Adcom structure could realize the goal of making test-free human judgments of merit.
Except, where exactly would the Adcoms look for evidence of merit, or more exactly, signs of superlative merit? Keep in mind that last year, UCLA’s medical school had space for only 175 of its 10,000 applicants. There weren’t very many incompetents in that pool, I’d wager. What’s required is human judgment capable of processing 10,000 applications, now lacking test scores and grades, and selecting 1 in 50 for admission.
How might this be done? Presumably, there would still be letters of recommendation. Any possible bias there? Hmm … would doctors and teachers, established under the old system, be scrupulously fair in evaluating candidates from marginalized groups who suffered under the old test system? Plus, letter writers would be drawn from a population of tens of thousands, themselves unvetted. Not good enough.
Alright then; candidates can certainly be asked to write personal statements. As anyone knows, essay writing is an ideal means of demonstrating the scientific and mathematical competence we expect of MDs … or maybe not.* And of course, never in the history of the world has it been the case that two people reading independently evaluated the same piece of writing as good (bad).
*This writing sample would be a much more meaningful and appropriate hurdle, of course, if we were evaluating applicants for a program in English Literature. And conversely, even less appropriate if we were evaluating applicants to a graduate program in engineering. Be careful when you see criticisms of statistical test bias coming from Humanities graduates and scholars. Is / should the admission selection process be the same, and use the same evidence weighted the same, across all disciplines?
When all else fails, we may fall back on the epitome of human judgment: the in-person interview, one human being interacting with another. Surely interviews provide a rich enough source of information with which to replace tests?
No? The phrase “interviewer bias” keeps cropping up unbidden? Are you sure that team of white interviewers, who succeeded in white society under its standards, can fairly judge the prospects of [ethnic group]?
A possible solution would be to insure that Admissions Committees are composed of [white] + [ethnic group 1] + [ethnic group 2] + … [ethnic group n], in proportion to the ratio of these groups in the population (not their ratio among existing MDs).
Except, it’s not clear that a straight member of [ethnic group n], the scion of two generations of Baptist preachers, could fairly judge the potential of a gay or transgender applicant, of whatever ethnicity. So really, we need a cross-tabulation of ethnicity X gender on the Adcom. For that matter, thinking back to my eastern Massachusetts childhood, it’s not clear that whites of Italian origin can feel confident they will be judged fairly by whites of Irish origin, and vice versa, suggesting a need to expand that cross-tabulation further.
Once we doubt that an interviewer can be ethnically unbiased, there is no stopping point short of a complete tribalism. The attempt to produce a test-free admission system devolves into an attempt to produce a particular ethnic-gender mix among medical students. Something like these concerns, and that desired endpoint, appears to be behind at least some of the college student protests of 2015-16.
Stepping back
Americans are committed to an ideal of merit-based social advancement
&
Americans are dubious about the fitness of the SAT, and any other kind of test, to serve as a reliable index of comparative merit
&
Americans have long been in the grip of a Lake Woebegone fantasy, where everyone can advance socially, and not just materially.
The purpose of the essay has been to hold the reader’s feet to the fire on this matter, to make it impossible to escape the contradictions inherent in the casual dismissal: “the SAT is a biased test, you know.”
The hard rock on which I attempted to shatter the shell of dismissal has been the impossibility of a fresh start in every generation. I tried to break the fantasy that the child’s social fate can be completely independent of the parent’s fate. In any non-caste social system where mobility is possible but not complete, there must be stickiness and inertia. Better off families will tend to stay that way, for indefinite lengths of time. They work at it.
Any system of advancement keyed to merit must distinguish gradations of merit. In a mass society of hundreds of millions, these gradations must often be fine. It is inherently difficult to reliably and validly measure fine gradations in merit. It may be impossible to devise a non-test-based measure of merit which both yields equally fine gradations, and does so with less bias, than a test constructed like the SAT, LSAT, GRE, MCAT or GMAT.
It is difficult to measure merit except by some form of test. It is almost impossible to prevent advantaged parents a leg up when it comes to demonstrating merit, by test or otherwise. And it is unpalatable in the extreme, in America, to imagine social preferment not keyed to merit.
What then is the alternative to the SAT, MCAT and the rest, understood as one item among several in the dossier submitted to an Admissions Committee?
There will be tests. And the scions of well-situated parents will have an advantage, whatever the test.
Be First to Comment