How Should Students at Elite Universities be Selected?

TL;DR: students with extremely impressive, hard to fake achievements (e.g. chess grandmaster, starting a successful business, skipping 3 grades) should be prioritized. After that, selecting for intelligence using test scores or GPA would be the most effective method of getting the most competent student body. Everything else (assessments of character/personality from personal essays, leadership, volunteering, extracurriculars, interviews) is worthless.

#######################################

A story of an Asian student who had a 97.3% GPA average, multiple extracurriculars (captain of NYC math team, volleyball player, and Jazz musician), and a 1560 SAT score who got rejected from many universities went viral recently. Responses to the story were quite polarized — some said he was trying too hard and deserved the rejections, others said it was an unfortunate incidence of universities discriminating against Asians.

From an objective standpoint, this is clearly a very impressive student. A 1560 on the SAT corresponds to the 99.75th percentile relative to the nation; in IQ notation that would correspond to a score of 142. He also has the close to perfect GPA, in one of the most competitive high schools in New York. Most people with this SAT score are not fantastic students because of regression to the mean, as SAT scores and GPA only correlate at .45. I’d guess that, in terms of latent academic aptitude, he would be about 3 to 4 standard deviations above the mean relative to American students.

from here

If I were the Fuhrer of admissions at an elite university, would I admit this student? Perhaps; It depends on what the rest of the pool looks like. If I were basing admissions on academic ability only then he would probably get in.

Consider that most Harvard students are, academically speaking, are not that impressive. In terms of IQ, the average Harvard students scores at an average of 122 (n = 86) after adjusting for the Flynn Effect. 25% of their students got an SAT score of 1460 or lower.

Many of these unimpressive students are not the “usual suspects”, in fact, race differences in SAT scores at Harvard are fairly small — only about 50 points on the composite scale, and they are only 20%-30% of the student body.

from here

II.

With that out of the way, how should America’s elite be selected? Even the most zealous intelligence researcher would have to admit that test scores only explain so much variance in competence. IQ and job performance, according to the most liberal reviews of the literature, correlate at about 0.5 - 0.6 depending on who you ask. There are a few dissenting studies that say the correlation is overstated by several biases, for what it is worth, both Hu and I have gone over the evidence, and concluded that the high correlations from the previous meta-analyses are defensible.

A correlation of .5 is fairly solid, but there is clearly much more to achievement than just intelligence, as a correlation of .5 can only explain 25% of the variance. So perhaps universities ought to look at more than just test scores, and opt for basing admissions on nonacademic measurements such as extracurricular activities, volunteering, references, and/or interviews to evaluate students based on their noncognitive skills.

Which would be a good idea if any of these methods worked.

Unstructured interviews don’t work well because raters cannot agree on how to rate the candidates. On average, interviewers agree at about .45 when it comes to assessments of traits, which is rather low.

from here

Because of that, interviews don’t predict job performance very well. And neither do references for that matter.

from here

Besides the issues with interviews, nonacademic ratings also suffer from the fact that they simply have little predictive validity in practice.

The following data I cite is from this excellent study which examines the effects of attending highly selective Ivy-Plus universities (Ivy league schools + MIT + Duke + Stanford + UChicago):

Abstract:

Leadership positions in the U.S. are disproportionately held by graduates of a few highly selective private colleges. Could such colleges — which currently have many more students from high-income families than low-income families — increase the socioeconomic diversity of America’s leaders by changing their admissions policies? We use anonymized admissions data from several private and public colleges linked to income tax records and SAT and ACT test scores to study this question. Children from families in the top 1% are more than twice as likely to attend an Ivy-Plus college (Ivy League, Stanford, MIT, Duke, and Chicago) as those from middle-class families with comparable SAT/ACT scores. Two-thirds of this gap is due to higher admissions rates for students with comparable test scores from high-income families; the remaining third is due to differences in rates of application and matriculation. In contrast, children from high-income families have no admissions advantage at flagship public colleges. The highincome admissions advantage at private colleges is driven by three factors: (1) preferences for children of alumni, (2) weight placed on non-academic credentials, which tend to be stronger for students applying from private high schools that have affluent student bodies, and (3) recruitment of athletes, who tend to come from higher-income families. Using a new research design that isolates idiosyncratic variation in admissions decisions for waitlisted applicants, we show that attending an Ivy-Plus college instead of the average highly selective public flagship institution increases students’ chances of reaching the top 1% of the earnings distribution by 60%, nearly doubles their chances of attending an elite graduate school, and triples their chances of working at a prestigious firm. Ivy-Plus colleges have much smaller causal effects on average earnings, reconciling our findings with prior work that found smaller causal effects using variation in matriculation decisions conditional on admission. Adjusting for the value-added of the colleges that students attend, the three key factors that give children from high-income families an admissions advantage are uncorrelated or negatively correlated with post-college outcomes, whereas SAT/ACT scores and academic credentials are highly predictive of post-college success. We conclude that highly selective private colleges currently amplify the persistence of privilege across generations, but could diversify the socioeconomic backgrounds of America’s leaders by changing their admissions practices.

[…]

2 Data

We construct a de-identified dataset on parent characteristics and student outcomes by linking five sources of data: (1) federal income tax records on parents and children’s incomes from 1996-2021; (2) 1098-T tax forms on college attendance from 1999-2015; (3) Pell grant records from the Department of Education’s National Student Loan Data System from 1999-2013; (4) standardized test score data from the College Board from 2001-2005 and every other year from 2007-15 and ACT from 2001-15; and (5) applications and admissions records for undergraduate first-year student admissions spanning subsets of years from 1998- 2015 from several Ivy-Plus colleges and highly selective public flagship universities, as well as data for all colleges in the University of California (UC) and California State University (CSU) systems and all four-year public colleges in Texas from the Texas Higher Education Coordinating Board (THECB). We include data from UC-Berkeley, UCLA, and UT-Austin among others in our sample of highly selective public flagship universities with internal data. These five sets of data were linked to each other at the individual level by social security number and/or identifying information such as name, date of birth, and gender.8

They tracked many of the student’s characteristics, such as their academic ratings, nonacademic ratings, earnings, test scores, and whatnot. Nonacademic ratings did predict reaching the top 1% in earnings after graduating, but this effect disappears after controlling for the effect of attending selective universities.

from here

This was the case for other outcomes of interest that were tracked:

from here

Test scores did predict achievement in this sample, even after controlling for attendance, race, gender, and GPA:

from here

Given that there are no threshold effects for cognitive ability, it should be expected that high scores exhibit no threshold effects. This appears to apply to SAT scores, but not GPA.

from here

As nonacademic ratings made by universities don’t predict anything, my priors are that most conventional selection methods (e.g. volunteering, extracurriculars, interviews, personal essay ratings, references) do not work, which corroborates the results from Hunter and Schmidt’s meta-analysis of predictors of job performance. I’m not sure if this is the case for athletics, though — I would consider it a decent signal of a prosocial and functional personality.

III.

There are other methods of evaluating students besides everything that I brought up earlier. Did the student start their own business? Did they win a national competition in physics? Did they become a chess GM at 14? Did they skip 3 grades? Are they a top athlete? Hard to fake signals of extreme competence and intelligence should be considered the gold standard when it comes to admitting students to elite universities.

After admitting these students, the next priority should be selecting for extremely high levels of measured ability. It’s the best predictor of competence that exists at the moment, and everything else seems to be fairly flimsy in terms of predictive validity.

The fact that the measured average IQ of Harvard students is even lower (122) than what would be expected from regression to the mean suggests that Harvard is probably not admitting students effectively. Consider this: if students were selected purely based on SAT scores, the expected average IQ would be roughly 130. Even if the relationship between IQ and scholastic test scores comes apart at the tails, it would be expected that selection for strong signals of competence should push the average up.

An experimental strategy for elite universities could be selecting for old money: this is not a good idea from an optical point of view, but I think it could work pretty well. Because regression to the mean happens once, this means that selecting for social class itself would not be effective, as there is a chance that the parents of the child succeeded due to environmental factors or non-additive genetic effects. Old money is a little different — if a lineage has managed to stay wealthy for a long time, then that perhaps speaks to their genetic quality. Or maybe it just speaks to a large inheritance that is well-managed. Who knows.

Asians and men should be discriminated against at elite universities — this is because a sex ratio of 50-50 is ideal for social purposes, and because Asians exhibit lower levels of eminence (e.g. extreme wealth, creative achievement) than what would be predicted from intelligence alone.

Under an ideal selection process, would there be room for this Asian student at an elite university? Probably not, now that I look through what admissions policies at elite universities would be in an ideal world. But I can’t exactly fault him for feeling slighted, since the evidence regarding whether non-academic selection mechanisms at Ivy league universities work is weak and favours the null of inefficacy.