I.
In the first LessWrong survey, the mean self-reported IQ score of the survey takers was 146 (n=166). According to Scott Alexander, this figure later fell to 140 in 2011, and 139 in 2012, and 138 in 2013. These high self-reported values do not depend on the method used to calculate them, as they were about equally high for all measures in 2012:
SAT (out of 1600): 1485.8 [n = 321]
SAT (out of 2400): 2319.5
ACT: 32.7 [n = 207]
IQ (on iqtest.dk): 125.63 [n = 378]
IQ (self-reported): 138.7 [n = 382]
Intuitively speaking, I doubt the average IQ of a rationalist is that high. Only three of eleven rationalist adjacent public figures score above a 138, based on my estimates of their IQs:
Hanania: 131 (136)
Aella: 132 (133)
Anatoly Karlin: 134 (135)
Lex Friedman: 135 (133)
Greg Cochran: 135 (136)
gwern: 136 (136)
Charles Murray: 136 (138)
Scott Alexander: 137 (135)
Vitalik Buterin: 139 (147)
Curtis Yarvin: 140 (143)
Eliezer Yudkowsky: 143 (143)
In parenthesis I also added my own guess as to what their actual IQ is, but even with these more generous estimates, only 3 score above the rationalist average of 138. I am not alone in thinking that they are not that smart - Emil Kirkegaard surveyed his twitter followers and calculated the median IQ rating of several public figures, and he found that the average rationalist-adjacent public figure (Yudkowsky, Hanson, Cochran, S. Alexander, Yarvin, gwern, C. Murray, Cremieux, Kirkegaard, Khan, G. Miller, Hanania, Karlin, Yglesias, Smith, Aella) had an IQ of 133. Even when you remove the low estimates given to Yglesias, Smith, Karlin, and Aella, the figure only rises to 136.
This is rather odd, as you would expect the leaders of a movement to be somewhat intelligent than their followers. I propose 4 potential solutions to the paradox:
This could be due to diminishing returns of high intelligence, but this is not found for mental health, educational attainment, or anything else. So this is probably not a solution.
Low sample size of public figures: I suppose this is always a possibility, but it doesn’t “feel” like a particularly satisfying solution. Especially when you consider that more intelligent public figures are more likely to have their IQs estimated.
Overreporting: Self-reported SAT scores are about 0.2 SDs higher than measured ones, which probably translates to IQ scores as well.
Selection bias: highly intelligent people are more likely to take tests like the SAT and have their IQ tested, thus they are more likely to report it.
To address the latter possibility, I have decided to analyze the survey of Scott Alexander’s readers to test whether individuals who reported and didn’t report test scores were different in variables that correlate with IQ.
II.
As of the current date, the average self-reported IQ score of one of Scott Alexander’s readers according to the most recent SSC survey is 137. Earlier surveys of the LessWrong community have about the same average IQ as Scott’s readers, which means that this could be used as a reference community for analysis.
The dataset also has a measure of self-reported SAT scores. Once subtest scores above 800 and below 200 are eliminated, the distribution of SAT scores is also skewed towards higher scores:
When these scores are corrected for age (details in the appendix), the average SSC reader would’ve achieved a score of 1524 on the SAT in 2019. This corresponds to the 99.375th percentile in comparison to the nationally representative sample of students. Believe it or not, this is almost exactly equivalent to the percentile of self-reported IQ scores (99.3) within this dataset. This indicates that if there are biases causing the IQ of the rationalists to be overestimated, these must apply to both the self-reported SAT and IQ scores.
Oddly enough, these two variables did not correlate very highly - only a correlation of (r=0.27, n=749) was observed. The correlation between the SATM and SATV scores was somewhat higher (r = 0.35, n=2555), but was still rather low. For reference, SAT scores and IQ correlate at 0.84 within representative samples, and the two sections of the SAT correlate at 0.73. I assume this is due to restriction of range, as correlations within highly selected groups tend to be much lower than what you observe within the general population.
To determine whether there were differences between SAT/IQ test takers and non-takers, the standardized difference (cohen’s d) was used. Individuals who were not American were excluded from the analysis which tested differences between SAT test takers and non takers.
I found that people who reported their SAT scores were different from non-takers in educational attainment, IQ, and mental illness:
no difference in age (p = 0.4)
higher educational attainment (d = 0.24, p < .001)
higher IQ scores (d = 0.46, p < .001)
no difference in happiness (p = 0.48)
possibly lower mental illness rates (d = -0.07, p = 0.03)
This also applied to IQ test reporters as well, though they only tend to be older and more mentally ill:
older (d = 0.35, p < .001)
no difference in educational attainment (lower after accounting for age, p < .01) (d = -0.04, p = 0.23)
lower SAT scores (no difference after controlling for age, p = 0.13) (d = -0.2, p < .001)
no difference in happiness (d = 0, p = 0.9)
more mentally ill (true after controlling for age) (d = 0.26, p < .001)
The IQ test score reporter appears to be different from the SAT test score reporter - IQ test reporters are more likely to be mentally ill, possibly due to the MENSA effect, while the SAT test score reporters are more consistent with what you would expect from a more intelligent population. Oddly enough, self-reported IQ scores do not correlate with mental illness (r = 0.0017, p = 0.94), but self-reported SAT scores do (r = -0.08, p < .001).
Adjusting the mean IQ of the American rationalists by weighing the SAT reporters and non-reporters equally changes the mean from 138.7 to 138.3 (135*0.405 + 140.6*0.595, where 0.595 is the reporting rate for SAT scores and 140.6 is the average IQ of those who reported them).
I can also sample the averages of variables that are highly correlated with intelligence - rationalists should have an educational attainment z-score of 1.2 if their average IQ is 137, if there is no selection for educational attainment, and if IQ and educational attainment correlate at 0.55. Given their distribution of educational attainment (available in Appendix), the average rationalist has an educational attainment z-score of 1.149, which would be consistent with an average IQ of 131.3.
This sets a plausible upper bound of the average IQ of an internet rationalist. There is bound to be some non-zero selection for educational attainment directly, so the true value is probably between 125-130.
IV.
Subtracting the overreporting bias (3 IQ points) and the test-reporting bias (0.4 IQ points) produces an estimated average IQ of about 133.6.
If the average IQ is inferred based on educational attainment and some plausible assumptions, the average IQ is 131.3 instead. Given that this is an upper bound estimate, and that there is probably some direct selection for educational attainment directly, I would estimate that it’s about 125-130.
Appendix
Proof SAT/IQ and being a rationalist do not relate within the SSC dataset:
Linear regressions predicting SAT scores and IQ using age (unstandardized)
calculate SAT score of 20yo: 1569-2.2468*20 = 1524. Note that IQ and age are positively correlated, but SAT and age are negatively correlated.
things i tested:
pfactor is not actually pfactor, it’s a composite variable of job/romantic/personal satisfaction and happiness
mental illness is an IRT score composite of all the mental illness self-reports in the dataset
newage is age
education is degree converted to a numeric variable
IQv is IQ
SAT is SAT
differences between SAT test takers and nontakers was only tested within americans
education distribution (1 - no degree, 2 - hs, 3 - 2 year, 4 - bach, 5 - mast, 6 - phd/prof):
z-scores for edattainment (USA, 2018):
edit - changed mind from average of 133 to average of 125-130
What this says more is that perhaps IQ isn't actually an indicator of being right. Or successful. Maybe it needs a certain personality to work...
I find it interesting that you're so sure about your original estimates for the IQ of rationalist public intellectuals. While you're trying to figure out why there's a difference between the average rationalist IQ and that of the more notable rationalist intellectuals, you're taking for granted that your estimates are right. Maybe it's worth considering that your initial guesses might not be as accurate as you think.
Shouldn't you trust high N, consistently-replicated survey data more than quick-and-dirty estimations?