My three kids did some prepping, with books and free online resources. My impression is that it does have an impact but with decreasing marginal returns. I am wondering how the stats account for the multiplicity of test taking including among high scorers. My older son was happy enough with 1590. An East-Asian classmate of his had gotten 1600 (after lots of practice) but did not have a perfect score at the optional Essay test so decided to take the SAT against (where she got a high score but not perfect). Other point: serious preppers will start before they take the PSAT, so the high correlation PSAT - SAT cannot demonstrate that coaching doesn’t work.
Does the data include tests taken abroad in East Asia? Because that draws a very self selected group of test takers and there have been many cheating scandals.
Good article, but you use a lot of studies based on the pre-2016 SAT (e.g correlation of SAT with IQ). I think we need to wait a couple years for good studies to be done on the psychometric properties of the current edition of the SAT before we can know for sure what's going on with rising Asian test scores.
Thoughts on practice effects:
One, I definitely believe they exists (talking from personal experience). Though I would caveat that the practice has to be truly *effortful*. Putting in the hours in half-assed manner because your parents are nagging you won't cut it (though I think aggressive tiger parenting would still work).
I worry a bit about endogeneity: whether or not a student decides to study hard might be related to their own self-assessment of their ability. In plainer language, what if the students who tend to practice the hardest are those who underperformed the most on the first test relative to their expectations. So they work hard to get the score they know they "should" get. That makes me wonder if good old-fashioned regression to the mean might also explain some of the effect of practice.
I also don't find twitter samples too convincing. The type of people to discuss psychometrics and g-loadings of various standardized tests on twitter are going to be, on average, high IQ. People with high IQs who underperform on their initial attempts are precisely those who will benefit the most from practicing.
I have a friend who made good money writing papers for Asian students. Whether they cheat on low stakes IQ tests, I don't know. They have a lot of self-esteem tied to being smart, so maybe. Undoubtedly, Asians are more likely to cheat on high stakes IQ tests, though, like the ones used in hiring.
More generally, I question using standardized test scores to estimate IQ. They are a result of cramming as much as G. If you spend twice as much time doing school work, or get a tutor, you're gonna do much better on the PISA or SAT. I'm not sure how anyone can think otherwise after having had a look at the questions. They're word problems. If education had no effect on PISA performance, education would be pointless.
The other comment section/rest of article are locked without submitting to a 7 day trial for membership thus I am commenting here as opposed to the previous article. Would you expand on your thoughts on Network States (https://thenetworkstate.com/ , it was mentioned in reference to different ways the political pendulum is swinging in a previous blog)?
My hope in asking is learning another way of looking at something where I might be too resolute in my own view of it.
You didn't mention that the SAT scoring method changed. They used to penalize wrong answers. But that was removed so if you can make educated guesses, you will come out ahead. Learning how to make educated guesses takes a lot of practice. So if you work hard at it, you will get better.
In this scenario, the hard working Asians would do a lot better than people who do not practice that much.
My three kids did some prepping, with books and free online resources. My impression is that it does have an impact but with decreasing marginal returns. I am wondering how the stats account for the multiplicity of test taking including among high scorers. My older son was happy enough with 1590. An East-Asian classmate of his had gotten 1600 (after lots of practice) but did not have a perfect score at the optional Essay test so decided to take the SAT against (where she got a high score but not perfect). Other point: serious preppers will start before they take the PSAT, so the high correlation PSAT - SAT cannot demonstrate that coaching doesn’t work.
Does the data include tests taken abroad in East Asia? Because that draws a very self selected group of test takers and there have been many cheating scandals.
Take. A. Look. At. Your. Fucking. Immigration. Policy. (Facepalm in Asian
Good article, but you use a lot of studies based on the pre-2016 SAT (e.g correlation of SAT with IQ). I think we need to wait a couple years for good studies to be done on the psychometric properties of the current edition of the SAT before we can know for sure what's going on with rising Asian test scores.
Thoughts on practice effects:
One, I definitely believe they exists (talking from personal experience). Though I would caveat that the practice has to be truly *effortful*. Putting in the hours in half-assed manner because your parents are nagging you won't cut it (though I think aggressive tiger parenting would still work).
I worry a bit about endogeneity: whether or not a student decides to study hard might be related to their own self-assessment of their ability. In plainer language, what if the students who tend to practice the hardest are those who underperformed the most on the first test relative to their expectations. So they work hard to get the score they know they "should" get. That makes me wonder if good old-fashioned regression to the mean might also explain some of the effect of practice.
I also don't find twitter samples too convincing. The type of people to discuss psychometrics and g-loadings of various standardized tests on twitter are going to be, on average, high IQ. People with high IQs who underperform on their initial attempts are precisely those who will benefit the most from practicing.
It's important to remember that Asian Americans are much smarter than Asians. They are in America due to brain drain.
East Asians are infamous for cheating on the SATs.
https://www.theatlantic.com/education/archive/2016/03/how-sophisticated-test-scams-from-china-are-making-their-way-into-the-us/474474/
https://restofworld.org/2022/chinese-software-cheat-sat-exams/
https://www.washingtonpost.com/news/answer-sheet/wp/2014/10/30/an-account-of-exactly-how-students-cheated-on-sat-in-asia/
I have a friend who made good money writing papers for Asian students. Whether they cheat on low stakes IQ tests, I don't know. They have a lot of self-esteem tied to being smart, so maybe. Undoubtedly, Asians are more likely to cheat on high stakes IQ tests, though, like the ones used in hiring.
More generally, I question using standardized test scores to estimate IQ. They are a result of cramming as much as G. If you spend twice as much time doing school work, or get a tutor, you're gonna do much better on the PISA or SAT. I'm not sure how anyone can think otherwise after having had a look at the questions. They're word problems. If education had no effect on PISA performance, education would be pointless.
The other comment section/rest of article are locked without submitting to a 7 day trial for membership thus I am commenting here as opposed to the previous article. Would you expand on your thoughts on Network States (https://thenetworkstate.com/ , it was mentioned in reference to different ways the political pendulum is swinging in a previous blog)?
My hope in asking is learning another way of looking at something where I might be too resolute in my own view of it.
You didn't mention that the SAT scoring method changed. They used to penalize wrong answers. But that was removed so if you can make educated guesses, you will come out ahead. Learning how to make educated guesses takes a lot of practice. So if you work hard at it, you will get better.
In this scenario, the hard working Asians would do a lot better than people who do not practice that much.