Discover more from sebjenseb
Statistical Malpractice: Small Penises and Fast Cars
A new preprint has come out alleging that men with small dicks are more likely to want a sports car. Here is their interpretation of their results:
Richardson, D. C., Devlin, J., Hogan, J. S., & Thompson, C. (2023, January 10). Small Penises and Fast Cars: Evidence for a Psychological Link.
In this experiment, we manipulated what men believed about their own penis size, relative to others. We gave them false information, stating that the average penis size was larger than it in fact is, reasoning that, on average, these males will feel that relatively and subjectively their own penis was smaller; compared to those told that the average penis size was smaller than true average. We then asked them to rate how much they would like to own a sports car. These facts and questions were buried amongst other items giving information and asking for product ratings, so that our hypothesis was masked from participants. We found that males, and males over 30 in particular, rated sports cars as more desirable when they were made to feel that they had a small penis.
They do this by telling men that the average penis size is very large (18cm) or very small (10cm), and then later ask them how much they want a sports car. If the men told that the average is large were more likely to want a sports car, that is evidence that insecurity about penis size changes the desire they have for a sports car.
Here is their scatterplot:
Figure 2. Participants' z-scored ratings of luxury sports cars, split by age and size manipulation shown by dots. There was strong evidence for a main effect of the experimental manipulation, as shown by the distribution of model estimates and MPE of 91% (left panel). Lines show the model's estimate for how ratings changed by participants' age (right panel). Grey shaded area shows that for every age under 29, there was little evidence of a difference between experimental groups (MPEs<90%), but for those 29 and an older, the model found strong evidence for a difference between small and large penis size conditions/groups. The effect of age had a strong effect on participants in the large penis group, but those in the small penis group were not affected by age.
There are several issues I have with this paper which I will describe below:
Issue 0: Bad choice of means
I doubt many men are actually going to believe the average dick size is 18cm or even 10cm. An effect would be more likely to be found if the participants were told the mean erect penis is 11cm or 15cm long.
Issue 1: Inconsistency in wording
In the paper, they claim that they either told the men the average penis size is 18cm or 10cm. It turns out this isn’t even true, and the subjects were notified the means were either 18.03 or 12.04 cm based on the dataset they provided.
Issue 2: Interpretation of results
They use Bayesian mixed modeling to determine the probability of direction for the interaction between age and penis size, as well as the main effect. This is the result of their model which they report in their supplemental analysis:
The probability of direction (probability an effect is positive or negative) for the small penis condition is 85%, for the interaction between age and small penis condition it is 94.3%, for the actual age effect it is 99.7%. I am not familiar with Bayesian statistics, but these are nothingburger tier results based on this table:
I’m pretty sure that these statistics aren’t actually interchangeable. The probability of direction is telling you the probability that an effect is positive or negative, while a p-value is telling you the probability of observing the given data based on the null hypothesis. I downloaded the data myself, and it turns out there is no difference in preference for sports cars between the men told the average dick size was small (bigd) and those told the average was large (smalld).
Even when adjusted for the average rating given to the products by raters, the same results were generated:
The interaction is not significant either. (penis$esteem)low here refers to the effect of the subject being told that the average dick size is large.
However, when the conditions were separated into two different datasets, some evidence of an interaction was found:
Regression model predicting preference for sports cars only in individuals told that the average dick size was small (12cm):
Regression model predicting preference for sports cars only in individuals told that the average dick size was big (18cm):
This is likely a function of the Elderly Hispanic Woman effect, where the probability of falsely detecting an interaction increases with the amount of statistical tests you can do. They asked about a lot of different products, so they could have easily reported the interaction that seemed the most plausible.
Summary: stop using Bayesian statistics to make your weak effects look stronger. p = .2 is not an effect.