Statistical Malpractice: Small Penises and Fast Cars
A new preprint has come out alleging that men with small genitals are more likely to want a sports car. Here is their interpretation of their results:
Richardson, D. C., Devlin, J., Hogan, J. S., & Thompson, C. (2023, January 10). Small Penises and Fast Cars: Evidence for a Psychological Link.
In this experiment, we manipulated what men believed about their own penis size, relative to others. We gave them false information, stating that the average penis size was larger than it in fact is, reasoning that, on average, these males will feel that relatively and subjectively their own penis was smaller; compared to those told that the average penis size was smaller than true average. We then asked them to rate how much they would like to own a sports car. These facts and questions were buried amongst other items giving information and asking for product ratings, so that our hypothesis was masked from participants. We found that males, and males over 30 in particular, rated sports cars as more desirable when they were made to feel that they had a small penis.
They tested this by testing different groups of men that the average penis size is either very large (18cm) or very small (10cm), and then later asked them how much they want a sports car. If the men told that the average penis is 18cm long were more likely to want a sports car, that is evidence that insecurity about penis size causes men to want a sports car.
This is the scatterplot, where the relationship between age and the desire for a sports care is regressed onto both groups:
Figure 2. Participants' z-scored ratings of luxury sports cars, split by age and size manipulation shown by dots. There was strong evidence for a main effect of the experimental manipulation, as shown by the distribution of model estimates and MPE of 91% (left panel). Lines show the model's estimate for how ratings changed by participants' age (right panel). Grey shaded area shows that for every age under 29, there was little evidence of a difference between experimental groups (MPEs<90%), but for those 29 and an older, the model found strong evidence for a difference between small and large penis size conditions/groups. The effect of age had a strong effect on participants in the large penis group, but those in the small penis group were not affected by age.
There are several issues I have with this paper — the main one is a bad choice of means. I doubt many men are actually going to believe the average erect penis is 18cm or even 10cm. An effect would be more likely to be found if the participants were told the mean was 11cm or 16cm.
Issue 1: Inconsistency in wording
In the paper, they claim that they told the men the average penis size is 18cm or 10cm. It turns out this isn’t even true, and the subjects were notified the means were either 18.03 or 12.04 cm based on the dataset they provided.
The second big issue is the way they modeled their results. They use Bayesian mixed modeling to determine the probability of direction for the interaction between age and penis size, as well as the main effect. This is the result of their model which they report in their supplemental analysis:
The probability of direction (probability an effect is positive or negative) for the small penis condition is 85%, for the interaction between age and small penis condition it is 94.3%, for the actual age effect it is 99.7%. I am not familiar with Bayesian statistics, but these are not impressive when converted to a p-value:
I’m pretty sure that these statistics aren’t interchangeable — the probability of direction is telling you the probability that an effect is positive or negative, while a p-value is telling you the probability of observing the given data based on the null hypothesis. I downloaded the data myself, and it turns out there is no difference in preference for sports cars between the men told the average dick size was small (bigd) and those told the average was large (smalld) (p = .26).
Even when adjusted for the average rating given to the products by raters, the same results were generated:
The interaction is not significant either. (penis$esteem)low here refers to the effect of the subject being told that the average dick size is large.