I first published this post on the Clearer Thinking blog on December 19, 2022, and first cross-posted it to this site on January 21, 2023.
You have probably heard the phrase "replication crisis." It refers to the grim fact that, in a number of fields of science, when researchers attempt to replicate previously published studies, they fairly often don't get the same results. The magnitude of the problem depends on the field, but in psychology, it seems that something like 40% of studies i...
More
social science
How can we look at the same dataset and come to wildly different conclusions?
Recently, a study came out where 73 research teams independently analyzed the same data, all trying to test the same hypothesis. Seventy-one of the teams came up with numerical results across a total of 1,253 models. Across these 1,253 different ways of looking at the data, about 58% showed no effect, 17% showed a positive effect, and 25% showed a negative effect. But that's not even the oddest part.
The oddest part is that despite a heroic attempt to do so, the study authors failed to...
More
It can be shockingly hard just to understand three variables
In science (and when developing hypotheses more generally), it is very common to come across situations where a variable of interest (let’s call this the dependent variable, “Y”) is strongly correlated with at least two other variables (let’s call them “A” and “B”). Here are some examples:
If you’re a psychology researcher investigating possible causes of depression (Y), you may have trouble disentangling the effects of poor sleep quality (A) and anxiety (B), both of which tend to be corre...
More