Is it a bad idea to broadly tell people to just "trust the science"? I think so.
The reason stems from my thinking that all of the following are important and true (and too often overlooked) regarding science:
1) A lot of science is real AND valuable to society.
2) A lot of "science" is actually fake - see, for instance, a decent percentage of papers in psychology 15 years ago.
3) "Science" (as an approach to knowledge discovery) is one of humanity's greatest inventions - but in pra...
More
social science
Importance Hacking: a major (yet rarely-discussed) problem in science
I first published this post on the Clearer Thinking blog on December 19, 2022, and first cross-posted it to this site on January 21, 2023.
You have probably heard the phrase "replication crisis." It refers to the grim fact that, in a number of fields of science, when researchers attempt to replicate previously published studies, they fairly often don't get the same results. The magnitude of the problem depends on the field, but in psychology, it seems that something like 40% of studies i...
More
How can we look at the same dataset and come to wildly different conclusions?
Recently, a study came out where 73 research teams independently analyzed the same data, all trying to test the same hypothesis. Seventy-one of the teams came up with numerical results across a total of 1,253 models. Across these 1,253 different ways of looking at the data, about 58% showed no effect, 17% showed a positive effect, and 25% showed a negative effect. But that's not even the oddest part.
The oddest part is that despite a heroic attempt to do so, the study authors failed to...
More
It can be shockingly hard just to understand three variables
In science (and when developing hypotheses more generally), it is very common to come across situations where a variable of interest (let’s call this the dependent variable, “Y”) is strongly correlated with at least two other variables (let’s call them “A” and “B”). Here are some examples:
If you’re a psychology researcher investigating possible causes of depression (Y), you may have trouble disentangling the effects of poor sleep quality (A) and anxiety (B), both of which tend to be corre...
More