Always Conduct the “Simplest Valid Analysis”

July 10, 2024

This piece was cross-posted on the Transprent Replications blog. A significant and pretty common problem I see when reading papers in social science (and psychology in particular) is that they present a fancy analysis but don’t show the results of what we have named the “Simplest Valid Analysis” – which is the simplest possible way of analyzing the data that is still a valid test of the hypothesis in question. This creates two potentially serious problems that make me less confident in th...

How to avoid feeding anti-science sentiments

August 13, 2023

A major mistake scientists sometimes make in public communication: they state things science isn't sure about as confidently as things it is sure about. This confuses the public and undermines trust in science and scientists. Some interesting examples: 1) As COVID-19 spread early in the pandemic, epidemiologists confidently stated many true things about it that were scientifically measured (e.g., rate of spread). Some of them were also equally confidently stating things that were just spec...

My rules for making great spreadsheets (in Google Sheets or Excel)

May 5, 2023

1) Round numbers: use "decrease decimal point" or "format" to automatically round numbers to the greatest number of decimal points that are truly useful (so 0.15, not 0.15121215 and 32%, not 32.42%). 2) Set units: use the "format" feature to make percentages into actual percentages (ending in %), to make dollar figures into actual dollar figures (starting with $), and so on. This makes it easier to interpret figures at a glance. 3) Use formulas: anything that can be calcul...

Importance Hacking: a major (yet rarely-discussed) problem in science

December 19, 2022

I first published this post on the Clearer Thinking blog on December 19, 2022, and first cross-posted it to this site on January 21, 2023. You have probably heard the phrase "replication crisis." It refers to the grim fact that, in a number of fields of science, when researchers attempt to replicate previously published studies, they fairly often don't get the same results. The magnitude of the problem depends on the field, but in psychology, it seems that something like 40% of studies i...

Seven reasons why you could be defining a concept ineffectively

March 2, 2022

Note (December 16, 2022): This piece is cross-posted from the Clearer Thinking blog, where it appeared on March 2, 2021. Can a chosen definition be "wrong"? No. If you choose a definition, then you can define a sound or series of characters to mean whatever you want them to mean. For instance, if you wanted, you could declare that whenever you say "phloop," you mean one of those little paper umbrellas that are sometimes found in Piña coladas. That would be weird, but it wouldn't be ...

A Common Problem with Debates

July 30, 2017

Opposing parties in debates over commonly-raised questions often fail to focus their arguments on the same concept. These "ambiguous questions" are usually those that are fundamentally unresolvable until one disambiguation of the question is agreed upon. Until that disambiguation is reached, the debate often swirls in circles as different parties effectively argue about distinct topics, seemingly discussing the same thing. Examples of common "ambiguous questions" include debates like: Is ...