Five rules for good science (and how they can help you spot bad science)

September 22, 2023

I have a few rules that I aim to use when I run studies. By considering what it looks like when these rules are inverted, they also may help guide you in thinking about which studies are not reliable. (1) Don't use a net with big holes to catch a small fish That means you should use a large enough sample size (e.g., number of study participants) to reliably detect whatever effects you're looking for! (2) Don't use calculus to help you assemble IKEA furniture That means...

Importance Hacking: a major (yet rarely-discussed) problem in science

December 19, 2022

I first published this post on the Clearer Thinking blog on December 19, 2022, and first cross-posted it to this site on January 21, 2023. You have probably heard the phrase "replication crisis." It refers to the grim fact that, in a number of fields of science, when researchers attempt to replicate previously published studies, they fairly often don't get the same results. The magnitude of the problem depends on the field, but in psychology, it seems that something like 40% of studies i...

Three big reasons we struggle to find the truth

June 3, 2021

As I see it, there are three main causes for our struggles to see the truth on any particular topic: 1. Mimicry: when our in-group promotes falsity that we copy 2. Incentives: when we predict that knowing the truth would feel bad or harm our objectives 3. Complexity: when the truth is hard to figure out Examples: 1. Mimicry • Some are Christians because all their friends and family are, too; some are atheists for the same reason. • Some think that it makes sense to ...

Many global challenges arise due to collective action problems or incentive misalignment

November 16, 2020

Many of the biggest challenges that we face in society are due to one or both of these types of problems: (A) Collective Action Problems, where many individuals or groups are currently better off taking action X, even though they'd be better off in the long-term if everyone agreed not to take action X. Some of the big challenges with Collective Actions Problems are (i) getting people or groups to agree to stop the behavior in the first place, and then (ii) creating a v...

Incentive misalignment and conflicts of interest

May 26, 2018

Written: May 26, 2018 | Released: June 11, 2021 In my view, fundamental incentive misalignment affects many parts of society. Companies: There are many ways to make money that are net harmful to the world (e.g., see here for 13 ways). Non-profits: (1) The people that sponsor the work of a non-profit organization aren’t the ones that benefit from it. (2) Donors can’t easily tell how much benefit their donation caused the organization to create. (3) Th...

Your Law Firm Does Not Have Your Incentives

August 15, 2011

If you hire a law firm, as an individual or the owner of a small business, there is a pretty good chance they will bill you by the hour. So if the work performed takes 100 hours rather than 50, you will pay them twice as much. From the law firm’s perspective, this is reasonable, because each one of their work hours is about as valuable to them as every other one (holding the specific employees on the project constant). However, if we are justified in assuming that law firms are entities that can...