Have a hypothesis about the world, society, human nature, physics, or anything else that nobody has directly tested before? It might seem like conducting a costly experiment would be required to find out whether it’s true. But a lot of the time, you can check your hypothesis easily using what I call an “Instantaneous Experiment.”
How to do an Instantaneous Experiment:
Step 1: Think of anything at all about the world that’s checkable that is likely to be true if your hypothesis is true, but that is likely to be false if your hypothesis is false.
Important: this checkable thing should be something that you have never investigated before – in other words, you don’t actually know if it’s true, and the only real reason you think it’s true is just because your hypothesis implies it would be. This is critical to help prevent bias from occurring during the process (for instance, this procedure doesn’t work if the fact you are checking is one that influenced your development of the hypothesis).
Step 2: Go check whether the checkable thing is true or not by trying to look the answer up (e.g., in an article or paper)!
The amount of evidence that the answer provides in favor (or against) your hypothesis precisely depends on how many times more likely you are to see that result if your hypothesis is true compared to if it’s not true. The bigger that number is, the greater the evidence!
Instantaneous Experiments work because, to get evidence for a theory or hypothesis, it is not necessary to directly check whether that thing is true. All you have to do is check something that is implied by that theory (that would be unlikely to be true otherwise).
Here’s an example:
Suppose you believe that “greater intelligence causes people to worry a lot more”
That’s very hard to test. But you can do an Instantaneous Experiment:
Step 1: if intelligence causes worry, then you might expect higher IQ people to agree more often with a statement like “I worry too much,” whereas if the theory is not true, you wouldn’t expect a positive correlation between IQ and agreement with that statement.
Step 2: We go check this, and we find a paper that measures both IQ and the level of agreement on the statement “I worry too much.” The correlation between them is essentially 0.
Result: We haven’t completely disproven the theory, but we should now reduce our confidence in it compared to what we thought before.
How much we reduce our confidence depends on how many times less likely we’d be to find no correlation between self-reported worry and IQ if our hypothesis “greater intelligence causes people to worry a lot more” is true, compared to if it’s false.
This piece was first written on August 24, 2024, and first appeared on my website on October 11, 2024.
Seems like a fun way to explore lots of phenomena that occur in life.