Self-reporting on surveys seems ridiculously unreliable. People can lie or may not pay attention. People misremember things. People often lack self-insight. And YET, self-reporting fairly often works remarkably well in measuring things. Here are some examples:
(1) In a large study we ran, IQ (measured by performance on intelligence tasks) had a strong correlation with self-reported (remembered) performance on the math portion of the SAT exam (r=0.61, n=714), which most participants would have taken MANY YEARS prior. This suggests that they were neither lying that often nor were their memories (of a long-ago event) that bad. Though I think there was likely some inflation in their self-reported scores, their reported scores contained a lot of signals.
(2) When we asked people how much they agreed with the extremely subjective statement “I feel satisfied with my income” (on a 7-point scale), their responses correlated quite strongly with their self-reported household income (r=0.43, n=639). This suggests that self-reported subjective feelings can map pretty well onto actual realities.
(3) When we asked, “Have you ever been diagnosed with depression by a medical or mental health professional?” it had a reasonable correlation with the level of agreement to the single statement, “I often tell myself that ‘I am not good enough’ (r=0.37, n=509). This suggests that agreement with vague, seeming statements can say a surprisingly large amount about a person.
(4) You might think that even if it’s an anonymous survey, people wouldn’t be willing to admit socially undesirable behaviors or traits, socially stigmatized events, or highly personal things. But, in our experience, many people are willing to indicate that these apply to them. In one study, when asked if they had ever cheated on a romantic partner, we found that 32% of respondents admitted to having done so. In another, when asked whether, in childhood, an adult in their home ever hit, beat, kicked, or physically hurt them, 26% agreed. When asked if they had been raped, more than 20% agreed. While it’s certain that some people lie, many people are willing to honestly talk about their experiences in anonymous surveys.
While it’s important to be cautious about self-reporting on surveys – people can lie, they may not be paying attention, they may not remember, and they may lack self-insight – in my experience, it often works (perhaps surprisingly) well to tap important traits!
This piece was first written on March 29, 2025, and first appeared on my website on April 24, 2025.
Comments