Overconfidence: it depends on how, what and whom you ask. (1999)

“Many studies note that peoples’ confidence in their judgments exceeds their accuracy and that overconfidence increases with the difficulty of the task. However, some common analyses confound systematic psychological effects with statistical effects

Klayman, J., University of Chicago
Soll, J., INSEAD
Gonzalez-Vallejo, C., Ohio University
Barlas, S., Experian Direct Tech

Organizational Behavior and Human Decision Processes 79: 216-247.

Many studies note that peoples’ confidence in their judgments exceeds their accuracy and that overconfidence increases with the difficulty of the task. However, some common analyses confound systematic psychological effects with statistical effects that are inevitable if judgments are imperfect. The authors present three experiments using new methods to separate systematic effects from the statistically inevitable.

They still find systematic differences between confidence and accuracy, including an overall bias toward overconfidence. However, these effects vary greatly with the type of judgment. There is little general overconfidence with two-choice questions and pronounced overconfidence with subjective confidence intervals. Over- and underconfidence also vary systematically with the domain of questions asked, but not as a function of difficulty. The researchers also find stable individual differences.

Determining why some people, some domains, and some types of judgments are more prone to overconfidence will be important to understanding how confidence judgments are made.

Access the full paper here: Overconfidence: it depends on how, what and whom you ask.