How do people weigh evidence when judging their own confidence?
When deciding whether a faint signal was present at all, people were paradoxically put off by extra evidence in the non-target location, the mirror image of the known bias to feel more confident in discrimination whenever overall evidence is high.
Source
Paradoxical evidence weighting in confidence judgments for detection and discrimination
Study at a glance
- Design
- Human experiment — Four psychophysics experiments comparing detection and discrimination tasks with noisy stimuli (random-dot motion in the lab; flickering luminance or hue patches online), analysed with reverse correlation and, in Exps 3–4, a direct manipulation boosting overall evidence; human results compared against four simulated Bayes-rational observer models
- N
- Separate samples: Exp 1 had ten lab participants each doing many sessions of trials; Exps 2, 3 and 4 were online with 102, 100 and 100 included participants respectively (after preregistered exclusions from 147, 173 and 117 recruited).
- Population
- UCL subject-pool volunteers (Exp 1) and native-English-speaking adults recruited via Prolific (Exps 2–4)
- Outcome
- Decision kernels and confidence kernels: how random fluctuations in evidence in the target and non-target channels (and their sum and difference) predicted detection and discrimination decisions and confidence ratings
Structured fields used in claim comparison tables when every cited study has a complete layer.
What they did
Participants did two kinds of perceptual task and rated their confidence after each choice: discrimination (which of two motion directions or two flickering patches was stronger) and detection (was any signal present at all). The stimuli were deliberately noisy, so the researchers could use reverse correlation to see how random moment-to-moment fluctuations in each channel shifted decisions and confidence. Experiment 1 tested ten people in the lab on moving dots; Experiments 2 to 4 tested about a hundred online participants each on flickering patches, and Experiments 3 and 4 also directly brightened or shifted the hue of both patches on half the trials. The human data were compared with four simulated observers that made optimal Bayesian decisions under different kinds of noise or attention.
What they found
Discrimination decisions depended only on the difference between the two channels, but discrimination confidence also rose with the total amount of evidence, replicating the 'positive evidence bias'; two of the four models (firing-rate noise and goal-directed attention) could reproduce this. In detection, however, all four models predicted that evidence in either channel should make a 'yes' more likely, yet in every experiment people were less likely to say 'present', and less confident when they did, when the non-target channel carried more evidence. In Experiments 3 and 4, boosting overall evidence made people say 'yes' more often and raised confidence in both tasks, and made them less confident in 'no' responses.
The limits
What it doesn't show
None of the four rational models explained the negative weighting in detection, and the authors' preferred explanation, that detection and discrimination confidence share representational resources, was not supported by individual-difference correlations, which the studies were not powered to detect. Some effects did not replicate across experiments (for example, the absence of a sum-evidence effect on detection confidence in Experiments 1 and 2 reversed in Experiments 3 and 4). Experiment 1 had only ten participants, the online experiments had few trials per person and no difficulty calibration, and the evidence-boost manipulation in Experiments 3 and 4 may itself have made focusing on differences between channels a reasonable strategy.
Key terms
- Detection vs discrimination
- Detection asks whether a signal is present at all; discrimination asks which of two alternatives the signal belongs to.
- Positive evidence bias
- The tendency for confidence to be driven more by evidence supporting the chosen option than by evidence against it, equivalent to feeling more confident when total evidence is high.
- Reverse correlation
- An analysis that uses random noise in the stimulus on each trial to work out which moments and features of the evidence pushed people toward a given decision or confidence level.
- Sum vs relative evidence
- Sum evidence is the total evidence in both channels; relative evidence is the difference between them. Only relative evidence matters for a rational discrimination, but sum evidence is informative for detection.
- Bayes-rational observer
- A model agent that makes the best possible decision and confidence judgement given the noisy evidence it has, without any biases or shortcuts.
Flashcards
0 of 11 answers reviewed
Research intelligence for this paper
See its role on concept claims, tensions it is part of, placement history, and related discoveries.
Quiz yourself
In discrimination tasks, what did decisions depend on?
Common questions
Why is it 'paradoxical' that people weigh non-target evidence negatively in detection?
If the question is whether any signal is present, evidence in either location should increase the chance it is. Treating non-target evidence as counting against presence is what you would do if you were discriminating between locations, not detecting.
Does the positive evidence bias mean people are irrational?
Not necessarily. The authors show that rational models with stimulus-dependent noise or goal-directed attention can produce it, so it can arise from optimal use of limited, noisy information.
Why run Experiment 4 with colour?
Luminance on an uncorrected monitor is not perceptually uniform, so a bright deviation could be more salient than an equal dark one. Using a perceptually uniform colour space and counterbalancing target hues ruled out that artefact.
More on Metacognition