Calibrate confidence to actual reliability.
Refrain from overstating certainty in system outputs.
Contents 9. AI-Mediated Interaction 5. Confidence Signaling
How certainty and uncertainty are conveyed.
First, ask yourself
“Does your system signal uncertainty to users honestly and actionably?”
Mission statement
Ensure users calibrate trust by communicating uncertainty appropriately.
Key heuristics Confidence Signaling
- Expressed confidence must reflect actual system reliability.
- Uncertainty must be surfaced directly, not implied through silence.
- Different sources of uncertainty must be clearly distinguished.
- Presentation polish must not imply greater accuracy than exists.
- Confidence signaling must increase in clarity and prominence as consequence increases.
- Confidence signals must inform recommended action thresholds.
- Expressed confidence levels must remain consistent across updates.
- Confidence signaling must strengthen as workflows become more automated.
- Confidence and uncertainty signals must remain perceivable under urgency.
Executive brief
AI systems must signal confidence in ways users can reliably act on.
Core questions Confidence Signaling
“Does the system clearly communicate how reliable its output is?”
“Do users know what to do when confidence is low?”
Focus areas
Calibration-first
“Is confidence proportional to certainty?”
“Does the system avoid overstating or understating reliability?”
Action-first
“What should users do when confidence is low?”
“Is uncertainty accompanied by clear guidance?”
Risk-aware
“Do higher-risk outcomes get stronger signaling?”
“Are low-confidence results clearly marked?”
AI-aware
“Does the system distinguish fact, inference, and guess?”
“Is uncertainty exposed, not hidden behind fluent output?”
UX directives Confidence Signaling
Surface uncertainty explicitly.
Ensure absence of signal does not imply correctness.
Differentiate uncertainty sources.
Clarify whether uncertainty arises from missing data, model limits, or ambiguity.
Separate presentation fluency from reliability.
Prevent polished output from implying unwarranted accuracy.
Scale confidence signaling to risk level.
Increase clarity and prominence for high-impact decisions.
Tie confidence signals to recommended action.
Provide guidance when reliability falls below threshold.
Maintain stable calibration across updates.
Safeguard against unexplained shifts in expressed confidence levels.
Increase signaling rigor in automated workflows.
Strengthen transparency when actions are delegated.
Ensure clarity under high-pressure conditions.
Design signals that remain perceivable during urgency.
Executive summary
- Confidence Signaling is calibrated reliability communication, not rhetorical certainty.
- It aligns expressed confidence with actual system performance and limitations.
- The system must make uncertainty explicit and differentiate its sources clearly.
- Fluency of presentation must never imply accuracy beyond verified reliability.
- Confidence signals must scale with risk, autonomy, and decision impact.
- Confidence Signaling succeeds when users interpret system outputs with appropriately calibrated trust.
Success indicators
- Confidence levels match the system’s actual reliability.
- Uncertainty is clearly indicated when results may be incorrect.
- Different sources of uncertainty are distinguishable.
- Polished presentation does not imply false certainty.
- Confidence signals clearly guide user decisions.
One-line summary
Before trust can be calibrated, confidence and uncertainty must be visible.