Align user reliance with actual system reliability.
Design mechanisms that prevent both overreliance and avoidance.
Contents 9. AI-Mediated Interaction 9. Trust Calibration
How reliance is adjusted over time.
First, ask yourself
“Is user trust aligned with actual system capability over time?”
Mission statement
Ensure users rely appropriately by aligning perceived and actual system capability without overtrust or undertrust.
Key heuristics Trust Calibration
- User reliance must match actual system reliability.
- Defaults, messaging, and permissions must track measured reliability over time.
- Trust mechanisms must adjust to task type and risk domain.
- Consistent experiential reliability must underpin trust.
- Reliance on delegated actions must be monitored and adjusted.
- Reliance mechanisms must be monitored and adjusted over time.
- Increased reliance must follow demonstrable performance improvement.
- Trust signals and permissions must decrease when performance declines.
- Overtrust and undertrust must be addressed before patterns solidify.
Executive brief
AI systems must actively maintain appropriate user trust over time.
Core questions Trust Calibration
“Is user trust proportional to actual system reliability?”
“Does trust stay aligned with reality as the system evolves?”
Focus areas
Calibration-first
“Are users overrelying or underrelying on the system?”
“Does observed behavior indicate misplaced confidence?”
Drift-aware
“How can trust drift be detected over time?”
“Do improvements or degradations adjust reliance appropriately?”
Transparency-aware
“Are changes in reliability communicated clearly?”
“Can users see when updates affect system behavior?”
AI-aware
“Does AI learning change reliability without changing signals?”
“Are updates recalibrating trust honestly and visibly?”
UX directives Trust Calibration
Keep trust signals current with measured performance.
Update defaults, messaging, and permissions as measured reliability changes over time.
Calibrate trust contextually.
Adjust signaling and controls according to task and risk domain.
Reinforce trust through consistent outcomes.
Prioritize experiential reliability over declarative assurance.
Track reliance in delegated workflows.
Monitor whether users over- or under-rely on delegated actions and adjust oversight accordingly.
Continuously reassess trust alignment.
Monitor and adjust reliance mechanisms over time.
Signal reliability improvements transparently.
Increase permitted reliance only when performance demonstrably improves.
Reduce trust signals when performance declines.
Promptly adjust confidence and permissions in response to degradation.
Provide immediate feedback to correct miscalibration.
Address overtrust or undertrust before patterns solidify.
Executive summary
- Trust Calibration aligns user reliance with actual system reliability, not perceived competence.
- It prevents both overreliance and avoidance by synchronizing signals with measured performance.
- The system must calibrate confidence, defaults, and permissions according to task risk and context.
- Trust must be reinforced through consistent outcomes, not declarative assurance.
- Reliance on delegated actions must be monitored, not assumed.
- Trust Calibration succeeds when reliance adjusts dynamically as performance improves or degrades.
Success indicators
- User reliance matches the system’s actual reliability.
- Confidence signals reflect real system performance.
- Trust signals adjust according to task risk and context.
- Consistent outcomes reinforce appropriate user trust.
- The system corrects overtrust or undertrust through clear feedback.
One-line summary
Before trust can be stable, it must match actual system capability.