Design explanations to enable informed judgment.
Ensure explanatory content directly supports user decisions.
Contents 9. AI-Mediated Interaction 4. Explainability
How results are understood.
First, ask yourself
“Can users understand and contest outcomes in your system?”
Mission statement
Ensure users understand outputs by providing clear reasoning and context.
Key heuristics Explainability
- Explanations must directly support informed user decisions.
- Explanations must surface outcome-influencing factors in user-relevant terms.
- Explanations must address the specific output, not only the system in general.
- Explanations must be faithful to what actually produced the output.
- Explanations must show which inputs would change the outcome.
- Explanation mechanisms must strengthen as automation increases.
- Users must be able to question and challenge outcomes with understanding.
Executive brief
AI systems must explain individual outputs that no fixed rule produced, well enough to accept or contest them.
Core questions Explainability
“Is the system’s output clear enough for users to accept, reject, or correct it?”
“Can users make reliable judgments about the result?”
Focus areas
Judgment-first
“Can users tell whether the output makes sense in this context?”
“Do they know when to rely on it and when not to?”
Action-first
“What can users do if they believe the result is wrong?”
“Is disagreement actionable or only theoretical?”
Boundary-aware
“Which parts of the output are certain? Which are inferred?”
“Where does explanation end? Is that limit clear?”
AI-aware
“Is AI’s reasoning transparent enough to support user judgment?”
“Are inferred or generated elements clearly distinguishable from deterministic results?”
UX directives Explainability
Surface causal factors rather than technical mechanisms.
Explain what influenced the outcome in user-relevant terms.
Explain the individual output.
Give the reasons for this result in this case, not only a general account of how the model works.
Keep explanations faithful to the model.
Do not present plausible rationales that did not produce the output.
Show what would change the outcome.
Indicate which inputs, if different, would have produced a different result.
Merged into 72/05—Increase explanatory depth for high-risk outcomes.
Increase transparency as autonomy increases.
Strengthen explanation mechanisms in automated workflows.
Enable informed disagreement.
Allow users to challenge outcomes with understanding.
Merged into 72/06—Enable correction mechanisms alongside explanation.
Executive summary
- Explainability is decision-supporting clarity, not technical disclosure.
- It reveals causal factors in user-relevant terms rather than exposing internal mechanisms.
- The system must explain individual outputs faithfully, including what would have changed them.
- Explanatory depth follows the general provisions on proportionality and automation rigor.
- Explanation must enable informed disagreement and meaningful corrective action.
- Explainability succeeds when users can understand, question, and responsibly act on system outcomes.
Success indicators
- Users can understand why the system produced a result.
- Explanations focus on factors that influenced the outcome.
- Explanations appear when users need to make decisions.
- Users can see what would have changed the result.
- Users can question or correct system outcomes with understanding.
One-line summary
Before users can trust outcomes, they must understand them well enough to agree or disagree.