HITL and the Confidence Mismatch: Why the Reviewer's Calibration Rarely Matches the Agent's
Every HITL system has two confidence scores on every action. The agent's confidence (the LLM's calibrated or uncalibrated probability that the action is correct). The reviewer's confidence (the human's calibrated or uncalibrated belief that the action should be approved).
The two scores are supposed to align. The system's logic assumes alignment: the agent is confident, the reviewer agrees; the agent is uncertain, the reviewer disagrees. When the alignment holds, the system works. When the alignment breaks, the system produces wrong decisions on both sides.
The alignment almost never holds. The agent is over-confident on actions that should have been reviewed. The reviewer is over-confident on actions they should have rejected. The two calibrations are misaligned. The mismatch is the most pervasive calibration failure in HITL — and the most invisible.
This post is about the confidence mismatch — why the two calibrations diverge, what the divergence costs, and how to design HITL systems that produce alignment between the agent's and the reviewer's confidence.
What the Two Confidences Measure
The two confidences are different in kind. The agent's confidence is computed. The reviewer's confidence is felt. The agent's confidence is reproducible. The reviewer's confidence is contextual. The two are measuring different things.
The Agent's Confidence
The agent's confidence is the LLM's output probability (or its surrogate after calibration). The probability is computed from the token distribution at the action's decision point. The probability reflects the model's "belief" that the action is correct.
The agent's confidence is a single number. The number is reproducible. The number can be calibrated against historical accuracy. The number is what the system uses to route the action to review or to autonomy.
The agent's confidence has known failure modes:
- Over-confidence on familiar patterns — the model is confident because the pattern is familiar, not because the action is correct
- Under-confidence on novel patterns — the model is uncertain because the pattern is novel, not because the action is wrong
- Confidence inflation from context — the model's confidence is higher when the context supports the action (which is when the agent constructed the context)
- Confidence on the wrong dimension — the model is confident about the action's structure, not about the action's correctness
The Reviewer's Confidence
The reviewer's confidence is the reviewer's felt certainty. The certainty is constructed from the reviewer's pattern recognition, the reviewer's experience, the reviewer's emotional state, the reviewer's cognitive load. The certainty is what the reviewer uses to make the decision.
The reviewer's confidence is qualitative. The certainty is reported as high, medium, low. The certainty is influenced by factors the reviewer doesn't recognize. The certainty is correlated with the reviewer's calibration.
The reviewer's confidence has known failure modes:
- Over-confidence on familiar patterns — the reviewer is confident because they've seen the pattern before
- Under-confidence on novel patterns — the reviewer is uncertain because the pattern is new
- Confidence inflation from the agent's reasoning — the reviewer's confidence is higher when the agent's reasoning is clear
- Confidence erosion from the reciprocity problem — the reviewer's confidence tilts toward approval
The two confidences have similar failure modes. The similarity is part of the problem — the two systems are miscalibrated in the same ways.
Why the Confidences Mismatch
The two confidences mismatch for five reasons:
Reason 1: The Confidence Is Computed vs Felt
The agent's confidence is computed from the model's internal state. The reviewer's confidence is felt from the reviewer's cognitive state. The two are computed by different mechanisms. The mechanisms don't share a calibration.
The mechanism difference means the same action can produce different confidence scores. The agent is confident because the model's internal state supports the action. The reviewer is uncertain because their pattern recognition doesn't support the action. The two diverge.
Reason 2: The Confidence Is About Different Things
The agent's confidence is about the action's correctness in the model's view. The reviewer's confidence is about the action's acceptability in the reviewer's view. The two can differ because the reviewer's acceptability may not match the model's correctness.
The action can be correct (in the model's view) but unacceptable (in the reviewer's view). The reviewer rejects an action the agent is confident about. The reviewer is right (from the customer's perspective) or the agent is right (from the policy's perspective). The two views differ.
Reason 3: The Confidence Is Anchored to Different Priors
The agent's confidence is anchored to the model's training data. The reviewer's confidence is anchored to the reviewer's experience. The two priors differ. The same action can have different confidence scores because the priors differ.
The prior difference is structural. The model's prior is across millions of training examples. The reviewer's prior is across hundreds of similar reviews. The two priors produce different confidence scores.
Reason 4: The Confidence Reflects the Decision Already Made
The reviewer's confidence is often the confidence in the decision the reviewer is about to make. The decision anchors the confidence. The confidence is high because the decision feels right. The confidence is low because the decision feels wrong.
The anchoring is post-hoc. The reviewer has already decided. The reviewer expresses confidence in the decision. The confidence is not an independent measure. The confidence is the rationalization of the decision.
The post-hoc anchoring is the most insidious failure mode. The reviewer's confidence is not a calibration signal. The reviewer's confidence is a rationalization of the decision. The confidence is not informative.
Reason 5: The Confidence Is Communicated to Different Audiences
The agent's confidence is communicated to the system. The system's routing logic uses the confidence. The system's calibration depends on the confidence being informative.
The reviewer's confidence is communicated to the system. The system's calibration depends on the reviewer's confidence being informative.
The two confidences are both informative when the two systems are well-calibrated. The two confidences are both uninformative when the two systems are miscalibrated. The confidence mismatch is the calibration failure of both systems.
The Mismatch Patterns
The mismatch has five distinct patterns. Each pattern has different causes, different costs, different remedies.
Pattern 1: The Over-Confident Agent, Calibrated Reviewer
The agent is confident (90%). The reviewer is uncertain (medium). The reviewer rejects. The action was correct.
The cost: the reviewer rejects correctly-acting actions. The customer is denied. The aggregated cost is the lost value to the customer. The remedy: calibrate the agent's confidence downward for this action type.
Pattern 2: The Under-Confident Agent, Calibrated Reviewer
The agent is uncertain (40%). The reviewer is uncertain (medium). The reviewer approves with note. The action was wrong.
The cost: the reviewer approves incorrectly-acting actions. The customer is harmed. The aggregated cost is the harm to the customer and the company's exposure. The remedy: calibrate the agent's confidence upward (or route this action type to a different model).
Pattern 3: The Over-Confident Agent, Over-Confident Reviewer
The agent is confident (90%). The reviewer is confident (high). The reviewer approves without deliberation. The action was wrong.
The cost: the most expensive pattern. The high agent confidence justifies the high reviewer confidence. The high reviewer confidence produces the rubber stamp. The action executes. The customer is harmed. The remedy: flag the action for additional review based on the double-confidence pattern.
Pattern 4: The Under-Confident Agent, Under-Confident Reviewer
The agent is uncertain (40%). The reviewer is uncertain (low). The reviewer rejects without deliberation. The action was correct.
The cost: the customer is denied. The aggregated cost is the lost value. The remedy: flag the action for additional review based on the double-uncertainty pattern.
Pattern 5: The Inverted Confidence
The agent is confident (90%). The reviewer is confident (high) but for a different reason. The reviewer is confident the action is wrong. The reviewer rejects. The action was correct.
The cost: the customer is denied. The remedy: surface the source of the reviewer's confidence. The reviewer's reasoning reveals the inversion. The system can correct.
The Cost of the Mismatch
The mismatch has five distinct costs:
Cost 1: The Wrong Routing
The agent's confidence drives the routing. The over-confident agent routes actions that should have been reviewed. The under-confident agent routes actions that should have been autonomous. The routing is wrong. The wrong routing produces the wrong outcome.
Cost 2: The Wrong Calibration
The team's calibration is based on the agent's confidence. The agent's confidence is miscalibrated. The team's calibration is based on miscalibrated data. The team's calibration is wrong. The team's improvements are misdirected.
Cost 3: The Reviewer's Misplaced Trust
The reviewer trusts the agent's confidence. The trust is misplaced. The reviewer approves when the agent is over-confident. The reviewer rejects when the agent is under-confident. The misplaced trust produces wrong decisions.
Cost 4: The Customer's Harm
The aggregated wrong decisions produce customer harm. The customer is denied when they should be approved. The customer is approved when they should be denied. The customer's experience is degraded. The customer's trust is eroded.
Cost 5: The System's Miscalibration
The system's calibration depends on the alignment between the agent's and the reviewer's confidence. The alignment is broken. The system's calibration is broken. The system's improvements are misdirected.
The Architecture for Confidence Alignment
The architecture that aligns the two confidences:
Layer 1: The Agent Confidence Calibration
The agent's confidence is calibrated against historical accuracy. The calibration produces a reliability diagram per action type. The calibrated confidence is what the system uses for routing.
The calibration is continuous. The agent's reliability is monitored. The calibration is updated. The calibration is the agent's calibration signal.
Layer 2: The Reviewer Confidence Calibration
The reviewer's confidence is calibrated against outcomes. The reviewer who expresses confidence on actions that turn out right is well-calibrated. The reviewer who expresses confidence on actions that turn out wrong is miscalibrated. The calibration is the reviewer's signal.
The calibration is private. The reviewer's calibration is not surfaced as a score. The reviewer's calibration is used to weight the reviewer's confidence in the aggregated metrics.
Layer 3: The Confidence Mismatch Detection
The mismatch between the two confidences is detected. The detection uses the agent's calibrated confidence and the reviewer's calibrated confidence. The detection produces a mismatch signal.
The mismatch signal is per action. The mismatch signal is aggregated over time. The mismatch patterns are surfaced to the team.
Layer 4: The Mismatch-Driven Routing
The routing is adjusted based on the mismatch. The actions where the agent is over-confident are routed to additional review. The actions where the agent is under-confident are routed to a different reviewer pool.
The routing is calibrated to the action's risk. The high-stakes actions get more routing based on the mismatch. The routine actions are not affected.
Layer 5: The Mismatch-Driven Improvement
The mismatch patterns drive improvements. The agent's calibration is improved. The reviewer's calibration is improved. The system becomes more aligned over time.
The improvements are measured. The mismatch rate decreases. The calibration improves. The system's quality improves.
Layer 6: The Confidence Communication
The reviewer is told about the mismatch. The reviewer's pattern is shown. The agent's calibration is shared (in aggregate). The reviewer can calibrate their confidence against the agent's calibration.
The communication is calibrated. The reviewer isn't shown every decision's mismatch. The reviewer is shown patterns. The patterns inform the reviewer's calibration.
The Anti-Pattern: The Single Confidence Display
The anti-pattern is the single confidence display. The system shows the agent's confidence. The system hides the reviewer's confidence. The reviewer is supposed to defer to the agent's confidence.
The single display creates the wrong incentive. The reviewer defers to the agent's confidence. The reviewer's own calibration is irrelevant. The reviewer's judgment is suppressed. The system has only one calibration — the agent's.
The single display is the most common HITL design. The single display is also the most damaging. The system loses the reviewer's calibration contribution. The system has a single point of failure (the agent's calibration). The system is fragile.
What Changes When the Confidences Are Aligned
When the confidences are correctly aligned:
- The agent's calibration is improved
- The reviewer's calibration is improved
- The mismatch is detected and acted upon
- The routing is calibrated to the action's true risk
- The customer harm is reduced
- The system's improvements are targeted
The system has two calibration signals, not one. The two signals cross-validate. The cross-validation catches the failures either signal would miss. The system is more robust.
Where Facio Fits
Facio's policy engine calibrates both confidences. The agent's confidence is calibrated against historical accuracy. The reviewer's confidence is calibrated against outcomes. The mismatch is detected.
Facio's metrics surface the mismatch patterns. The over-confident agent, the under-confident agent, the over-confident reviewer, the under-confident reviewer. The patterns are visible.
Placet.io's review interface shows the calibrated confidence. The reviewer sees the agent's calibrated confidence, not the raw LLM probability. The reviewer can calibrate their own confidence against the calibrated signal.
The audit trail captures both confidences. The agent's calibrated confidence, the reviewer's reported confidence, the outcome. The alignment is measurable.
Facio is built for confidence alignment. The alignment is the calibration's foundation.
Key Takeaways
- The confidence mismatch is the most pervasive calibration failure in HITL — agent and reviewer confidences diverge on most actions
- Five mismatch patterns: over-confident agent + calibrated reviewer, under-confident agent + calibrated reviewer, double over-confidence, double under-confidence, inverted confidence
- Five reasons for the mismatch: computed vs felt confidence, about different things, anchored to different priors, post-hoc rationalization, communicated to different audiences
- Five costs: wrong routing, wrong calibration, misplaced trust, customer harm, system miscalibration
- Six architecture layers: agent calibration, reviewer calibration, mismatch detection, mismatch-driven routing, mismatch-driven improvement, confidence communication
- The anti-pattern is the single confidence display — the reviewer defers, the calibration is fragile, the system has one point of failure
- Facio + Placet.io align the confidences — the engine calibrates both, the metrics surface patterns, the interface shows calibrated confidence, the audit trail measures alignment
Sources: The confidence mismatch analysis draws on calibration research in machine learning (the documented over-confidence of LLMs, the calibration gap between raw probabilities and reliability diagrams), the established psychology of confidence judgment in human experts (the documented biases in expert confidence calibration), the operational research on confidence aggregation in human-AI teams, and the production observations of HITL systems in 2025-2026 where confidence mismatches produced wrong decisions on both sides of the routing boundary.