Every approved action will fail sometimes. The question is not if, but when and how. Most HITL systems optimize for the action's success. The team designs the action. The review process checks the action. The metrics measure the action's success rate. The team's attention is on the success.
The recovery-first principle says: optimize for the action's failure, because failure is inevitable and recovery is optional. Every action will fail. The failures are the system's most informative events. The recovery from the failures is the system's most important capability.
This is counterintuitive. The team's instinct is to optimize for success. The optimization is real. The optimization is reasonable. The success rate is what the customer experiences. But the success rate hides the failure design. The failure design is what determines the system's long-term quality.
This post is about the recovery-first principle — why designing every action as if it will fail produces better HITL decisions than designing for success, how to design HITL systems that ask the recovery question first, and what changes when the team's mental model shifts from success-optimization to recovery-design.
The Success-Optimization Model vs The Recovery-First Model
The two models produce different systems:
The Success-Optimization Model
The team's primary metric is the success rate. How many actions succeeded? The metric is the team's quality measure. The metric drives the team's behavior. The metric rewards the action's success.
The success-optimization model has three problems:
Problem 1: The Success Rate Has Selection Bias
The success rate is biased toward the successful actions. The failed actions are not in the success count. The count is biased toward what worked, not toward what was tried.
The bias is structural. The metrics show the wins. The metrics hide the failures. The team's view is the wins.
Problem 2: The Success Rate Hides the Failure Design
The success rate doesn't measure the recovery design. The action that succeeds has a recovery that's never tested. The recovery's design is unmeasured. The unmeasured design can't be improved.
The hiding is the success-optimization's blind spot. The recovery is invisible. The recovery is unmeasured. The recovery is unimproved.
Problem 3: The Success Rate Punishes Conservative Actions
The reviewer who rejects more actions has a lower success rate but a higher failure prevention rate. The reviewer who approves more actions has a higher success rate but a higher failure occurrence rate. The success rate rewards the approval. The approval may be wrong.
The punishment is the success-optimization's incentive problem. The reviewer optimizes for the success rate. The reviewer approves more. The reviewer is rewarded for the wrong behavior.
The Recovery-First Model
The team's primary metric is the recovery design quality. How well-designed is the recovery for each action? The metric is the team's defense measure. The metric drives the team's behavior. The metric rewards the recovery design.
The recovery-first model has five components:
Component 1: The Recovery Question
Every action has a recovery question: "If this action is approved and fails, what is the recovery?" The question is asked before the action is approved. The question shapes the action's design.
The recovery question is the recovery-first model's foundation. The question is asked early. The question shapes the action.
Component 2: The Recovery Design
The action is designed with the recovery in mind. The action's parameters are chosen to enable recovery. The action's interface is designed for rollback. The action's documentation is prepared for the failed case.
The recovery design is the recovery-first model's output. The design is the action's defensive layer. The layer is the customer's safety net.
Component 3: The Recovery Testing
The recovery is tested before the action is approved. The recovery's mechanism is verified. The recovery's window is measured. The recovery's cost is estimated.
The recovery testing is the recovery-first model's validation. The validation is the action's survivability guarantee.
Component 4: The Recovery Documentation
The recovery is documented. The documentation is in the audit trail. The documentation is the future operator's guide. The documentation is the team's institutional memory.
The recovery documentation is the recovery-first model's record. The record is the action's defensive memory.
Component 5: The Recovery Outcome Loop
The recovery is executed when the action fails. The execution produces learning. The learning is captured. The captured learning is the system's improvement.
The recovery outcome loop is the recovery-first model's mechanism. The mechanism turns the failure into improvement. The improvement reduces future failures.
Why the Recovery-First Model Is Better
The recovery-first model is better for six reasons:
Reason 1: It Captures the Inevitable Event
The failure is inevitable. The recovery is designed for the inevitable. The recovery-first model captures the inevitable event. The success-optimization model ignores it.
The capture is the recovery-first model's primary advantage. The most common event (the eventual failure) is accounted for. The system's design reflects the reality.
Reason 2: It Surfaces the Hidden Risk
The hidden risk is in the recovery. The recovery's mechanism is the hidden risk. The recovery's window is the hidden constraint. The recovery's cost is the hidden limitation. The recovery-first model surfaces all three.
The surfacing is the recovery-first model's secondary advantage. The hidden risks are visible. The visible risks are addressable.
Reason 3: It Drives the Action's Design
The action is designed for recovery. The action's parameters are chosen to enable rollback. The action's interface is designed for the failed case. The design is recovery-aware.
The design is the recovery-first model's tertiary advantage. The action is recoverable by design. The recoverability is the action's quality.
Reason 4: It Enables Honest Decisions
The reviewer can approve riskier actions when the recovery is solid. The reviewer's decision is informed by the recovery design. The decision is honest about the risk and recovery.
The honesty is the recovery-first model's quaternary advantage. The reviewer can be honest. The honest decisions are better than the pretending-to-be-safe decisions.
Reason 5: It Protects the Customer
The customer experiences the failure eventually. The recovery is the customer's protection. The recovery-first model ensures the customer's protection is solid.
The protection is the recovery-first model's fifth advantage. The customer is protected. The protection is the system's quality.
Reason 6: It Builds Institutional Memory
The recovery documentation is the institutional memory. The memory survives reviewer turnover. The memory is the team's long-term capability.
The memory is the recovery-first model's sixth advantage. The institutional memory is built. The memory is the team's resilience.
How to Implement the Recovery-First Principle
The design patterns that implement the recovery-first model:
Pattern 1: The Recovery Question Prompt
The interface prompts the reviewer: "If this action is approved and fails, what is the recovery?" The prompt is structured. The prompt is mandatory for high-stakes actions.
The prompt is the recovery-first model's invitation. The reviewer is invited to engage. The engagement is the recovery-first's foundation.
Pattern 2: The Recovery Field
The interface has a structured recovery field. The reviewer can record the action's recovery mechanism, window, cost, stakeholders. The field is recorded in the audit trail.
The field is the recovery-first's documentation. The aggregated recovery fields are the team's defensive inventory.
Pattern 3: The Recovery Testing Trigger
The interface triggers recovery testing for high-stakes actions. The recovery is tested. The test's outcome is recorded.
The trigger is the recovery-first's validation. The recovery is tested before the action is approved. The validation is the action's survivability guarantee.
Pattern 4: The Recovery Aggregation
The system aggregates the recovery designs. The patterns are surfaced. The team sees which action types have weak recoveries. The improvements are targeted.
The aggregation is the recovery-first's intelligence. The aggregated recoveries are the team's defensive inventory.
Pattern 5: The Recovery Outcome Loop
The system closes the recovery loop. The recovery is executed when the action fails. The execution is captured. The capture is the learning.
The loop is the recovery-first's mechanism. The mechanism turns the failure into improvement. The improvement is the system's quality.
Pattern 6: The Recovery Documentation
The recovery is documented. The documentation is in the audit trail. The documentation is the future operator's guide.
The documentation is the recovery-first's memory. The memory is the team's long-term resilience.
Pattern 7: The Recovery Reward
The reviewer who designs for recovery is recognized. The recognition is in the metrics (the recovery design score). The recognition is in the performance review.
The reward is the recovery-first's motivation. The motivation is the team's alignment with the principle.
The Anti-Pattern: The Success-Only System
The anti-pattern is the success-only system. The system measures the success rate. The system doesn't measure the recovery design. The system rewards the success. The system punishes the failure.
The success-only system is the default. The success is the easy metric. The recovery is the hard metric. The system optimizes for the easy metric.
The success-only system is the most damaging pattern in HITL at scale. The system ignores the most common event. The system punishes the most informative event. The system optimizes for the wrong thing. The system degrades.
The Recovery-First Review Process
The review process that implements the recovery-first principle:
Step 1: The Recovery Question
The reviewer asks: if this action fails, what is the recovery? The question is the recovery-first's foundation.
Step 2: The Recovery Design
The action is designed with the recovery in mind. The parameters are chosen to enable recovery. The interface is designed for the failed case.
Step 3: The Recovery Testing
The recovery is tested. The mechanism is verified. The window is measured. The cost is estimated.
Step 4: The Recovery Documentation
The recovery is documented. The documentation is in the audit trail. The documentation is the future operator's guide.
Step 5: The Decision
The reviewer decides based on the action's success probability AND the recovery design. The decision is honest about the risk and recovery.
Step 6: The Recovery Outcome Loop
When the action fails, the recovery is executed. The execution is captured. The capture is the learning.
What Changes When the Recovery-First Principle Is Implemented
When the recovery-first principle is correctly implemented:
- The action's failures are accounted for in the design
- The hidden risks are surfaced
- The action's design is recovery-aware
- The reviewer's decisions are honest about risk
- The customer's protection is solid
- The institutional memory is preserved
The system is designed for the inevitable. The design is the system's quality. The quality is the customer's experience.
Where Facio Fits
Facio's policy engine encodes the recovery-first trigger. The manifest specifies which action types require the recovery question. The trigger is automatic based on the action's risk.
Facio's metrics measure the recovery design quality. The recovery design score per action type. The patterns are surfaced. The improvements are targeted.
Placet.io's review interface presents the recovery question. The question is calibrated to the action. The recovery field is structured. The documentation is recorded.
The audit trail captures the recovery design. The recovery design, the testing outcome, the execution outcome. The audit trail is the recovery-first's institutional memory.
Facio is built for the recovery-first principle. The failure is inevitable. The recovery is optional. Facio makes the recovery designed.
Key Takeaways
- The recovery-first principle: design every approved action as if it will fail
- Five components: recovery question, recovery design, recovery testing, recovery documentation, recovery outcome loop
- Six reasons the recovery-first model is better: captures the inevitable event, surfaces the hidden risk, drives the action's design, enables honest decisions, protects the customer, builds institutional memory
- Seven design patterns: recovery question prompt, recovery field, recovery testing trigger, recovery aggregation, recovery outcome loop, recovery documentation, recovery reward
- The anti-pattern is the success-only system — measures success rate, ignores recovery design, punishes failure
- Six-step recovery-first review process: recovery question, recovery design, recovery testing, recovery documentation, decision, recovery outcome loop
- Facio + Placet.io implement the recovery-first principle — the trigger is encoded, the metrics measure it, the interface prompts it, the audit trail captures it
Sources: The recovery-first principle analysis draws on the established research on failure-driven design (the documented advantages of designing for failure over designing for success), the operational research on rollback-first architecture in production systems (the documented patterns of systems that recover vs systems that don't), the systems thinking research on inevitable failures (the documented patterns of organizations that plan for failure vs organizations that don't), and the production observations of HITL systems where the recovery-first principle was implemented and produced measurable improvements in system resilience during 2025-2026.