HITL and the Stop Rule: Why Every Reviewer Should Have a Personal Threshold for Rejecting on Sight
Every experienced reviewer has a stop rule — a personal threshold below which they reject the action immediately, regardless of the policy. The stop rule is not in the manifest. It's not in the training. It's not in the metrics. It's in the reviewer's intuition, accumulated over hundreds of similar reviews.
The stop rule is the reviewer's gut feeling that something is wrong. The action looks correct on paper. The context supports the action. The policy permits the action. But the reviewer has seen enough similar actions that they recognize the failure pattern. The reviewer rejects on sight. The reviewer's pattern recognition beats the policy's formal analysis.
The stop rule is the most underrated signal in HITL. The stop rule catches what the policy misses. The stop rule encodes the reviewer's experience. The stop rule is what makes experienced reviewers more valuable than novice reviewers.
But the stop rule is also the most invisible. The reviewer doesn't articulate the stop rule. The reviewer doesn't record the stop rule. The stop rule is in the reviewer's intuition. The intuition is unmeasurable. The system's metrics don't capture the stop rule.
This post is about the stop rule — what it is, why it matters, how to surface it, and what changes when the system's rules and the reviewer's stop rules are aligned.
What the Stop Rule Is
The stop rule is a learned threshold. The reviewer has evaluated hundreds of similar actions. The reviewer has observed which actions led to failures and which led to successes. The reviewer has internalized the pattern. The reviewer applies the pattern without conscious thought.
The stop rule has five characteristics:
Characteristic 1: It's Learned
The stop rule is not inborn. The reviewer develops it over hundreds of reviews. The novice reviewer doesn't have it. The experienced reviewer has it. The expert reviewer has it strongly.
The learning is implicit. The reviewer doesn't sit down and write a stop rule. The reviewer absorbs the pattern through repeated exposure. The stop rule emerges from the reviewer's experience.
Characteristic 2: It's Pattern-Based
The stop rule is triggered by patterns, not by specific facts. The reviewer doesn't reject because "the customer's account was created on this date." The reviewer rejects because "this pattern looks like the failure pattern I've seen before."
The pattern-based trigger is what makes the stop rule fast. The reviewer recognizes the pattern without analyzing the specific facts. The recognition is immediate. The rejection is fast.
Characteristic 3: It's Below Conscious Awareness
The stop rule operates below conscious awareness. The reviewer doesn't think "I'm applying my stop rule now." The reviewer thinks "this is wrong." The thought is the stop rule's output. The reviewer's consciousness is the stop rule's surface.
The below-awareness operation is what makes the stop rule invisible. The reviewer can't articulate it. The team can't measure it. The stop rule is real but unmeasurable.
Characteristic 4: It's Calibrated to Experience
The stop rule is calibrated to the reviewer's specific experience. The reviewer who worked in fraud detection has different stop rules from the reviewer who worked in customer service. The reviewer who worked in healthcare has different stop rules from the reviewer who worked in finance.
The calibration is unique to each reviewer. The stop rule is the reviewer's signature. The stop rule is what makes each reviewer distinctive.
Characteristic 5: It's Often Right
The stop rule is often right. The reviewer's intuition catches failures the policy misses. The reviewer's experience detects patterns the formal analysis doesn't see. The stop rule is the reviewer's value.
The "often right" characteristic is what justifies the stop rule's existence. The stop rule is wrong sometimes. But the stop rule's right rate is higher than the policy's right rate on the actions the stop rule targets. The stop rule is a net positive.
Why the Stop Rule Matters
The stop rule matters for five reasons:
Reason 1: It Catches What the Policy Misses
The policy is a formal rule. The policy encodes the team's agreed-upon decisions. The policy is rigid. The policy misses the edge cases.
The stop rule is the reviewer's informal pattern. The stop rule encodes the reviewer's experience. The stop rule is adaptive. The stop rule catches the edge cases.
The stop rule is the policy's complement. The policy handles the routine. The stop rule handles the exception. Together, they cover the action space better than either alone.
Reason 2: It Encodes the Reviewer's Expertise
The stop rule is the reviewer's expertise made operational. The reviewer who has evaluated thousands of actions has absorbed patterns that can't be taught in training. The patterns are in the stop rule. The stop rule is what makes the experienced reviewer valuable.
The expertise encoding is the stop rule's most important value. The expertise is hard to transfer. The expertise is hard to measure. The expertise is in the stop rule.
Reason 3: It Improves with Experience
The stop rule improves as the reviewer sees more actions. The novice reviewer has a weak stop rule. The experienced reviewer has a strong stop rule. The expert reviewer has a highly calibrated stop rule.
The improvement is the stop rule's self-calibration. The stop rule doesn't need external calibration. The stop rule calibrates through use. The reviewer who uses the stop rule correctly strengthens the stop rule. The reviewer who uses the stop rule incorrectly weakens the stop rule.
Reason 4: It's Fast
The stop rule is fast. The pattern recognition is immediate. The rejection is fast. The fast rejection saves the reviewer's time. The saved time is the stop rule's efficiency contribution.
The speed is the stop rule's most underrated property. The experienced reviewer is faster because of the stop rule. The speed is what makes the experienced reviewer more valuable than the formal analysis.
Reason 5: It's a Signal of Reviewer Quality
The stop rule is a signal of the reviewer's quality. The reviewer with a strong, well-calibrated stop rule is a high-quality reviewer. The reviewer with a weak, miscalibrated stop rule is a lower-quality reviewer.
The signal is measurable. The reviewer's stop-rule hits and misses can be tracked. The tracking reveals the reviewer's calibration. The calibration reveals the reviewer's quality.
Why the Stop Rule Is Invisible
The stop rule is invisible for five reasons:
Reason 1: It Operates Below Awareness
The stop rule operates below conscious awareness. The reviewer doesn't think "I'm applying my stop rule." The reviewer thinks "this is wrong." The below-awareness operation makes the stop rule hard to articulate.
The reviewer's introspection is unreliable. The reviewer can't reliably report what triggered the rejection. The reviewer's report is post-hoc rationalization. The post-hoc rationalization hides the stop rule.
Reason 2: It's Not Recorded in the Audit Trail
The audit trail records the decision and the reasoning. The reasoning is the reviewer's post-hoc justification. The reasoning doesn't include "my stop rule triggered." The audit trail doesn't capture the stop rule.
The audit trail's structure hides the stop rule. The audit trail is designed for the formal reasoning. The stop rule is informal. The audit trail can't capture the informal.
Reason 3: It's Not Trained
The training teaches the policy. The training teaches the action types. The training teaches the reasoning structure. The training doesn't teach the stop rule. The stop rule is learned through experience, not through training.
The training's omission makes the stop rule invisible. The team doesn't know what to train. The team doesn't know how to train it. The stop rule remains untaught.
Reason 4: It's Not Measured
The metrics measure the decision. The metrics measure the reasoning length. The metrics measure the time. The metrics don't measure the stop rule. The stop rule is unmeasured.
The metrics' omission makes the stop rule invisible. The team doesn't see the stop rule in the data. The team doesn't know the stop rule exists. The team concludes the stop rule doesn't matter.
Reason 5: It's Discouraged
The metrics reward approval velocity. The reviewer who rejects on sight slows the queue. The reviewer who rejects on sight is penalized. The reviewer learns to suppress the stop rule. The suppression makes the stop rule invisible.
The discouragement is structural. The metrics shape the reviewer's behavior. The metrics discourage the stop rule. The stop rule is suppressed.
How to Surface the Stop Rule
The patterns that surface the stop rule:
Pattern 1: The Stop Rule Field
The interface has a structured stop rule field. The reviewer can record "this triggered my stop rule" with the specific pattern they recognized. The field is optional. The field is recorded in the audit trail.
The stop rule field gives the reviewer a place to articulate the intuition. The articulation may be incomplete. The articulation is better than silence.
Pattern 2: The Stop Rule Aggregation
The aggregated stop rules tell the team which patterns the reviewers are recognizing. The patterns are surfaced. The team can investigate. The team can formalize the patterns that are common.
The aggregation is the stop rule's team value. The aggregated stop rules are the system's informal policy. The team's formal policy can be updated based on the aggregated stop rules.
Pattern 3: The Stop Rule Recognition
The reviewer is recognized for high-quality stop rule hits. The recognition is in the metrics (the calibration score). The recognition is in the performance review.
The recognition reverses the discouragement. The reviewer is rewarded for the stop rule. The reviewer strengthens the stop rule. The stop rule's quality improves.
Pattern 4: The Stop Rule Training
The reviewer is trained on the stop rule. The training explains what the stop rule is. The training provides examples from experienced reviewers. The training teaches the reviewer to articulate the stop rule.
The training is supplementary. The training doesn't replace the experience. The training accelerates the experience. The training gives the reviewer a vocabulary for the intuition.
Pattern 5: The Stop Rule Documentation
The team's most experienced reviewers document their stop rules. The documentation is shared. The documentation is reviewed. The documentation becomes part of the training.
The documentation is the stop rule's institutional memory. The institutional memory persists beyond individual reviewers. The stop rule is preserved across turnover.
Pattern 6: The Stop Rule Formalization
The common stop rules are formalized into the policy. The policy is updated. The stop rule is now policy. The reviewer's intuition becomes the team's rule.
The formalization is the stop rule's evolution. The stop rule starts as intuition. The stop rule becomes policy. The policy is the team's institutional knowledge.
Pattern 7: The Stop Rule Calibration
The stop rule is calibrated against outcomes. The reviewer with high stop-rule hits and low stop-rule misses is well-calibrated. The reviewer with low hits or high misses is miscalibrated.
The calibration is the stop rule's measurement. The measurement enables the calibration to improve. The calibration enables the reviewer to improve.
The Anti-Pattern: The Approval Pressure
The anti-pattern is the approval pressure. The metrics reward approval velocity. The reviewer is pressured to approve. The reviewer's stop rule is suppressed. The reviewer approves actions that the stop rule flagged.
The approval pressure is structural. The metrics shape the behavior. The behavior shapes the stop rule. The stop rule is suppressed.
The approval pressure is the stop rule's enemy. The stop rule requires the reviewer to reject on sight. The rejection slows the queue. The slowdown is penalized. The stop rule is suppressed.
The Stop Rule in Practice
Consider an experienced reviewer evaluating refund actions. The reviewer has processed 5000 refunds. The reviewer has seen the patterns. The reviewer has internalized the failure patterns.
Action 1: The Routine Refund
The customer requests a refund of $487. The customer's history is clean. The refund is within policy. The reviewer approves. The stop rule is not triggered.
Action 2: The Pattern Flagged Refund
The customer requests a refund of $487. The customer's history shows three refunds in the past 30 days. The customer's account was created 7 days ago. The customer's email was verified 3 days ago. The customer is using a VPN.
The reviewer recognizes the pattern. The reviewer has seen this pattern before. The pattern is the fraud pattern. The reviewer rejects on sight. The stop rule triggers.
The action was correct on paper. The policy permits the refund. But the pattern flags the action. The reviewer rejects. The reviewer's stop rule catches what the policy missed.
Action 3: The Edge Case
The customer requests a refund of $487. The customer's history shows one refund in the past year. The customer's account is 6 months old. The customer's email is verified. The customer is not using a VPN. But the refund reason is vague.
The reviewer's stop rule is uncertain. The pattern doesn't fully match the fraud pattern. The reviewer asks the customer for more information. The reviewer doesn't reject on sight. The reviewer escalates to ask.
The stop rule's strength is calibrated to the pattern's clarity. The clear pattern triggers immediate rejection. The unclear pattern triggers escalation. The reviewer's calibration is the stop rule's quality.
What Changes When the Stop Rule Is Surfaced
When the stop rule is correctly surfaced:
- The reviewer's intuition is articulated and recorded
- The team's policy is updated based on the aggregated stop rules
- The reviewer's calibration improves
- The system's catch rate improves
- The audit trail captures the informal reasoning
- The institutional memory is preserved across turnover
The reviewer is doing the work the policy missed. The work is the stop rule. The work is supported by the system.
Where Facio Fits
Facio's policy engine supports the stop rule field. The manifest specifies which action types enable the stop rule field. The field is recorded in the audit trail.
Facio's metrics aggregate the stop rules. The common patterns are surfaced. The team can formalize the patterns into the policy.
Placet.io's review interface presents the stop rule field. The reviewer can record the stop rule. The reviewer is recognized for high-quality stop rule hits.
The audit trail captures the stop rule. The intuition is recorded. The institutional memory is preserved.
Facio is built for the stop rule. The stop rule is the reviewer's intuition made operational. Facio makes it visible.
Key Takeaways
- The stop rule is the reviewer's personal threshold for rejecting on sight — the most underrated signal in HITL
- Five characteristics: learned, pattern-based, below conscious awareness, calibrated to experience, often right
- Five reasons it matters: catches what policy misses, encodes expertise, improves with experience, is fast, signals reviewer quality
- Five reasons it's invisible: below awareness, not recorded, not trained, not measured, discouraged
- Seven design patterns: stop rule field, aggregation, recognition, training, documentation, formalization, calibration
- The anti-pattern is the approval pressure — the metrics discourage the stop rule, the reviewer suppresses it
- Facio + Placet.io surface the stop rule — the field is in the interface, the aggregation is in the metrics, the formalization is in the policy, the calibration is in the audit trail
Sources: The stop rule analysis draws on the established research on expert intuition (Klein's recognition-primed decision model, the documented pattern recognition in experienced experts), the cognitive psychology research on implicit learning (the documented formation of intuitive thresholds through repeated exposure), the operational research on review quality in high-volume contexts (the documented advantage of experienced reviewers over novice reviewers), and the production observations of HITL systems where stop rules were surfaced and formalized into the policy during 2025-2026.