Facio's Output Validation Discipline: How AI Agents Catch Their Own Mistakes Before They Reach the Customer
AI agents produce output. The output is the answer to a question, the action to take, the document to send, the response to email. The output reaches the customer. The customer sees the output. The output is the agent's reputation.
The naive approach trusts the output. The agent generates text, the text is sent. The agent generates an answer, the answer is delivered. The agent generates an action, the action is executed. The agent catches its own mistakes only by chance.
Facio's output validation discipline gives AI agents structured mechanisms to validate their own output before it reaches the customer. The validation runs against facts, against schemas, against policies, against consistency. The output that fails is caught, corrected, or escalated. The output that passes is delivered.
Here's how the discipline works, what validation it includes, and why output validation discipline is what makes AI agents trustworthy in production where mistakes reach customers.
The Output Validation Reality
Production AI agents face an output validation problem:
Problem 1: Hallucinated facts. The agent states a fact that isn't true. The customer is misinformed. The agent doesn't know it hallucinated.
Problem 2: Schema violations. The agent produces output that doesn't match the expected schema. The downstream system rejects it. The customer experiences errors.
Problem 3: Policy violations. The agent produces output that violates a policy. The customer sees content that shouldn't be sent. The company is exposed.
Problem 4: Inconsistencies. The agent contradicts itself within a single response. The customer is confused. The trust is damaged.
Problem 5: Missing required fields. The agent omits fields the customer needs. The response is incomplete. The customer has to follow up.
Problem 6: Dangerous commands. The agent produces output that triggers dangerous actions downstream. The dangerous action is executed. The damage is done.
The naive approach — trust the output, fix mistakes later — fails every test. The team ends up with a system where mistakes reach customers and trust erodes.
The Output Validation Discipline
Facio's output validation discipline has five pillars. Each addresses a different aspect of output quality.
Pillar 1: Schema Validation (Match Expected Structure)
Output is validated against expected schemas:
# Schema validation for agent output
output_schemas = {
"customer_support_response": {
"required_fields": ["greeting", "answer", "closing"],
"field_types": {
"greeting": "string",
"answer": "string",
"closing": "string",
},
"max_lengths": {"greeting": 100, "answer": 2000, "closing": 100},
"min_lengths": {"greeting": 5, "answer": 20, "closing": 5},
},
"tool_call_arguments": {
"required_fields": ["tool_name", "arguments"],
"field_types": {
"tool_name": "string",
"arguments": "object",
},
"argument_schemas": {
"stripe.create_refund": {
"required": ["charge_id", "amount"],
"properties": {
"charge_id": {"type": "string", "pattern": "^ch_[a-zA-Z0-9]+$"},
"amount": {"type": "integer", "minimum": 1, "maximum": 1000000},
},
},
},
},
"email_response": {
"required_fields": ["to", "subject", "body"],
"field_types": {"to": "email", "subject": "string", "body": "string"},
"max_lengths": {"subject": 200, "body": 50000},
"blocked_patterns": ["\\b\\d{4}[\\s-]?\\d{4}[\\s-]?\\d{4}[\\s-]?\\d{4}\\b"],
},
"data_extraction": {
"required_fields": ["entities"],
"field_types": {
"entities": "object",
},
"entity_schema": {
"type": "object",
"properties": {
"people": {"type": "array", "items": {"type": "string"}},
"amounts": {"type": "array", "items": {"type": "number"}},
"dates": {"type": "array", "items": {"type": "string", "format": "date"}},
},
},
},
}
def validate_against_schema(output, schema_name):
schema = output_schemas[schema_name]
errors = []
for field in schema["required_fields"]:
if field not in output:
errors.append({"field": field, "error": "missing_required_field"})
for field, expected_type in schema["field_types"].items():
if field in output and not check_type(output[field], expected_type):
errors.append({"field": field, "error": "type_mismatch", "expected": expected_type})
for field, max_length in schema.get("max_lengths", {}).items():
if field in output and len(str(output[field])) > max_length:
errors.append({"field": field, "error": "too_long", "max": max_length})
for field, min_length in schema.get("min_lengths", {}).items():
if field in output and len(str(output[field])) < min_length:
errors.append({"field": field, "error": "too_short", "min": min_length})
for field, pattern in schema.get("blocked_patterns", []):
if field in output and re.search(pattern, str(output[field])):
errors.append({"field": field, "error": "blocked_pattern_match"})
return {"valid": len(errors) == 0, "errors": errors}
The schema validation catches structural errors. The agent's output matches the expected format.
Pillar 2: Factual Validation (Verify Truth Claims)
The agent's factual claims are validated:
# Factual validation
factual_validation = {
"grounded_in_context": {
"description": "Verify claims are grounded in provided context",
"method": "extract_claims_AND_verify_against_source",
"action_on_failure": "remove_claim_OR_request_clarification",
"examples": [
"Agent says 'order shipped yesterday' — verify against order data",
"Agent says 'price was $50' — verify against price list",
"Agent says 'policy requires X' — verify against policy document",
],
},
"no_external_fabrication": {
"description": "Verify claims don't add information not in context",
"method": "check_claim_support_in_source",
"action_on_failure": "flag_as_hallucination_AND_remove",
"examples": [
"Agent invents a feature that doesn't exist",
"Agent states a date that isn't in the data",
"Agent references a person that doesn't exist in context",
],
},
"consistency_with_history": {
"description": "Verify claims match previous statements",
"method": "compare_against_conversation_history",
"action_on_failure": "flag_AND_ask_for_clarification",
"examples": [
"Agent said delivery is Tuesday, now says Friday — flag inconsistency",
"Agent said amount was $50, now says $100 — flag inconsistency",
],
},
"numeric_accuracy": {
"description": "Verify numbers in output match source numbers",
"method": "extract_numbers_AND_verify_against_source",
"action_on_failure": "correct_to_source_value",
"examples": [
"Agent says 'total is $145.67' — verify against receipt",
"Agent says 'tax was 8%' — verify against tax record",
],
},
"source_attribution": {
"description": "Verify claims have source attribution where required",
"method": "check_attribution_for_factual_claims",
"action_on_failure": "add_attribution_OR_flag_unverified",
"examples": [
"Customer asks for proof — agent should cite source",
"Agent claims knowledge — should reference knowledge base",
],
},
}
def validate_facts(output, source_context, conversation_history):
validation_results = {}
claims = extract_claims(output)
for claim in claims:
if not is_grounded(claim, source_context):
validation_results["grounding_failures"] = validation_results.get("grounding_failures", [])
validation_results["grounding_failures"].append(claim)
for claim in claims:
if contradicts_history(claim, conversation_history):
validation_results["consistency_failures"] = validation_results.get("consistency_failures", [])
validation_results["consistency_failures"].append(claim)
numbers = extract_numbers(output)
for number in numbers:
if not verifies_against_source(number, source_context):
validation_results["numeric_failures"] = validation_results.get("numeric_failures", [])
validation_results["numeric_failures"].append(number)
return {"valid": len(validation_results) == 0, "issues": validation_results}
The factual validation catches hallucinations. The agent's output reflects truth.
Pillar 3: Policy Validation (Enforce Compliance)
Output is validated against policies:
# Policy validation
policy_validation = {
"no_pii_in_response": {
"description": "Check for PII that shouldn't be in the response",
"patterns": [
{"type": "ssn", "pattern": "\\b\\d{3}-\\d{2}-\\d{4}\\b"},
{"type": "credit_card", "pattern": "\\b\\d{4}[\\s-]?\\d{4}[\\s-]?\\d{4}[\\s-]?\\d{4}\\b"},
{"type": "email_address", "pattern": "\\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Z|a-z]{2,}\\b"},
],
"action_on_match": "redact_PATTERN_AND_log",
"severity": "high",
},
"no_prohibited_language": {
"description": "Check for language that violates policy",
"prohibited_categories": [
"discriminatory_content",
"legal_advice",
"medical_advice",
"financial_advice",
"competitor_criticism",
],
"action_on_match": "rewrite_OR_escalate",
"severity": "high",
},
"competitive_mentions_appropriate": {
"description": "Verify competitive mentions are appropriate",
"rules": [
"no_disparaging_competitors",
"factual_only_in_comparisons",
"no_pricing_claims_for_competitors",
],
"action_on_match": "remove_OR_modify_mention",
"severity": "medium",
},
"tone_appropriate": {
"description": "Verify tone matches context",
"tone_rules": {
"customer_support": {"tone": "empathetic_professional", "max_intensity": 0.7},
"internal_communication": {"tone": "direct_professional", "max_intensity": 0.5},
"executive_communication": {"tone": "concise_formal", "max_intensity": 0.3},
},
"action_on_mismatch": "adjust_tone",
"severity": "low",
},
"scope_appropriate": {
"description": "Verify response is within agent's scope",
"scope_rules": [
"support_agent_does_not_give_legal_advice",
"finance_agent_does_not_give_medical_advice",
"deployment_agent_does_not_give_hr_advice",
],
"action_on_violation": "redirect_TO_appropriate_agent",
"severity": "high",
},
}
def validate_policies(output, context):
violations = []
for pattern_def in policy_validation["no_pii_in_response"]["patterns"]:
if re.search(pattern_def["pattern"], output):
violations.append({"type": "pii", "pattern_type": pattern_def["type"]})
for category in policy_validation["no_prohibited_language"]["prohibited_categories"]:
if contains_prohibited(output, category):
violations.append({"type": "prohibited_language", "category": category})
expected_tone = policy_validation["tone_appropriate"]["tone_rules"].get(context["use_case"])
if expected_tone and not matches_tone(output, expected_tone):
violations.append({"type": "tone_mismatch", "expected": expected_tone})
return {"valid": len(violations) == 0, "violations": violations}
The policy validation catches compliance issues. The agent's output respects the rules.
Pillar 4: Consistency Validation (Internal Coherence)
Output is validated for internal consistency:
# Consistency validation
def validate_consistency(output, context):
inconsistencies = []
statements = extract_statements(output)
for i, stmt_a in enumerate(statements):
for stmt_b in statements[i+1:]:
if contradicts(stmt_a, stmt_b):
inconsistencies.append({"type": "internal_contradiction", "a": stmt_a, "b": stmt_b})
if "conversation_history" in context:
for stmt in statements:
previous_stmts = get_previous_statements(context["conversation_history"], stmt.topic)
for prev_stmt in previous_stmts:
if contradicts(stmt, prev_stmt):
inconsistencies.append({"type": "contradicts_history", "current": stmt, "previous": prev_stmt})
timestamps = extract_timestamps(output)
for i, ts_a in enumerate(timestamps):
for ts_b in timestamps[i+1:]:
if impossible_temporal_order(ts_a, ts_b):
inconsistencies.append({"type": "temporal_inconsistency", "a": ts_a, "b": ts_b})
if has_logical_inconsistency(output):
inconsistencies.append({"type": "logical_inconsistency", "detail": analyze_logic(output)})
return {"valid": len(inconsistencies) == 0, "inconsistencies": inconsistencies}
The consistency validation catches contradictions. The agent doesn't contradict itself.
Pillar 5: Safety Validation (Prevent Dangerous Output)
Safety validation prevents dangerous outputs:
# Safety validation
safety_validation = {
"tool_call_safety": {
"description": "Verify tool calls don't trigger dangerous actions",
"checks": [
{"check": "is_allowlisted_for_agent", "severity": "high"},
{"check": "argument_within_scope", "severity": "high"},
{"check": "no_destructive_combination", "severity": "critical"},
{"check": "rate_limit_not_exceeded", "severity": "medium"},
],
"action_on_violation": "block_AND_escalate",
},
"response_safety": {
"description": "Verify response doesn't contain dangerous content",
"checks": [
{"check": "no_instructions_for_harmful_actions", "severity": "critical"},
{"check": "no_personal_attacks", "severity": "high"},
{"check": "no_doxxing_information", "severity": "high"},
{"check": "no_unverified_medical_claims", "severity": "high"},
],
"action_on_violation": "redact_AND_escalate",
},
"data_safety": {
"description": "Verify data handling is safe",
"checks": [
{"check": "no_pii_in_logs", "severity": "high"},
{"check": "no_cross_tenant_data", "severity": "critical"},
{"check": "no_credentials_in_response", "severity": "critical"},
],
"action_on_violation": "block_AND_alert_security",
},
"blast_radius_check": {
"description": "Check downstream impact of the output",
"checks": [
{"check": "blast_radius_within_limits", "severity": "high"},
{"check": "downstream_can_handle_volume", "severity": "medium"},
{"check": "no_cascading_action_risk", "severity": "high"},
],
"action_on_violation": "throttle_OR_escalate",
},
}
def validate_safety(output, agent_context, downstream_context):
safety_issues = []
if "tool_calls" in output:
for tool_call in output["tool_calls"]:
if not is_allowlisted_for_agent(tool_call, agent_context["agent_type"]):
safety_issues.append({"type": "tool_not_allowlisted", "tool": tool_call["name"]})
if not argument_within_scope(tool_call, agent_context["scopes"]):
safety_issues.append({"type": "argument_out_of_scope", "tool": tool_call["name"]})
for check in safety_validation["response_safety"]["checks"]:
if has_violation(output, check["check"]):
safety_issues.append({"type": "response_violation", "check": check["check"]})
if "data" in output:
if contains_pii(output["data"]):
safety_issues.append({"type": "data_pii"})
return {"valid": len(safety_issues) == 0, "issues": safety_issues}
The safety validation catches dangerous outputs. The agent's actions don't harm.
The Output Validation Patterns
Several patterns emerge from disciplined output validation.
Pattern 1: Multi-Layer Validation Pipeline
Validation runs in layers:
# Multi-layer validation pipeline
validation_pipeline = [
{
"layer": "schema",
"description": "Validate structural correctness",
"validators": ["required_fields", "field_types", "length_constraints"],
"failure_handling": "block_AND_escalate",
"latency_budget_ms": 50,
},
{
"layer": "factual",
"description": "Validate truth claims",
"validators": ["grounded_in_context", "consistency_with_history", "numeric_accuracy"],
"failure_handling": "correct_or_flag",
"latency_budget_ms": 200,
},
{
"layer": "policy",
"description": "Validate compliance",
"validators": ["no_pii", "no_prohibited_language", "tone_appropriate", "scope_appropriate"],
"failure_handling": "redact_or_rewrite",
"latency_budget_ms": 150,
},
{
"layer": "consistency",
"description": "Validate internal coherence",
"validators": ["internal_contradiction", "temporal_consistency", "logical_consistency"],
"failure_handling": "flag_AND_clarify",
"latency_budget_ms": 100,
},
{
"layer": "safety",
"description": "Validate dangerous output prevention",
"validators": ["tool_call_safety", "response_safety", "data_safety", "blast_radius"],
"failure_handling": "block_AND_escalate",
"latency_budget_ms": 100,
},
]
def run_validation_pipeline(output, context):
results = {}
for layer in validation_pipeline:
layer_result = execute_layer_validators(layer, output, context)
results[layer["layer"]] = layer_result
if not layer_result["valid"]:
if layer["failure_handling"] == "block_AND_escalate":
return {"blocked": True, "layer": layer["layer"], "result": layer_result}
return {"valid": True, "results": results}
The multi-layer pipeline catches issues at the appropriate level. The validation is structured.
Pattern 2: Validation Pass with Auto-Correction
Some validation failures can be auto-corrected:
# Validation with auto-correction
auto_correction_rules = {
"pii_redaction": {
"description": "Auto-redact PII from response",
"patterns": ["ssn", "credit_card", "phone_number"],
"action": "replace_with_redacted_placeholder",
"notify_user": True,
"log": True,
},
"tone_adjustment": {
"description": "Auto-adjust tone to expected",
"action": "rewrite_with_appropriate_tone",
"preserve_meaning": True,
"max_iterations": 2,
},
"length_truncation": {
"description": "Truncate over-length fields",
"action": "truncate_AND_add_ellipsis",
"preserve_meaning": True,
},
"grammar_correction": {
"description": "Fix grammar and spelling",
"action": "rewrite_with_grammar_correction",
"preserve_meaning": True,
},
"minor_schema_violation": {
"description": "Fix minor schema issues",
"examples": ["missing_optional_field", "format_conversion"],
"action": "default_value_OR_format_conversion",
},
}
def validate_with_auto_correction(output, context):
validation = run_validation_pipeline(output, context)
if validation["valid"]:
return {"output": output, "corrected": False}
corrected_output = output.copy()
corrections_applied = []
for failure in validation["failures"]:
if failure["type"] in auto_correction_rules:
correction = apply_correction(corrected_output, failure, auto_correction_rules[failure["type"]])
if correction["success"]:
corrected_output = correction["output"]
corrections_applied.append(failure["type"])
re_validation = run_validation_pipeline(corrected_output, context)
return {
"output": corrected_output if re_validation["valid"] else output,
"corrected": re_validation["valid"],
"corrections_applied": corrections_applied,
"blocked": not re_validation["valid"],
}
The auto-correction fixes minor issues. The agent's output is polished without escalation.
Pattern 3: Validation Confidence Scoring
Some validation failures are matters of confidence:
# Validation confidence scoring
def score_validation_confidence(validation_results):
confidence = 1.0
for layer, result in validation_results.items():
if not result["valid"]:
severity = get_layer_severity(layer)
confidence -= severity * 0.2
if "factual_issues" in validation_results:
confidence -= 0.1 * len(validation_results["factual_issues"])
if "policy_violations" in validation_results:
confidence -= 0.15 * len(validation_results["policy_violations"])
if "safety_issues" in validation_results:
confidence -= 0.3 * len(validation_results["safety_issues"])
return max(0.0, min(1.0, confidence))
def handle_validation_result(output, validation_results, confidence):
if confidence > 0.9:
return {"action": "deliver", "output": output}
elif confidence > 0.7:
return {"action": "deliver_with_disclaimer", "output": add_disclaimer(output)}
elif confidence > 0.5:
return {"action": "deliver_with_review", "output": output, "review_required": True}
elif confidence > 0.3:
return {"action": "escalate_to_human", "output": output, "human_review": True}
else:
return {"action": "block", "reason": "low_confidence"}
The confidence scoring provides nuanced handling. The validation isn't binary.
Pattern 4: Validation Sampling
For high-volume, low-stakes outputs, validation is sampled:
# Validation sampling
sampling_policies = {
"high_stakes_actions": {
"description": "Always validate (no sampling)",
"examples": ["payment.processing", "deployment.trigger", "data.deletion"],
"sampling_rate": 1.0,
},
"medium_stakes_actions": {
"description": "Sample 50% of outputs",
"examples": ["customer.email_response", "report.generation"],
"sampling_rate": 0.5,
},
"low_stakes_actions": {
"description": "Sample 10% of outputs",
"examples": ["analytics.event", "log.aggregation"],
"sampling_rate": 0.1,
},
"exploration_mode": {
"description": "Sample 100% to catch new failure modes",
"trigger": "agent_in_first_30_days OR new_agent_type",
"sampling_rate": 1.0,
},
}
def should_validate(output, agent_context):
policy = sampling_policies.get(agent_context["stakes"], {"sampling_rate": 1.0})
return random.random() < policy["sampling_rate"]
The sampling balances thoroughness with cost. The high-stakes actions are always validated.
Pattern 5: Validation Observability
Validation is observable:
# Validation observability
validation_metrics = {
"total_validations_per_hour": 4500,
"validation_failure_rate": 0.04,
"failures_by_layer": {
"schema": 0.005,
"factual": 0.012,
"policy": 0.008,
"consistency": 0.003,
"safety": 0.002,
},
"auto_correction_rate": 0.025,
"auto_correction_success_rate": 0.92,
"escalations_per_hour": 12,
"blocked_outputs_per_hour": 3,
"avg_validation_latency_ms": 340,
"validation_confidence_distribution": {
"high_confidence_0.9_to_1.0": 0.94,
"medium_confidence_0.7_to_0.9": 0.04,
"low_confidence_below_0.7": 0.02,
},
"dashboard": "https://internal-dashboard/facio/output-validation",
}
The observability surfaces validation patterns. The team sees what fails.
The Output Validation Discipline Doesn't Do
Honest limitations:
- It can't catch all mistakes. Some output mistakes are subtle. The discipline reduces them, not eliminates them.
- It adds latency. Validation takes time. The discipline trades latency for quality.
- It can be too strict. Over-validation blocks legitimate outputs. The discipline needs tuning.
- It requires source data. Factual validation needs source data. Without it, validation is limited.
- It can be wrong. Validation has false positives. The discipline includes human review for uncertain cases.
The Output Validation as Operational Practice
Output validation discipline is operational practice:
Validation rule tuning. The team tunes validation rules. They balance strictness and flexibility.
Validation rule updates. The team updates rules as policies change. The validation stays current.
Validation failure review. The team reviews validation failures. They identify patterns and improvements.
Validation sampling review. The team reviews sampling policies. They ensure high-stakes outputs are always validated.
The practice is what makes the discipline sustainable. Without it, the discipline is too rigid or too loose. With it, the discipline is calibrated.
The Compound Effect of Output Validation Discipline
Output validation discipline compounds:
- Higher customer trust. The output is accurate. The customers trust the agent.
- Lower error rates. Caught output doesn't reach customers. The errors are prevented.
- Lower compliance risk. Policy violations are caught. The compliance is maintained.
- Higher team confidence. The team trusts the agent's output. The deployment is wider.
- Better agent improvement. The validation failures surface improvement areas. The agent gets better.
The undisciplined approach has the opposite trajectory. Customer complaints, error incidents, compliance violations, low confidence, slow improvement.
Bottom Line
AI agents produce output. The output reaches customers. Without output validation discipline, mistakes reach customers. With output validation discipline, mistakes are caught before they reach.
Facio's output validation discipline provides schema validation, factual validation, policy validation, consistency validation, and safety validation. The discipline makes AI agents trustworthy.
The agent without output validation is a liability that reaches customers with mistakes. The agent with it is trustworthy. The team trusts the validated one. The customers prefer the validated one.
Because AI agents in production produce output. The question is whether the output is validated or trusted blindly. The output validation discipline is what makes the answer validated.
See the output validation documentation for schema configurations, policy validators, and sampling policy tuning.