AI agents produce work. The work flows downstream — to databases, to APIs, to humans, to other agents. The downstream systems have capacity limits. The databases can handle N writes per second. The APIs throttle at M requests per minute. The human reviewers can review K items per hour. The dependent agents can process P tasks per minute.
The naive approach floods the downstream regardless. The agent produces work as fast as it can. The downstream gets backed up. The timeouts start. The errors cascade. The system collapses.
Facio's backpressure discipline gives AI agents the mechanisms to sense downstream capacity and adjust their production rate. The agent doesn't outrun the downstream. The agent doesn't outrun the humans. The agent flows at the pace the system can absorb.
Here's how the discipline works, what backpressure it includes, and why backpressure discipline is what makes AI agents sustainable in production where downstream capacity is finite.
The Backpressure Reality
Production AI agents face a backpressure problem:
Problem 1: Downstream is finite. Every downstream system has limits. Databases have write throughput. APIs have rate limits. Humans have review capacity. The agent ignores the limits at the system's peril.
Problem 2: Errors cascade under overload. When the downstream is overloaded, it returns errors. The agent sees errors, retries, makes the overload worse. The cascade spreads.
Problem 3: Latency degrades. When the downstream is at capacity, latency degrades. The agent waits. The user's request takes longer. The user is unhappy.
Problem 4: Data loss in queues. When queues are full, the oldest items get dropped. The agent's work is lost. The user is affected.
Problem 5: Human review burnout. When humans are flooded with review requests, the quality of review drops. The humans approve things they shouldn't. The system becomes unsafe.
The naive approach — produce as fast as possible — fails every test. The team ends up with a system where the agent is the bottleneck for cascading failures.
The Backpressure Discipline
Facio's backpressure discipline has five pillars. Each addresses a different aspect of capacity-aware flow.
Pillar 1: Capacity Sensing (Know the Downstream)
The agent senses the capacity of downstream systems:
# Capacity sensing
downstream_capacity = {
"postgres.production_db": {
"current_writes_per_second": 1200,
"max_sustained_writes_per_second": 1500,
"current_utilization_pct": 80,
"queue_depth": 45,
"p99_latency_ms": 120,
"degraded": False,
},
"stripe.api": {
"current_requests_per_second": 95,
"max_requests_per_second": 100,
"current_utilization_pct": 95,
"queue_depth": 230,
"p99_latency_ms": 850,
"degraded": True,
"rate_limit_remaining": 50,
},
"human_review.queue": {
"pending_items": 1247,
"max_pending_items": 1500,
"current_utilization_pct": 83,
"avg_review_time_seconds": 240,
"reviewers_available": 8,
"reviewers_at_capacity": False,
},
"elasticsearch.index": {
"current_writes_per_second": 4500,
"max_writes_per_second": 5000,
"current_utilization_pct": 90,
"queue_depth": 1200,
"p99_latency_ms": 450,
"degraded": True,
},
}
def get_downstream_capacity(service):
return downstream_capacity.get(service, {"utilization_pct": 0, "degraded": False})
The sensing provides real-time visibility into downstream capacity. The agent knows what's possible.
Pillar 2: Adaptive Throttling (Adjust Production Rate)
The agent adjusts its production rate based on downstream capacity:
# Adaptive throttling
def calculate_production_rate(agent_id, targets):
rates = {}
for target in targets:
capacity = get_downstream_capacity(target)
if capacity["degraded"]:
utilization = capacity["current_utilization_pct"]
if utilization > 95:
rates[target] = {"rate_per_second": 10, "reason": "near_capacity"}
elif utilization > 80:
rates[target] = {"rate_per_second": 50, "reason": "high_load"}
else:
rates[target] = {"rate_per_second": 100, "reason": "normal"}
else:
rates[target] = {"rate_per_second": 500, "reason": "healthy"}
return rates
# Example: agent producing data to three downstream systems
production_rates = calculate_production_rate(
agent_id="agent-data-pipeline-007",
targets=["postgres.production_db", "stripe.api", "elasticsearch.index"]
)
# Result:
# {
# "postgres.production_db": {"rate_per_second": 500, "reason": "healthy"},
# "stripe.api": {"rate_per_second": 10, "reason": "near_capacity"},
# "elasticsearch.index": {"rate_per_second": 50, "reason": "high_load"},
# }
# The agent's effective rate is the minimum: 10/second
The adaptive throttling ensures the agent doesn't outrun the slowest downstream. The flow is balanced.
Pillar 3: Queue Management (Buffer Without Drowning)
When production temporarily exceeds capacity, queues buffer the work:
# Queue management
queue_policies = {
"postgres.write_queue": {
"max_size": 10000,
"overflow_strategy": "reject_oldest",
"overflow_action": "alert_AND_fail_fast",
"drain_strategy": "fifo_with_priority",
"estimated_drain_time_seconds": 8,
},
"human_review.queue": {
"max_size": 1500,
"overflow_strategy": "reject_new",
"overflow_action": "alert_AND_route_to_secondary_reviewer",
"drain_strategy": "priority_based",
"estimated_drain_time_seconds": 1800,
},
"stripe.api_queue": {
"max_size": 500,
"overflow_strategy": "block_producer",
"overflow_action": "agent_throttles_AND_reports",
"drain_strategy": "fifo_with_retry_backoff",
"estimated_drain_time_seconds": 5,
},
"elasticsearch.ingest_queue": {
"max_size": 50000,
"overflow_strategy": "drop_lowest_priority",
"overflow_action": "log_drops_AND_alert",
"drain_strategy": "bulk_optimized",
"estimated_drain_time_seconds": 12,
},
}
def enqueue_with_backpressure(queue_name, item):
queue = queue_policies[queue_name]
if queue_size(queue_name) >= queue["max_size"]:
if queue["overflow_strategy"] == "reject_oldest":
evict_oldest(queue_name)
elif queue["overflow_strategy"] == "reject_new":
return {"rejected": True, "reason": "queue_full"}
elif queue["overflow_strategy"] == "block_producer":
wait_for_drain(queue_name)
elif queue["overflow_strategy"] == "drop_lowest_priority":
drop_lowest_priority(queue_name)
return enqueue(queue_name, item)
The queue management prevents catastrophic overload. The strategies differ by downstream characteristics.
Pillar 4: Flow Control Signals (Communicate Pressure)
The agent and downstream exchange flow control signals:
# Flow control signals
flow_control_signals = {
"ready_to_receive": {
"description": "Downstream is ready for more work",
"signal": "actor_ready",
"trigger": "queue_depth < 50% of max",
"agent_action": "increase_production_rate",
},
"slow_down": {
"description": "Downstream is approaching capacity",
"signal": "actor_slow_down",
"trigger": "queue_depth > 70% of max OR utilization > 80%",
"agent_action": "reduce_production_rate",
},
"near_capacity": {
"description": "Downstream is near capacity",
"signal": "actor_near_capacity",
"trigger": "queue_depth > 90% of max OR utilization > 95%",
"agent_action": "minimal_production_rate",
},
"overflow_imminent": {
"description": "Downstream is about to overflow",
"signal": "actor_overflow_imminent",
"trigger": "queue_depth > 95% of max",
"agent_action": "stop_production_AND_alert",
},
"recovered": {
"description": "Downstream has recovered from pressure",
"signal": "actor_recovered",
"trigger": "queue_depth < 30% of max AND utilization < 50% for 5 minutes",
"agent_action": "gradually_increase_production_rate",
},
}
The signals allow the downstream to communicate pressure. The agent responds to the actual state, not a stale measurement.
Pillar 5: Backpressure-Aware Retries (Don't Make Overload Worse)
Retries are backpressure-aware:
# Backpressure-aware retries
def retry_with_backpressure(action, max_attempts=5):
for attempt in range(max_attempts):
try:
return execute_action(action)
except CapacityError as e:
if attempt == max_attempts - 1:
raise
if e.signal == "actor_overflow_imminent":
wait_time = exponential_backoff(attempt) * 2
elif e.signal == "actor_near_capacity":
wait_time = exponential_backoff(attempt) * 1.5
else:
wait_time = exponential_backoff(attempt)
time.sleep(wait_time)
except RateLimitError as e:
time.sleep(e.retry_after_seconds)
except TransientError as e:
time.sleep(exponential_backoff(attempt))
raise RetryExhaustedError(action)
The retries respect the downstream's capacity. Retries don't make overload worse.
The Backpressure Patterns
Several patterns emerge from disciplined backpressure.
Pattern 1: Circuit Breaker Pattern
When downstream is consistently failing, the circuit breaker trips:
# Circuit breaker pattern
circuit_breaker_states = {
"stripe.api": {
"state": "open",
"last_failure_timestamp": "2026-07-27T09:23:45Z",
"consecutive_failures": 12,
"failure_threshold": 10,
"open_until": "2026-07-27T09:33:45Z",
"half_open_attempts": 0,
},
"postgres.production_db": {
"state": "closed",
"consecutive_failures": 0,
"failure_threshold": 10,
"last_failure_timestamp": None,
},
}
def execute_with_circuit_breaker(service, action):
breaker = circuit_breaker_states[service]
if breaker["state"] == "open":
if time.now() < breaker["open_until"]:
raise CircuitBreakerOpenError(service)
else:
breaker["state"] = "half_open"
breaker["half_open_attempts"] = 0
try:
result = execute_action(action)
if breaker["state"] == "half_open":
breaker["state"] = "closed"
breaker["consecutive_failures"] = 0
return result
except Exception as e:
breaker["consecutive_failures"] += 1
if breaker["consecutive_failures"] >= breaker["failure_threshold"]:
breaker["state"] = "open"
breaker["open_until"] = time.now() + 60
raise
The circuit breaker prevents the agent from hammering a failing downstream. The downstream gets room to recover.
Pattern 2: Load Shedding
When the system is overloaded, low-priority work is shed:
# Load shedding
load_shedding_policy = {
"priority_levels": {
"critical": {"sheddable": False, "examples": ["payment.processing", "auth.login"]},
"high": {"sheddable": False, "examples": ["customer.support_response", "order.processing"]},
"medium": {"sheddable": True, "examples": ["email.notification", "analytics.event"]},
"low": {"sheddable": True, "examples": ["report.generation", "log.aggregation"]},
},
"shedding_strategy": {
"healthy_load": {"shed": [], "process": "all"},
"moderate_load": {"shed": ["low"], "process": ["critical", "high", "medium"]},
"high_load": {"shed": ["low", "medium"], "process": ["critical", "high"]},
"critical_load": {"shed": ["low", "medium", "high"], "process": ["critical"]},
},
}
def should_process(work_item, system_load):
priority = work_item.priority
shed_list = load_shedding_policy["shedding_strategy"][system_load]["shed"]
return priority not in shed_list
The load shedding protects critical work. The system stays functional under stress.
Pattern 3: Bulk Operations When Possible
The agent batches work to reduce downstream pressure:
# Bulk operations
bulk_patterns = {
"postgres.bulk_insert": {
"max_batch_size": 500,
"trigger": "queue_depth >= 50 OR scheduled_flush",
"benefit": "100x throughput vs single inserts",
},
"elasticsearch.bulk_index": {
"max_batch_size": 1000,
"trigger": "queue_depth >= 200 OR 5 second elapsed",
"benefit": "100x throughput vs single indexes",
},
"email.bulk_send": {
"max_batch_size": 100,
"trigger": "queue_depth >= 20 OR 30 second elapsed",
"benefit": "10x throughput vs single sends",
},
"s3.bulk_put": {
"max_batch_size": 1000,
"trigger": "queue_depth >= 500 OR 60 second elapsed",
"benefit": "50x throughput vs single puts",
},
}
The bulk operations reduce the per-item overhead. The downstream gets fewer, larger requests.
Pattern 4: Workload Smoothing
Bursts are smoothed to avoid rapid capacity swings:
# Workload smoothing
workload_smoothing = {
"detect_burst": {
"window_seconds": 60,
"burst_threshold": "production_rate > 3x average_for_5_minutes",
"action": "smooth_to_average_rate",
},
"smoothing_strategies": {
"leaky_bucket": {
"description": "Drip work at constant rate",
"use_case": "API calls with strict rate limits",
},
"token_bucket": {
"description": "Burst within quota, drip otherwise",
"use_case": "Mixed workloads with quota",
},
"jitter": {
"description": "Add random delay to spread load",
"use_case": "Periodic batch jobs",
},
},
}
def smooth_workload(work_items, strategy="token_bucket"):
if strategy == "leaky_bucket":
return drip_at_constant_rate(work_items, rate=10)
elif strategy == "token_bucket":
return allow_burst_within_quota(work_items, quota=100, refill_rate=10)
elif strategy == "jitter":
return add_random_delay(work_items, max_delay_seconds=30)
The smoothing prevents bursts that overwhelm downstream. The work flows steadily.
Pattern 5: Backpressure Observability
Backpressure is observable:
# Backpressure observability
backpressure_metrics = {
"downstream_utilization_pct": {
"postgres.production_db": 78,
"stripe.api": 92,
"human_review.queue": 83,
"elasticsearch.index": 90,
},
"queue_depths": {
"postgres.write_queue": 245,
"stripe.api_queue": 230,
"human_review.queue": 1247,
"elasticsearch.ingest_queue": 1200,
},
"circuit_breaker_states": {
"stripe.api": "open",
"postgres.production_db": "closed",
"auth.service": "closed",
},
"throttled_agents": 4,
"load_shedding_active": True,
"avg_wait_time_seconds": 23,
"max_wait_time_seconds": 180,
"backpressure_events_per_hour": 47,
"dashboard": "https://internal-dashboard/facio/backpressure",
}
The observability surfaces backpressure patterns. The team sees what's stressed.
The Backpressure Discipline Doesn't Do
Honest limitations:
- It doesn't eliminate capacity limits. The downstream still has limits. The discipline works within the limits, not beyond them.
- It requires downstream cooperation. The downstream must communicate state. Some systems don't.
- It can be too aggressive. Sometimes the agent should push through. The discipline needs tuning.
- It adds latency. Waiting for capacity adds latency. The discipline trades latency for stability.
- It can drop work. Some load shedding drops work. The discipline accepts that some work is non-essential.
The Backpressure as Operational Practice
Backpressure discipline is operational practice:
Capacity monitoring. The team monitors downstream capacity. They see trends before they become problems.
Threshold tuning. The team tunes throttling thresholds. They balance throughput and stability.
Circuit breaker review. The team reviews circuit breaker events. They identify systematic downstream issues.
Load shedding review. The team reviews load shedding events. They ensure critical work isn't shed.
The practice is what makes the discipline sustainable. Without it, the discipline is too rigid or too loose. With it, the discipline is calibrated.
The Compound Effect of Backpressure Discipline
Backpressure discipline compounds:
- Higher availability. The system doesn't crash under load. The availability is higher.
- Better latency under stress. The system degrades gracefully. The latency is bounded.
- Lower error rates. Cascading failures are prevented. The errors are contained.
- Better human review quality. Humans aren't flooded. The review quality is higher.
- Lower operational cost. The team isn't firefighting. The focus is on improvement.
The undisciplined approach has the opposite trajectory. Cascading failures, latency spikes, high error rates, review burnout, constant firefighting.
Bottom Line
AI agents produce work. The downstream has capacity. Without backpressure discipline, the agent overwhelms the downstream. With backpressure discipline, the agent flows at the pace the system can absorb.
Facio's backpressure discipline provides capacity sensing, adaptive throttling, queue management, flow control signals, and backpressure-aware retries. The discipline makes AI agents sustainable in production.
The agent without backpressure discipline is a liability that overwhelms the system. The agent with it is a managed flow. The team trusts the managed one. The downstream accepts the managed one.
Because AI agents in production produce work. The question is whether the work is absorbed or floods. The backpressure discipline is what makes the answer absorbed.
See the backpressure documentation for capacity sensing, throttling configuration, and circuit breaker tuning.