Facio's Token Discipline: How AI Agents Spend Their Context Budget Like Engineers Spend Cloud Credits

An AI agent's context window is its working memory — and like cloud credits, it has a budget. A team that ignores the budget wakes up to a $50k monthly bill. An agent that ignores the context budget forgets, hallucinates, and produces degraded output by mid-session.

The naive approach is to give the agent whatever context it asks for and hope for the best. The team dumps system prompts, conversation history, retrieved documents, tool results, and metadata into the context. The model processes all of it. The token bill grows. The quality drops. The team wonders why the agent "got worse" this week.

Facio's token discipline treats the context window as a managed resource: tracked per session, allocated by priority, optimized by structure, and accounted for like cloud credits. The agent spends tokens intentionally. The team knows where tokens go. The budget is real, not aspirational.

Here's how the discipline works, what it tracks, and why treating context as a budget is what keeps AI agents sharp at scale.

The Context Window Reality

Modern LLMs have large but finite context windows: 128k tokens for GPT-4, 200k for Claude, up to 1M for Gemini. The number sounds large until the agent actually uses it.

The agent's context fills up with:

• System prompt. 2-5k tokens for a well-engineered system prompt.
• Tool definitions. 1-3k tokens per tool, multiplied by 10-20 tools = 10-60k tokens.
• Conversation history. Grows with every user message and agent response.
• Retrieved documents. RAG pipelines dump thousands of tokens of "context" into the prompt.
• Tool results. Each tool call returns data; the agent keeps the data in context for future reasoning.
• Reasoning scratchpad. The agent's chain-of-thought, scratch work, intermediate reasoning.
• Working memory. Notes the agent makes about the current task.

The total adds up fast. A typical mid-session agent has 30-80k tokens in context. A long session approaches the limit. When the limit is reached, the agent has three bad options: crash, truncate (and lose important context), or compress (and potentially lose important information).

The context window is also quality-bound. Models perform worse on information buried in the middle of long contexts ("lost in the middle" problem). Models forget earlier parts of long conversations. Models hallucinate details that seem consistent with the context but aren't there. The longer the context, the more degraded the output.

So the context window has two limits: a hard limit (max tokens) and a soft limit (quality degrades before the hard limit). Both need management.

The Token Discipline

Facio's token discipline has four pillars. Each addresses a different aspect of context management.

Pillar 1: Token Tracking (Per Session, Per Component)

The discipline tracks token usage at every level:

# Per-session token tracking
session_tokens = {
    "system_prompt_tokens": 2450,
    "tool_definitions_tokens": 18500,
    "conversation_history_tokens": 4200,
    "retrieved_documents_tokens": 12300,
    "tool_results_tokens": 8700,
    "reasoning_scratchpad_tokens": 1850,
    "working_memory_tokens": 920,
    "total_tokens": 48920,
    "context_window_max": 200000,
    "context_utilization_pct": 24,
    "remaining_tokens": 151080,
    "quality_zone": "green",   # green = optimal, yellow = degraded, red = critical
}

The token tracking shows the team exactly where tokens are spent. The agent, the operator, and the dashboard see the same numbers.

Pillar 2: Allocation Priorities

Not all context is equally important. The discipline allocates tokens by priority:

# Token allocation priorities
priority_allocation = [
    {"category": "system_prompt",          "priority": "critical", "min_tokens": 2000, "max_tokens": 5000},
    {"category": "tool_definitions",       "priority": "critical", "min_tokens": 5000, "max_tokens": 30000},
    {"category": "working_memory",         "priority": "high",     "min_tokens": 500,  "max_tokens": 5000},
    {"category": "current_task_context",   "priority": "high",     "min_tokens": 1000, "max_tokens": 10000},
    {"category": "recent_tool_results",    "priority": "medium",   "min_tokens": 1000, "max_tokens": 15000},
    {"category": "conversation_history",   "priority": "medium",   "min_tokens": 1000, "max_tokens": 20000},
    {"category": "retrieved_documents",    "priority": "low",      "min_tokens": 0,    "max_tokens": 30000},
    {"category": "reasoning_scratchpad",   "priority": "low",      "min_tokens": 0,    "max_tokens": 10000},
]

The allocation ensures critical context (system prompt, tools, working memory) is always preserved. Less critical context (retrieved documents, scratchpad) is the first to be trimmed.

Pillar 3: Token Optimization (Strategies)

When the context approaches the limit, the discipline applies optimization strategies:

# Token optimization strategies
strategies = [
    {
        "name": "summarize_old_history",
        "trigger": "context_utilization_pct > 60",
        "action": "Compress conversation history older than 10 messages into summary",
        "expected_savings_pct": 30
    },
    {
        "name": "drop_unused_tools",
        "trigger": "context_utilization_pct > 70",
        "action": "Remove tool definitions for tools not used in last 5 turns",
        "expected_savings_pct": 15
    },
    {
        "name": "compact_retrieved_docs",
        "trigger": "context_utilization_pct > 75",
        "action": "Compress retrieved documents, keep only top-k relevant",
        "expected_savings_pct": 25
    },
    {
        "name": "checkpoint_and_truncate",
        "trigger": "context_utilization_pct > 85",
        "action": "Save full state to checkpoint, truncate context to recent + summary",
        "expected_savings_pct": 60
    },
    {
        "name": "session_split",
        "trigger": "context_utilization_pct > 95",
        "action": "Split session into two, preserve critical context in both",
        "expected_savings_pct": 80
    },
]

The optimization strategies fire in order of intrusiveness. Mild optimization (summarization) happens early. Aggressive optimization (checkpoint + truncate, session split) happens late.

Pillar 4: Cost Accounting

The discipline accounts for token usage as cost:

# Per-session cost accounting
session_cost = {
    "tokens_in": 48920,
    "tokens_out": 1840,
    "model": "claude-opus-4",
    "cost_input_usd": 0.73,        # $15/M tokens
    "cost_output_usd": 0.092,      # $50/M tokens
    "cost_total_usd": 0.82,
    "cost_per_decision_usd": 0.012,  # 68 decisions made
    "budget_remaining_usd": 4.18,
}

The cost accounting makes tokens visible as money. The team sees the dollar cost of context choices. Decisions about context allocation are informed by cost.

The Token-Aware Patterns

Several patterns emerge from disciplined token usage.

Pattern 1: Tool List Minimization

The agent has access to many tools, but most aren't needed for the current task. The discipline loads only relevant tools:

# Tool list minimization
available_tools = ["slack.send_message", "slack.read_channel", ...]  # 50 tools

# For this task, only some tools are relevant
relevant_tools = select_relevant_tools(task, available_tools, max_count=10)
# Selected: ["slack.send_message", "postgres.execute_query", "github.create_issue"]

# Token savings: 50 tool definitions (~50k tokens) → 10 tool definitions (~10k tokens)
# Quality: Agent chooses correctly from relevant tools, less confused by irrelevant ones

The minimized tool list saves tokens AND improves quality. The agent doesn't get distracted by irrelevant tools.

Pattern 2: Retrieval Quality Over Quantity

The discipline prefers fewer, high-quality retrieved documents over many low-quality ones:

# Retrieval quality over quantity
retrieved_docs = retrieve(query, k=20)  # Naive: retrieve 20 documents

# Filter to high-quality
high_quality_docs = filter_by_relevance(retrieved_docs, query, threshold=0.7)
# Filtered: 5 documents

# Add metadata about what was filtered out
context_note = f"Note: Retrieved 20 documents, kept 5 most relevant. Others were lower relevance."

# Token savings: 20 docs (~30k tokens) → 5 docs (~7k tokens)
# Quality: Model focuses on relevant content, not distracted by noise

The discipline treats retrieval as a quality discipline, not a "give me everything" discipline.

Pattern 3: Reasoning Compression

The agent's chain-of-thought can be verbose. The discipline compresses reasoning while preserving decisions:

# Verbose reasoning
reasoning_verbose = """
Let me think about this carefully. The user is asking about order status.
First, I should check what orders exist for this customer. To do that, I need to
look up the customer ID. The customer is identified by their email or phone. Let me
use the customer lookup tool. Actually, I already have the customer ID from the
context, so I can use that directly. Let me proceed with the order lookup.
[... 500 more tokens of reasoning ...]
"""

# Compressed reasoning
reasoning_compressed = """
Plan: Use existing customer_id from context → order lookup → return status
"""

# Token savings: 800 tokens → 30 tokens
# Quality: Decision preserved, verbosity removed

The compressed reasoning preserves the "what" (decisions) while removing the "how" (process narrative). The agent can still reason; it just doesn't store every word.

Pattern 4: Context-Aware Model Selection

Different parts of the workflow have different context needs. The discipline selects models accordingly:

# Context-aware model selection
context_strategy = {
    "initial_planning": "use_cheap_model_with_summary",
    "complex_reasoning": "use_capable_model_with_full_context",
    "final_response": "use_cheap_model_with_structured_output",
    "validation": "use_judge_model_with_minimal_context",
}

# Example execution:
# 1. Initial planning: cheap model reads 5k tokens, produces 200-token plan
# 2. Complex reasoning: capable model reads 25k tokens + plan, produces decision
# 3. Final response: cheap model reads decision + minimal context, produces response
# 4. Validation: judge model reads response + criteria, produces score

# Total: Mix of models, total cost lower than always-capable-model

The model selection reduces cost while preserving quality. The expensive model is used where it matters; cheap models are used where they suffice.

Pattern 5: Checkpoint-Based Context Reset

When context grows too large, the discipline saves state and resets:

# Checkpoint-based context reset
if session.context_utilization > 85:
    # Save full state
    checkpoint = {
        "session_id": session.id,
        "full_history": session.history,
        "all_tool_results": session.tool_results,
        "agent_reasoning": session.reasoning,
        "current_state": session.state,
    }
    save_checkpoint(checkpoint)

    # Reset active context to essentials
    new_context = {
        "system_prompt": essentials_only,        # Compressed system prompt
        "recent_messages": last_5_turns,        # Only recent turns
        "task_summary": "Working on X, currently at step Y",
        "checkpoint_reference": "checkpoint-12345",  # Pointer to full state
        "next_actions": [...],                  # What's pending
    }

    session.context = new_context
    # Now we have plenty of room to continue

The checkpoint-based reset is dramatic but recoverable. The agent can always load the full state from the checkpoint if needed.

The Token Discipline Doesn't Do

Honest limitations:

• It doesn't eliminate the limit. Token discipline manages context; it doesn't make the context window infinite. Hard limits remain.
• It can lose important context. Summarization and compression can lose details. The discipline preserves what matters; it can't preserve everything.
• It can hurt reasoning quality. Aggressive optimization (especially session splits) can break the agent's chain of thought. The discipline trades continuity for capacity.
• It requires good priorities. The priority allocation is configuration. Misconfigured priorities protect the wrong context. The team has to set priorities correctly.
• It doesn't fix bad prompts. Token discipline optimizes context usage; it doesn't fix bloated system prompts or poorly designed tools. Source-level fixes are also needed.

The Token Discipline as Operational Practice

Token discipline is an operational practice:

Continuous monitoring. The team tracks token usage over time. They see trends. They identify workflows that are inefficient.

Optimization culture. The team asks "how can we use fewer tokens for the same quality?" They measure. They improve.

Budget-driven design. When designing new workflows, the team considers token budget upfront. The workflow fits within the budget.

Cost transparency. The team shares token costs with stakeholders. The cost of features is visible. The trade-offs are explicit.

The practice is what makes the discipline sustainable. Without it, the team optimizes for capability (more context = better agent) and ignores cost. With it, the team optimizes for capability per token.

The Compound Effect of Token Discipline

Token discipline compounds:

• Lower costs. Disciplined token usage reduces spend by 40-70%. The savings compound over time.
• Better quality. Minimized, focused context produces better outputs than bloated context. The agent is sharper.
• More sessions. Disciplined usage means more sessions per budget. The team can serve more users.
• More autonomy. With predictable token costs, the team gives the agent more autonomy. The autonomy is affordable.
• Better planning. Token budgets are predictable. The team plans capacity. The plans are accurate.

The undisciplined approach has the opposite trajectory. Bloated context, high costs, degraded quality, limited autonomy, firefighting.

Bottom Line

An AI agent's context window is its working memory. Like cloud credits, it has a budget. Without management, the budget is exceeded and quality degrades.

Facio's token discipline treats the context window as a managed resource: tracked per component, allocated by priority, optimized by strategies, and accounted as cost. The agent spends tokens intentionally. The team knows where tokens go.

The agent without token discipline is a budget runaway. The agent with it is a budget optimizer. The finance team prefers the optimizer. The engineering team trusts the optimizer.

Because AI agents in production have context limits. The question is whether the limits are managed or exceeded. The token discipline is what makes the limits manageable.

---

See the token discipline documentation for token tracking setup, allocation configuration, and optimization strategies.