AI Cost Engineering: Token Budgeting, Prompt Caching, and Three-Tier Model Routing

Reading Your AI Bill: Where the Money Actually Goes

Most developers who hit a surprise AI bill make the same mistake: they assume output tokens are the problem. They're not. Understanding where your tokens actually go is the first step to controlling costs.

Here's what Anthropic charges per million tokens as of April 2026 — verified against the Anthropic platform pricing docs:

| Model | Input (per MTok) | Output (per MTok) |

|---|---|---|

| Claude Haiku 4.5 | $1.00 | $5.00 |

| Claude Sonnet 4.6 | $3.00 | $15.00 |

| Claude Opus 4.7 | $5.00 | $25.00 |

At a glance, this looks like output tokens are 5x more expensive than input. And yes, per token they are — but the distribution of where tokens land in a real production system is surprising.

Consider a customer support agent that uses a large system prompt (2,000 tokens), retrieves 3 docs from RAG (1,500 tokens each), receives a user message (100 tokens), and produces a reply (400 tokens). Your token breakdown:

• System prompt: 2,000 tokens (input)
• RAG context: 4,500 tokens (input)
• User message: 100 tokens (input)
• Output: 400 tokens (output)

Total: 6,600 input + 400 output. On Sonnet 4.6 that's ($3 × 6.6 / 1,000) + ($15 × 0.4 / 1,000) = $0.0198 + $0.006 = $0.0258 per request.

At 100,000 requests/month, that's $2,580/month — and 77% of that cost is input tokens you're re-sending every single time.

In the Anthropic Console, filter by model and look at two ratios:

• Input-to-output ratio: If it's consistently above 10:1, your context window management is the primary cost driver — prompt caching will help immediately.
• Cache hit rate: If this is zero or near zero, you're paying full price for every token. Module 2 addresses this.
• Model distribution: What percentage of your requests hit Opus vs. Sonnet vs. Haiku? If it's mostly Opus, you almost certainly have routing opportunities.

The mental model to carry forward: input tokens are a tax on context; output tokens are a tax on verbosity; model choice is a multiplier on both. Engineer all three.

Prompt Caching: The 90% Cost Reduction Pattern

Prompt caching is the single highest-leverage cost optimization in the Anthropic API. A cache read costs 10% of the standard input price — meaning every cached token you re-read costs one-tenth of what it would without caching. For workloads with large, stable context (system prompts, RAG documents, conversation history), this translates directly to 60–90% input cost reduction.

When you mark a content block with cache_control, Anthropic stores that processed KV state server-side. On the next request, if the same prefix appears, the API reads from cache instead of re-processing. You pay a write premium (1.25x base for 5-minute cache; 2x base for 1-hour cache), then 0.1x base for every subsequent read.

• 5-minute cache: pays for itself after 1 cache read (write cost 1.25x, first read saves 0.9x → net positive at 2 total uses)
• 1-hour cache: pays for itself after 2 cache reads

For any stable content used more than twice in a session, caching is the correct choice.

| Operation | Price | When |

|---|---|---|

| Standard input | $3.00/MTok | No cache |

| 5-min cache write | $3.75/MTok | First request, 5-min TTL |

| 1-hour cache write | $6.00/MTok | First request, 1-hour TTL |

| Cache hit (read) | $0.30/MTok | All subsequent requests |

response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": your_large_system_prompt,
            "cache_control": {"type": "ephemeral"}  # 5-minute TTL
        }
    ],
    messages=messages
)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": retrieved_document_1,
                "cache_control": {"type": "ephemeral"}
            },
            {
                "type": "text",
                "text": retrieved_document_2,
                "cache_control": {"type": "ephemeral"}
            },
            {"type": "text", "text": user_question}
        ]
    }
]

For multi-tenant SaaS products where many users share the same system prompt, use the 1-hour cache duration. A 3,000-token system prompt on Sonnet 4.6:

• Without caching: $3.00/MTok × 100,000 requests × 3,000 tokens = $900/month
• With 1-hour caching (1 write per hour, reads for the rest): ≈ $90/month

The exact savings depend on your request frequency, but high-traffic shared prompts are where 1-hour caching pays for itself fastest.

• Non-deterministic prefixes: If your system prompt includes a timestamp or session ID that changes per request, you'll never hit the cache. Keep dynamic content at the end of your context, after the cached prefix.
• Cache TTL expiry: The 5-minute TTL is short. For low-traffic agents or long-running sessions with gaps, use the 1-hour TTL or implement refresh logic.
• Token ordering: Cache reads only apply when the cached prefix matches exactly. Don't shuffle or modify content before the breakpoint marker.

Check usage.cache_read_input_tokens and usage.cache_creation_input_tokens in the API response. Your target: cache reads should exceed cache writes by at least 5:1 in steady state. Below 2:1, your caching architecture likely has a TTL or prefix consistency problem.

Three-Tier Routing: Opus 4.7 / Sonnet 4.6 / Haiku 4.5

The most impactful cost optimization after prompt caching isn't a pricing feature — it's architecture. Routing requests to the cheapest model that can handle them reliably is where teams cut their bills in half without touching quality.

| Model | Input | Output | Best For |

|---|---|---|---|

| Haiku 4.5 | $1/MTok | $5/MTok | Classification, extraction, simple Q&A, formatting |

| Sonnet 4.6 | $3/MTok | $15/MTok | Complex reasoning, code generation, multi-step analysis |

| Opus 4.7 | $5/MTok | $25/MTok | Highest-stakes judgment, novel reasoning, frontier tasks |

Opus costs 5x Haiku on input and 5x on output. If 60% of your traffic can be handled by Haiku, you've just reduced that portion's cost by 80%.

Use this framework to route requests:

• Task is a single-turn classification or label extraction
• Output is structured (JSON, boolean, category)
• No tool chaining or multi-step reasoning required
• High volume, latency-sensitive, cost-sensitive path
• Examples: intent classification, sentiment tagging, entity extraction, templated email fill

• Task requires code generation or debugging
• Multi-document synthesis or summarization
• Multi-step tool use with conditional branching
• User-facing content where quality visibly matters
• Default tier for most production agent tasks

• Novel problem where even Sonnet makes structural errors
• High-stakes decisions (financial, legal, medical adjacent)
• Complex long-form reasoning where correctness > cost
• Research or planning tasks that downstream other agents
• Volume is low but quality is non-negotiable

The simplest router is a lightweight classifier that runs on Haiku itself — using a cheap model to decide which expensive model to call:

def route_request(user_message: str, context: dict) -> str:
    """Returns model ID for this request."""
    
    # Fast heuristics first (no LLM call)
    if context.get("task_type") in ["classify", "extract", "format"]:
        return "claude-haiku-4-5"
    
    if len(user_message) > 5000 or context.get("requires_code_generation"):
        return "claude-sonnet-4-6"
    
    # Use Haiku as a classifier for ambiguous cases
    classification = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=10,
        system="Classify the complexity: respond with only 'simple', 'moderate', or 'complex'.",
        messages=[{"role": "user", "content": user_message[:500]}]
    )
    
    tier = classification.content[0].text.strip().lower()
    return {
        "simple": "claude-haiku-4-5",
        "moderate": "claude-sonnet-4-6",
        "complex": "claude-opus-4-7"
    }.get(tier, "claude-sonnet-4-6")

The Haiku classifier call costs ~$0.001 at 1,000 input + 10 output tokens. If it routes even 30% of traffic away from Sonnet, the savings pay for the classifier call thousands of times over.

Another approach: run Haiku first, then escalate if quality is insufficient:

def generate_with_escalation(prompt: str) -> str:
    result = call_model("claude-haiku-4-5", prompt)
    
    if confidence_score(result) < 0.85 or result.stop_reason == "max_tokens":
        result = call_model("claude-sonnet-4-6", prompt)
    
    return result

This works well when you have a reliable quality signal (structured output validation, length checks, confidence scores from the model). It's less useful when quality is subjective.

For any task that doesn't need a real-time response — nightly content processing, bulk analysis, large-scale evaluation — the Batch API cuts all token costs in half:

| Model | Batch Input | Batch Output |

|---|---|---|

| Haiku 4.5 | $0.50/MTok | $2.50/MTok |

| Sonnet 4.6 | $1.50/MTok | $7.50/MTok |

| Opus 4.7 | $2.50/MTok | $12.50/MTok |

Combine Batch API with prompt caching for large offline workloads and you can achieve 60–75% total cost reduction vs. naive real-time Opus calls.

Token Budget Patterns: Measuring, Alerting, Controlling

Knowing pricing and routing strategy gets you 70% of the way. The remaining 30% is instrumentation: actually measuring what you spend, building alerts before surprises hit, and enforcing budgets at the code level so a rogue agent loop can't drain your account overnight.

Every Anthropic API response includes a usage object. Log it for every production call:

import logging
from dataclasses import dataclass

@dataclass
class TokenUsageEvent:
    request_id: str
    model: str
    input_tokens: int
    output_tokens: int
    cache_read_tokens: int
    cache_write_tokens: int
    user_id: str | None
    feature: str  # e.g. 'support-agent', 'code-review', 'summary'

def log_usage(response, request_id: str, user_id: str, feature: str):
    usage = response.usage
    event = TokenUsageEvent(
        request_id=request_id,
        model=response.model,
        input_tokens=usage.input_tokens,
        output_tokens=usage.output_tokens,
        cache_read_tokens=getattr(usage, 'cache_read_input_tokens', 0),
        cache_write_tokens=getattr(usage, 'cache_creation_input_tokens', 0),
        user_id=user_id,
        feature=feature
    )
    logging.info("token_usage", extra=vars(event))

Send these events to your observability stack (Datadog, Grafana, CloudWatch). Tag by model, feature, and user_id so you can slice costs three ways.

The max_tokens parameter is your primary control knob. Set it deliberately, not arbitrarily:

OUTPUT_BUDGETS = {
    "classify": 20,       # Single label
    "extract": 200,       # Structured JSON
    "summarize": 400,     # Paragraph summary
    "code_review": 800,   # Detailed feedback
    "plan": 2000,         # Multi-step plan
}

def call_with_budget(task_type: str, prompt: str, model: str) -> str:
    max_tokens = OUTPUT_BUDGETS.get(task_type, 500)
    return client.messages.create(
        model=model,
        max_tokens=max_tokens,
        messages=[{"role": "user", "content": prompt}]
    )

For conversational agents, context window management is where costs compound over time. A conversation that runs for 30 turns without trimming can accumulate 40,000+ input tokens — even if the last 20 turns are irrelevant.

A practical trimming strategy:

MAX_CONTEXT_TOKENS = 8000
SYSTEM_PROMPT_TOKENS = 2000  # reserved

def trim_conversation(messages: list, max_tokens: int = MAX_CONTEXT_TOKENS) -> list:
    """Keep most recent messages within token budget."""
    reserved = SYSTEM_PROMPT_TOKENS
    available = max_tokens - reserved
    
    trimmed = []
    running_count = 0
    
    for msg in reversed(messages):
        estimated = len(msg["content"]) // 4  # rough token estimate
        if running_count + estimated > available:
            break
        trimmed.insert(0, msg)
        running_count += estimated
    
    return trimmed

Set hard stops in your Anthropic Console (Usage → Spend limits), and mirror them in your own application logic. A defense-in-depth approach: Console limit as the emergency brake, application-level token accounting as the normal operating control. Never rely on only one layer.

Track cumulative spend per billing period in your own DB, not only via the Console. The Console resets monthly and provides no per-feature breakdown — you need your own attribution to know whether your support agent or your code review tool is the real cost driver.