Token Cost Optimization

Overview

Every token costs money and latency. This skill encodes battle-tested techniques for cutting LLM API spend without degrading output quality. Covers both design-time strategies (how you structure prompts and tools) and runtime strategies (caching, batching, model routing).

Applies to any provider (OpenAI, Anthropic, Mistral, Gemini, etc.) and also to local inference where you pay in GPU time instead of dollars.

---

When to Use

• You are building an LLM-powered feature and want to keep costs low from day one.
• An existing app's API bill is unexpectedly high and you're diagnosing why.
• You want to set a per-request token budget and enforce it programmatically.
• You are selecting which model tier to use for a given task.
• You want to add semantic caching or prompt caching to an existing pipeline.

Don't use for:

• One-off interactive chat (costs are negligible; don't over-engineer).
• Tasks where output quality is the only constraint (compression may hurt).

---

Strategy 0 — headroom-ai: Drop-In Context Compression (60–95% savings)

headroom is an open-source library (17k+ ⭐, Apache 2.0) that automatically compresses tool outputs, logs, RAG chunks, files, and conversation history before they reach the LLM — with no accuracy loss.

Benchmarked savings on real agent workloads:

Workload	Before	After	Savings
Code search (100 results)	17,765	1,408	92%
SRE incident debugging	65,694	5,118	92%
GitHub issue triage	54,174	14,761	73%
Codebase exploration	78,502	41,254	47%

Accuracy on benchmarks: unchanged. (GSM8K ±0.000, TruthfulQA +0.030, SQuAD v2 97%, BFCL 97% with 19–32% compression.)

Installation

pip install "headroom-ai[all]"    # Python — everything included
npm install headroom-ai           # TypeScript / Node (requires proxy)
docker pull ghcr.io/chopratejas/headroom:latest

Granular extras: [proxy], [mcp], [ml], [code], [memory], [image], [langchain], [agno], [evals]. Requires Python 3.10+.

Three usage modes

Mode 1 — Library (minimal code change):

from headroom import SmartCrusher, SmartCrusherConfig

crusher = SmartCrusher(SmartCrusherConfig(
    max_items_after_crush=20,   # keep top-N rows by importance
    first_fraction=0.30,        # always keep first 30% of remaining budget
    last_fraction=0.15,         # always keep last 15%
    factor_out_constants=True,  # hoist repeated fields to save tokens
))

# ✅ Use .crush() — returns CrushResult
result     = crusher.crush(tool_output_str, query="error root cause")
compressed = result.compressed   # str — drop-in replacement for tool output

# CrushResult attributes (confirmed v0.23.0):
#   .compressed   — str, the compressed content
#   .original     — str, the original content unchanged
#   .strategy     — str, e.g. "lossless:table(200->len=31363)"
#   .was_modified — bool, False if content was too small to compress

# ⚠️  DO NOT use crush_array_json() — it returns a metadata dict that
#     wraps the content in extra keys, increasing token count by 50%+.
#     Always use crush() for tool outputs.

# Inject compressed result into messages
msgs_after = [
    *msgs_before[:-1],   # keep all but the last tool-result message
    {"role": "user", "content": [
        {"type": "tool_result", "tool_use_id": tool_use_id,
         "content": compressed}
    ]},
]
response = client.messages.create(model="claude-sonnet-4-5", messages=msgs_after, ...)

# Check what happened
print(f"Strategy  : {result.strategy}")
print(f"Modified  : {result.was_modified}")

Mode 2 — Proxy (zero code changes):

# Start local proxy on port 8787
headroom proxy --port 8787

# Point any OpenAI-compatible client at it
OPENAI_BASE_URL=http://localhost:8787/v1 your-app
ANTHROPIC_BASE_URL=http://localhost:8787 claude

# Stats endpoint
curl http://localhost:8787/stats
# {"requests_total": 42, "tokens_saved_total": 125000, ...}

Mode 3 — Agent wrap (one command):

headroom wrap claude     # wraps Claude Code with --memory and --code-graph
headroom wrap codex      # shares memory with Claude
headroom wrap cursor     # prints config, paste once
headroom wrap aider      # starts proxy + launches
headroom wrap copilot    # starts proxy + launches

What headroom compresses (and what it doesn't touch)

Content type	Compressor	Typical savings
JSON arrays of dicts	SmartCrusher .crush() (lossless table reformat + statistical sampling)	42–95%
JSON arrays of strings	Dedup + adaptive sampling	60–90%
Build/test logs	Pattern clustering	85–94%
Source code	CodeCompressor (AST-aware)	40–70%
Plain text	Kompress-base (HuggingFace model)	30–50%
HTML	Article extraction (trafilatura)	~95%
Short content (<200 tokens)	Not compressed (overhead > savings)	—
User messages	Never compressed (intent preserved)	—
System prompt content	Preserved; only dynamic parts relocated	—

⚠️ crush_array_json() pitfall: This method returns a metadata dict with an items key containing the original JSON as a string. It does NOT reduce tokens — it increases them by 50%+. Always use crush() for tool outputs.

How the pipeline works internally

Three-stage compression pipeline:

1. CacheAligner — extracts dynamic content (dates, UUIDs, tokens) from system prompt prefix and moves it to the tail. Stabilizes prefixes so provider KV caches (Anthropic cache_control, OpenAI prefix cache) actually hit. Sub-millisecond overhead.

2. ContentRouter → SmartCrusher / CodeCompressor / Kompress-base — detects content type, selects best compressor. SmartCrusher applies field-level statistical analysis (variance, uniqueness, Kneedle algorithm on bigram coverage). Errors and anomalies are always kept regardless of budget. Overhead: 1–50ms.

3. Context Manager — ensures final message array fits within the model's context window.

   • Rolling Window (default): drops oldest messages first, preserving system prompt and recent turns. Tool call + response pairs dropped atomically.
   • Intelligent Context (advanced): scores messages on 6 dimensions (recency, semantic similarity, TOIN importance, error indicators, forward references, token density).

CCR (Compress-Cache-Retrieve) — reversible compression: originals are never deleted; they're stored in a local SQLite cache. If the LLM needs full data it calls ccr_retrieve("<hash>") to get it back. Headroom is the only major compressor that is fully reversible.

SDK integrations

# Wrap existing client — Anthropic or OpenAI
from headroom.integrations import withHeadroom
from anthropic import Anthropic

client = withHeadroom(Anthropic())   # all calls auto-compressed

# LangChain
from headroom.integrations.langchain import HeadroomChatModel
llm = HeadroomChatModel(your_llm)

# Agno
from headroom.integrations.agno import HeadroomAgnoModel
model = HeadroomAgnoModel(your_model)

# ASGI middleware
from headroom.integrations.asgi import CompressionMiddleware
app.add_middleware(CompressionMiddleware)

# MCP server (for any MCP client)
headroom mcp install
# Exposes: headroom_compress, headroom_retrieve, headroom_stats

Provider cache optimization (built-in)

After compression, headroom automatically applies provider-specific cache hints:

Provider	Mechanism	Cache savings
Anthropic	cache_control blocks on stable prefix	Up to 90% on repeated tokens
OpenAI	Prefix alignment for automatic caching	Up to 50% on repeated tokens
Google	CachedContent API	Up to 75% on repeated tokens

Cross-agent shared memory

from headroom.memory import SharedContext

ctx = SharedContext()
ctx.put("key", value, agent="claude")      # store from any agent
ctx.get("key")                              # retrieve from any agent (Claude, Codex, Gemini)

headroom learn mines failed agent sessions and writes corrections to CLAUDE.md / AGENTS.md / GEMINI.md automatically.

headroom vs. alternatives

Tool	Scope	Local	Reversible
headroom	All context — tools, RAG, logs, files, history	✅	✅
RTK	CLI command outputs only	✅	❌
lean-ctx	CLI commands, MCP tools, editor rules	✅	❌
Compresr / Token Co.	Text sent to hosted API	❌	❌
OpenAI Compaction	Conversation history only	❌	❌

headroom ships RTK internally and can also use lean-ctx as the context tool (HEADROOM_CONTEXT_TOOL=lean-ctx).

Performance check

headroom perf    # benchmark compression on your actual workloads

---

Strategy 1 — Measure First

Never optimize blind. Instrument token usage before changing anything.

# openai SDK: response.usage is always present
response = client.chat.completions.create(...)
print(response.usage)  # CompletionUsage(prompt_tokens=..., completion_tokens=..., total_tokens=...)

# anthropic SDK
response = client.messages.create(...)
print(response.usage)  # Usage(input_tokens=..., output_tokens=...)

Key metrics to track per request:

Metric	Formula
Prompt tokens	tokens used by the input
Completion tokens	tokens in the output
Cost	prompt_tokens × price_in + completion_tokens × price_out
Prompt-to-completion ratio	prompt / completion — high ratios mean you're paying for context

Use tiktoken (OpenAI) or anthropic.count_tokens() to estimate costs before sending:

import tiktoken
enc = tiktoken.encoding_for_model("gpt-4o")
n = len(enc.encode(your_prompt_text))
print(f"{n} tokens ≈ ${n / 1_000_000 * 5:.4f} at $5/1M input")

---

Strategy 2 — Model Tiering

Route tasks to the cheapest model that can do the job. Typical tiers:

Tier	Examples	Use for
Micro / fast	GPT-4o-mini, Claude Haiku, Gemini Flash	Classification, extraction, summarization, short Q&A
Standard	GPT-4o, Claude Sonnet, Gemini Pro	Multi-step reasoning, coding, analysis
Frontier	o3, Claude Opus, Gemini Ultra	Research, long-horizon agents, highest accuracy

Rule of thumb: Start with the cheapest tier. Only escalate if evals fail.

# Simple routing: route by estimated complexity
def pick_model(prompt: str) -> str:
    n_tokens = len(enc.encode(prompt))
    if n_tokens < 500 and "summarize" in prompt.lower():
        return "gpt-4o-mini"    # 15x cheaper than gpt-4o
    return "gpt-4o"

---

Strategy 3 — Prompt Compression

3a. Remove Boilerplate

Every word in the system prompt costs tokens on every call. Ruthlessly remove:

• "You are a helpful AI assistant" — LLMs know they're LLMs.
• Long preambles explaining what the AI should generally do.
• Repeated instructions that are already default behavior.

3b. Use Terse Formatting

BEFORE (47 tokens):
"Please carefully analyze the following customer feedback and provide
a detailed response identifying the main issues."

AFTER (12 tokens):
"Analyze feedback. List issues."

3c. Few-Shot → Zero-Shot

Each example in a few-shot prompt costs its full token count. Test if zero-shot works first. If accuracy suffers, add one example, not five.

3d. Compress Reference Documents

When inserting long documents into context, summarize or extract only the relevant sections:

# Instead of stuffing 10k-token PDF into context:
# 1. Chunk it
# 2. Embed chunks
# 3. Retrieve top-k by similarity
# 4. Inject only top-k (typically 3-5 chunks × ~300 tokens = ~1k tokens total)

---

Strategy 4 — Context Window Management

Sliding Window (for long conversations)

Keep only recent messages when history grows too large:

MAX_HISTORY_TOKENS = 4000

def trim_history(messages: list[dict], enc) -> list[dict]:
    # Always keep system prompt (index 0)
    system = messages[:1]
    history = messages[1:]
    
    # Count from most recent until budget exhausted
    kept, budget = [], MAX_HISTORY_TOKENS
    for msg in reversed(history):
        cost = len(enc.encode(msg["content"]))
        if cost > budget:
            break
        kept.append(msg)
        budget -= cost
    
    return system + list(reversed(kept))

Summarize Instead of Truncate

When truncating loses important context, summarize old history with a cheap model first:

def summarize_old_history(old_msgs: list[dict]) -> str:
    response = client.chat.completions.create(
        model="gpt-4o-mini",  # cheap model for summarization
        messages=[
            {"role": "user", "content": f"Summarize this conversation in ≤100 words:\n{old_msgs}"}
        ],
        max_tokens=150,
    )
    return response.choices[0].message.content

---

Strategy 5 — Prompt Caching

Anthropic and OpenAI offer explicit prompt caching — identical prefix tokens are cached at a 90% discount on repeat calls.

Anthropic cache_control

response = anthropic.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": LARGE_SYSTEM_PROMPT,        # long static prefix
            "cache_control": {"type": "ephemeral"}  # mark for caching
        }
    ],
    messages=[{"role": "user", "content": user_question}]
)
# cache_read_input_tokens shows how many were served from cache
print(response.usage.cache_read_input_tokens)

Rules for Anthropic prompt caching:

• Minimum cacheable prefix: 1024 tokens (Sonnet/Haiku) or 2048 tokens (Opus).
• Cache TTL: 5 minutes (refreshed on each cache hit).
• Cost: cache write = 1.25× normal; cache read = 0.1× normal.
• Structure your prompt so the STATIC part comes first (system prompt, documents) and the DYNAMIC part (user input) comes last.

OpenAI Automatic Caching

OpenAI caches automatically — no code changes needed. The first 1024+ token prefix of any request is cached. You get a discount automatically if the same prefix is reused within ~1 hour.

Check with: response.usage.prompt_tokens_details.cached_tokens

---

Strategy 6 — Semantic Caching

Cache at the semantic level — if a new query is very similar to a past query, return the cached answer.

# Minimal semantic cache with sentence-transformers + numpy
# pip install sentence-transformers numpy
from sentence_transformers import SentenceTransformer
import numpy as np

model = SentenceTransformer("all-MiniLM-L6-v2")  # 80MB, fast
cache: list[tuple[np.ndarray, str]] = []          # (embedding, answer)

SIMILARITY_THRESHOLD = 0.92

def cached_query(question: str) -> str | None:
    q_emb = model.encode(question)
    for emb, answer in cache:
        similarity = np.dot(q_emb, emb) / (np.linalg.norm(q_emb) * np.linalg.norm(emb))
        if similarity > SIMILARITY_THRESHOLD:
            return answer
    return None

def store_in_cache(question: str, answer: str):
    cache.append((model.encode(question), answer))

For production: use Redis with vector search or Qdrant/Weaviate as the cache backend.

---

Strategy 7 — Output Length Control

Completion tokens are often more expensive per-token than input tokens. Control output length explicitly:

# Hard limit
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    max_tokens=256,    # hard cutoff — set per task
)

# Soft limit via prompt instruction
system = "Respond in ≤3 sentences. Be direct."

# Structured output reduces verbose prose
# Ask for JSON instead of paragraphs — usually 2-5× fewer tokens
system = "Respond as JSON: {\"issue\": str, \"severity\": 1-5, \"fix\": str}"

Use JSON/structured output mode for extraction tasks — it forces concise, parseable answers.

---

Strategy 8 — Tool Call Minimization

Each tool call round-trip adds tokens (tool schema + tool result injected back into context). Minimize by:

1. Batch tool calls — request multiple results in one call instead of chaining.
2. Filter tool schemas — only include tools relevant to the current task. Every tool definition in the schema costs ~100-300 tokens per call.
3. Inline small lookups — if a tool just returns a short string, compute it before the LLM call and inject it directly into the prompt.
4. Return minimal tool results — truncate or summarize large tool outputs before injecting them:

def truncate_tool_result(result: str, max_chars: int = 2000) -> str:
    if len(result) <= max_chars:
        return result
    return result[:max_chars] + f"\n[...truncated {len(result) - max_chars} chars]"

---

Strategy 9 — Batching

If you're making many independent LLM calls, batch them:

# OpenAI Batch API — 50% discount, 24h turnaround
# Good for: eval pipelines, bulk classification, offline enrichment
import json

requests = [
    {"custom_id": f"req-{i}", "method": "POST", "url": "/v1/chat/completions",
     "body": {"model": "gpt-4o-mini", "messages": [{"role": "user", "content": q}]}}
    for i, q in enumerate(questions)
]

# Write JSONL file, upload, then call client.batches.create(...)

For Anthropic, use the Message Batches API (client.beta.message_batches.create(...)).

---

Strategy 10 — Token Budget Enforcement

Fail fast rather than blowing the budget silently:

TOKEN_BUDGET = 8_000  # per request hard limit

def safe_complete(messages: list[dict]) -> str:
    total = sum(len(enc.encode(m["content"])) for m in messages)
    if total > TOKEN_BUDGET:
        raise ValueError(f"Prompt exceeds budget: {total} > {TOKEN_BUDGET} tokens")
    
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        max_tokens=min(1024, TOKEN_BUDGET - total),
    )
    return response.choices[0].message.content

---

Publishing a Skill to GitHub for Peer Sharing

When sharing a skill via hermes skills install <url>, the SKILL.md must have complete frontmatter or peers get a warning:

"The file lacks proper Hermes SKILL.md frontmatter."

Required frontmatter fields for clean peer install:

---
name: skill-name
description: Use when ...   # ≤1024 chars, starts with "Use when"
version: 1.0.0
author: your-github-username
license: MIT
platforms: [linux, macos, windows]   # ← REQUIRED — missing = warning
metadata:
  hermes:
    tags: [tag1, tag2]
    homepage: https://github.com/you/skill-name
    related_skills: [other-skill]
---

Publish workflow:

# 1. Create repo with the skill files
mkdir my-skill && cd my-skill
cp ~/.hermes/skills/<category>/<name>/SKILL.md .
cp -r references/ templates/ .        # include support files

# 2. Write README.md with install command, smoke test results, file list
# 3. Init, create remote, push
git init && git add .
git commit -m "feat: initial release"
gh repo create my-skill --public --source . --push

# 4. Peers install with:
hermes skills install https://raw.githubusercontent.com/<user>/<repo>/master/SKILL.md

Validate frontmatter before pushing:

import yaml, re, pathlib
content = pathlib.Path("SKILL.md").read_text(encoding="utf-8")
assert content.startswith("---")
m = re.search(r'\n---\s*\n', content[3:])
fm = yaml.safe_load(content[3:m.start()+3])
assert "name" in fm and "description" in fm and "platforms" in fm
assert len(fm["description"]) <= 1024
print("✅ Frontmatter valid")

---

Smoke Test Output Format

When reporting smoke test results, use this format (emoji headers, "Prompt tokens" label, system counted as a message):

==============================================================
🧪 TOKEN COST OPTIMIZATION — SMOKE TEST
   Model  : <model-name>
   Pricing: $X/1M in · $Y/1M out · $Z/1M cache-read
   Scenario: <description>
==============================================================

──────────────────────────────────────────────────────────────
📥 BEFORE  (raw, no optimization)
──────────────────────────────────────────────────────────────
   Messages          : N  (incl. system)
   Prompt tokens     : XX,XXX
   ↳ tool output     : XX,XXX  (XX% of total)
   ↳ system prompt   : XX
   Est. cost / call  : $X.XXXXX

──────────────────────────────────────────────────────────────
📤 AFTER   (headroom SmartCrusher — Strategy 0)
──────────────────────────────────────────────────────────────
   Messages          : N  (incl. system)
   Prompt tokens     : X,XXX
   ↳ tool output     : X,XXX  (XX% of total)
   Est. cost / call  : $X.XXXXX
   Compression time  : X.XX ms
   Rows kept         : XX/XXX  (XX% removed)  |  errors always kept: XX

==============================================================
📊 DELTA
==============================================================
   Tokens saved            : XX,XXX  (XX.X%)
   Cost saved / call       : $X.XXXXX
   Cost saved / 1,000 calls: $XX.XX
   Cost saved /10,000 calls: $XXX.XX

Key formatting rules:

• Use model's actual pricing — not GPT-4o pricing if user is on Claude
• Count system prompt as message 1 → Messages: 5 (incl. system)
• Label is Prompt tokens not Total tokens
• Rows format: kept/original not original → kept

---

Quick-Reference Checklist

Run this mental checklist before deploying any LLM feature:

• headroom installed? For any app with tool calls, RAG, or log output, pip install "headroom-ai[all]" and wrap the client or start the proxy — typically yields 60–95% savings with zero accuracy loss.
• Measured baseline token usage per request type (response.usage, headroom perf)
• Chose the cheapest model tier that passes evals
• System prompt is ≤ 500 tokens (unless prompt caching covers the excess)
• Long static context is marked for prompt caching (or headroom CacheAligner handles it)
• History trimming / summarization in place for multi-turn chat (or headroom Rolling Window)
• max_tokens set per task (not left at API default)
• Tool schemas pruned to only relevant tools
• Extraction tasks use JSON/structured output
• Repeated queries go through semantic cache
• Bulk offline tasks use the Batch API

---

Smoke Test Report Format

When producing a token cost smoke test report, always use this exact format:

• Header emoji: 🧪 for test header, 📥 BEFORE, 📤 AFTER, 📊 DELTA, 🗄️ caching strategy, 🏷️ model tiering, ✅ checklist
• Label: Prompt tokens (NOT Total tokens)
• Messages count: include system as message 1 → Messages: N (incl. system)
• Compression rows: Rows kept: N/M (X% removed) | errors always kept: N
• Model/pricing: ALWAYS use the user's actual active model and Anthropic pricing — never default to GPT-4o or OpenAI pricing
• Indent: 3 spaces inside each section block

Example BEFORE block:

📥 BEFORE  (raw, no optimization)
──────────────────────────────────────────────────────────────
   Messages          : 5  (incl. system)
   Prompt tokens     : 24,186
   ↳ tool output     : 23,665  (98% of total)
   ↳ system prompt   : 24
   Est. cost / call  : $0.07256  (@ $3.0/1M in)

---

Strategy 11 — Measure Actual Spend from the Hermes Session DB

When running through Hermes, every session's token counts are written to a local
SQLite DB. Query it directly to see exactly how much you spent today, last 7
days, or last 30 days — no API key or dashboard required.

DB location:

• Windows: %LOCALAPPDATA%\hermes\state.db
• Linux/Mac: ~/.local/share/hermes/state.db

Key columns in sessions: input_tokens, output_tokens,
cache_read_tokens, cache_write_tokens, model, started_at.

⚠️ Copilot billing caveat: when billing_provider = 'copilot',
estimated_cost_usd is always 0.0. Compute cost yourself from token counts
using Anthropic API-equivalent pricing — this shows what it would cost on the
direct API, not the Copilot subscription fee.

Cost formula:

cost = (
    input_tokens        / 1e6 * price_in   +
    output_tokens       / 1e6 * price_out  +
    cache_write_tokens  / 1e6 * price_cw   +
    cache_read_tokens   / 1e6 * price_cr
)

Cache-read tokens can reach millions per session (every KV-cache hit is counted)
but are cheap ($0.30/1M). Always include them — omitting understates cost by ~30%.

See templates/spend_report.py for a full CLI tool with --today / --7d / --30d
flags and a per-model daily bar-chart breakdown.
See references/hermes-session-db.md for the full DB schema and query patterns.

---

Support Files

File	Purpose
templates/smoke_test_token_opt.py	Runnable BEFORE/AFTER smoke test (tiktoken only, no headroom install needed). Uses emoji headers and Anthropic claude-sonnet-4-5 pricing by default. Copy and run: python smoke_test_token_opt.py
templates/cost_count.py	CLI token counter — counts tokens and estimates cost for any text/JSON conversation. Accepts inline text, file, or stdin. Flags: -f FILE, -m MODEL, --system TEXT, --list-models. Requires tiktoken.
templates/spend_report.py	CLI spend reporter — queries Hermes state.db for actual session token counts and computes $ cost. Flags: --today, --7d, --30d, --all, --from/--to. No extra deps beyond stdlib + sqlite3.
references/headroom-ai.md	headroom-ai architecture, benchmarks, confirmed Windows uv-venv install (no MSVC), SDK integrations
references/hermes-session-db.md	Hermes state.db schema reference — cost-relevant columns, copilot billing caveat, cost computation pattern, model-name variations, cache-read token note.

---

Common Pitfalls

1. Not trying headroom first. For any app that has tool calls, RAG results, or log output in context, headroom is almost always the highest-leverage first move. Install it and wrap your client before hand-tuning anything else.

2. headroom-ai on Windows — use uv venv, NOT bare pip install. Bare pip install headroom-ai / uv pip install headroom-ai --system tries to compile Rust crates and fails without MSVC. But creating an isolated venv first pulls pre-built wheels and succeeds without any build tools:

   uv venv headroom-env --python 3.11
   source headroom-env/Scripts/activate   # bash/git-bash
   # OR: headroom-env\Scripts\activate.bat  (cmd)
   uv pip install "headroom-ai[all]"
   python -c "import headroom; print(headroom.__version__)"  # ✅ 0.23.0

   Confirmed working on Windows 10 with Python 3.11.15, uv, no MSVC installed.
   Full details: references/headroom-ai.md.

3. crush_array_json() inflates tokens — use crush() instead. crush_array_json() returns a dict with items, ccr_hash, dropped_summary, strategy_info, compacted, and compaction_kind keys. Serialising this dict adds ~50% overhead. Confirmed with real data: 17,062 raw tokens → 25,589 after crush_array_json() vs → 10,688 after crush(). Always call .crush(text, query="...") and use .compressed from the result.

4. Compressing user messages. headroom never touches user messages for a reason — user intent must be preserved exactly. Don't manually truncate or rewrite user input.

5. Optimizing before measuring. You often don't know where tokens actually go until you log response.usage or run headroom perf. Profile first.

6. Caching dynamic content. Prompt caching only helps if the prefix is truly static. If user ID / session info is in the system prompt prefix, the cache never hits. headroom's CacheAligner automatically moves dynamic content to the tail to fix this.

7. Setting max_tokens too low. If the model hits the limit mid-sentence the output is truncated and often useless. Set it generously for tasks that need complete responses.

8. Ignoring completion-token cost. For some providers, completion tokens cost 3-5× more per token than input tokens. A 100-token completion can cost more than a 400-token prompt. Control output length aggressively.

9. Stripping context that's actually needed. Over-aggressive trimming causes the model to lose track of key facts, leading to retries that cost more than you saved. headroom's CCR (Compress-Cache-Retrieve) avoids this by storing originals and letting the LLM retrieve them on demand.

10. Forgetting tool schema tokens. Each tool in the tools= array is tokenized into the prompt. 10 tools × 200 tokens = 2,000 tokens added to every request silently.

11. Not testing after compression. Always re-run evals after changing prompts. headroom's python -m headroom.evals suite and benchmark table can validate accuracy preservation automatically.

---

Verification Checklist

• headroom installed and wrapping LLM client (or proxy running) for any tool/RAG/log-heavy app
• headroom perf run to measure actual savings on real workloads
• Baseline cost/token logged before optimization
• Model tier justified by task complexity
• Prompt caching applied to static prefix (≥1024 tokens for Anthropic; headroom CacheAligner handles alignment automatically)
• max_tokens explicitly set per task
• Tool schemas filtered to only relevant tools
• History trimming strategy in place for chat pipelines (headroom Rolling Window or manual)
• Semantic cache hit rate measured after deployment
• Evals re-run after prompt compression to confirm no quality regression (python -m headroom.evals suite --tier 1)