Trimming the Fat, Not the Signal

Most writing about coding agents focuses on the loop: call the model, run its tools, feed results back, repeat. codeloom.engine's loop is deliberately unremarkable — an OpenAI-style tool-calling loop with a turn cap. The real engineering is in what enters the model's context, what's kept out, and how tool results are shaped.

The engine itself is a headless backend: a long-lived Python process bound to a single workspace, speaking newline-delimited JSON over a Unix socket, exposing 24 tools backed by tree-sitter (syntax) and LSP (types). Clients never touch agent internals — just commands and events. The core idea: context window is a budget, and every byte spent on stale, oversized, or unactionable content is wasted.

The three ways context rots

It fills with output no longer needed — an old search result, a file window read once for a function name, a diff for an edit since undone. It accumulates claims that are no longer true — the model read auth.py at turn 3, a rename touched it at turn 9, but the model still "remembers" the old version. It receives results it can't act on — a full traceback where one line would do, or a hallucinated-argument TypeError.

Every decision below targets one or more of these. Together they form a funnel: disk → tool-level caps → registry cap (80k chars) → history → compactor (120k token budget) → model.

1. Caps at the source, a fuse at the registry Tool ceilings are tuned per tool:

Tool / surface Limit read_file window 200 lines default, 400 max search (ripgrep) 80 matches default, 200 max find_references / goto_definition / document_symbols 50 / 20 / 200 query_tree captures 80 parse_file 200 nodes, depth 8 Diff in a tool result 4,000 characters run_command output 30k per stream, 60k total Any tool result reaching the model 80,000 characters (registry)

Tool-level caps shape the result (e.g., read_file returns a numbered, page-able window; hitting search's cap signals "narrow the query"). The registry cap is a backstop that should rarely fire — it exists so a misbehaving tool can't flood context alone.

Two smaller but easy-to-miss decisions: the read window and edit window share one coordinate system — read_file gives 1-based numbered lines, and replace_lines/insert_at_line take matching 1-based ranges, eliminating an off-by-one bug class. And the model gets a truncated diff (≤4,000 chars, enough to confirm the edit landed) while the client gets the full diff via a FileEdited event — different audiences, different payloads.

1. A cost hierarchy written into the system prompt The prompt tells the model the order to spend its budget: search/list_files to locate a file → list_symbols to see what's in it → find_symbol for a definition's source (which also returns its 1-based line/character position) → that position feeds goto_definition, find_references, or hover → get_diagnostics for errors → read_file windows only when surrounding context is truly needed.

This is a cost gradient: tree-sitter tools are instant, need no server, and work on broken files; LSP tools are authoritative but costlier; read_file is most expensive per unit of information since most of a window is usually irrelevant. The hinge is find_symbol returning a coordinate directly — in many harnesses, getting from "the function is called X" to querying the LSP requires reading the file first to find the line number. Here that step is skipped entirely: one fewer read, one fewer turn, hundreds fewer lines in context.

The prompt also states an editing contract the runtime enforces anyway (read before editing, use str_replace with unique surrounding text, use rename_symbol not search-and-replace, use undo_edit on mistakes) — stating rules the runtime enforces means the model learns them once, not through repeated refusals.

1. Failures are messages, not exceptions An aborted turn is the costliest thing that can happen to context. If any turn raises, history rolls back to a pre-message marker, leaving no partial state — but the bigger point is how rarely that's needed, since almost every failure becomes a readable string instead:

A throwing tool returns error: {message} instead of propagating. Hallucinated arguments that don't match a tool's signature are dropped, not raised as TypeError. A None return becomes an empty string. str_replace refuses on zero or multiple matches rather than guessing. apply_patch tolerates up to five lines of line-number drift per hunk, but any failing hunk rejects the whole patch — a clean failure, never a half-applied one. Editing an unread file yields "read {path} before editing it." Editing a file changed on disk since it was read yields "{path} changed on disk since you read it." The syntax gate rejects edits that introduce new tree-sitter ERROR/MISSING nodes (but not edits to already-broken files — otherwise the agent could never fix one). undo_edit refuses if the file's current hash no longer matches the journaled post-edit hash.

Every case is a one-line instruction telling the model what to do next (re-read, narrow, fix syntax, back off). The turn continues — the 16-turn budget goes to recovery, not to re-establishing context from scratch.

1. Feedback arrives inside the same tool result After a committed write, the engine resyncs the language server (pushing a didChange when the on-disk SHA moved), waits briefly for fresh diagnostics, and appends any new diagnostics to that same tool result — so the model learns about a type error it just introduced without spending a turn on get_diagnostics. This is filtered to a short delta, not the full diagnostic list. Same principle for run_command: a non-zero exit is reported as information, not failure; stdout/stderr stream live to clients while the model gets capped, complete output at the end.

2. History is curated, not appended

Tool messages aren't rehydrated on resume. hydrate() replays only user/assistant messages; tool results are dropped since they're stale workspace snapshots. Cost: the model re-reads files it already saw. The engine judges correctness worth the tokens. The FileTracker resets on every session bind. A resumed session starts with no assumptions about disk state, reinforcing the read-before-edit rule — two independent mechanisms both rebuild the model's picture of the workspace from disk, not memory. Model and client see different representations. Tool messages go to clients as 400-character previews plus structured ToolCallStarted/ToolCallFinished events for UI traces; the model gets the full capped result. Durable memory lives outside the window. {workspace}/.engine/context.md is long-term memory appended across sessions — a home for things summarization would otherwise lose. Only necessary state is persisted. Sessions save to SQLite after each message/edit, but file tree, git state, and detected language are stripped before writing since they're recomputed from disk on load — stale derived state never gets trusted.

1. Compaction as an explicit, observable state At a 120,000-token budget (ENGINE_CONTEXT_BUDGET), history goes to agents/compactor.py, which handles tool-result trimming, summarization, and overflow detection — a tiered strategy: trim old tool results first (cheapest, least valuable post-use), summarize when trimming isn't enough, and mark overflow explicitly rather than silently dropping content. A history-invariant test suggests the compactor must keep tool calls and results paired in a format the API will accept.

Two things matter more than the algorithm itself:

Compaction is a state, not a side effect — AgentStateChanged includes compacting alongside thinking, calling_tool, waiting_for_user, aborting, so a client can show the user the agent is compressing memory. Compaction is reported — a ContextCompacted event carries the strategy, counts, and summary that replaced trimmed history. The difference between an agent that mysteriously forgets and one that says what it forgot; also makes compaction debuggable externally.

StatsUpdated streams token counts, cost, and elapsed time after each exchange — the budget is visible on the wire, not hidden.

1. The response path is budgeted too Output gets the same discipline as input:

Streaming with canonical fallback — ChatMessageStarted opens a message, ChatMessageDelta streams incremental text, ChatMessageAdded follows as the complete canonical version. Clients that miss deltas still land on the right message. Deltas are droppable; canonical messages aren't — each client has a bounded event queue (4,096 items / 1 MiB) with a delta-drop policy; a slow client loses smoothness, never correctness. Oversized events are replaced, not truncated — a 512 KiB per-event soft limit and 8 MiB stream fuse mean an oversized event becomes an ErrorOccurred rather than corrupting the NDJSON stream. Snapshots stream their history — SnapshotReady ships with messages emptied and a message_count for instant UI sizing; history replays as individual ChatHistoryAdded events from a background task, with a generation counter to cancel stale replays.

Trade-offs worth naming

Static caps push work into the prompt — if the 51st reference mattered, the model must notice and re-query narrower. Dropping tool messages on resume costs tokens — the right default for a coding agent, but not free. Summarization is lossy — context.md mitigates this, but nothing prevents it from growing into its own context problem over many sessions. The 16-turn cap is tight for large refactors — rename_symbol's server-computed, batched rename across files is the main lever for staying inside it. The 120k-token budget is a heuristic, assuming headroom above it and a token estimate close enough to the provider's actual tokenizer.

What to take from it

Cap at the tool, then again at the registry — the first shapes, the second protects. Give the model a cost gradient with coordinate handoffs from cheap structural tools to expensive semantic ones. Convert every failure into a one-line instruction — an error string keeps the turn alive; an exception resets it. Put the consequences of an action inside its own result (new diagnostics after a write, a capped diff, exit code as information). Treat history as a curated document — roll back failed turns, drop stale results on resume, keep durable memory outside the window. Make compaction a state with a report, so forgetting is visible, not mysterious. Budget the output path with the same rigor as the input path.

The engine's README frames its architecture as a hard boundary: clients speak JSON, nothing else crosses the line. These context decisions form a second, less visible boundary — between what the workspace actually contains and what the model is allowed to believe about it.