Skip to main content
A pressure test of a clean end-to-end run surfaced five gaps in the pipeline (throughput, an optimistic estimate, and dishonest “$0” cost framing). Before writing a single fix, we pointed the tool at its own codebase and asked it why the code was the way it was. The recorded intent, extracted by whyfile from its own ADRs, shaped every fix and in one case had already recorded the exact concern we were about to raise. This is that account: the queries we ran, what the intent layer returned, and how each answer changed the fix.

​
Setup

The base graph was graphify’s AST extraction of src/ (199 code nodes). The intent layer was whyfile’s own enrichment of its docs/adr/ trail (the decisions, mechanisms, constraints, and trade-offs behind the pipeline). Every query below is whyfile why / explain / coverage / list-intent, read-only and LLM-free, run against that enriched graph.

​
The loop, gap by gap

​
Gap: the pre-flight time estimate is optimistic

The tool told us the estimate is known to be imperfect (the token breaker exists precisely to catch what the estimate misses), and, crucially, that exact token accounting is blocked upstream because graphify.llm discards usage. That decided the shape of two fixes: the honest estimate had to stay an estimate (we made the constants and the concurrency divisor honest instead of pretending precision), and the “honest cost” fix had to report estimated tokens, not real ones.

​
Gap: throughput / concurrency does not help

The recorded design intent was that --max-concurrency is a latency lever. Our pressure test found it delivers no latency on the subscription backend (a short investigation confirmed graphify’s upstream _call_llm holds a Lock around the claude-cli subprocess, serializing calls). So the intent named the promise, and reality broke it. The tool also pointed at the real lever: “per-section fan-out multiplies the call count.” That reframed gap #1 entirely: the fix is not “make it parallel” (impossible from here, the Lock is upstream) but “make fewer calls” (batch Pass B), plus correcting the docs to say concurrency only helps the API backend.

​
Gap: honest cost (subscription is not “$0”)

The strongest signal in the whole exercise: the intent layer had already recorded the concern that advertising subscription runs as zero-cost is misleading, because they still consume tokens with no visibility. We were about to make exactly that correction. The tool did not just permit the fix, it had documented the problem in its own words. It also warned (via “cache-aware cost accounting”) to keep cache hits out of token totals, which the reporting now respects.

​
Gap: Pass C is one giant call

Silence is a signal. Pass C’s single-call design has no recorded rationale in any ADR. Where the other gaps came with constraints to respect, this one came with none, which told us two things: changing it (to chunk the call) violates no prior decision, and the new decision is worth recording. ADR-0019 now documents the chunking and its cross-chunk trade-off, filling the dark spot the query exposed.

​
The guardrails we had to respect

Before touching the pipeline we pulled the full constraint list, the “do not break” set. None of the fixes touch the structural Pass A gate, the cache-as-persistence contract, or the edge-format rules, because the tool made those constraints explicit up front.

​
Two things this demonstrates about the product

  1. The intent layer answered “why is the code like this?” for real, un-cherry-picked work. The fixes were designed from the tool’s output: the estimate stayed an estimate because the tool said real tokens are upstream-blocked; gap #1 became “batch, do not parallelize” because the tool named the upstream serialization and the call-count lever; the cost-honesty fix was validated by a constraint the tool had already recorded.
  2. Coverage and silence are findings too. why returning no strong match for Pass C told us, correctly, that its design was undocumented, so we knew the change was safe and that it needed a new ADR. A tool that surfaces where the recorded reasoning is thin is as useful as one that surfaces where it is rich.
The honest footnote, in the spirit of the cost fix: this enrichment run itself consumed tokens (claude-cli, subscription) to produce the intent layer we queried. Not free, just not billed per token. The tool now says so.