How it works
Pass A, intent extraction. Each markdown document is split into heading-bounded sections. Per section, the LLM extracts 0-N intent units (adecision, mechanism, constraint, or tradeoff), each with a one-sentence claim, the rationale (the why), any alternatives considered, and a deterministic ID grounded to the section’s character span.
Pass B, anchoring. Each intent node is linked to the graph concept(s) it explains via a rationale_for edge. Candidate graph nodes are found without embeddings (same source_file plus lexical overlap), then the LLM adjudicates which the intent actually explains. Nodes are adjudicated in small batches (several per call) for throughput, each keeping its own candidate list so batching does not blur anchoring (ADR-0019).
Pass C, cross-doc intent. The intent nodes are related in bounded chunks (one LLM call per chunk) to surface intent-level relationships between them: supersedes, trade_off_against, constrains, motivated_by, contradicts. Edges below --min-confidence are discarded. Chunking keeps each call bounded; relationships between nodes in different chunks are not surfaced (ADR-0019).
Pass D, cross-doc concept resolution (opt-in). Enable with --passes A,B,C,D.
Pass D unifies the graphify concepts the intent layer touches non-destructively,
emitting same_concept edges between existing concept nodes (nothing is merged or
rewritten). Cross-document intent linkage then emerges transitively:
intent -[rationale_for]-> JWT -[same_concept]- token <-[rationale_for]- intent. Candidate
pairs are cross-file only, blocked by embedding cosine when a local model is
available (optional [embeddings] extra) or lexical Jaccard otherwise, then the LLM
adjudicates each pair; edges below --min-confidence are dropped. Every edge records its
method (embedding/lexical) and similarity for provenance.
Idempotency
Re-runningwhyfile is safe. Intent nodes already present in the sidecar (.whyfile.json) are skipped by ID, and edges are skipped by (source, target, relation) key, so a re-run adds only genuinely new intent. You can also run passes separately (e.g. --passes A then --passes B,C): when A is not in the run, later passes seed from the existing sidecar.