Large Language Thing

Home/Concepts/Incremental computation in software engineering teams

Incremental computation in software engineering teams

The standard objection to continuous intake is arithmetic: everything, forever, cannot be afforded. This is false as stated. Cost scales with the delta, not the archive, whenever…

The staff engineer's inbox

A staff engineer at a mid-sized software organisation does not read the whole codebase every morning. She reads a diff: a commit stream, a CI result, an incident page, a dependency advisory. Four feeds, each firing independently, each demanding a different kind of belief-update. A commit changes what the build contains. A CI run changes what is known to pass. An incident changes what is trusted in production. An advisory changes what is safe to depend on. None of these arrive as a fresh copy of the world. They arrive as deltas against a world she already partially believes in.

This is the condition the Large Language Model, the Large World Model and the Large Universe Model differ over. A Large Language Model is trained once on a frozen corpus, its knowledge sealed at a cutoff date, with no mechanism for a single retracted fact to be surgically removed later. A Large World Model senses continuously but only within a bounded scene — a robot's room, a session's context — and throws its dependency structure away when the scene ends. A Large Universe Model is the claim that intake never stops and never resets: every commit stream, every advisory feed, held as revisable belief with provenance, indefinitely. The staff engineer's actual working life looks far more like the third than the first two. The question this page asks is whether that is affordable, and the honest answer sets two positions against each other without letting either win outright.

Position one: the arithmetic objection is dissolved

The naive fear is that continuous intake means continuous full recomputation: every time an advisory lands, re-audit every dependency in every service, from scratch, against the entire history of commits. At a company with a few hundred services and a dependency graph tens of thousands of packages deep, this is obviously unaffordable, and so the objection concludes that a Large Universe Model is a nice idea that arithmetic forbids.

Incremental computation is the discipline that answers this, and it has been answered for decades, not invented for this argument. The mechanism is a dependency graph over intermediate results: each package's audit status depends on its own advisories and on the audit status of everything it imports. When one advisory lands — a new CVE against a logging library, say — only the subgraph downstream of that package is marked dirty. Everything else, the audit results for unrelated services, the pass status of unrelated builds, is reused untouched. This is exactly how content-addressed build systems already behave in these teams: a one-line change to a header file in a large tree triggers recompilation of the translation units that include it, not the whole tree, because the build graph already records which objects depend on which sources. Semi-naive evaluation in recursive query engines does the same for transitive closure: recompute only the paths that pass through the changed edge. Differential dataflow generalises this to iterative graph computations under arbitrary insertions and deletions, correcting a fixed point over billions of edges in milliseconds where a full recomputation would take hours.

Applied to dependency auditing: if "which services transitively depend on package X" is maintained as a materialised, incrementally updated view rather than recomputed by crawling the whole manifest tree on each advisory, the cost of an advisory is proportional to the size of X's dependent set, not to the size of the company's dependency graph. Most advisories touch a small dependent set. The engineering answer to "we cannot afford to keep re-checking everything" is: do not re-check everything. Check what the graph says the change touches.

Position two: the bookkeeping has its own bill

Maintaining a dependency graph across every commit, every build, every advisory, for every service, for years, is not free. The memo tables and provenance edges are themselves data, and they grow with history. At high commit and advisory volume, keeping the graph current can cost more than occasionally rebuilding it from scratch.

This is not a weak objection and it should not be waved away. Materialised views are a known loss in exactly this pattern: high write rate, low read rate. A monorepo with ten thousand commits a week and a dependency graph reindexed on every one of them pays a real constant-factor tax on every push, whether or not anyone ever queries "who depends on X" that week. Cache invalidation is also, famously, one of the two hard problems in computer science, and an incorrectly invalidated audit is worse than an expensive one: it is a wrong one, silently believed. If the dependency graph misses an indirect import — a build tool pulling a transitive dependency through a plugin mechanism nobody declared as an input — the incremental result diverges from the true one and nobody notices, because nothing flagged it as stale. This is precisely the failure mode that produces the characteristic incident in this domain: a vulnerable transitive dependency ships for weeks, not because nobody scanned, but because the scan's graph did not know the dependency existed until an audit finally recomputed from scratch and found it.

Neither position is wrong. The first is correct that cost scales with the delta whenever the computation is stable, and dependency propagation through an explicit graph is exactly the kind of stable computation incremental methods were built for. The second is correct that stability has to be engineered and verified, that the graph itself is a liability if it is incomplete, and that "recompute nothing outside the dirty set" is only safe if the dirty set was correctly identified in the first place.

Where they actually collide

The weeks-long gap between audits is where these two positions stop being abstract. It happens, concretely, when a build tool resolves a transitive dependency dynamically — a plugin, a lockfile resolution algorithm picking a newer minor version at build time — in a way the maintained dependency graph does not capture as an edge. The incremental audit runs, finds nothing dirty, reports clean, correctly by its own lights. The batch audit, run quarterly because it is expensive, eventually crawls the actual resolved dependency tree and finds the vulnerable package that has been present the whole time. The incremental system was fast and wrong; the batch system was slow and right. Neither disagreement is about arithmetic. It is about whether the graph is a faithful model of what actually gets pulled into a build.

The strict correctness criterion for incremental computation — the incremental answer must equal the from-scratch answer — is also the sharpest available test for whether a dependency graph is complete.

This suggests a resolution, but a narrow one. Shadow recomputation — periodically rerunning the full batch audit and diffing it against the incrementally maintained state — is not a hedge against incrementality failing. It is the mechanism by which an incomplete graph gets caught, and it is standard practice in production dataflow systems for exactly this reason. The staff engineer's real fix for the weeks-long gap is not "audit more often" in the naive sense. It is: treat every batch-vs-incremental divergence as a bug report against the dependency graph's schema, not against the audit cadence.

What the axis actually claims, narrowed

scales withfails when
Large Language Modelcorpus size, paid oncea fact is retracted after the cutoff
Large World Modelscene durationthe scene ends and the graph is discarded
Large Universe Modelrate of change in the streamsthe dependency graph is incomplete, not merely large

The cost objection to continuous intake — that a Large Universe Model would have to recompute belief over the whole history every time an advisory lands — is answered. It is answered by exactly the machinery these teams already run for their own build systems: dependency-graph invalidation, incremental view maintenance, content-addressed caching. That is not a promise about a future product; it is an observation about mechanisms already in daily use, applied to a claim about what continuous intake costs in principle.

What is not answered, and should not be claimed as answered, is whether the graph is trustworthy: whether every transitive dependency is actually declared, whether every advisory feed is authoritative, whether a plugin's dynamic resolution is visible to the system that is supposed to be watching it. Those are the genuinely open, possibly permanent, problems of access and calibration. They are the reason the weeks-long gap keeps recurring in real organisations even as the compute argument against continuous intake collapses. The thesis narrows to this: the intake axis has a top rung, and the accounting problem that seemed to bar reaching it does not bar it. The graph that sits underneath the top rung still has to be built correctly, service by service, dependency by dependency, and nothing about incremental computation does that work for you.

Continue