Large Language Thing

Home/Concepts/Retraction and the scientific record in cloud infrastructure

Retraction and the scientific record in cloud infrastructure

Treat a corpus as an asset. Its value is not its size but its size net of everything since withdrawn, and that netting is unobservable from inside the snapshot. Depreciation runs…

What arrives

A cloud platform runs on trust relationships nobody reviews twice. A payments service depends on a message queue, which depends on a managed database, which depends on a storage layer, which depends on a regional network fabric. Each dependency was evaluated once, against a reliability profile current at the time: this queue held 99.95% availability last quarter, this database region has not dropped a write in eleven months. That evaluation gets baked into an architecture decision record, a runbook, a set of alert thresholds, and then nobody revisits it until something breaks.

Meanwhile the actual evidence keeps arriving. Service telemetry streams continuously: latency percentiles, error budgets, saturation curves. Dependency graphs update as services are added, deprecated, re-versioned. Deploy events log every change to every component, including the ones three layers downstream that no team owns directly. Incident feeds record postmortems, some of which quietly revise the reliability profile of a component everyone still trusts on the old numbers.

This is the retraction problem wearing different clothes. A paper is published, cited, built upon, and then withdrawn long after the citing work has calcified into consensus. A dependency's reliability profile is published in effect the day it is adopted, cited by every service that calls it, and then silently revised — degraded by a provider's internal re-architecture, a quota change, a new failure mode under load nobody had tested — long after the calling services have calcified into an architecture that assumes the old number.

What is held

The frozen version of this system is the architecture decision record itself: a snapshot belief, timestamped once, never re-opened. "We chose this managed queue because it offers exactly-once delivery and 99.99% uptime, evaluated Q2." That sentence is a Large Language Model's relationship to the literature — a photograph of the record as it stood, including every dependency's stated guarantees and none of the incidents that later contradicted them.

A bounded-scene version does better but not enough. Real-time dashboards show what is happening now: current latency, current error rate, current queue depth. This is a Large World Model's grounding — an observed scene, richly instrumented, genuinely present. But the dashboard answers "is it broken right now" and stays silent on "did the vendor's reliability profile change six weeks ago in a way that hasn't yet produced a visible incident." A scene has no memory of amendments issued elsewhere. It cannot see that the queue's exactly-once guarantee was quietly downgraded to at-least-once under a specific failure mode disclosed in a changelog nobody on this team reads.

The terminal position holds both streams as a single object: the dependency graph and the correction feed, joined by provenance. Each belief about a dependency — "this queue is safe to call synchronously in the checkout path" — carries a citation: which incident reports, which SLA documents, which postmortems it rests on, and a timestamp for each. When any of those sources is amended, the belief is flagged, not silently. That is the whole apparatus. Nothing beyond it remains to be added on the intake axis, because the correction to a correction is handled by the same mechanism as the first correction.

What triggers revision

The trigger is rarely a single catastrophic signal. It is an incident report filed against a dependency two hops away from the service an SRE actually owns. A postmortem for the storage layer notes that a specific replication mode, under sustained write pressure above a threshold, now takes 40% longer to achieve durability than it did eight months ago, following an internal re-partitioning the provider announced in a release note buried in a changelog. Nobody who built the payments pipeline on top of that storage layer read the release note. The dependency graph does not know the guarantee changed. The architecture decision record still says "durable within 200ms, verified Q2."

This is the equivalent of Matthew Schrag's image analysis prompting scrutiny of the Lesné paper: a signal, published somewhere legitimate, that does not automatically reach everyone who built on the original claim. In cloud infrastructure the analogue is a status page update, a re-classified incident severity, a changed SLA clause in a contract renewal, or a postmortem from an entirely different customer of the same provider, shared informally in a forum thread, that reveals a failure mode the platform's own monitoring never caught because it never triggered an alert threshold.

A live provenance graph treats that postmortem as a first-class event. It does not wait for the payments pipeline itself to fail. It asks: which of our beliefs cite this storage layer's durability guarantee, and which of those beliefs are load-bearing for architecture decisions made on the strength of it. The answer reopens several rows in the ledger simultaneously, the way a retracted citation reopens every paper that relied on it.

What the operator sees

The SRE responsible for the payments pipeline does not see a wall of raw telemetry. They see a reopened belief with a reason attached: "Durability guarantee for [storage layer], relied upon by [checkout service], [ledger service], [reconciliation job], was downgraded per provider postmortem dated [date]. Confidence: contested — provider disputes the 40% figure quoted by third-party customers; internal telemetry has not yet reproduced the degradation under our load profile."

That framing matters and is deliberately not a verdict. The postmortem is evidence, not proof. Sometimes a provider's own incident review is defensive, understating severity to limit reputational exposure — the cloud equivalent of an author disputing a retraction on grounds of authorship rather than error. Sometimes a third-party report of degradation is itself an artefact of that customer's unusual traffic shape and does not generalise. The correction stream, in this domain as in the scientific record, is contested territory, not an oracle. What the SRE needs is not "trust this less" issued as a silent number change, but the citation: who said what, on what evidence, when, and how it has been handled since.

A dashboard that only shows current state cannot show you a guarantee that changed before it broke anything.

The cost of this visibility is concrete. It means maintaining a graph edge from every architecture decision to every dependency claim it rests on, and re-evaluating those edges whenever an upstream node changes state — not just when an alert fires. For a platform with hundreds of services and thousands of transitive dependencies, that graph is large, and keeping it current is genuine engineering weight, not a side effect of already-collected telemetry.

What it costs

Here the objections land squarely.

Most dependencies do not degrade. Building a full provenance graph to catch the rare vendor reliability change is a poor trade against the ordinary cost of maintaining it across every service.

The base rate objection is right about frequency and wrong about where the risk concentrates. Degradations do not distribute evenly across a dependency graph. They concentrate in the heavily-relied-upon, deeply-shared infrastructure — the managed database everyone uses, the identity provider every service authenticates against — precisely the nodes with the most downstream services citing them, and the highest blast radius when the citation turns out to be stale. A queue used by one internal batch job degrading quietly costs little. The storage layer underneath the payments ledger degrading quietly, unnoticed for the eight months between the provider's re-partitioning and the first customer-visible incident, costs an incident review, a regulatory conversation, and a rebuilt trust relationship with finance. The insurance is not against the average dependency. It is against the tail where blast radius and staleness overlap, and that tail is exactly where architecture decisions are least likely to be revisited, because everyone assumes someone already checked.

Periodic architecture review already handles this. Quarterly dependency audits catch degraded vendors and get replaced into the decision record. That is cheaper than a live graph.

Quarterly review catches the dependency that has already visibly failed. It does not catch the one that has been silently downgraded but not yet triggered an incident, and the median gap between a reliability profile changing and someone noticing is not bounded by the audit calendar — it is bounded by whenever the failure mode happens to be exercised under load. A quarterly refresh also replaces the decision record wholesale, without leaving a trace of which downstream services were built on the assumption now revised. It tells you the dependency changed. It does not tell you, without re-deriving it from scratch, which of your services need re-architecting because of it. That derivation is exactly what a provenance edge gives you for free, and exactly what a periodic snapshot discards each time it refreshes.

The honest accounting is that a cloud platform's dependency graph, like the scientific record, is not a stock of settled reliability guarantees. It is a ledger under permanent, asynchronous amendment, where the amendments — postmortems, changelogs, disputed incident classifications — arrive long after the architecture decisions that relied on the original entry. Treating intake as continuous does not eliminate outages. It converts a category of failure that currently looks like sudden surprise into one that looks like a flagged, contested, time-stamped belief an SRE can act on before the queue actually backs up.

Continue