The efficient market hypothesis in cloud infrastructure

The dependency that used to be reliable

Somewhere in most production systems there is a call to a service that was, eighteen months ago, rock solid. It shipped few changes, its on-call rotation never got paged, and someone wrote its 99.99 percent availability into a design doc that nobody has opened since. The team that depends on it stopped watching it, because watching costs attention and attention is the scarcest resource an SRE has. Then that service migrates its backing store, or its owning team gets reorganised, or a new engineer inherits it without inheriting the context, and its reliability profile quietly degrades. Nothing in the dependent system's model of the world changes to reflect this. The dependency graph, as drawn, is a portrait of an arrangement that no longer holds. The incident that follows is not a failure of the degraded service. It is a failure of intake: someone kept trading on a price that had moved.

This is the efficient market hypothesis, transplanted. The hypothesis, formulated by Eugene Fama in 1970 out of earlier work by Louis Bachelier and Paul Samuelson, holds that asset prices reflect the information available to the market — not that prices are correct, only that they cannot be predictably beaten given what is known. Fama distinguished three grades by what "available" includes: weak form covers past prices, semi-strong form all public disclosure, strong form private information too. The measurable content is speed. When new information arrives, how long before the price moves, and who eats the gap in between?

What an SRE is actually pricing

An SRE holds a position on every dependency their service calls, whether they think of it that way or not. The position is a set of assumptions: this queue drains in under a second, this API is idempotent, this database fails closed rather than open. Those assumptions are prices. They were set at some point, based on observation, and they are correct only for as long as nobody profits from them being wrong. The profit, here, is not measured in currency but in incident minutes, customer-visible errors, and the particular exhaustion of a 3 a.m. page for a fire that a stale mental model made unpredictable.

Service telemetry, dependency graphs, deploy events and incident feeds are the tape. A team that reads the tape continuously is trading close to the true, current state of its dependencies. A team that consults a design doc from a year and a half ago is trading on a snapshot, and the market — in this case, the actual behaviour of actual systems under actual load — will eventually collect the difference, usually during an incident review, usually phrased as "we didn't know it had changed."

Two positions worth taking seriously

Set two claims against each other, because both are defensible and the discipline lives in the friction between them.

The first: continuous intake is worth its cost. A large deploy-event and incident-feed pipeline, correlated against a live dependency graph, catches exactly the failure mode described above — a reliability profile that shifted without anyone re-pricing it. Organisations that maintain service catalogues with freshness SLAs, that page on dependency-graph drift rather than only on service-level errors, are doing the SRE equivalent of watching the tape. They pay in tooling, in cardinality costs for telemetry storage, in engineer hours spent maintaining the catalogue rather than shipping features. They are, in Fama's terms, buying informational advantage, and the advantage is real: fewer incidents caused by trusting something that stopped being trustworthy.

The second, opposing claim: most of that observability spend is waste, because the actual bottleneck in large incidents is rarely missing data. It is the failure to correlate data that was already there. A postmortem that reads "the metrics showed the anomaly forty minutes before the page fired" is not a story about insufficient intake. It is a story about a dashboard nobody was looking at, or an alert threshold nobody re-tuned after the traffic pattern changed. On this view, chasing ever more complete, ever more continuous telemetry is chasing the wrong axis. The axis that is actually broken is inference over the data already collected, and no amount of additional streaming fixes a broken correlation rule.

If you had simply looked at the graph you already had, you would have caught this. The problem was never that we lacked a feed. The problem is that nobody built the query.

Both positions are right about something real. The question is which failure mode is dominant in a given system, and that is an empirical question about a specific incident review, not a theorem.

What the cost of watching concedes

Sanford Grossman and Joseph Stiglitz proved, in 1980, that markets cannot be perfectly informationally efficient: if prices reflected all available information instantly and completely, nobody would have any reason to pay for research, and without anyone paying for research, prices would stop reflecting anything. There must be a permanent, positive return to gathering information, or the gathering stops and the efficiency collapses with it.

Applied here, this names something every SRE organisation already knows in its budget line for observability: the last mile of intake never gets cheaper for free. Someone has to instrument the new service, tag the new deploy pipeline, wire the dependency graph into the alerting system, and keep doing that as the architecture changes underneath them. That work has a permanent cost, and teams that skip it revert, quietly, to trading on stale information. Grossman-Stiglitz does not undercut the case for continuous intake. It prices it. It says the terminal information position — every stream, live, correlated — is never reached costlessly and by everyone at once, and that the market for observability tooling and platform-engineering time will persist indefinitely for exactly this reason. That is a claim about a cost curve, not a claim that some fifth kind of signal exists beyond service telemetry, dependency graphs, deploy events and incident feeds. The paradox concedes the shape of the argument. It just insists, correctly, that nobody gets the last mile for free.

Full dashboards, wrong model

The sharper objection is different and it should be conceded without qualification. Complete telemetry does not guarantee correct inference. Some of the worst production incidents on record happened with the relevant data sitting, fully collected, in a dashboard nobody correlated correctly — the pattern is structurally identical to 2008, where the mortgage exposure data existed in the prospectuses and the failure was in the models built on top of it, not in the data's absence. A cascading outage caused by a retry storm is frequently visible, after the fact, in metrics that were being ingested the entire time. What was missing was not intake. It was the mental model that would have told an engineer, at 2:47 a.m., that this particular latency spike on this particular dependency was the leading edge of a feedback loop rather than noise.

This matters for how much weight the intake argument should carry. It says only that a system's data completeness has a ceiling worth naming, and that the ceiling is real: whatever a monitoring stack does not observe is a quantified liability, discoverable, in retrospect, in the postmortem's "known unknowns" section. It says nothing about whether the humans and automated systems reasoning over that data will draw the right conclusion. Reasoning under complete information is a separate, unfinished problem, and no dependency graph, however fresh, resolves it by itself.

The claim, narrowed

The defensible version is smaller than it first sounds. There is no fifth category of signal past service telemetry, dependency graphs, deploy events and incident feeds, streamed continuously with provenance attached so a reliability assumption can be traced back to the observation that justified it and revised the moment that observation is contradicted. That is the terminal position on the intake axis for infrastructure reliability, in the same sense that strong-form efficiency is the terminal position for markets: all available information, incorporated as it arrives.

Beyond it lies the work that never closes: designing alert logic that turns a fresh feed into a correct inference before the incident, not during the retrospective. That work is not made easier by more data. It is made possible by it, and no more than that.