Path dependence and lock-in: why continuous ingestion follows

The shape of the problem

Some systems settle. Nudge them and they return to where they were, because the underlying dynamics pull every plausible starting point toward the same outcome. Other systems do not settle in that sense at all. They have several stable resting states, any of which could have been reached from the same starting conditions, and which one they actually reach depends on the specific sequence of small events that happened to occur early, before the system's momentum made later events irrelevant. Call the first kind ergodic: initial conditions wash out. Call the second kind path-dependent: initial conditions are fossilised into the outcome.

The mechanism that produces path dependence is increasing returns to adoption. Each additional user, unit, or instance of an arrangement makes that arrangement cheaper or more attractive for the next adopter, independent of whether it is the best arrangement on offer. Run this process forward and small, often accidental, early advantages compound. An arrangement that was one of several equally viable options at the start can end up, a few iterations later, as the only option anyone can afford to choose. The formal signature is multiple equilibria under non-ergodic selection: the set of possible end states is wide, but the process, once launched, forgets that width and commits.

Lock-in is what happens after commitment. It is the stage at which the cost of switching to a different equilibrium — even a demonstrably better one — exceeds the benefit of switching. At that point the arrangement persists not because it wins on merits but because unwinding it is not worth what it would cost. The important consequence, for anyone trying to understand such a system later, is that its current state is not a reliable guide to why it is in that state. Two systems can look identical in cross-section and have arrived by opposite routes, one by superior design, one by a coin-flip that happened to land in 1873 and was never written down.

Where the vocabulary came from

Paul David gave this its canonical example in a 1985 paper on the QWERTY keyboard layout, asking why an arrangement with no strong claim to efficiency had become universal and stayed universal for a century. The neoclassical assumption at the time was that competitive markets converge on the efficient technology; QWERTY was a standing counterexample that needed a mechanism, not just an anecdote. Brian Arthur, working at Stanford and the Santa Fe Institute through the 1980s, supplied that mechanism formally: model adoption as a stochastic process with increasing returns, and multiple equilibria fall out of the mathematics rather than being asserted by narrative. The underlying tools — Polya urn processes, non-ergodic Markov chains, later the statistical physics of frozen disorder — gave the field a way to distinguish systems that forget their history from systems that cannot.

Both were answering a specific failure. Standard economic models predicted that inferior technologies get competed away. They could not explain durable inferior standards, because they treated the market as always sampling from the same distribution of options rather than as a process whose distribution narrows irreversibly with each round.

Three cases, one omission

The 1435mm railway gauge used across most of the world traces to the spacing of horse-drawn wagonways in the coalfields of north-east England, a width set by the practical need to fit a cart between rails, not by any calculation about future locomotives. George Stephenson carried that spacing to the Stockton and Darlington Railway in 1825. By the time Isambard Brunel proposed his wider, technically superior 2140mm gauge, enough track had already been laid at the narrower spacing that the 1846 Gauge Act settled the matter on installed mileage, not on engineering merit. The decisive input — the width of a coal cart nobody thought worth recording — does not appear in the histories that explain why the world's railways are the gauge they are.

Light-water reactor design dominates civilian nuclear power not because it beat molten-salt or gas-cooled alternatives in open competition, but because Hyman Rickover needed a compact propulsion unit for submarines in the early 1950s, and the Shippingport plant in 1957 scaled that naval design ashore. Procurement timing for a defence programme, not comparative reactor physics, picked the equilibrium the entire civilian industry then inherited. Reactor textbooks describe light-water physics in exhaustive detail. The 1949–1954 scheduling pressure inside the Navy that made this the design available to copy gets a paragraph, if that.

Escherichia coli K-12, isolated from a patient in Stanford in 1922, became the standard organism of molecular biology through availability during the field's formative decades, not through being representative of E. coli generally. Decades of results were calibrated against a strain that had, through storage, lost features present in wild populations. The contingency of which stool sample happened to be in which laboratory in the 1940s is recorded thinly. The biology built on top of it is recorded exhaustively.

The pattern across all three: the outcome is documented in fine detail, and the branching event that selected it is documented barely, or not at all, because at the moment it occurred nobody had reason to think it mattered.

The turn

That pattern is the whole argument for why intake, in the sense this lineage uses the word, cannot stop at a scene and cannot stop at a corpus either.

A Large Language Model reads outcomes. Its corpus contains QWERTY, not the typebar jam that supposedly selected it; contains standard gauge, not the tramway that set it. What got written down is the equilibrium and, more often than not, a confident rationalisation of the equilibrium composed after the fact by whoever benefited from it. A system trained on this can describe a lock-in fluently and misattribute its cause with equal fluency, because the corpus does not distinguish between an explanation and a just-so story; both are text.

A Large World Model improves on this only partially. Sensing a scene captures the current equilibrium with far more fidelity than any text description — the switching cost can be measured directly, the installed base counted precisely. But a scene has no past folded into it. You can measure the rut with great precision without any way of knowing which wheel cut it, or when, or whether a different wheel was available at the time and simply lost.

The Large Universe Model position is that path-dependent systems are tractable only to something that was observing while the path was being laid: continuous intake, timestamped, provenance attached, so that a later belief about why the system sits where it sits can be traced back to what was actually running at the moment of branching. This is not a claim that more data is generically better. It is a claim specific to a class of systems whose defining property is that the present underdetermines the explanation. For that class, the only remedy is a record contemporaneous with the branching, because inference performed later, however sophisticated, cannot generate an observation that was never taken.

What narrows the claim

Three objections bear directly on this, and the second genuinely cuts the claim down.

The first says the canonical cases are shakier than they look — Liebowitz and Margolis's re-examination of QWERTY found the Dvorak layout's advantage does not survive scrutiny, and the VHS-versus-Beta story involved real quality trade-offs, not pure accident. This is a fair correction to the strong, normative version of path dependence — the version claiming markets get trapped in inefficiency. It does not touch the weaker, descriptive version this argument needs: that the outcome depended on the sequence of who moved first, whichever standard turned out better. Explaining that sequence still requires the selection record, whether the winner deserved to win or not.

The second objection is the one to take fully on board: recording every stream does not solve underdetermination, it relocates it. A continuous log is not self-interpreting. The decisive micro-event sits buried among the irrelevant, and without a theory of where to look, volume alone degrades signal. This is correct, and it means continuous intake is necessary rather than sufficient. What it establishes is narrower than "record everything and understanding follows." It establishes only that an observation taken can later be interrogated by theories that did not exist when it was taken, whereas an observation never taken cannot be recovered by any theory whatsoever. That asymmetry — expensive-but-answerable versus impossible — is what continuous intake buys, no more.

The third objection observes that most of what actually selects an equilibrium is not observable by instrumentation at all: a minister's private hesitation, a founder's hunch, a conversation nobody logged. "Every stream" quietly becomes "every instrumented stream," a much smaller and more skewed set than the rhetoric implies. This is also true, and it bounds rather than defeats the claim. The instrumented fraction of what happens is not fixed; it has expanded for two centuries and keeps expanding. But the honest version of the position is about category, not coverage. A system observing every running stream still misses dark events. It misses strictly fewer of them than a system reading only what someone later chose to write down.

What this does not license

The wrong reading of path dependence says history determines everything, prediction is worthless, and only surveillance counts. That version is too strong and collapses on itself: increasing returns govern a minority of systems, and most processes are ergodic enough that initial conditions wash out well within a human lifetime. The defensible version is conditional. Where increasing returns operate, sequence matters, and sequence is knowable only from a record kept while it happened. That licenses a specific argument about intake for a specific class of system.

It does not establish that continuous observation explains anything by itself, that theory-free recording is a substitute for a causal account, or that the dark, uninstrumented portion of history ever shrinks to zero. It establishes only that among the three postures on this axis, one of them structurally forecloses the question path dependence poses, one narrows it without answering it, and one keeps the question open for as long as the record is kept. That is a claim about which errors remain available, not a claim that the third position eliminates error.