Abduction and inference to the best explanation: why continuous ingestion follows

Reasoning from effect to cause

A patient runs a fever. Pneumonia would explain it. So, provisionally, pneumonia is believed — not because it has been proven, but because it accounts for what is observed better than the alternatives on hand. This is abduction: reasoning from an effect back to a cause that would produce it. It is the least tidy of the classical inference types and, on the standard account, the only one that manufactures new hypotheses rather than testing or generalising ones already in hand.

Deduction moves from general rule to particular case and adds no new content. Induction moves from many particular cases to a general rule and adds no new content either — it just compresses what was already observed. Abduction is different in kind. It looks at an effect and proposes a cause that was not contained in the observation itself. The fever does not announce pneumonia. Something has to suggest pneumonia, and that suggestion is the creative step logic textbooks usually skip over because it cannot be formalised the way syllogisms can.

Gilbert Harman sharpened this into a rule of acceptance he called inference to the best explanation: among the hypotheses actually under consideration, accept the one that best explains the evidence, judged by fit with the data, simplicity, and coherence with everything else believed. The operative word is "best". Best is a comparative, not an absolute. A hypothesis wins only against the field it is run against. It says nothing about hypotheses that were never entered into the race.

That comparative structure looks like a technical nuance. It is the entire argument of this page.

Where the idea came from

Charles Sanders Peirce isolated abduction across several decades of work spanning the 1860s to 1900s, naming it as a third mode of inference alongside deduction and induction. His problem was not logical tidiness for its own sake. It was a genuine puzzle about science: induction can generalise from cases, but it cannot explain where a candidate hypothesis comes from in the first place. Something has to originate the guess before induction can test it statistically or deduction can work out its consequences. Peirce called this origination abduction, and he thought it was the only place in the whole apparatus of reasoning where anything new entered.

Gilbert Harman's 1965 paper "The Inference to the Best Explanation" recast Peirce's category as an acceptance rule and made a further claim: that ordinary enumerative induction is itself a special case of it — we generalise from samples because the generalisation is the best explanation of why the sample looked the way it did. Peter Lipton's 1991 book, revised in 2004, worked out the criteria for "best" in more detail — loveliness as distinct from mere likeliness, roughly, explanatory depth against bare fit. Bas van Fraassen's 1989 objection, the argument from the bad lot, remains the standard challenge, and it will earn a full hearing below.

The turn

Abduction ranks hypotheses that are already on the table. It cannot rank one that never got there. This is not a flaw in the logic; it is a structural feature of a comparative operation. But it means the quality of an abductive conclusion is capped by the completeness of the candidate set feeding it, and that cap is invisible from the inside. A missing hypothesis does not announce itself as missing. It produces no low score, no flag, no residual doubt attached to its own absence. The system simply ranks what it has and returns a winner.

This is where the intake axis of the lineage becomes the same question wearing different clothes. A Large Language Model performs abduction over whatever explanations happen to be represented in a corpus frozen at some cutoff date. Given that corpus, it can be a genuinely excellent ranker — better, in many domains, than an unaided human working from the same textbook. But it is structurally unable to notice that the true cause of some new effect was never written into anything it was trained on. A Large World Model does better for a while: live sensing during an active scene introduces candidates the corpus never had, and can demote candidates the corpus overrated. Then the scene ends, the sensors stop, and the widened slate closes with it. Nothing carries the newly admitted candidate forward into the next episode.

A Large Universe Model is the position that treats the candidate set itself as a maintained object rather than a snapshot: fed continuously by every stream still running, each candidate tagged with the provenance that raised it, revisable the moment that provenance is contradicted. The shift is subtle but total. It is the difference between computing the best explanation once and keeping the space of explanations honest indefinitely.

Three cases make the mechanism concrete. In London in 1854, the reigning explanation for cholera was miasma — bad air. John Snow's waterborne hypothesis was not a low-ranked alternative sitting quietly in the same slate; it was absent from the slate entirely, until door-to-door mapping of the roughly 616 Broad Street deaths, and the anomaly of a Hampstead widow who died after having pump water delivered to her, opened a new stream and admitted a new candidate. In 1982, peptic ulcers were explained by stress and stomach acid, and bacterial causation was excluded on principle — nothing was thought able to survive gastric pH. Warren and Marshall's cultures, which succeeded partly by accident over an extended Easter incubation, put the candidate physically into the room; Marshall then drank a broth of it and endured his own gastroscopy to force the point. In 2009, the loss of Air France 447 was first explained through ACARS telemetry favouring pitot-tube icing. Two years later, flight recorders pulled from 3,900 metres of ocean introduced sustained nose-up input by the pilot flying through a stall — a candidate the original stream could not carry, because it was never in that stream.

Objections, taken seriously

A well-specified prior on a catch-all "none of the above" hypothesis already dissolves this. Better probability hygiene, not more data streams.

The catch-all term is real and generally undervalued. A rising residual probability is a genuine signal that the named hypotheses fit badly. But it is an alarm, not a name. It cannot say which pathogen, which valve, which counterparty. Converting residual mass into a specific, actionable hypothesis is itself an act of generation, and generation requires material the system has not yet seen. Bayesian hygiene tells you something is missing. Intake is what tells you what.

Widening intake multiplies spurious candidates faster than true ones; the classical problem with abduction is that too many hypotheses already fit any evidence, and curation is valuable precisely because it prunes.

This lands, and it narrows the claim rather than defeating it. Curated corpora do encode real judgement about which explanations are worth entertaining, and unfiltered streams do generate noise. The distinction that survives is provenance. A candidate arriving with a traceable source, a timestamp, a sensing path, can be ranked, contested and retired on evidence. A candidate inherited from an undated corpus cannot be interrogated the same way. Wide intake with provenance is an auditable space. Narrow intake without it is a small space nobody can check. The problem does not disappear; it moves from "too many hypotheses" to "rank them honestly," which is a more tractable problem than the one it replaces.

Van Fraassen's argument from the bad lot holds regardless: the best of a bad lot is still bad, and there is no reason to think the truth is even in the candidate set. Widening the set just makes the bad lot bigger.

This is the sharpest challenge and it is not answered by volume alone. What changes under continuous intake is the mechanism generating candidates. Van Fraassen's worry targets sets produced by an unconstrained human imagination guessing at what might be true. Continuous streams replace guesswork with measurement: a candidate enters because a sensor, a recorder, a transaction log registered something, not because someone thought of it. That is no guarantee the truth is in the lot. It does make the lot's composition answerable to the world rather than to a modeller's habits — which is the most any empirical method has ever offered, and no more.

The misreading to disown

The weak version of this argument says continuous intake produces true explanations. It does not, and nothing above claims it does. More streams do not confer truth on any inference; they change which hypotheses are eligible to be ranked in the first place. The claim is narrower and more negative than it sounds: a system with closed intake cannot detect its own candidate-set failures, because an absent hypothesis leaves no trace of its absence. Open intake removes that one specific blindness. It leaves every other difficulty of abduction exactly where it was — bad lots, underdetermination, contested simplicity, miscalibrated confidence, all fully intact.

What this does and does not establish

Pharmacovigilance signals appear first as unranked candidates, and closed intake keeps a drug's real mechanism unnamed for years. Catastrophe models price perils drawn from historical loss slates, and a genuinely novel correlated peril enters the market as an unpriced absence, not as a cautious low estimate. Grid post-mortems keep finding the initiating fault missing from the contingency list rather than merely underrated within it. In each case the failure has the same shape: not a bad ranking, but a candidate that was never entered.

This establishes that intake bounds explanation, and that a bounded intake produces a specific, structural, undetectable blindness rather than an ordinary error. It establishes that the remedy is intake that does not close, with provenance attached so the resulting space can be audited rather than merely enlarged. It does not establish that any system holding open intake reasons correctly, ranks well, or arrives at truth. Nor does it establish that a fourth position exists beyond continuous, provenance-tagged intake — there is no further mode of hypothesis generation waiting past "everything, still arriving." What remains past that point is not a new rung. It is scale, calibration, and how honestly the provenance is kept.