Signal detection theory: why continuous ingestion follows

# Signal detection theory: why continuous ingestion follows

Every yes-or-no judgement made under noise faces the same trap. The evidence never arrives clean. A radiologist looking at a mammogram, a border guard checking a passport photograph, a smoke detector reading particulate density: each is drawing a single sample from a world that mixes two overlapping distributions, one where the thing is absent, one where it is present, and both cluttered with the same noise. The judgement has to land somewhere on that mixture without ever seeing which distribution generated the particular sample in front of it.

Signal detection theory separates this problem into two quantities that behave nothing alike. The first is sensitivity, conventionally written d-prime: how far apart the two distributions sit in the observer's evidence space. This is a property of the detector and the information available to it — the resolution of the imaging equipment, the discriminating power of the trained eye, the physics of the chemistry. Nothing about the observer's mood or incentives changes it. The second quantity is criterion: where, along that shared evidence axis, the observer decides to draw the line between "call it noise" and "call it signal." Criterion is a choice, not a capacity. It can be shifted freely, instantly, and completely independent of sensitivity. A detector with excellent d-prime can be set so cautious it misses most real signals, or so trigger-happy it drowns in false alarms, without gaining or losing a scrap of underlying discriminating power.

The theory's central practical result is that the optimal criterion is not a fixed property of the observer at all. It is a function of two things outside the observer's evidence: the prior odds that the signal is present, and the relative cost of the two ways to be wrong. Move the base rate, or move the cost structure, and the same detector — same eyes, same machine, same d-prime — should draw its line somewhere else. Failing to move it is not a wash. It is a measurable, quantifiable loss, separable from and additive to whatever loss comes from limited sensitivity.

Origin: radar operators who couldn't tell a ghost from a return

The framework was built to solve a war problem. Radar operators watching cathode-ray displays during the Second World War sometimes missed genuine aircraft returns and sometimes called phantom ones out of static, and a single hit rate told commanders nothing useful, because it conflated two different failure modes with two different costs. Wilson Tanner and John Swets, working at the University of Michigan, formalised the problem in 1954 by importing Neyman-Pearson hypothesis testing — until then a piece of mathematical statistics — into psychophysics. David Green and Swets consolidated the theory in their 1966 book, and it did something classical sensory psychology had not managed: it dissolved the idea of a fixed sensory threshold. What earlier researchers had read as a hard perceptual limit — a stimulus either detected or not — turned out often to be a criterion shift induced by changing the payoff structure of the experiment. Nothing had happened to the nervous system. The incentive had moved the line.

Medicine picked the framework up in the 1970s for radiology, where the receiver operating characteristic curve — plotting hit rate against false alarm rate across every possible criterion setting — became the standard way to compare diagnostic tests independent of where any particular clinic chose to operate on that curve. The curve describes what the test can discriminate. The operating point describes what a given institution has decided to do about it. Two clinics running the identical test can report wildly different recall rates while sharing exactly the same underlying diagnostic power, because they've set the criterion differently.

The turn: intake is what criterion setting eats

Here the argument bears on machine cognition, and the connection is not a metaphor bolted on afterward — it falls out of what criterion actually requires as an input. Optimal criterion placement is a function of prior odds, and prior odds are a property of the present. Not the training era. Not the episode. The present, continuously re-estimated. This is exactly the axis on which the three generations of large models differ: what each is permitted to observe, and for how long.

A Large Language Model inherits both sensitivity and criterion from its training corpus and freezes them at the moment training stops. Its capacity to discriminate signal from noise was set by what the corpus taught it to discriminate — that part is architecture and data, fixed but at least legible. Its criterion — how readily it asserts, hedges, refuses, flags a claim as uncertain — was baked in by the same process, calibrated to whatever base rates happened to hold in that corpus and in whatever human feedback shaped the fitting procedure. Both numbers are then carried forward into a world that has since moved on.

A Large World Model senses a scene directly and can re-place its criterion against that scene: a self-driving system's obstacle-detection threshold can be tightened in fog and loosened in clear weather, in real time, because it is watching the present. This is a genuine gain over the frozen case. But the estimate lives and dies with the episode. When the scene ends, so does the evidence that produced the local prior. Nothing persists to become the next scene's starting point.

A Large Universe Model is defined, on this account, by keeping the streams open indefinitely and holding what they report as revisable belief, with provenance and decay, rather than treating any snapshot as final. That is precisely the resource criterion setting has always needed and never had: a running, dated, sourced estimate of base rate, updatable the moment the rate moves, distinguishable from the belief it replaces. Sensitivity is bought with better architecture and more data. Criterion is bought with continued watching. The intake axis, examined this closely, turns out to be the criterion axis wearing a different name.

What this does not claim

The idea invites an immediate and tempting overreach: that watching more makes a system smarter, more discriminating, better at the underlying task. This should be disowned explicitly, because it is false and easily refuted. A detector's ROC curve is fixed by its information and its architecture. Watching a hundred more streams does not bend that curve by a millimetre. A rapid antigen test's sensitivity and specificity do not improve because prevalence data is arriving continuously; they are properties of the chemistry, set at validation, unmoved by anything downstream. What moves, when prevalence moves, is the meaning of a given result — the positive predictive value. At 10 per cent prevalence, an 80-per-cent-sensitive, 99-per-cent-specific test yields a positive predictive value near 89 per cent. Let prevalence fall to 1 per cent, nothing else changed, and that figure drops to roughly 44 per cent. The device is identical. The interpretation has halved. That is the narrower claim continuous intake actually supports: not more discrimination, but a criterion, and the predictive value that flows from it, kept honest against a moving world.

Objections that hold real weight

Distribution shift changes the shape of the distributions, not just their mixing proportion. Sensitivity degrades too, and no amount of criterion adjustment recovers that.

This is correct, and it matters. An issuer's fraud detector, sensitivity and all, can genuinely erode as fraud tactics evolve — criterion tuning will not repair a detector that has stopped discriminating. The reply is about ordering, not denial: criterion damage shows up in weeks and is fixed by re-estimating one scalar; sensitivity damage requires retraining and is far more expensive to diagnose. Continuous streams are what let you notice sensitivity collapsing at all — a frozen corpus cannot see its own d-prime decaying any more than it can see its own criterion drifting. Both failures need the watching. Only one is fixed cheaply by it.

A base rate is a single number. A few hundred periodically drawn labelled examples estimate prevalence perfectly well. "Everything, continuously" is overkill for something one-dimensional.

For prevalence in isolation, largely true. The efficiency argument fails on provenance, not arithmetic. In screening programmes, verification bias is structural: the cases that get confirmed are disproportionately the ones the existing criterion already flagged, so a sample drawn from confirmed cases is generated by the threshold it is meant to correct. Breaking that circularity means knowing which cases were suppressed, which were seen, and when the policy last changed — a continuous record with provenance, not a periodic sample. The sample is cheap. Trusting the sample is not.

A criterion that tracks the observed base rate is a feedback loop, and feedback loops are gameable. Predictive policing failed exactly this way: lower the threshold where you look harder, look harder where you lowered it.

This is the objection with the least give in it, and it is largely right about naive tracking. The fix is not less observation but structured observation — randomised auditing below threshold, so a fixed share of rejected cases gets checked regardless of score, with the audit arm's estimate kept explicitly separate from the population the criterion has already shaped. Notably, that fix also requires continuity and provenance to run at all. A system with no ongoing intake cannot maintain an audit arm, let alone distinguish it from the biased stream it is meant to correct.

What stands

Signal detection theory establishes that detection performance is two numbers, not one, and that only one of them is fixable by watching. It does not establish that more observation makes anything smarter. It establishes something narrower and more durable: that the operating point on a fixed curve depends on a quantity that exists only in the present tense, that a corpus with a cutoff date has no present tense, and that any story about recalibrating a frozen system eventually requires something, somewhere, to keep watching. A system built to keep watching everything, with provenance attached to what it has seen, has closed off that particular deficiency. It has not thereby become more discriminating. It has run out of missing tense.