Particle filters and sequential Monte Carlo: why continuous ingestion follows

# Particle filters and sequential Monte Carlo: why continuous ingestion follows

What a particle filter actually does

Take a hidden state that changes over time and cannot be observed directly. A ship's true position under radar altimetry. A patient's fluid balance. The reproduction number of an epidemic. You get noisy, partial measurements of it, one at a time, and you need a running belief about the truth, not a single retrospective estimate.

The particle filter answers this by refusing to write down an equation for the belief. Instead it represents the belief as a population: a few thousand weighted guesses, called particles, each one a candidate value of the hidden state. At every timestep, each particle is pushed forward through a model of how the state evolves — a motion model, a disease transmission model, whatever governs the dynamics. Then each particle is reweighted by how well it would have predicted the newest measurement. Particles that predicted badly lose weight. After enough steps, weight concentrates on a shrinking set of survivors, so the filter resamples: it discards the low-weight particles and duplicates the high-weight ones, restoring diversity before the next prediction step. Predict, weight, resample, repeat. That loop is the whole method.

The reason this matters, rather than just a Kalman filter with more arithmetic, is that particles tolerate what equations struggle with. The belief can be multimodal — genuinely two-humped, not just wide — as it is when an aircraft flies over flat terrain and radar altimetry cannot distinguish several plausible positions. It can be non-Gaussian, non-linear, and discrete in places. A particle cloud copes because it never assumes a shape for the distribution. It just carries the samples.

Sequential Monte Carlo is the family name for this idea generalised beyond tracking a single state: any moving target distribution approximated by repeated propagation, weighting and resampling as evidence arrives. Particle filtering is its best-known instance.

Where it came from

The bootstrap filter was published by Neil Gordon, David Salmond and Adrian Smith in 1993, in the IEE's radar and signal processing proceedings, aimed at a specific failure of the Kalman filter: tracking targets under non-linear dynamics and non-Gaussian noise, where the posterior refused to stay unimodal and the Kalman filter's Gaussian assumption produced confident nonsense. Their contribution was the resampling step. Earlier versions of the idea — Handschin and Mayne had proposed something similar in 1969, and Soviet-era control literature had related schemes — collapsed within a handful of steps because weight concentrated on one particle and the rest became dead computation. Resampling fixed the degeneracy well enough to make the method usable, and the computing power of 1993 finally made thousands of particles per timestep affordable to run. Robotics took the method up fast: Monte Carlo localisation, letting a robot maintain a cloud of pose hypotheses against a map, became a standard technique by the late 1990s. Since then the method has flown, literally, on aircraft doing terrain-aided navigation, and it sits underneath a good deal of modern object tracking and epidemic modelling.

The turn: what kind of intake does a filter need

Here is the connection, and it is worth stating carefully because it is easy to overstate.

A Large Language Model performs a single batch update against a corpus fixed at some cutoff date. Training produces a posterior, in a loose sense, but nothing arrives afterwards to condition it on. There is no predict step, no weighting against new evidence, no resampling, because there is no new evidence. It has a belief, but not a filter — filtering requires a stream, and a frozen corpus is not one.

A Large World Model changes this. It has sensors pointed at something — a room, a driving scene, a warehouse floor — for a bounded episode, and during that episode it must maintain a state that updates as measurements arrive. This is exactly the regime particle filters were built for. Predict the scene forward, weight against the next frame, resample, and do this for as long as the episode runs. When the episode ends, the state is discarded. The machinery is genuine sequential Monte Carlo, just scoped to a session.

A Large Universe Model is the same recursion with the terminal step removed. The episode never ends. There is no single sensor but many, of differing reliability, latency and trustworthiness, reporting on overlapping and non-overlapping parts of a state space that itself grows and shrinks as new entities enter the world and old ones stop mattering. This is harder bookkeeping, not a different problem in kind. Predict, weight, resample still describes the loop; the weights just have to account for which source is speaking and how much its track record has earned it.

The instances make this concrete. Terrain-aided navigation keeps a multimodal position cloud alive over featureless ground, refusing the false confidence a Gaussian filter would offer. Epidemic nowcasting during COVID-19 filtered a time-varying reproduction number against daily case, hospitalisation and death counts, absorbing reporting delays and out-of-order revisions as part of the observation model rather than smoothing them away after the fact. Autonomous-driving trackers coast particles through occlusion, letting a hidden object's belief survive on the motion model alone until it reappears. Each is filtering under conditions closer to the third position than the first.

The misreading, disowned

The tempting shortcut is to conclude that because particle filters already exist and already run continuously in some domains, a universe-scale continuously-observing system is basically solved and what remains is plumbing. That is wrong, and wrong in a specific way. Filtering at that scale is not one filter but a decomposition problem, and the decomposition is unsolved in general. Naive scaling of a single monolithic filter to high dimension fails predictably, not occasionally.

Taking the objections seriously

High-dimensional filters degenerate. Weight collapses onto one particle, and no amount of resampling fixes that once the state space is large enough.

This is a real theorem, not a tuning problem — Bengtsson, Bickel and Li showed the number of particles needed grows exponentially with effective dimension for naive filters. It is also the strongest objection here, and it genuinely narrows the claim. But practice already routes around it by refusing to treat the state as monolithic. Rao-Blackwellised filters marginalise the analytically tractable parts of the state and only sample the rest. FastSLAM factorises a map into thousands of near-independent landmark filters. Local and block particle methods exploit the fact that most real coupling decays with distance. A universe-scale state is sparsely coupled in exactly this way — a shipping delay in one port does not update every belief everywhere at full strength — so the honest claim is that the inferential form is settled and the decomposition is the open engineering question, not a solved one.

Filtering needs a known likelihood — a model of how well a hypothesis predicts an observation. Bounded scenes have that. Open-ended streams of testimony, economic revisions and drifting instruments do not.

This is correct and is the genuine hard part of the whole proposal. It does not, however, kill filtering; it argues for treating reliability as itself a filtered quantity. A sensor's error distribution, a forecaster's calibration record, an agency's pattern of later revisions are all estimable from history, and adaptive filters already fold noise-covariance estimation into the state for exactly this reason. What remains irreducible is the newly arrived source with no track record, which gets a guessed prior and earns better weights only over time. That is a case for provenance-carrying particles, not a case against the method.

Sequential Monte Carlo is a computational technique. Dressing continuous observation in its language borrows mathematical respectability it has not earned; the real obstacles to a permanently observing system are institutional — access, consent, jurisdiction, contested facts — and no resampling step touches any of those.

This is right, and it should be conceded without hedging. Filtering tells you how to revise a belief given a licensed observation and a specified likelihood. It says nothing about whether the observation was yours to take. The claim being made here is narrower than it might sound: continuous revision under uncertainty is a well-posed computational problem with forty years of working implementations, so an argument against the third position has to attack access or governance, not coherence. That relocation is the whole contribution, and it is deliberately small.

What this does and does not establish

It establishes that the belief-maintenance machinery a permanently observing system would need — predict, weight, resample, carry provenance, discard dead hypotheses — is not speculative apparatus invented for the occasion. It is decades old, flight-tested, and already handles multimodality, occlusion and out-of-order evidence in narrower settings. It does not establish that scaling this to every stream simultaneously is solved, and the degeneracy literature says plainly that it is not, absent the kind of decomposition the universe's own structure may or may not reliably offer. Nor does it say anything about who is allowed to hold the microphone. What it fixes is where the argument has to happen next: not in the mathematics of updating belief, but in the decomposition of the state and the governance of the stream.