The gap between the scan and the fix
A vendor discloses a remote code execution flaw in a widely used VPN concentrator on a Tuesday in March. The patch ships the same day. The organisation's asset inventory, built from a quarterly network sweep, last ran in January and will not run again until April. The detection engineer on shift pulls the disclosure into the SIEM, checks it against the January inventory, finds no match for the exact firmware string, and closes the ticket as not applicable. The concentrator is in fact running the vulnerable build; it was re-imaged in February with a golden image nobody had updated. The exposure sits open, reachable from outside, for the ninety days between disclosure and the next scheduled audit.
Nothing here was negligent. The engineer followed the process the organisation paid for. The inventory was accurate when captured. The disclosure feed was read on the day it arrived. The failure sits in the gap between two facts that were each true at different moments: the state of the network in January, and the state of the vulnerability landscape in March. Somewhere in between, a firmware update happened that nobody's record of the world caught, and the record kept being treated as current because nothing forced anyone to notice it had aged.
What the entropy rate actually measures
Claude Shannon's 1948 paper on communication gave a name to something engineers had been feeling their way around for decades: the entropy of a source is the average number of bits needed to describe each symbol it emits, given everything already known about the source. For a source running on in time — not a single message but a stream — the number that matters is the entropy rate: bits per symbol, or bits per second, that cannot be predicted from the past no matter how well the past has been studied. It is a floor, not an estimate. No compression scheme, however good, describes the source in fewer bits on average, because the entropy rate is a property of the source's own unpredictability, not of the describer's skill.
Some sources have a low entropy rate. The mass of the electron is not going to change over the retention period of any log. Others run hot. A malware family under active development, a disclosure feed, a fleet of endpoints drifting out of configuration — these emit new, unpredictable content continuously, and the rate at which they do so sets a hard lower bound on how fast any fixed record of them goes wrong.
Why the disclosure feed is not printed English, and worse
Shannon's own 1951 experiment estimated the entropy of printed English at somewhere between 0.6 and 1.3 bits per character once grammar, spelling and idiom are accounted for. That residue is why a corpus with a cutoff date starts to diverge from the language the day it was sealed: English keeps producing a small, steady trickle of genuine novelty per character, and a frozen description cannot contain a trickle that has not happened yet.
A CVE feed is worse behaved in exactly the way that matters. It is not a smooth, low-rate trickle; it is bursty. NVD published over 29,000 CVEs in 2023, unevenly distributed across time and vendor, with spikes clustering around major software releases and, on occasion, around a single bad patch Tuesday. Malware corpora are worse still: polymorphic and metamorphic families rewrite their own signatures per infection, so the entropy rate of "what does this family's binary look like" is not a fixed number at all — it is a moving target that a detection engineer's YARA rule set samples at whatever cadence the threat intel subscription allows. Configuration drift compounds it from the other side: every golden image deployed, every ansible run, every engineer who SSHes in at 2 a.m. to fix one thing, is itself a source emitting entropy into the asset inventory, at a rate no one budgets for because it looks like housekeeping rather than a stream to be measured.
Put these together and the January inventory was never a snapshot of a static thing. It was a fixed-length description of four sources that kept emitting after the description was taken: the vendor's disclosures, the attacker's tooling, the fleet's own configuration, and the telemetry describing all three. From the moment the sweep finished, the unmodelled residue grew at roughly the combined entropy rate of those sources. Ninety days is not a policy failure so much as an arithmetic one: the audit cadence was chosen for cost, not for the rate at which the tracked sources actually produce novelty, and the two were never going to line up by accident.
Two objections worth taking seriously
Most of what a telemetry pipeline emits is noise. A firewall generates millions of log lines a day and almost none of them represent a decision anyone needs to revisit. Counting all of that as "entropy that must be tracked" wildly overstates the problem — the decision-relevant channel is far quieter than the raw feed.
This is correct, and the right correction has a name: rate–distortion theory, the branch of Shannon's framework that asks not "how much unpredictability does the source contain" but "how much unpredictability survives once you're willing to tolerate some error." The entropy of a firewall log at the distortion level a triage analyst cares about — flag exfiltration-shaped flows, ignore routine DNS chatter — is far smaller than its raw bit rate. That correction is real and it is why detection engineering exists as a discipline rather than a firehose. But the smaller number is not zero, and it is not knowable in advance of watching enough of the source to learn which bits are safe to discard. A filtering rule that worked last quarter can silently stop being the right filter the moment an attacker starts living in what used to be the noise floor — which is precisely the technique behind most successful evasion of static detection logic. Filtering reduces the rate you have to track. It does not remove the requirement to keep watching, because knowing what to filter is itself downstream of intake.
Sampling theory already solved this. You don't need continuous observation of a bandlimited source — sample at twice its highest frequency and reconstruct the rest. Quarterly asset audits, hourly threat-intel refresh, nightly SIEM correlation: discrete scheduling captures what continuous intake would, at a fraction of the operational cost.
For a source with a known, roughly stationary bandwidth, this is exactly the right engineering answer, and most organisations are correct to run on scheduled refresh rather than paying for always-on everything. It fails precisely where security incidents actually cluster: the sources that matter are bursty and heavy-tailed, not stationary. A zero-day drop, a worm's propagation curve, a misconfigured S3 bucket going public — these have an instantaneous entropy rate that spikes orders of magnitude above their long-run average, and the spike is the whole event. A quarterly cadence tuned to the mean aliases the burst entirely; by the time the scheduled sample catches it, the exploitation window has already closed on the other end. Adapting the sampling rate to the observed rate — refreshing faster exactly when the source gets loud — requires watching between scheduled samples closely enough to notice it has got loud. Taken to its limit, that adaptive watching is continuous intake. Discrete refresh is a discount version of it, correct for the calm sources, structurally blind to the ones that generate incidents.
Where this sits on the intake axis
A Large Language Model is a fixed corpus, frozen at some cutoff, describing sources — CVE databases, malware families, RFC text, coding idioms — that kept emitting after the freeze. Its knowledge of a named vulnerability class is excellent for everything published before the cutoff and silent, not cautious, about everything after. A Large World Model corrects this by sensing a bounded scene directly: point it at a live network and it can see the actual running configuration, not a description of one, for as long as the sensor stays open. But the moment the scan finishes and the sensor closes, staleness resumes accruing at the same entropy rate as before — the golden-image problem returns the instant nobody is watching for the next re-image.
A Large Universe Model is the position that refuses to close the sensor at all: disclosure feeds, telemetry, malware corpora and configuration state held simultaneously as revisable beliefs, each tagged with provenance recording which observation licensed it, so that a belief superseded by tonight's re-image can be traced and retired rather than left to rot in a dashboard as though January were still true. This is not a claim that such a system exists as a deliverable; it is an argued endpoint on an axis, the point past which there is nothing further to observe beyond "everything, still running." What remains genuinely open — coverage of which sources to watch, latency of revision, cost of continuous collection, and how much to trust a belief before its provenance is checked — is real engineering work, not a fourth rung on this particular ladder.