Home/Concepts/Paradigm shifts and normal science in software engineering teams
Paradigm shifts and normal science in software engineering teams
Kuhn's anomaly problem is an intake problem before it is an epistemic one. A frame does not fail because a decisive experiment is run; it fails because residuals that were each…
The objection that should win
Here is the strongest version of the case against this whole framework, and it deserves to be stated in its own terms before any defence is offered.
Every engineering organisation already observes continuously. Commit streams, CI results, incident feeds, dependency advisories — these are logged, timestamped, retained, often for years, sometimes indefinitely under compliance policy. If continuous observation were sufficient to prevent frame-blindness, no team with a functioning observability stack would ever ship a known-vulnerable dependency. Yet they do, constantly. The failure is not one of intake. It is one of interpretation: engineers see the data and decide it does not matter yet. Piling more streams onto an already-drowning on-call rotation does not fix that. It makes the drowning worse.
This is not a strawman. It is close to the correct account of most real incidents. A staff engineer reviewing a dependency-audit dashboard is not short of information. The advisory for the transitive package was published. The CVE score was computed. The CI pipeline had the manifest in front of it. The problem was that none of these facts crossed a threshold that triggered action, and thresholds are set by a framework — in Kuhn's sense — of what currently counts as urgent. More logging does not change the threshold. If anything, an organisation that streams everything and retains everything has simply built a bigger haystack.
Where the objection is right
Thomas Kuhn's 1962 argument was never really about data volume. Normal science, in his account, is efficient precisely because it does not look at everything: it measures what the accepted framework says is worth measuring, and it is entitled to treat the rest as noise, error, or someone else's problem. That is not a bug in scientific practice. It is what makes steady, cumulative engineering work possible at all — in laboratories and in codebases alike. A team that tried to evaluate every advisory, every flaky test, every marginal latency blip with equal seriousness would ship nothing. Triage is normal science, and normal science is where nearly all real engineering happens.
So the objection is correct that intake alone does not produce judgement. A dependency graph with ten thousand transitive packages, each emitting advisories at its own cadence, is not rendered safe by recording every advisory. Someone still has to decide which ones matter this sprint. That decision is made inside a frame — call it the team's current model of what a "serious" vulnerability looks like — and the frame is exactly what filters a slow-burning risk into the bucket marked "later."
The Challenger case generalises here more cleanly than any software example could invent on its own. Thiokol's engineers had O-ring erosion data across several flights before the 1986 disaster. Each instance was judged, individually, to be within the accepted experience base. The organisation had not failed to observe. It had learned, flight by flight, to widen its definition of acceptable risk, and no amount of additional telemetry would have interrupted that drift on its own. A dependency team that reviews each new CVE against last quarter's baseline is doing the same thing: normalising deviance one audit cycle at a time, each decision locally defensible, the trend invisible because no one is looking at the series.
What still survives
Two things follow, and they are narrower than "watch everything."
First: retention is not the same claim as neutrality, and the objection conflates them. Nobody is arguing that a continuously streaming system sees the vulnerability free of any frame. The claim is that a system which keeps the advisory, with its provenance — which registry flagged it, at what severity, against which manifest hash, on what date — makes that advisory available to a later frame, one in which the threshold has shifted because three other things also turned out to be true. A team that discards low-severity advisories after each sprint's triage cannot later reconstruct that the same transitive package has thrown low-severity flags in four consecutive quarters, each individually ignorable, together a pattern. Nimbus-7's ozone readings sat in an archive for years, flagged as instrument error and set aside by the processing chain, and were only recognised as real once Halley Bay's continuous ground station forced a re-examination. The recognition took years. The values survived because they were kept, error-flag and all, rather than deleted at ingestion.
This is the actual shape of the transitive-dependency failure. It is rarely one advisory a team missed. It is usually a package three levels down the graph that has quietly accumulated four separate low-severity CVEs over five months, none individually worth an emergency patch, together a package that should have been replaced in month one. A staff engineer looking at this month's dashboard sees this month's severity score. A staff engineer able to query the retained history — every advisory that ever touched that package, with the commit and audit-run provenance attached — sees the accumulation. The frame did not change. What changed is that the evidence for changing it was still there to look at.
Second: the objection's institutional point — that Challenger-style failures are about incentives, not instruments — bounds the claim rather than defeating it. Continuous, provenance-bearing intake does not fix an organisation that has learned to treat deviance as normal. It does something more modest: it makes the normalisation auditable. A postmortem that can query five years of dependency-audit history and show the drift as a trend line is a different postmortem from one that can only reconstruct what people remember about "how we usually handle these." Ignoring evidence is a correctable failure of will. Discarding evidence is not correctable at all, because there is nothing left to reconsider once the incentives improve. Fixing a team's triage culture six months from now recovers nothing if the audit logs from this quarter have already rolled off retention.
Why the three intake classes still separate cleanly here
| Generation | What it can see about the dependency graph | Characteristic miss |
|---|---|---|
| Large Language Model | Whatever was true, and considered worth writing down, about packages and advisories up to a training cutoff | Cannot see this morning's advisory at all; its picture of "risk" was filtered by writers who had their own frame of what mattered, then frozen |
| Large World Model | The current manifest, current CI run, current advisory feed, as a scene | Sees this morning's advisory, but a package that has thrown four small, individually-tolerable flags over five months looks, in any single scene, like a package with one small flag |
| Large Universe Model | Every commit, every CI result, every incident, every advisory, held as revisable beliefs with provenance, never truncated | Nothing structurally — the miss becomes a matter of calibration and elapsed time, not of a class of evidence that is unavailable in principle |
The middle row is the one worth dwelling on, because it is where most tooling actually sits today. A dashboard that shows "current known vulnerabilities" is doing exactly what a Large World Model does: reporting a bounded scene, accurately, at this moment. It genuinely admits surprise — an advisory that lands today will show up today. What it cannot do is notice that this package's advisory count has a slope, because the scene has no memory of its own earlier scenes unless someone has separately built that memory, with provenance, on purpose. That separate structure — the thing that turns a sequence of scenes into a series a later frame can re-read — is precisely the Large Universe Model posture, whether or not anyone building it calls it that.
The claim that actually holds
Kuhn's asymmetry is an intake problem before it is an epistemic one, and the software case makes that unusually legible because the streams involved — commits, CI, incidents, advisories — are already machine-legible and already timestamped. The frame that decides "this dependency is fine for now" is not going to be dissolved by more data arriving faster. What continuous, retained, provenance-bearing intake does is narrower and more defensible: it keeps the residuals that the current triage frame calls noise, attached to enough origin information that a later frame — a new staff engineer, a postmortem, an auditor with a different severity model — can re-read them as a pattern rather than reconstruct them from memory or find they were never kept at all.
That does not shorten the argument a team has to make to justify an emergency patch, and it does not replace the institutional work of getting a team to take slow-burning risk seriously before the audit rather than after the incident. It removes one specific excuse: that the evidence was not there. Whether that evidence gets looked at in time remains, as it always was, a question of judgement, incentive, and how much a staff engineer is willing to argue against a triage board's settled sense of what counts as urgent this sprint.