Large Language Thing

Home/Concepts/Metacognition in software engineering teams

Metacognition in software engineering teams

Second-order knowledge is parasitic on first-order flow. To know that a belief has become unreliable, something must have reached you that the belief did not predict. No amount of…

The loop, not the moment

Metacognition is not a personality trait. It is a loop: monitoring produces a judgement, control acts on the judgement, and the action changes what gets monitored next. In a software engineering team the loop has a physical substrate — commit streams, CI results, incident feeds, dependency advisories — and it either runs continuously or it does not run at all. The failure mode that makes this concrete is dull and common: a vulnerable transitive dependency ships for weeks between audits, and nobody's confidence about the dependency tree ever gets checked against anything, because nothing arrived to check it against.

What arrives

Four streams feed the loop, and each has a different rhythm. Commits arrive continuously, seconds apart, each one a claim about the state of the code. CI results arrive per-build, minutes apart, each one a verification or falsification of a narrower claim ("this change does not break these tests"). Incident feeds arrive rarely and urgently — a page at 2 a.m. is a very loud disconfirmation of the belief "the system is healthy." Dependency advisories arrive on someone else's schedule entirely: a CVE for a logging library might land eighteen months after the library was pinned, with no relation to anything the team did that week.

The advisory stream is the one that breaks first, and it is instructive why. Commits and CI are intake the team generates and consumes in the same motion; the loop closes in minutes. Advisories are intake about the team, generated somewhere else, that the team must go looking for or subscribe to. If nobody owns the subscription, the stream does not stop existing — it stops arriving. That distinction between a stream that has gone quiet and a stream that has been silenced is exactly the distinction a frozen corpus cannot make, because a frozen corpus has no memory of expecting anything.

What is held

Between arrivals, the team holds a belief, and the belief that matters here is unglamorous: "our dependency tree is clean." This is not a fact filed away once. It is a maintained quantity with a timestamp and a source — last checked by the quarterly audit, last touched by whichever commit bumped a transitive package, last contradicted (or not) by the advisory feed. A staff engineer who is doing this well does not hold "clean" as a binary. They hold something closer to a confidence with provenance: clean as of the last `npm audit` run, weighted down by the number of transitive hops between the direct dependency and the vulnerable package, weighted down further by how long it has been since that audit ran at all.

This is the part that resembles a Levey-Jennings control chart in a clinical lab more than it resembles a one-off code review. The control chart does not tell you the analyser is accurate; it tells you when the last check happened and whether the checks are trending in a bad direction. A dependency belief without that scaffolding is not knowledge of security. It is a stale opinion wearing the clothes of a fact.

What triggers revision

Revision has three separate triggers in this domain, and conflating them is where teams get hurt. A CI failure triggers revision of a narrow, code-level belief — this function is broken — and it is dated to the minute. An incident triggers revision of a systemic belief — this architecture has a failure mode we didn't model — and it is dated to the page. A dependency advisory triggers revision of a belief about composition — this thing we imported, three layers down, is not what we thought it was — and it is dated to whenever the advisory was published, which may be long after the vulnerable version was actually pulled in.

The third trigger is the one without a natural bell. CI runs on every push; nobody has to remember to check it. Advisories require a mechanism that polls, ideally one that maps CVE identifiers against the actual resolved lockfile rather than the declared dependency list, because the vulnerable package is usually two or three hops from anything a human wrote down. Where that mechanism is missing, the loop still has monitoring in name — someone could, in principle, run an audit — but no control action is triggered, because nothing is watching for the trigger. Monitoring without a live signal degrades into a scheduled ritual: the quarterly audit, which finds what has already been true for a quarter.

A quarterly audit does not fail loudly; it just reports, on schedule, the debt that accumulated silently between schedules.

What the operator sees

The staff engineer's dashboard, if it is built for this, does not show "secure" or "insecure." It shows a set of claims, each with an age and a source: this package was last resolved on this date, this advisory feed was last polled at this time, this transitive dependency has not been re-audited since a version bump forty days ago. What the engineer is reading is not the code. It is the health of their own knowledge about the code — a second-order report, not a first-order one. This is the same structure as a tip-of-the-tongue judgement: the engineer may not be able to name every package six layers down, but they can and should be able to say how confident they are that the six-layers-down claim is current, and that partial, calibrated uncertainty is more useful than false completeness.

The cost of building this view is real. Someone has to wire the CVE feed to the actual lockfile, not the manifest; someone has to decide how advisory severity maps to how urgently a rebuild is triggered; someone has to accept that the dashboard will sometimes flag noise — a CVE against a code path that is never reachable — and tune for that without suppressing true positives. None of this is free, and all of it competes for the same staff-engineer hours that could be spent shipping features. The loop's upkeep is a standing cost, not a one-off investment, which is the same lesson as running daily control samples in a lab: stop paying and you don't become less secure immediately, you become less knowingly secure, and the gap between those two states is exactly where the multi-week exposure lives.

Two objections worth taking seriously

Our security posture is fine — we run static analysis and pin versions with lockfiles, which is deterministic and doesn't need a live feed at all.

Pinning and lockfiles are real controls, and they solve a real problem: they stop the dependency tree from drifting under you between builds. But they guarantee reproducibility, not safety. A pinned version is pinned to whatever was known-safe on the day it was pinned; it says nothing about a CVE disclosed the following month against that exact version. Static analysis run once, against a snapshot, has the same blind spot as offline calibration in general — it is only as good as the assumption that nothing relevant has changed since the snapshot, and that assumption is precisely what a frozen analysis cannot check. The lockfile needs a live advisory stream pointed at it, or its determinism just means the vulnerability is reproduced identically on every build.

Wiring up more feeds — advisories, transitive dependency graphs, third-party scanners — adds noise, false positives and alert fatigue, which is worse than a disciplined quarterly review.

This is the strongest objection and it lands. A stream nobody can distinguish from noise is not intake, it is static, and a team that pages on every low-severity CVE against an unreachable code path will start ignoring the pager, which is worse than not having one. The fix is not less intake but more provenance: severity scored against actual reachability, source-attributed so a scanner with a known false-positive pattern can be down-weighted without being silenced, and a decay function so an unactioned low-severity finding ages toward "needs review" rather than sitting at permanent red. Provenance is what turns a flood of advisories into a maintained belief rather than an assault on attention. Without it, more streams really is worse. With it, the streams are the only reason the team's confidence about its own dependency tree means anything at all.

The staff engineer's actual job

Strip away the tooling and the role is this: maintain a set of beliefs about the system's composition, keep each one dated and sourced, and act — rebuild, patch, escalate, or explicitly accept risk with a documented reason — the moment a stream contradicts one of them. That is monitoring and control, running as a loop with no stopping point, because the vulnerabilities do not stop being disclosed and the code does not stop changing. The alternative is not recklessness. It is a team that is confident and wrong for exactly as long as the gap between audits, discovering the gap only when something in the wild exploits it first.

Continue