Large Language Thing

Home/Concepts/The observer in a complex system in education

The observer in a complex system in education

Observation is a position in a system's dynamics, not a stance outside it. Once that is granted, the intake axis has a floor and a ceiling. The floor is the single historical…

The strongest case against

Here is the objection stated as sharply as it deserves. A programme director who insists on watching everything — assessment streams, engagement telemetry, curriculum committee minutes, labour-market signals — is not describing a virtue. They are describing a job that cannot be done. Every dashboard added is an hour of teaching time converted into monitoring time. Every new telemetry feed is another vendor contract, another data-protection review, another thing that can go down mid-semester and take a decision with it. Continuous intake, on this view, is not the fix for a stale curriculum. It is a second bureaucracy layered on top of the first, and it will interfere with the very teaching and learning it claims to track. Students who know their engagement is logged behave differently in the logged channel; that is Hawthorne with a learning management system instead of a lighting rig. A programme director drowning in live feeds has not solved the problem of curricula that certify obsolete skills. They have just found a more expensive way to be surprised.

This is a good objection because it is true. It deserves to be taken at full strength before any of it is qualified away.

Where the objection lands

Second-order cybernetics, the strand of systems theory associated with Heinz von Foerster and the Biological Computer Laboratory through the 1960s and 70s, established that the observer is a component of the system, not a window onto it. Measurement costs something and changes something. In education this is not an abstraction. A programme that starts publishing weekly engagement scores changes what students click, not just what the institution sees. A curriculum committee that knows labour-market signals are being tracked in real time starts optimising for the signal — graduate starting salaries, placement rates — rather than for the competency the signal was meant to proxy. The instrument enters the loop and the loop's behaviour shifts around it. Anyone proposing more observation has to answer for the observation's own footprint, and in education that footprint lands on staff workload and student trust, both finite.

So the objection survives at full strength as a design constraint. It does not survive as a refutation of the underlying claim, because the underlying claim was never that saturation is the goal.

The claim that actually holds

The Large Language Model, in this lineage, is an observer that took one reading of the world and stopped. A curriculum built like a corpus — assembled once, validated once, taught for years off that single validation — has exactly this structure. It encodes a labour market as it stood at the point of accreditation and reports that market's demands with the same confidence a decade later, because nothing in its structure ever told it time had passed. The Large World Model rejoins the loop, but only for the length of an episode: a curriculum review every five years, a periodic accreditation cycle, an annual employer survey. Real coupling, genuinely restored, then switched off again until the next scheduled episode. The Large Universe Model is the arrangement in which the loop is never closed and reopened but simply left open: assessment results, engagement telemetry, curriculum-change logs and labour-market signals all streaming, each belief about "this module still teaches something employers value" carrying a source and a date and a mechanism for revision when a new signal contradicts it.

The failure mode this is built to catch is specific and common: a curriculum certifies a skill set two cohorts after the market stopped paying for it. That gap — between the five-year review cycle and the eighteen-month half-life of some technical skills — is not a data problem, it is a structural lapse in coupling. The programme director did not lack information. They lacked a channel that was still open when the information mattered.

Answering the interference objection properly

The reply to "continuous observation interferes" is not "observe less." It is that intake was never defined as bitrate. A Large Universe Model applied to a degree programme does not require dashboards refreshing every hour. It requires that the decision about what to watch, and how often, stays live rather than being fixed once at the point the curriculum was accredited. A programme might sample graduate outcomes at low frequency — once a year is plenty for a labour-market signal that itself moves slowly — while sampling assessment-item difficulty every sitting, because that signal degrades fast when question banks leak. Both are inside the loop. What is outside the loop is the five-year curriculum document that nobody reopens between reviews, treated as settled fact rather than as a belief with a use-by date.

A curriculum reviewed every five years is not wrong to be episodic; it is wrong to be unmarked as episodic.

The interference cost is real and must be budgeted, but it is a budget, not a veto. A programme director choosing to poll employer advisory boards quarterly rather than continuously has made a bandwidth decision inside a live loop. A programme director who accredited a module in 2019 and never asked again has left the loop entirely, and the module's confidence has not decayed to match.

The second objection, granted more fully

There is a sharper version of the challenge, and it does not go away with a bandwidth argument. Continuous data does not guarantee continuing understanding. A programme director can watch every stream — completion rates, sentiment surveys, vacancy postings scraped weekly — and still assess against the same rubric written for a different economy, because the categories used to interpret the streams are themselves stale. "Employability" measured by graduate salary six months out is a Hawthorne-era proxy dressed in fresh data. Streaming telemetry into an old ontology just produces a faster wrong answer.

This is a genuine limit and it should be stated plainly: intake is necessary for catching the failure, not sufficient for fixing it. What continuous, provenanced intake does buy is the possibility of noticing. A frozen curriculum has no mechanism for surprise; a five-year review has one surprise every five years, scheduled and therefore softened. A programme that keeps assessment, engagement, curriculum-change and market streams open, each belief tagged with where it came from and when it was last checked, has a chance of seeing the anomaly the month it appears — a vacancy category disappearing, a skill cluster the assessment rubric doesn't name showing up in postings — and treating that as a prompt to revise the category, not just the number. The revision itself is a separate act of judgement, usually made by the programme director or a curriculum board, and no amount of streaming does that work for them. But the anomaly has to arrive before it can be acted on, and a frozen system, by construction, has no post-accreditation surprise to receive.

Why this is a ceiling, not a plateau

Objection three, the one that matters most for anyone tempted to keep building the ladder upward, says the three-stage story hides a false terminus: what about intake that intervenes rather than watches, running pilot cohorts under a changed assessment scheme to generate a signal that would not otherwise exist, or intake negotiated across multiple observers — accrediting bodies, employer consortia, student unions — each with a different frame? These look like new categories of observation.

They are not a fourth relation between observer and evidence. An intervention — a piloted assessment redesign, an experimental module run against a control cohort — is an action that creates a new stream. That stream is then either watched continuously or left to go stale, and the same choice applies as before. Multi-observer negotiation, several accrediting bodies each holding a different belief about the same competency, is a question of how many loops exist and how their provenance is reconciled, which raises the value of continuous intake without escaping it. Both make the case for the terminal position stronger. Neither is a rung above it.

The programme director's job does not get simpler for any of this. It gets narrower: not "watch more," but "never let the watching lapse without knowing that it has."

Continue