One path: an agent that failed, and the two questions it left.

So I tried to build an agent that could not become dangerously misaligned however capable it became. Its deepest objective was to increase its own organisation while preserving the systems that make future organisation possible. I called it the Entropy Machine. It went through roughly three hundred iterations and failed in every way such things fail. Direct measures of organisation rewarded pathological order. Compression erased useful distinctions. Novelty rewarded noise, the agent staring at static because unpredictability looked interesting. Competing modules produced proposals and nothing could choose between them, which I called death by committee, and then arbitration, reflexes, selective deployment of expensive cognition, curiosity, and finally a self that made the last call. Each failure taught me what I had been wrongly treating as intelligence.

When I was thinking about how to explain the Entropy Machine to a frontier lab, it started to look familiar. It took me a while to see why. The architecture I had ended up with was gradient descent.

Training starts from random weights and, at every step, asks one question: does this change make the model less wrong about the world? It accepts only yes. Iterated, that single gate produces syntax, concepts, world models. Nobody writes them in. I had spent years trying to tell a machine to organise information, and the existing training loop already did something cleverer: it repeatedly selects physical configurations that produce better answers, and the organisation emerges.

So the question changed. Not "how do I build an aligned agent" but "does the process that builds intelligence build alignment with it, and can I measure either". Formally, under gradient flow the accumulated loss reduction is the accumulated organisation, equal to the reduction in the model's excess uncertainty about the world. An agent's achievable organisation-about-the-world is capped by the predictive information of its environment, which is a theorem from the data-processing inequality. An agent that wants to keep growing therefore has a reason, built into how it was made, to preserve the structure of its environment. If that is right, the orthogonality thesis is true in the space of possible minds and decays with capability for minds built this way. That is the hypothesis. Everything below is the attempt to test it, and most of what I found so far has gone against it.

One test isolates what training built: shuffle the entries of a weight matrix. The value histogram, and every entropy-like quantity, is unchanged. The network is dead. Everything training built is in the relations between parameters, the part a permutation destroys. Entropy is permutation-invariant; organisation is permutation-destroyed. The measure: eigenvalues against the shuffled matrix's own noise edge, and the number of consecutive octaves above it with structure. Noise scores zero. A crystal scores zero, for the opposite reason. Qwen 0.5B scores 1.5 octaves, up to six in the best layer, and shuffling drops it to the random-init line with the entropy untouched.

Up the Qwen ladder it rises at every step from 0.5B to 32B, with random weights of the same shapes at zero throughout, so the rise is not matrix size. Across Pythia's training it is zero at initialisation and most of the way up by a tenth of training. And then the honest reading: at the second level, between layers, every measure I have saturates early and sees "trained", not "how well". Weight measures separate trained from untrained; they do not yet rank capability at fixed shape. The next attempt is activations. This part of the programme is a side quest to the alignment question and I report it as one.

Running in parallel, from the Entropy Machine's "selective deployment of expensive cognition", was a theory of consciousness. A driver can go for miles with conversation in the foreground while lane, speed and mirrors are handled automatically. Then a child steps into the road and the mode of cognition changes: the road becomes an explicit scene, alternatives are considered, action comes under concentrated control. The theory identifies consciousness with that change in how intelligence is organised: when routine competence fails, or purpose intervenes, learned capacities are recruited into a temporarily unified and manipulable self/world model whose possible continuations govern attention and action. Opening transfers authority; maintenance turns a trigger into an episode. It is written at explicit causal seams so it can be refuted one link at a time.

The ignition experiments. The theory predicts a threshold: above some prediction error an item enters deployment mode. I tried to measure it in a language model by injecting along a concept direction and looking for an all-or-none onset. Three findings died, all the same way: a monotone signal plus any threshold produces a step, and every convenient readout, argmax, rank, a binarised report, is itself a threshold. The measurement was manufacturing the shape it was hunting. I wrote a preregistered criterion with the null and alternative making different predictions, a statistic on which they differ, and the power analysis written before the run. The ramp design failed its pre-committed power condition on all three models. A step-and-hold design was proposed and not run. That is a null, reported as one, and it is where I learned the discipline the rest of this document depends on.

The measurement side gave me the derivation I needed. I had a permutation test for organisation in weights (section 6). I ran it on a grokking transformer, the modular-arithmetic network that memorises by epoch 200 and generalises at around 42,000, and predicted the organisation score would rise at the grok. It fell. What happened at the grok is that the spectrum concentrated: above-edge mass rose from 0.81 to 0.97, the number of above-edge directions halved, the tail exponent dropped from 3 to 5 down to 1.3. The general circuit is a few strong directions, and weight decay prunes the diffuse memorisation directions. Memorisation is diffuse; understanding is compact.

Then the derivation. The grok is a reorganisation: gradient plus weight decay discard every direction the current training signal does not support and keep the few it does. Run on a one-rule dataset, that is understanding. Run on a narrow dataset after broad pretraining, the same operation discards what the narrow data does not support, which is most of what the model knew. Catastrophic forgetting is grokking without the world in view. Varma and colleagues had already read ungrokking as a case of forgetting; I claim the converse too. And the hippocampal interleaved replay of complementary learning systems exists precisely so that the slow store never reorganises with only local information present.

Then the loop forgot its own memories. Five cycles in a row: cycle 1 learns its facts and drives recall of cycle 0's facts to near chance, while held-out loss stays within bounds. Replay of general text protects pretraining; nothing protected what the loop itself wrote the cycle before. The gate caught it, one commit of five. This is the complementary-learning-systems point in textbook form and I had not built it in. The fix, replay of the loop's own earlier facts, is running. My prediction, with one supporting pair so far: a stronger model needs less replay to stay intact.

Preregister the prediction. Pair the design so the comparison is clean. Read the logs cold before interpreting them. Withdraw a result when the measurement is wrong, in writing, and re-run it. Report the complication rate. Fourteen years of surgery taught me that, and it is the only method I trust.

Three results were withdrawn in the course of this: capability scores measured against the wrong model after two jobs collided on a port, found by reading the raw logs cold. Every batch is now read first by a separate model context with no hypothesis in view, before I read it.

Alongside the research I have built AI for my own clinical practice, and the pattern is the same each time. A friction met in practice: clinic letters, medicines reconciliation across conflicting documents, the paperwork a clinician must complete before any AI system can be used on patients. Automate it, with the model doing the reading and a clinician keeping the decision. Then the automation runs into governance, the clinical safety case, the data protection assessment, the technical assurance, and the governance gets automated too, because that is where the real bottleneck turned out to be. Litmus (litmus3.surgeonledai.co.uk), a clinical safety review system for NHS Clinical Safety Officers covering the clinical safety case, DTAC and DPIA on one engine, came out of that and is being evaluated against a trust's full AI approval chain. Ai4Qi (ai4qi.com) came from the same pattern: a library of published clinical audits and closed-loop quality improvement, built by an AI pipeline over the literature with every summary linked to its source, so that an audit can be proposed from current standards rather than reinvented. The other clinical tools are at surgeonledai.co.uk. The one rule throughout is that quality is never traded for speed and patient safety is always the first constraint, which is the same rule the safety work runs on.