The Model Should Not Be the System of Record

Models are powerful because they compress experience. A changing reality is not only more experience to compress.

Reality has identity, history, disagreement, and time. These properties require an external system of record.

Training is not enough

Training gives models broad capabilities and useful prior knowledge. It is a good way to learn patterns, abstractions, procedures, and language.

Weights are a poor place to keep knowledge that must stay current, remain open to inspection, combine with other knowledge, and survive repeated change.

Updating a model is expensive. More importantly, a weight update does not naturally preserve when a claim was true, where it came from, what contradicts it, or which later update replaced it.

These properties matter little when a model writes a poem. They matter when an agent conducts research, advises a company, operates software, or makes decisions over time.

A more capable model can still reason from the wrong premises.

Storage is not reachability

Information can be present in a model without being reliably available to the task.

Recent work on continual factual learning measures the gap directly. A model was trained to write new facts into its weights, then asked questions that required combining two of them. When both facts lived in the weights, it answered correctly 32 percent of the time. When the same two facts were supplied in the prompt instead, it answered 91 percent of the time.¹

The bottleneck was not knowledge. It was reach. Asked simply to state a fact it had been trained on and could answer questions about, the model produced the correct content 34 percent of the time. Its failures inverted relations and blended facts together.

Interference compounds this. Where training was a plain restatement of each fact, 70 percent of wrong answers about an apparently forgotten fact contained the content of the most recently written one. Asked about the first fact it learned, the model answered with the twentieth. Where the same facts were trained with varied restatement, that confusion nearly vanished, at 1 percent.

Supplying the fact in context repairs much of this, but how much depends on how the fact was written. Under varied restatement, questions returned to 77 to 80 percent. Under plain restatement, to 14 percent.

The system needs more than storage. It needs stable addresses.

Context supplies an explicit address at the moment of reasoning. It places relevant knowledge into the active computation, where the model can use it, combine it, and compare it with its prior beliefs.

But context alone does not maintain the knowledge. Someone still has to decide what belongs there.

From context to Worlds

Keeping knowledge outside the weights is necessary, but it is not sufficient.

A pile of documents is not a representation of reality. Search can locate evidence. Retrieval can place passages into context. Neither one decides what is current or preserves the understanding built across earlier tasks.

A useful external system must:

• keep canonical knowledge outside the model
• preserve evidence and provenance
• represent change over time
• show disagreement instead of flattening it
• give entities, claims, and relationships stable addresses
• assemble the relevant state when reasoning begins

We call this system a World.

A World is a maintained representation of a real domain. It stays outside the model but remains available to the model's computation.

Models and Worlds learn differently

A model learns when its weights change. A World learns when its state changes.

These operations are not interchangeable.

A weight update can change the behavior of the whole model in ways that are hard to locate or reverse. A World update can preserve the new state, the state it replaced, the supporting evidence, and the unresolved disagreement.

Training compresses. World maintenance preserves structure.

A different architecture for continual intelligence

The dominant architecture puts intelligence and knowledge inside the same object. The model reasons because it contains knowledge. It contains knowledge because it was trained. When knowledge changes, the system tries to train again.

There is another architecture.

The model learns general reasoning.

The World holds the changing state of the domain.

Context brings the relevant parts of the World into the model's active computation.

Under this design, the system can keep learning without continually rewriting the reasoner. People, evidence, and operating processes maintain the World around it.

The field has treated continual learning as a problem of changing the model. It is also a problem of maintaining the world around it.

Models reason. Worlds orient.

Sources

1. Charles O'Neill, Can a Language Model Learn Facts Continually in Its Weights? Preprint, arXiv:2607.11020v2, July 2026. Figures are from the study and bare-statement training conditions as reported; the 91 percent figure is the both-facts-in-prompt reference condition, not a recovery. ↩