Lit is not a framework in the way React or Angular is. It is a thin, roughly 5 KB layer that automates browser features you already have. Once that clicks, almost every confusing Lit behavior becomes obvious.

When I started with Lit, I treated it like “lightweight React.” I looked for a virtual DOM, expected synchronous renders, and fought the shadow DOM instead of using it. Most of my bugs came from that wrong mental model, not from Lit itself.

This article rebuilds Lit from the ground up. We start with the problem it solves, look at the browser primitives underneath, and then add Lit’s pieces one at a time. At the end is the list of gotchas I wish someone had handed me on day one.

1. The problem: components without a library

Browsers can already define their own HTML tags. The catch is that doing it by hand means writing all the plumbing yourself. Here is a simple counter with no library:

class MyCounter extends HTMLElement {
static observedAttributes = [‘count’];

constructor() {
super();
this.attachShadow({ mode: ‘open’ });
this._count = 0;
}

connectedCallback() {
this.render();
}

attributeChangedCallback(name, oldValue, newValue) {
if (name === ‘count’) {
this._count = Number(newValue);
this.render();
}
}

render() {
this.shadowRoot.innerHTML = `
<button>Clicked ${this._count} times</button>
`;
this.shadowRoot.querySelector(‘button’)
 .addEventListener(‘click’, () => {
this._count++;
this.render();
});
}
}

customElements.define(‘my-counter’, MyCounter);

It works, but look at what we had to do manually:

• Re-render everything on every change. innerHTML throws away the whole DOM, including focus, selection and input state.

• Re-attach event listeners after every render, because the old button no longer exists.

• Sync attributes and properties by hand. Setting el.count = 5 from JavaScript does nothing here.

• Avoid render storms. Change three values in a row and you render three times.

• Escape user input yourself, or innerHTML becomes an XSS hole.

Every one of these is solvable. Lit is simply the collection of those solutions, done efficiently.

2. The three browser primitives underneath

Lit sits on top of three standards that ship in every modern browser. Lit does not replace any of them.

Custom Elements let you register a class as a new tag with customElements.define(‘my-tag’, MyClass). The browser then calls lifecycle hooks on your class:

• constructor() when the element is created

• connectedCallback() when it is added to the document

• disconnectedCallback() when it is removed

• attributeChangedCallback() when an observed attribute changes

A LitElement is still an HTMLElement. These callbacks still fire, and you can still override them (just call super).

Shadow DOM gives an element a private DOM subtree. Styles inside it do not leak out, and page styles do not leak in. Light DOM children are projected into it through <slot> elements. This is where Lit renders by default.

HTML templates are inert chunks of DOM, parsed once and cloned cheaply. The browser can clone a <template> much faster than it can parse an HTML string. Lit leans on this heavily, as the next section shows.

The practical consequence: anything you know about these standards applies to Lit directly. When Lit docs feel thin on a topic, MDN usually has the answer.

3. How lit-html actually renders

This is the single most important idea in Lit: the static parts of a template are processed once, and only the dynamic values are touched on updates. There is no virtual DOM and no tree diffing.

It starts with a JavaScript feature called tagged template literals. When you write

html`<button class=${cls}>Clicked ${count} times</button>`

the html function does not receive one string. It receives two things separately:

• strings: [‘<button class=”’, ‘“>Clicked ‘, ‘ times</button>’]

• values: [cls, count]

The strings array is the same object every time that line of code runs. JavaScript guarantees it. Lit uses that array as a cache key.

First render. Lit joins the strings with marker comments, builds a real <template> element, and walks it once to record where each value goes. Each of those spots becomes a part: an attribute part for class, a child part for the text. Lit then clones the template into the DOM and fills in the values.

Every later render. Lit looks up the cached template by its strings array, skips parsing and cloning, and walks the list of parts. For each part, it compares the new value with the last one. If it is identical, nothing happens. If it changed, Lit updates exactly that attribute or text node.

lit-html render path · first render vs every later render

The expensive work (building and cloning the template) happens once; every later render is just a comparison per value.

Compare the cost:

Approach

Work on each update

innerHTML

Parse the whole string, rebuild every node

Virtual DOM

Build a new tree in memory, diff it against the old tree, patch

lit-html

Compare N values, touch only the ones that changed

This is also why Lit templates are safe by default. Values are set as text nodes or properties, never parsed as HTML, so ${userInput} cannot inject markup.

4. Reactive properties and the update cycle

Here is the same counter in Lit:

import { LitElement, html, css } from ‘lit’;
import { customElement, property } from ‘lit/decorators.js’;

@customElement(‘my-counter’)
export class MyCounter extends LitElement {
@property({ type: Number }) count = 0;

render() {
return html`
<button @click=${() => this.count++}>
Clicked ${this.count} times
</button>
`;
}
}

Every problem from section 1 is gone. The interesting question is how this.count++ turns into a DOM update.

Properties become accessors. @property replaces the class field with a getter/setter pair. The setter stores the value, checks whether it actually changed, and calls requestUpdate().

Updates are asynchronous and batched. requestUpdate() does not render. It marks the element dirty and schedules one update in a microtask. Set ten properties in the same tick and you still get one render.

@property vs @state. Both are reactive. @property is public API: it can be set from outside and can map to an attribute. @state is private internal state with no attribute. Use @state for anything a consumer should not set.

When the microtask runs, Lit walks this lifecycle in order:

1. shouldUpdate(changed): return false to skip this update.

2. willUpdate(changed): compute derived values before rendering. Setting properties here does not trigger another update.

3. update(changed): reflects properties to attributes, then calls render().

4. render(): returns the template. Keep it pure, with no side effects.

5. firstUpdated(changed): runs once, after the first render. Good for querying the shadow DOM.

6. updated(changed): runs after every render. Setting properties here schedules another update, so be careful.

7. updateComplete: a promise that resolves when the cycle finishes.

The changed argument is a Map of property names to their previous values. It is how you react only to specific changes:

willUpdate(changed: PropertyValues<this>) {
if (changed.has(‘items’)) {
this.sorted = […this.items].sort();
}
}

5. Attributes vs properties

This distinction causes more Lit confusion than anything else, and it comes straight from the DOM.

• Attributes live in HTML. They are always strings: <my-counter count=”5">.

• Properties live on the JavaScript object. They can be anything: el.count = 5, el.items = […].

By default, @property observes an attribute with the lowercased property name, and converts the string using type. So count=”5" becomes the number 5. A few options control this:

@property({ type: Boolean, reflect: true }) open = false;
@property({ attribute: ‘max-items’ }) maxItems = 10;
@property({ attribute: false }) data: Item[] = [];

• reflect: true copies the property back to the attribute, which lets you style with :host([open]).

• attribute: ‘max-items’ uses a dash-case attribute name, since maxItems becomes maxitems by default.

• attribute: false skips the attribute entirely. Use it for objects and arrays, which should never round-trip through a string.

Inside templates, the binding prefix decides which one you set:

Syntax

Sets

Example

name=${v}

Attribute (as a string)

<img src=${url}>

.name=${v}

Property (any value)

<my-list .items=${items}>

?name=${v}

Boolean attribute (added or removed)

<button ?disabled=${busy}>

@name=${fn}

Event listener

<button @click=${this.save}>

The rule of thumb: pass primitives as attributes when they come from HTML, and pass everything else with a . property binding. Passing an array without the dot gives the child the string “[object Object]”.

6. Styling in the shadow DOM

Lit components declare styles once per class, not per instance:

static styles = css`
 :host {
display: block;
padding: var( — card-padding, 16px);
}
 :host([open]) .body { display: block; }
 .body { display: none; }
`;

Under the hood, Lit turns this into a constructable stylesheet and shares it across every instance via adoptedStyleSheets. A thousand cards share one parsed stylesheet. That is why styles should be static: interpolating per-instance values into css is blocked on purpose.

Because the shadow root is encapsulated, your global CSS (Tailwind, Bootstrap, a reset) does not reach inside. That is the feature, not a bug. To let consumers customize a component, use the escape hatches the platform provides:

• CSS custom properties inherit through shadow boundaries. Expose — card-padding and let the page set it. This is the main theming mechanism.

• ::part() lets you expose a specific internal element: <div part=”header”> can be styled from outside with my-card::part(header).

• ::slotted() styles light DOM children projected into a slot, but only the top-level slotted nodes.

• :host and :host(…) style the element itself, and the page can still override them since outer styles win over :host.

For dynamic per-instance styling, use the classMap and styleMap directives in the template instead of building CSS strings.

7. Composition: slots, events, controllers, directives

Data down, events up. Parents pass data to children through properties. Children report back by dispatching DOM events. There is no special Lit mechanism; it is just the DOM.

component communication · properties down, events up

The child never reaches up to change the parent’s state; it announces what happened and lets the parent decide.

// child
this.dispatchEvent(new CustomEvent(‘item-selected’, {
detail: { id: this.itemId },
bubbles: true,
composed: true,
}));

// parent template
html`<item-row .itemId=${id} @item-selected=${this.onSelect}></item-row>`

Slots vs properties. Use a <slot> when the consumer should supply markup (a card body, a button label, an icon). Use a property when the component needs data it will render itself. Named slots (<slot name=”footer”>) handle multiple insertion points.

Reactive controllers are Lit’s answer to hooks or mixins. A controller is a plain object that attaches to a host, receives its lifecycle callbacks, and can call host.requestUpdate():

class ClockController implements ReactiveController {
value = new Date();
private timer?: number;

constructor(private host: ReactiveControllerHost) {
host.addController(this);
}
hostConnected() {
this.timer = setInterval(() => {
this.value = new Date();
this.host.requestUpdate();
}, 1000);
}
hostDisconnected() {
clearInterval(this.timer);
}
}

Any component can now write clock = new ClockController(this). Setup and cleanup stay together, which is the main thing hooks gave React.

Directives customize how a single binding renders. The built-ins worth knowing first:

• repeat(items, keyFn, template): keyed lists that move DOM nodes instead of rewriting them.

• classMap and styleMap: toggle classes and inline styles from objects.

• ifDefined: omit an attribute when the value is undefined.

• until: render a placeholder until a promise resolves.

• ref: get a handle to a rendered element.

• live: compare against the live DOM value, useful for inputs the user can edit.

• guard and cache: skip or keep expensive sub-templates.

8. Gotchas I learned the hard way

Mutating arrays and objects does nothing. Lit compares values with !==. Pushing into an array keeps the same reference, so no update is scheduled.

this.items.push(newItem); // no re-render
this.items = […this.items, newItem]; // re-renders

Treat reactive properties as immutable, or call this.requestUpdate() after mutating.

The DOM is not updated right after you set a property. Updates run in a microtask. If your next line measures or queries the DOM, it sees the old state.

this.open = true;
await this.updateComplete;
this.renderRoot.querySelector(‘input’)?.focus();

Events stop at the shadow boundary. A CustomEvent without composed: true never leaves the shadow root, so the page never hears it. Native events like click are composed already, but they get retargeted: outside the component, event.target is the host element, not the inner button. Use event.composedPath() when you need the original target.

Forms do not see your inputs. An <input> inside a shadow root is invisible to the surrounding <form>. To participate in forms, make the element form-associated with static formAssociated = true and this.attachInternals(), then call internals.setFormValue().

IDs do not cross shadow roots. <label for=”x”> outside the component cannot point at id=”x” inside it, and aria-labelledby has the same problem. Keep labels and their targets in the same root, or accept a label through a property or slot.

Class fields can shadow reactive properties. With TypeScript useDefineForClassFields: true (the default for modern targets), a plain class field overwrites Lit’s accessor. Use the declare keyword or accessor with standard decorators, and check the Lit docs for your compiler setup.

Setting properties in updated() causes a second render. It works, but it is a render loop waiting to happen. Compute derived state in willUpdate() instead.

Registering the same tag twice throws. If two bundles both include a component, the second customElements.define fails. This is common in micro-frontends. Guard with customElements.get() or make sure one copy owns the definition.

SSR is a separate concern. @lit-labs/ssr renders declarative shadow DOM on the server, but it does not run browser-only code like connectedCallback the same way. Plan for hydration from the start rather than bolting it on.

9. When Lit is (and isn’t) the right tool

Lit’s strength follows directly from what it is: a standard custom element works anywhere HTML works.

Lit shines for:

• Design systems shared across teams using React, Angular, Vue or plain HTML. You write the button once.

• Embeddable widgets dropped into pages you do not control, where style isolation matters.

• Long-lived code. Web Components are a browser standard, so they do not get rewritten when a framework’s major version changes.

• Performance-sensitive UI where bundle size and update cost matter.

Think twice for:

• Large, state-heavy SPAs. Lit has no built-in router, global store or data-fetching layer. You can add them (Lit’s own @lit/task, @lit/context, signals), but you are assembling the stack yourself.

• Teams deep in a framework ecosystem that rely on its component libraries and tooling.

• Projects needing mature SSR today. It exists, but it is less battle-tested than Next.js or Nuxt.

A common and healthy pattern is both: Lit for the shared component library, and an app framework on top for routing and state.

10. The mental model in one picture

Read Lit as a stack of thin layers, each automating the one below it:

1. The browser provides Custom Elements, Shadow DOM and <template>.

2. lit-html turns tagged template literals into cached templates with parts, and updates only the values that changed.

3. ReactiveElement turns properties into accessors, batches changes into one async update, and runs the lifecycle.

4. LitElement connects the two: render() output goes into the shadow root, and static styles become shared stylesheets.

5. Your component declares properties, returns a template, and communicates through properties down and events up.

Lit’s layers · from the browser up to your component

LitElement is simply ReactiveElement and lit-html wired together on top of the platform.

When something behaves strangely, ask which layer it belongs to. An event that never arrives is a Shadow DOM question. A stale value is a reactivity question. A slow list is a lit-html question. Nine times out of ten, the answer is a browser standard doing exactly what it was designed to do.

That is the real lesson of Lit: learn the platform, and the library mostly explains itself.

Lit from First Principles: Everything I Wish I Knew Before Using It was originally published in Tutorial Savvy on Medium, where people are continuing the conversation by highlighting and responding to this story.

Your next most important user will never see your homepage.

It is 9:14 on a Tuesday morning. Someone types one sentence into an AI assistant: “Book me a flight to Austin next Thursday, aisle seat, under $400, and add it to my calendar.”

Thirty seconds later it is done. The user never opened an airline app. They never saw the carousel your design team spent a quarter perfecting. They never hovered over the button your growth team A/B tested eleven times.

An agent did all of it. It read your docs, called your API, parsed your error messages, retried when your rate limiter pushed back, and moved on. If any of that had been confusing, it would have quietly picked a competitor instead.

That agent is your customer now. And how it experiences your product has a name: AX, or Agent Experience.

The new chain of use: person → agent → product

What is AX?

Agent Experience is the holistic experience an AI agent has when it uses your product, platform, or service on behalf of a human.

The phrase gained traction in early 2025, when Netlify co-founder Mathias Biilmann argued that companies needed to design deliberately for agents the way they once designed for users and developers. The idea spread fast because it named something many teams were already feeling: a growing share of their traffic, signups, and API calls were not coming from people at all.

If UX asks “Can a person understand and enjoy this?” and DX asks “Can a developer build with this?”, AX asks a third question:

Can an autonomous agent discover, understand, and reliably act on this, without a human holding its hand?

UX, DX and AX: three users, three questions

Why now?

For twenty years, software had two kinds of users: people who clicked, and developers who coded. Agents are a third kind, and they arrived quickly.

Coding agents now open pull requests, run tests, and deploy to cloud platforms. Browsing agents fill in forms and compare prices. Workplace assistants read your tickets, draft your emails, and update your CRM. Protocols like the Model Context Protocol (MCP) have made it routine to plug an agent straight into a SaaS product.

This changes the funnel. When a developer asks an assistant to “set up auth for my app,” the agent effectively chooses the vendor. It picks the one whose docs it can parse, whose SDK it already knows, and whose signup does not stall on a CAPTCHA. Products that are hard for agents to use are becoming invisible, the same way sites that ignored mobile became invisible a decade ago.

The shift is subtle but real: you are no longer only designing for the person. You are designing for the delegate the person trusts.

Six principles of great AX

1. Make it discoverable. Agents cannot use what they cannot find. Publish clean, crawlable docs, an llms.txt file, OpenAPI specs, and an MCP server. If your best capabilities live behind a JavaScript-rendered dashboard, they effectively do not exist.

2. Make it legible. Write for a reader that takes you literally. Use consistent names, explicit units, and examples for every endpoint. Clever marketing copy confuses agents; precise descriptions guide them.

3. Make errors teach. “Error 400” is a dead end. “start_date must be ISO 8601, for example 2026–10–15” is a path forward. Every error message is now a prompt, so write it like one.

4. Make actions safe. Agents make mistakes. Give them dry-run modes, idempotency keys, undo, and clear confirmation steps for anything destructive or expensive. Good AX makes the right action easy and the catastrophic one hard.

5. Make identity and permission clear. An agent acts on someone’s behalf, so it needs scoped, revocable credentials, not a shared password. OAuth flows, fine-grained scopes, and audit logs let humans delegate without losing control.

6. Make it predictable. Stable schemas, versioned APIs, and deterministic behavior matter more than ever. An agent that worked yesterday should work today. Silent changes break thousands of automated workflows at once.

The six principles of AX at a glance

What to do on Monday morning

You do not need a new department to start. You need to watch an agent try to use your product, the same way you once watched users in a usability lab.

• ☒ Ask three different AI assistants to complete your product’s core task from scratch. Record where each one gets stuck.

• ☒ Audit your docs for content that only renders with JavaScript, lives in images, or hides behind a login.

• ☒ Publish an OpenAPI spec and an llms.txt file, and consider shipping an MCP server for your top workflows.

• ☒ Rewrite your 20 most common error messages so each one says what went wrong and how to fix it.

• ☒ Find every step that requires a human (CAPTCHAs, email verification, manual approval) and decide which ones truly need to stay.

• ☒ Add agent traffic to your analytics so you can see how much of your usage is already delegated.

• ☒ Name an owner. AX falls between product, design, and platform teams, so without one it belongs to no one.

But wait, does UX still matter?

Yes. “AX is the new UX” is a provocation, not an obituary.

Humans still decide what to delegate. They still review what the agent did, approve the purchase, and judge whether the result was any good. They still need to trust your brand enough to let a machine act in their name. And plenty of moments, like choosing a wedding venue or reading a story, are ones people want to experience themselves.

The better framing is this: AX is now part of UX. The human experience increasingly runs through an agent. If the agent struggles with your product, the human feels it as delay, error, or a vague sense that your product “didn’t work.” Good AX is how you deliver good UX in a delegated world.

AX lives inside UX, not beside it

The work also overlaps more than it seems. Clear language, honest errors, safe defaults, and predictable behavior have always been good design. Agents just stopped letting us get away without them.

The interface is becoming a conversation

Every major platform shift created a new discipline. The web gave us information architecture. Mobile gave us touch-first design. APIs gave us developer experience.

Agents are giving us AX.

The companies that win the next decade will not just have beautiful screens. They will have products an agent can understand on the first try, act on safely, and recommend with confidence. Their users may never see the interface at all, and they will still have a great experience.

So the next time your team reviews a new feature, add one question to the checklist: “How would an agent use this?”

If nobody knows the answer, you have found your next design problem.

If this resonated, give it a clap and share how your team is thinking about agent experience in the comments.

AX Is the New UX was originally published in Tutorial Savvy on Medium, where people are continuing the conversation by highlighting and responding to this story.

Models don’t have that. If you want one that’s better at chemistry, you fine-tune the whole thing or train a new one. Two labs can’t each build a piece and snap them together, because every layer inside a trained model speaks a private dialect. It only makes sense next to the exact weights it grew up with.

The one test that matters

Two people train two modules separately. They never see each other’s code or data. Then we plug both modules into the same model, and it does nearly as well as a model where everything was trained together.

That’s it. If that works on a toy problem, I’ll believe there’s something here. If it doesn’t, at least it fails fast and cheap.

I like this test because of what it leaves out. The modules don’t need to be small or efficient. There’s no new training algorithm involved. It’s just asking whether two strangers can build parts that understand each other, because if they can’t, nothing else I’d want to build on top of this matters.

A whiteboard everyone shares

The first piece is a shared workspace: a fixed set of slots, say 64 vectors, that every module reads from and writes to.

The picture in my head is a whiteboard in a meeting room. Nobody whispers to anyone else. If you have something to say, you write it on the board, and if you want to know what’s going on, you read the board. Modules work the same way. They never call each other directly.

The board is the socket. Its shape doesn’t change: same number of slots, same width, same spot in the forward pass. As long as a module fits that shape, it can plug in.

What a module promises

The second piece is a contract every module signs. It reads the workspace through attention, does some work, and writes results back.

The middle part is nobody’s business. A module could be a transformer, a CNN, a lookup table, or a hand-written symbolic solver. I only care about the read and the write. If you’ve written a function against someone else’s API, it’s the same deal: agree on what goes in and what comes out, and do whatever you want in between.

The whiteboard and the contract are the easy bits. This next one is where I think the whole idea lives or dies.

Agreeing on the shape of the board doesn’t mean agreeing on what’s written on it. Say two modules were trained apart, and both learned to use slot 12. One uses it to mean “the subject of the sentence.” The other uses it to mean “something red.” Plug them in together and they’ll happily read each other’s notes and get everything wrong. Same alphabet, different language.

So something has to pin down what each slot means. The simplest version I can think of is a frozen reference model, an anchor. Every module trains against it: it learns from workspace states the anchor produced, and whatever it writes has to stay readable by the anchor. Since the anchor never changes, a module built this week and one built next year should still share a dictionary.

What I genuinely don’t know is whether that’s enough. A frozen anchor might also be a ceiling. If the anchor has no way to represent some concept, can a new module still get that idea across? Or does every module end up limited to the vocabulary of whatever we froze on day one? If I had to pick one question to spend a year on, it’d be this one.

Who gets called

The last piece is a router that decides which modules run for a given input. Routers aren’t new. The awkward requirement here is that this one has to accept modules that didn’t exist when it was trained. If you need to retrain the router every time someone ships a module, you’ve lost most of the point.

One way I can imagine it working: each module publishes a small key, a vector living in the same space as the workspace. The router compares the current state of the board against those keys and picks the closest matches. Joining means registering a key. Nobody retrains anything.

How I’d test it

Putting it together, the first experiment looks something like this:

Two teams, separate data, separate code. The only thing they share is the frozen anchor. Plug both modules in, then compare against a baseline with the same parts trained together. A small gap means the idea works. A big gap tells us where it breaks, which is useful too.

Hasn’t someone done this?

Sort of, and I’d be surprised if nobody had come close. The nearest work I know of:

• “Coordination Among Neural Modules Through a Shared Global Workspace” (Goyal et al., 2021). This is almost exactly the whiteboard idea. But the modules are trained together.
• Recurrent Independent Mechanisms (Goyal et al., 2019). Separate modules that compete to update a shared state. Also trained jointly.
• Perceiver (Jaegle et al., 2021). Reads and writes through a fixed latent array, which is very close to how I picture the workspace.

There’s also a nearby line of work on model stitching and on aligning representations between separately trained networks, plus methods that train expert models in parallel and merge them afterward. Those feel relevant to the anchor question in particular.

As far as I can tell, none of them tests the specific thing I care about: modules trained independently, by people who never coordinated, plugged in after the fact. That looks like the gap. I’m going from memory on these citations, though, so if you know of work that already does this, please tell me. I’d honestly rather find out now.

The stuff I’m deliberately ignoring

There are two bigger ideas sitting behind this one, and I’m parking both on purpose.

The first is training without backpropagation. Backprop needs the whole network held in memory while gradients flow backward, which makes training expensive and pretty much rules out a device learning on the fly. The dream is a network where each layer learns from local signals only, so layers can train in parallel and out of sync, a bit like the brain seems to. Hinton’s Forward-Forward algorithm and predictive coding networks are early steps, but as far as I know nothing matches backprop at scale yet. Whoever cracks that changes what hardware we build.

The second is an asynchronous framework where modules update on their own schedules

What If Neural Networks Came in Parts? was originally published in Tutorial Savvy on Medium, where people are continuing the conversation by highlighting and responding to this story.

Persistent memory made AI coding agents useful across sessions. Without maintenance, that memory slowly turns against you. Here’s how Claude’s reported Auto Dream feature works, how it compares to other approaches, and what it teaches us about designing agent memory.

TL;DR

• The problem: AI agents that write their own notes accumulate contradictions, stale facts, and meaningless time references, so memory quality degrades with every session.
• The fix: Auto Dream is a periodic background pass in which Claude reviews, merges, prunes, and re-indexes its memory files, much like the brain consolidating memories during sleep.
• How it works (reportedly): A four-phase cycle (Orient → Gather Signal → Consolidate → Prune & Index), triggered when 24+ hours and 5+ sessions have passed since the last run.
• Safety: It writes only to memory files, never your code, and runs in the background with a lock against concurrent runs.
• The bigger lesson: Agent memory needs separate collection and maintenance layers, the same way databases need writes and compaction.

1. The Problem: Memory That Only Grows

Claude Code’s auto memory lets the agent write notes about your project as it works: build quirks, debugging insights, architecture decisions, and your preferences. Early on, this feels like a superpower.

But memory that only grows has a predictable lifecycle

Usefulness
   ^
   |        .------.
   |      .-'      '--.                <- Without consolidation:
   |    .-'           '---.               useful early, then noise
   |   .'                 '----.          overwhelms signal
   |  .'                       '----------
   | .'
   |.'     .--.   .--.   .--.   .--.   .--.
   |'   ---'  '---'  '---'  '---'  '---'  '-- <- With consolidation:
   |                                          quality resets each cycle
   +------------------------------------------> Sessions
     1     5     10    15    20    25    30

After roughly 20 or more sessions, memory files typically develop four kinds of rot:

FAILURE MODE    EXAMPLE                          WHY IT HURTS
--------------  -------------------------------  ------------------------
Contradiction   "API uses Express" AND           Agent picks the wrong
                "API uses Fastify"               one, or hedges

Staleness       Fix notes for auth-legacy.ts,    Wastes tokens, suggests
                which was deleted                dead paths

Temporal rot    "Yesterday we decided to         "Yesterday" means
                use Redis"                       nothing a week later

Duplication     Same build quirk recorded        Bloats the index past
                in three sessions                what loads at startup

The underlying insight is one every engineer building agentic systems eventually learns: the hard part of memory isn’t storage, it’s curation.

2. Why “Dreaming”? The Neuroscience Parallel

The name is a deliberate analogy, and it maps surprisingly well.

HUMAN BRAIN                        CLAUDE CODE
---------------------------------  ---------------------------------
Waking experience -> short-term    Each session writes new notes
Sleep replays the day's events     Dream scans recent sessions
Important connections strengthen   Relevant facts merged into files
Irrelevant details fade            Stale/contradicted notes deleted
Filed into long-term storage       MEMORY.md rebuilt as clean index
Sleep deprivation hurts memory     No consolidation = memory decay

Before Auto Dream, Claude Code was effectively sleep-deprived: it kept taking notes and never processed them.

3. How Auto Dream Works: The Four Phases

+--------------------------------+
              |  TRIGGER: 24h + 5 sessions     |
              |  (or manual: "dream")          |
              +---------------+----------------+
                              |
                              v
+------------------------------------------------------------+
| PHASE 1: ORIENT                                            |
| ls memory dir -> read MEMORY.md -> skim topic files        |
| "What do I already know, and how is it organized?"         |
+-----------------------------+------------------------------+
                              v
+------------------------------------------------------------+
| PHASE 2: GATHER SIGNAL                                     |
| Daily logs -> drifted facts -> narrow grep of transcripts  |
| Targets: corrections, "remember this", themes, decisions   |
+-----------------------------+------------------------------+
                              v
+------------------------------------------------------------+
| PHASE 3: CONSOLIDATE                                       |
| Merge into existing files, relative -> absolute dates      |
| Delete contradicted facts, drop stale notes, dedupe        |
+-----------------------------+------------------------------+
                              v
+------------------------------------------------------------+
| PHASE 4: PRUNE & INDEX                                     |
| Keep MEMORY.md under ~200 lines, fix dead links            |
| Move verbose entries to topic files, resolve conflicts     |
+-----------------------------+------------------------------+
                              v
              +--------------------------------+
              |  Summary: what was merged,     |
              |  updated, or pruned            |
              +--------------------------------+

Phase 1: Orient

Claude lists the memory directory, reads the MEMORY.md index, and skims existing topic files. The goal is to improve what exists rather than create near-duplicates, which is the most common failure of naive memory systems.

Phase 2: Gather Signal

This is where the design gets clever. Claude stores every session as a local JSONL transcript, and active projects can have hundreds of them. Reading them all would be slow and token-expensive, so Auto Dream reportedly searches narrowly, in a rough priority order:

PRIORITY  SOURCE                  WHAT IT LOOKS FOR
--------  ----------------------  -----------------------------------
1         Daily logs (if any)     Append-only stream of recent work
2         Existing memories       Facts that now contradict the code
3         Transcript search       Targeted grep for specific terms

The high-value signals it targets:

• User corrections, the moments you told Claude it was wrong
• Explicit saves, such as “remember this” or “save to memory”
• Recurring themes, patterns that show up across multiple sessions
• Decisions, including architecture, tooling, and workflow changes

The instruction is roughly “look only for things you already suspect matter.” That’s a retrieval strategy, not a reading strategy, and it’s a big reason the process scales.

Phase 3: Consolidate

The core cleanup step performs four operations:

OPERATION        BEFORE                        AFTER
---------------  ----------------------------  -----------------------
Absolute dates   "Yesterday we chose Redis"    "On 2026-03-15 we
                                               chose Redis"

Contradictions   "API uses Express" +          "API uses Fastify"
                 "Migrated to Fastify"

Staleness        Debug notes for deleted file  (removed)

Merging          3 entries, same build quirk   1 clean entry

Notably, contradictions are fixed at the source rather than flagged for a human. The wrong file gets corrected directly.

Phase 4: Prune & Index

MEMORY.md is the file loaded at startup, and reportedly only about the first 200 lines make it in. Everything past that line is effectively invisible to the next session, so this phase:

• Removes pointers to deleted or superseded topic files
• Adds pointers to newly important ones
• Moves verbose detail out of the index and into topic files
• Resolves disagreements between the index and the files it points to

Files that don’t need changes are left alone. The process is surgical, not a full rewrite.

4. When It Runs: The Dual-Gate Trigger

A dream cycle reportedly runs only when both conditions are true:

+--------------------------+
            | 24h+ since last dream?   |
            +------------+-------------+
                 no <----+----> yes
                 |              |
                 v              v
             +------+   +--------------------------+
             | wait |   | 5+ sessions since last?  |
             +------+   +------------+-------------+
                             no <----+----> yes
                             |              |
                             v              v
                         +------+      +---------+
                         | wait |      |  DREAM  |
                         +------+      +---------+

Here’s how that plays out across realistic usage patterns:

SCENARIO                         TIME   SESSIONS  DREAMS?  WHY
-------------------------------  -----  --------  -------  ------------
One long session over 2 days     48h    1         NO       Too few
                                                           sessions
10 quick sessions, 1 afternoon   3h     10        NO       Too little
                                                           time
Daily work, ~2 sessions a day    72h    6         YES      Both gates
Side project touched weekly      7d     2         NO       Too few
                                                           sessions
Heavy sprint week                5d     25        YES      Both gates

The dual gate avoids wasted runs on lightly used projects while making sure active projects get regular cleanup.

5. By the Numbers

The write-up reports one consolidation covering 913 sessions in roughly 8–9 minutes. That works out to about 100 sessions per minute, which is only feasible because Phase 2 greps narrowly instead of reading transcripts in full.

It also reports an index that shrank from 280 lines to 142, bringing it back under the roughly 200-line startup limit:

MEMORY.md size vs. startup load limit

Before |######################################## 280 lines
Limit  |############################|            200 lines (loads)
After  |####################                      142 lines
       +------------------------------------------>
        0         100        200        300

Before: ~80 lines (29%) never reached the model at startup
After:  everything loads, with ~58 lines of headroom

That 29% figure is the hidden cost of unmaintained memory: knowledge the agent technically has but never sees.

6. Before and After

BEFORE (30+ sessions, no consolidation)
~/.claude/projects/<project>/memory/
|-- MEMORY.md            [!] 280 lines, over the load limit
|-- debugging.md         [!] 3 contradictory API error entries
|-- api-conventions.md   [!] still says Express (migrated 2 wks ago)
|-- random-notes.md      [!] mix of stale and current info
|-- build-commands.md    [!] "yesterday" x6, no dates
`-- user-preferences.md  [!] duplicates MEMORY.md entries

AFTER (one dream cycle)
~/.claude/projects/<project>/memory/
|-- MEMORY.md            [ok] 142 lines, clean index
|-- debugging.md         [ok] deduplicated, current fixes only
|-- api-conventions.md   [ok] reflects Fastify migration
|-- build-commands.md    [ok] absolute dates, no duplicates
`-- user-preferences.md  [ok] merged with relevant index entries

random-notes.md -> dissolved (merged into topic files or pruned)

7. Safety: What a Dream Can and Can’t Touch

• Will it change my code? No. Writes are limited to the memory directory.
• What if two sessions are open? A lock file allows only one dream at a time.
• Will it block my work? No. It runs in a separate background process.
• Can I undo its choices? Yes. Memory files are plain Markdown you can edit.
• Does it override my rules? No. CLAUDE.md remains the human-owned source of truth.

8. Auto Memory vs. Auto Dream

These aren’t competing features. They’re two layers of one system.

ASPECT            AUTO MEMORY                 AUTO DREAM
----------------  --------------------------  ---------------------------
Role              Collection                  Maintenance
Runs              Continuously, in sessions   Periodically (24h + 5)
Output            New notes                   Merged, pruned, re-indexed
Reads transcripts No (writes as it goes)      Yes, narrowly via grep
Trigger           Automatic                   Automatic or manual
If skipped        Lose useful learnings       Memory quality decays
Analogy           Taking notes in class       Rewriting notes for exam

9. The Full Memory Stack in Claude Code

With Auto Dream, Claude Code reportedly has four memory mechanisms:

CLAUDE.md     AUTO MEMORY   SESSION MEM.   AUTO DREAM
-------------  ------------  ------------  -------------  ------------
Written by     You           Claude        Claude         Claude
Purpose        Rules         Learnings     Continuity     Consolidation
Cadence        Manual        Every session ~Every 5K tok  24h + 5 sess.
Scope          Project/      Project       Session        Project
               global
At startup     Full file     First ~200    Relevant past  N/A (runs
                             index lines   sessions       between)
Authority      Highest       Learned       Contextual     Maintenance
Analogy        Manual        Day notes     Short-term     REM sleep

Authority / priority
                      ^
  CLAUDE.md           |  ############  You own it. Always wins.
                      |
  Auto Memory         |  ########      What Claude learned.
                      |      ^
  Auto Dream          |      | maintains -- keeps Auto Memory accurate
                      |
  Session Memory      |  #####         What happened recently.
                      +---------------------------------------->

10. How It Compares to Other Agent Memory Strategies

Auto Dream isn’t the only way to manage long-running agent memory. Here’s how the common approaches stack up:

STRATEGY             HOW IT WORKS           STRENGTH          WEAKNESS
-------------------  ---------------------  ----------------  ----------------
Append-only notes    Keep adding entries    Simple, no loss   Unbounded bloat,
                                                              contradictions

Rolling summaries    Summarize and replace  Bounded size      Lossy; details
                     old context                              vanish silently

Vector retrieval     Embed notes, fetch     Scales to huge    Returns stale and
(RAG)                by similarity          stores            current alike

Human-curated rules  Engineer maintains     Accurate,         Misses what the
                     a rules file           authoritative     agent learns

Periodic             Review, merge, prune   Bounded AND       Token cost per
consolidation        on a schedule          current           cycle; can err
(Auto Dream)

The key distinction is that most approaches handle size, but few handle truth. Retrieval systems will happily surface an outdated fact if it’s semantically similar to your query. Consolidation actively deletes facts that the codebase has disproven.

11. An Engineer’s Lens: This Is Compaction

If you’ve worked with storage systems, Auto Dream will look familiar. It’s compaction for agent memory.

SYSTEM          WRITE PATH          MAINTENANCE PATH     WHY IT EXISTS
--------------  ------------------  -------------------  ------------------
LSM-tree DBs    Append SSTables     Compaction merges,   Reads slow as
                                    drops tombstones     files pile up

PostgreSQL      MVCC row versions   VACUUM reclaims      Dead rows bloat
                                    dead tuples          tables

Git             Loose objects       git gc packs and     Repo slows with
                                    prunes               loose objects

GC runtimes     Allocate objects    GC frees             Memory fills up
                                    unreachable ones

Claude Code     Auto memory notes   Auto Dream           Context fills
                                    consolidation        with noise

The pattern is universal: any system with a cheap append path eventually needs an expensive cleanup path. Agent memory is no exception. The difference is that here the “garbage” isn’t just wasted space. It’s actively misleading information that shapes what the model does next.

12. Practical Recommendations

• Normal active development: Let it run automatically. The defaults are sensible.
• Right after a framework migration: Trigger a dream manually by saying “dream” or “consolidate my memory files.”
• After a large rename or refactor: Trigger manually, since stale file references do the most damage here.
• Every few weeks: Skim MEMORY.md and correct any consolidation choices you disagree with.
• Team-wide conventions: Put them in CLAUDE.md, not memory. Rules belong to humans.
• Checking your setup: Run /memory and look for the auto-dream toggle.

# Check whether Auto Dream is enabled
/memory

# Look for:
# Auto-dream: on

13. Design Lessons for Anyone Building Agents

Even if you never use Claude Code, Auto Dream encodes useful principles for agent memory design:

1. Separate collection from maintenance. Writing memories and cleaning them are different jobs with different cadences.
2. Retrieve narrowly, not exhaustively. Searching for known high-value signals beats replaying full history.
3. Normalize time. Relative dates are a bug waiting to happen, so store absolute timestamps.
4. Fix at the source. Don’t just flag contradictions. Resolve them where they live.
5. Respect the context budget. If only about 200 lines load at startup, everything past that line doesn’t exist.
6. Keep a human-owned layer on top. Learned memory should never outrank explicit rules.
7. Gate maintenance on activity. Time-plus-usage triggers avoid wasting compute on idle projects.

Closing Thought

We’ve spent a lot of effort making AI agents remember more. Auto Dream points to the harder and more important goal of making them remember better. Forgetting the right things, resolving contradictions, and keeping knowledge aligned with reality isn’t a polish feature for long-running agents. It’s what stops memory from becoming a liability.

Even AI, it turns out, needs a good night’s sleep.

Claude Is Learning to Sleep: A Deep Dive into “Dreaming,” Memory Decay, and How AI Agents Keep… was originally published in Tutorial Savvy on Medium, where people are continuing the conversation by highlighting and responding to this story.

Ask a good LLM what happens if you nudge a glass off the edge of a table and you’ll get a flawless answer: it falls, it probably shatters, there’s water on the floor. Now ask it to show you the next three seconds, from the other side of the room, after you’ve nudged the glass two centimetres to the left instead. It can’t. It knows the sentence. It doesn’t have the scene.

That gap is what large world models are trying to close.

In the last 18 months, world models have gone from a research curiosity to one of the most heavily funded ideas in AI. Google DeepMind’s Genie 3 generates explorable worlds in real time. Fei-Fei Li’s World Labs raised $1 billion in February 2026 and, on September 28, agreed to join AMD. Yann LeCun left Meta to found AMI Labs, which raised $1.03 billion to build world models. NVIDIA’s Cosmos models, trained on 20 million hours of video, are being used to train robots and self-driving cars. Gartner put “physical AI” on its list of top strategic technology trends for 2026.

This post is a practical guide. We’ll cover:

• what a world model is, and what makes one “large”
• how LWMs work under the hood, with block diagrams
• the four families of world models and the flagship systems in each
• where they’re used today, and what’s still hard
• how LWMs differ from LLMs, and why nearly every AI product you use still runs on an LLM

TL;DR: An LLM predicts the next token of text. An LWM predicts the next state of a world (video frames, a 3D scene or an abstract latent state) given an action. That makes LWMs the right tool for robotics, driving, games and simulation, and the wrong (and far more expensive) tool for most knowledge work. The future isn’t one replacing the other. It’s a stack where each does what it’s best at.

1. What is a world model?

A world model is a learned simulator. Give it the current state of an environment and an action, and it predicts what happens next:

next_state = f(current_state, action)

The idea is older than deep learning. In 1943 the psychologist Kenneth Craik suggested that the mind carries a “small-scale model” of external reality, which lets us try out alternatives before committing to one. You use yours constantly. To catch a ball, you predict where it will be, not where it is. To cross a dark room, you remember where the coffee table was.

A machine world model has three jobs:

1. Represent. Compress raw observations (pixels, depth, lidar, joint angles) into a compact state.
2. Predict. Roll that state forward in time, conditioned on actions.
3. Stay consistent. Remember what it has seen, so the wall you walked past is still there when you turn around.

The core loop. The agent imagines many futures cheaply, then acts on the best one in the real world.

The payoff is imagination. An agent with a good world model can try a thousand candidate actions “in its head” and execute only the one that looks best. In 2018, David Ha and Jürgen Schmidhuber showed that an agent could learn to play a video game entirely inside “its own hallucinated dream,” then carry that skill back to the real game. They split the system into Vision (V), Memory (M) and Controller ©. That three-part shape still describes most world models today.

2. So what makes it “large”?

Classic world models were small and narrow: one model per game or robot, trained on that environment’s data. Large world models apply the recipe that made LLMs work:

• Internet-scale data: millions of hours of video instead of one game’s replays.
• Big transformer backbones: billions of parameters.
• Pretrain, then adapt: one general model, then fine-tune or prompt it for driving, robotics or games.
• Prompting: describe a world in text, or hand it a photo, and it generates one.

A note on naming, because it’s messy. World Labs describes its work as building “large world models.” NVIDIA calls Cosmos “world foundation models.” DeepMind called Genie 2 a “foundation world model.” And a 2024 Berkeley paper literally named “Large World Model” (LWM) was a long-context model that reads video and text across a million-token window, which is closer to a multimodal LLM than to a simulator. In this post, LWM means the broad category: large, general-purpose models that learn how worlds look and behave.

3. A short history

World models have deep roots in cognitive science and reinforcement learning, but most of the progress has happened since 2024.

Three threads came together.

Model-based reinforcement learning gave us the loop. Richard Sutton’s Dyna (1990) learned a model of the environment and planned inside it. Ha and Schmidhuber (2018) trained agents in their dreams. DeepMind’s MuZero mastered Go, chess and Atari by planning with a learned model it was never told the rules of. DreamerV3 used one configuration across more than 150 tasks and became the first algorithm to collect diamonds in Minecraft from scratch, with no human data. It was published in Nature in 2025.

Video generation gave us scale. OpenAI’s February 2024 Sora report argued that scaling video models is “a promising path towards building general purpose simulators of the physical world.”

Interactivity tied them together. DeepMind’s Genie (February 2024, 11 billion parameters) learned playable 2D worlds from unlabeled internet video, without any action labels. Genie 2 (December 2024) made playable 3D worlds from a single image, consistent for up to a minute. Genie 3 (August 2025) made them real-time.

4. Inside a large world model

There’s no single LWM architecture, but most systems fill the same four slots: inputs, tokenizers, a prediction backbone and outputs.

Step 1: Tokenize the world

Raw video is enormous. One minute of 720p video at 24 frames per second is 1,440 frames and more than 1.3 billion pixels. So the first job is compression. A video tokenizer (usually a VAE or a vector-quantized encoder) chops a clip into small space-time patches and maps each one to a compact latent token. The rest of the model never sees raw pixels; it works in this compressed space.

Other inputs get their own encoders: depth and camera pose, text prompts (often via a pretrained language model), and, most importantly, actions.

Step 2: Condition on actions

This is what separates a world model from a video generator. A video model answers “what might happen next?” A world model answers “what happens next if I do this?”

Actions can be explicit: key presses, steering angles, robot joint commands. But most internet video has no action labels. Nobody logged which buttons the player pressed. Genie’s clever trick was a latent action model that watches consecutive frames and infers a small set of discrete actions that explain the change. Train it on enough gameplay footage and those latent actions end up meaning things like “jump” or “move left,” without anyone labeling them.

Step 3: Predict the next state

The backbone is usually a large spatiotemporal transformer. It attends within each frame (space), across frames (time, usually causally so the future can’t leak into the past), and to the actions and text it’s conditioned on. Three training objectives dominate:

• Autoregressive prediction: predict the next video tokens one step at a time, the way an LLM predicts the next word. Genie’s dynamics model works this way.
• Diffusion or flow matching: start from noise and iteratively denoise the next frames, conditioned on history and actions. Most photoreal video world models work this way. NVIDIA released Cosmos in both diffusion and autoregressive versions.
• Embedding prediction (JEPA): don’t predict pixels at all. Predict the representation of the future.

That last choice is one of the biggest philosophical splits in the field.

Yann LeCun’s case for JEPA (Joint-Embedding Predictive Architecture) is that the world is full of details nobody can predict, like the exact pattern of leaves in the wind or ripples on a pond, and forcing a model to generate them wastes capacity. Predict an abstract summary instead, and the model can focus on what matters: the glass is falling, the car is braking. The trade-off is that you can’t watch a JEPA model’s imagination. There’s no video to inspect.

Step 4: Remember

Memory is the hardest part. Generate a world frame by frame and small errors compound: a door drifts, a sign changes, a building disappears when you look away. There are three main approaches:

• Frame cache: attend to recent frames. Genie 3’s visual memory reaches back about a minute.
• Pose-indexed spatial memory: store frames along with the camera position they were taken from, and retrieve the nearby ones when you come back. World Labs’ RTFM model does this and runs at interactive frame rates on a single H100 GPU.
• Explicit 3D: build a persistent 3D scene (Gaussian splats or meshes) so the world can’t forget. World Labs’ Marble takes this route.

The imagination loop, in code

Once you have a world model, planning is surprisingly simple. Here’s the core idea in pseudocode. It’s roughly how Meta’s V-JEPA 2-AC plans robot arm movements toward a goal image:

# Planning with a world model (simplified pseudocode)
z    = encoder(camera_frame)       # 1. compress what you see
goal = encoder(goal_image)         #    ...and what you want to see

best_plan, best_score = None, float("-inf")
for plan in sample_action_sequences(n=256, horizon=10):
    z_sim = z
    for action in plan:            # 2. imagine; don't act yet
        z_sim = dynamics(z_sim, action)
    score = -distance(z_sim, goal) # 3. how close did we get?
    if score > best_score:
        best_plan, best_score = plan, score

robot.execute(best_plan[0])        # 4. take one step, observe, repeat

No hand-written reward function and no real-world trial and error. The robot rehearses in latent space, takes one step, looks again, and replans.

5. The four families of world models

What a model predicts determines what it’s good at.

Interactive video world models (Genie 3, Runway GWM, Odyssey, Decart, Cosmos Predict, Waymo’s world model, Wayve GAIA) generate pixels in response to actions. They’re the most visually impressive and the most general. They’re also the most expensive to run, and they drift over time.

Spatial or 3D world models (World Labs Marble, Tencent HunyuanWorld) produce an explicit 3D scene you can walk through, edit and export to other tools. Persistence comes for free. Rich dynamics don’t.

Latent predictive models (Meta’s V-JEPA 2, AMI Labs) predict in embedding space. They’re efficient and good for planning, but hard to show off, since there’s nothing to watch.

Agent-centric RL world models (DreamerV3, MuZero) learn compact dynamics for one environment. They’re narrow but proven, and they’re the source of most of the field’s core ideas.

Real systems increasingly mix families. Cosmos pairs video models with a reasoning vision-language model, and World Labs’ RTFM renders frames in real time on top of a spatial memory.

6. The flagship systems, as of September 2026

Google DeepMind: Genie 3 and Project Genie

Genie 3, announced in August 2025, generates worlds from a text prompt in real time at 720p and 24 fps, and keeps them consistent for a few minutes. It introduced “promptable world events”: type a text command to change the weather or add a new object or character, and the world changes mid-session. DeepMind also uses Genie 3 worlds as a training ground. Its SIMA 2 agent plays and improves itself inside them.

In January 2026, Google opened Project Genie, a prototype for US Google AI Ultra subscribers with a 60-second limit per world. In May it added Street View, so you can generate worlds from real streets. DeepMind’s Jack Parker-Holder was candid about where it stands, calling it “maybe six to 12 months behind video” in quality. And in February, Waymo unveiled the Waymo World Model, built on Genie 3. It generates both camera and lidar data and can simulate rare events like a tornado or an elephant on the road.

World Labs: Marble, RTFM and the AMD deal

Fei-Fei Li’s World Labs came out of stealth in September 2024 with $230 million to build “large world models” with spatial intelligence. Its product Marble (generally available since November 2025) turns text, images, video or rough 3D layouts into persistent 3D worlds, and exports them as Gaussian splats, meshes or video. RTFM, its real-time frame model, renders explorable worlds on a single H100.

In 2026 World Labs launched a World API, raised $1 billion from backers including AMD, Autodesk and NVIDIA, and released Atlas, which generates up to a minute of 1440p video. On September 28, 2026, it announced an agreement to join AMD, with Fei-Fei Li set to become AMD’s Chief Scientist. The deal is expected to close by the end of 2026. TechCrunch reports its value at $8.2 billion.

NVIDIA: Cosmos

NVIDIA launched Cosmos at CES in January 2025 as a platform of open world foundation models for “physical AI,” trained on 9,000 trillion tokens drawn from 20 million hours of real-world data. The family grew into three lines: Predict (generate future video), Transfer (turn structured inputs like depth maps, segmentation or simulator output into photoreal video) and Reason (a vision-language model that reasons about physical scenes). Cosmos 3, released in May 2026, merges text, image, video, sound and action into one “omnimodel” in 4B, 16B and 64B sizes.

Cosmos’s pitch is less “play a world” and more “make training data for robots and cars.”

Meta and AMI Labs: the JEPA bet

Meta’s V-JEPA 2 (June 2025) is a 1.2-billion-parameter model trained on more than a million hours of video. Its action-conditioned version, V-JEPA 2-AC, was trained on just 62 hours of robot data and then planned pick-and-place tasks zero-shot in new labs, with 65–80% success. V-JEPA 2.1 (March 2026) improved real-robot grasping by about 20 points.

Yann LeCun, who championed JEPA at Meta, left to found AMI Labs in Paris. In March 2026 it raised $1.03 billion at a $3.5 billion pre-money valuation to build JEPA-style world models. As of September 2026, it hasn’t released a product and describes itself as still in a research and building phase.

Driving: Waymo, Wayve and Tesla

Self-driving is the most mature commercial use of world models. Besides Waymo’s Genie-based model, Wayve’s GAIA-3 (15 billion parameters, December 2025) and GAIA-4 (August 2026, closed-loop simulation) generate driving scenarios for training and testing, and Tesla has described a “neural world simulator” for evaluating its driving software. The value is obvious: you can’t wait for a real child to run into a real road to test your car, but you can simulate it a million times.

Also worth knowing

Runway’s GWM-1 (December 2025) and GWM Worlds 2 (September 2026) target media and robotics. Microsoft’s Muse, published in Nature, generates gameplay from a real game’s data. Odyssey-3 (September 2026) aims to cover robots, cars and games in one model. Tencent’s HunyuanWorld line offers open-source 3D world generation. And OpenAI, whose Sora report helped popularize the “world simulator” framing, discontinued the Sora app in April 2026.

7. Where world models are used today

The common thread is that reality is a slow and expensive teacher. With a world model, robots can rehearse before they move, cars can crash safely, agents can train on endless worlds, and game studios and VFX teams can prototype in minutes instead of weeks.

8. What’s still hard

• Long-horizon consistency. DeepMind describes Genie 3 as supporting “a few minutes of continuous interaction, rather than extended hours.” Small errors compound.
• Physics that looks right but isn’t. Visual plausibility is not physical accuracy. TechCrunch described Project Genie as “not yet physics-aware.” A robot trained on subtly wrong physics learns subtly wrong skills.
• Limited actions and multiple agents. DeepMind lists a limited action space and interactions between multiple agents as Genie 3 limitations, along with text legibility and geographic accuracy.
• Data. There’s plenty of video, but very little action-labeled video that records what caused each change.
• Evaluation. There’s no MMLU for worlds. Meta released three new benchmarks (IntPhys 2, MVPBench and CausalVQA) alongside V-JEPA 2 because existing ones didn’t measure physical understanding well.
• Compute. This is the biggest reason LWMs aren’t everywhere yet. More on it below.
• Trust and safety. Photoreal, controllable video of real places raises obvious questions about deepfakes and misuse.

9. LLMs vs LWMs: what’s actually different?

At a high level they share a recipe: big transformers, scaling laws, and pretraining on huge datasets followed by post-training. The difference is what they predict.

An LLM lives in a one-dimensional world of text. Its state is the text so far, its unit of prediction is a sub-word token, and a hundred tokens a second feels instant. An LWM lives in space and time. Its state is a whole scene, including whatever is currently off-screen. Its unit is a space-time patch, and it has to produce 24 to 60 frames every second to feel real.

The most important differences, in words:

• The core question. An LLM asks “what word comes next?” An LWM asks “what happens next if I do this?”
• Training data. Recent open LLMs train on roughly 15 to 36 trillion tokens of text and code (Llama 3 used 15T+, Qwen3 about 36T). Cosmos trained on about 9,000 trillion tokens from 20 million hours of video. Each video token carries far less meaning than a text token, so the numbers aren’t directly comparable.
• The scarce ingredient. For LLMs, it’s fresh, high-quality human text. For LWMs, it’s action-labeled data.
• What they understand. LLMs: language, facts, code and symbolic reasoning. LWMs: geometry, intuitive physics, cause and effect, object permanence.
• Time. LLMs are turn-based, and a second of latency is fine. LWMs are continuous and real-time.
• How they fail. LLMs hallucinate facts. LWMs hallucinate physics: drift, melting objects, things that vanish.
• Maturity. LLMs are everywhere, with mature APIs and tooling. LWMs are mostly research previews and early APIs.

Don’t LLMs already have a world model? Partly. Trained on enough text, LLMs absorb a lot about how the world works, which is why they can explain why a glass breaks. But that knowledge comes from descriptions of the world, not from observing it, and it isn’t spatial, continuous or tied to actions. An LLM knows the glass falls. An LWM can show you where the shards land.

10. So why do we still use LLMs, not LWMs?

If world models are closer to how intelligence works in the physical world, why does nearly every AI product (chat assistants, coding tools, support bots, search) still run on LLMs? There are seven reasons.

1. Most valuable digital work is language-shaped

Code, contracts, emails, tickets, spreadsheets, reports: most knowledge work is text in, text out. A world model can simulate a kitchen, but it can’t write your SQL query, summarize a contract or fix a failing test. The problem picks the model, and most software problems are language problems.

2. Text is pre-compressed meaning

“The glass fell and shattered” is five words. Rendering it as video takes hundreds of thousands of tokens. When people write, they’ve already done the hard work of abstraction: deciding what matters and naming it. That’s why an LLM learns so much per token, and why a world model has to wade through huge volumes of redundant pixels to learn the same concept. Worse, most video lacks the thing a world model needs most: a record of the action that caused each change.

3. Serving a world costs orders of magnitude more

World-model figures from World Labs’ RTFM post (October 2025). LLM figures are typical 2026 values.

A chatbot streams about a hundred tokens a second and can pause to think. A world model has to generate frames continuously, for every user, against hard real-time deadlines. World Labs has written that world models “will be very computationally demanding, much more so than LLMs today.” By its estimate, 4K video at 60 fps needs more than 100,000 tokens per second, and keeping a world consistent for an hour would take more than 100 million tokens of context, which it calls “neither feasible nor economically viable” today.

You can see the economics in how these systems are offered. Genie 3 launched as a limited research preview, and Project Genie caps each world at 60 seconds for top-tier subscribers.

4. You can check an LLM’s work

When an LLM writes code, you run the tests. When it states a fact, you can look it up. Checking whether a simulated world obeys physics is much harder, and the benchmarks to do it are still being invented. For a business, being able to verify output is what turns a demo into a product.

5. The ecosystem is years ahead

LLMs come with everything a team needs to ship: stable APIs, fine-tuning, retrieval, function calling, agent frameworks, evaluation harnesses, guardrails, and lots of engineers who know how to use them. LWMs mostly offer research previews, waitlists and first-generation APIs. AMI Labs, with more than a billion dollars raised, hasn’t released a product yet.

6. Language is still the interface

People specify goals in words. Even world models are prompted in text (“a rainy street in Tokyo”), and many use a pretrained language model as their text encoder. NVIDIA built Cosmos Reason, a vision-language model, because a physical-AI system still needs something that can follow instructions and explain its reasoning. Here LLMs aren’t competing with world models; they’re a component inside them.

7. They aren’t substitutes

“Why LLMs and not LWMs?” assumes one should replace the other. They do different jobs. Asking an LWM to draft a strategy memo is like asking a flight simulator to write the pilot’s manual. The useful question is where each one fits.

11. Which one should you reach for?

A practical rule of thumb:

• Use an LLM when the input and output are language, code or structured data, and the job is reasoning, retrieval, writing or tool use.
• Use a vision-language model when you need to understand images or video but can answer in text: inspection, captioning, visual Q&A.
• Use a world model when you need to predict what physically happens next under different actions: generating training data, testing robot or driving policies, rehearsing plans, or building explorable 3D and interactive media.
• Use both when an agent has to act in the physical world. That’s where the field is heading.

12. The future is a stack, not a winner

Each layer does the job it’s best at. The loop between imagination and action is the key idea.

The emerging architecture for physical AI looks less like one giant model and more like a team. A language model understands the request and plans the steps. A world model imagines what each candidate step would do. An action model, often a vision-language-action (VLA) model like Physical Intelligence’s π series, turns the chosen plan into motor commands. Sensors stream back, and the loop repeats.

You can already see the pieces converging. Cosmos 3 folds text, video, sound and action into one model. DeepMind trains SIMA 2 agents inside Genie 3 worlds. Waymo tests its cars against Genie-based simulations of events it may never record in reality. The lines between “language model,” “video model” and “world model” get blurrier every quarter.

References

1. Google DeepMind, “Genie 3: A new frontier for world models” (Aug 2025): https://deepmind.google/discover/blog/genie-3-a-new-frontier-for-world-models/
2. Google DeepMind, “Genie 2: A large-scale foundation world model” (Dec 2024): https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/
3. Bruce et al., “Genie: Generative Interactive Environments” (Feb 2024): https://arxiv.org/abs/2402.15391
4. Google, “Project Genie” (Jan 2026): https://blog.google/innovation-and-ai/models-and-research/google-deepmind/project-genie/
5. TechCrunch, “Google’s Genie world model can now simulate real streets with Street View” (May 2026): https://techcrunch.com/2026/05/19/googles-genie-world-model-can-now-simulate-real-streets-with-street-view/
6. Google DeepMind, “SIMA 2” (Nov 2025): https://deepmind.google/blog/sima-2-an-agent-that-plays-reasons-and-learns-with-you-in-virtual-3d-worlds/
7. Waymo, “The Waymo World Model” (Feb 2026): https://waymo.com/blog/2026/02/the-waymo-world-model-a-new-frontier-for-autonomous-driving-simulation
8. World Labs, “Marble” (Nov 2025): https://www.worldlabs.ai/blog/marble-world-model
9. World Labs, “RTFM: A Real-Time Frame Model” (Oct 2025): https://www.worldlabs.ai/blog/rtfm
10. World Labs, 2026 funding announcement: https://www.worldlabs.ai/blog/funding-2026
11. World Labs, AMD announcement (Sep 2026): https://www.worldlabs.ai/blog/amd-announcement
12. TechCrunch, “AMD will acquire Fei-Fei Li’s World Labs for $8.2 billion” (Sep 2026): https://techcrunch.com/2026/09/28/amd-will-acquire-fei-fei-lis-world-labs-for-8-2-billion/
13. TechCrunch, World Labs comes out of stealth (Sep 2024): https://techcrunch.com/2024/09/13/fei-fei-lis-world-labs-comes-out-of-stealth-with-230m-in-funding/
14. NVIDIA, Cosmos launch (Jan 2025): https://nvidianews.nvidia.com/news/nvidia-launches-cosmos-world-foundation-model-platform-to-accelerate-physical-ai-development
15. NVIDIA, “What are world foundation models?”: https://blogs.nvidia.com/blog/cosmos-world-foundation-models/
16. NVIDIA, Cosmos 3 launch (May 2026): https://nvidianews.nvidia.com/news/nvidia-launches-cosmos-3-the-open-frontier-foundation-model-for-physical-ai
17. Meta AI, “V-JEPA 2” (Jun 2025): https://ai.meta.com/blog/v-jepa-2-world-model-benchmarks/
18. V-JEPA 2.1 (Mar 2026): https://arxiv.org/abs/2603.14482
19. TechCrunch, “Yann LeCun’s AMI Labs raises $1.03 billion to build world models” (Mar 2026): https://techcrunch.com/2026/03/09/yann-lecuns-ami-labs-raises-1-03-billion-to-build-world-models/
20. TechCrunch, “World model companies are keeping a lot of secrets” (Sep 2026): https://techcrunch.com/2026/09/20/world-model-companies-are-keeping-a-lot-of-secrets/
21. OpenAI, “Video generation models as world simulators” (Feb 2024): https://openai.com/index/video-generation-models-as-world-simulators/
22. OpenAI, “Sora 2 is here” (Sep 2025; product discontinued Apr 2026): https://openai.com/index/sora-2/
23. Wayve, GAIA-3 (Dec 2025): https://wayve.ai/thinking/gaia-3/ and GAIA-4 (Aug 2026): https://wayve.ai/thinking/gaia-4/
24. Runway, “Introducing GWM-1” (Dec 2025): https://runwayml.com/research/introducing-runway-gwm-1
25. Microsoft Research, “Introducing Muse” (Feb 2025): https://www.microsoft.com/en-us/research/blog/introducing-muse-our-first-generative-ai-model-designed-for-gameplay-ideation/
26. Odyssey, “Introducing Odyssey-3” (Sep 2026): https://odyssey.systems/introducing-odyssey-3
27. Liu et al., “World Model on Million-Length Video and Language with Blockwise RingAttention” (Feb 2024): https://arxiv.org/abs/2402.08268
28. Ha & Schmidhuber, “World Models” (2018): https://arxiv.org/abs/1803.10122
29. Hafner et al., “Mastering diverse control tasks through world models,” Nature (2025): https://www.nature.com/articles/s41586-025-08744-2
30. Schrittwieser et al., “Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model” (MuZero): https://arxiv.org/abs/1911.08265
31. Meta, “Introducing Meta Llama 3”: https://ai.meta.com/blog/meta-llama-3/ · Qwen, “Qwen3”: https://qwenlm.github.io/blog/qwen3/
32. Gartner, “Top Strategic Technology Trends for 2026” (Oct 2025): https://www.gartner.com/en/newsroom/press-releases/2025-10-20-gartner-identifies-the-top-strategic-technology-trends-for-2026

Large World Models(LWM): Teaching AI to Imagine the World was originally published in Tutorial Savvy on Medium, where people are continuing the conversation by highlighting and responding to this story.

Anyone who has shipped something built on an LLM has had this happen. You ask a simple question, like which API version added a certain endpoint, and you get a clean, confident, nicely formatted answer.

It’s wrong. The version number doesn’t exist, and the function it cites was never in the SDK. What bothered me most the first time was the tone: the model sounded exactly as sure as it does when it’s right.

For a while I treated hallucination as a bug that the next model release would fix. It hasn’t gone away, and I now think that’s expected. Hallucination is a side effect of how these models are trained and evaluated. Once you see where it comes from, it’s much easier to tell which mitigations actually work and which ones just make you feel better.

This post covers both. First the why, then the fixes that have held up for me and in the research, and a few that are overrated.

First, what are we actually talking about?

“Hallucination” has turned into a catch-all for “the AI said something wrong,” and that’s too broad to be useful. OpenAI’s definition is the one I like best: plausible but false statements generated by language models. The key word is plausible. A typo is a mistake. A made-up but perfectly formatted citation is a hallucination.

So why do models make things up?

Short version: next-word prediction produces hallucinations naturally, and the way we grade models keeps them around. There’s no single cause. Each stage of a model’s life adds a bit of pressure.

1. Next-word prediction has no concept of “false”

During pretraining the model learns one thing: given this text, what comes next? Nobody labels any sentence as true or false. All it ever sees are examples of fluent language.

The OpenAI paper by Kalai et al. uses an analogy that made this click for me. Train an image model to label photos as “cat” or “dog” and it gets very good. Train it to label each pet photo with the pet’s birthday and it will always make mistakes, because nothing in the photo predicts the birthday.

Lots of facts work like birthdays. Spelling and bracket-matching follow consistent patterns, so those errors go away with scale. A specific person’s birth date, a niche API parameter, or the exact title of a paper doesn’t follow a pattern. Either the model saw it enough times to memorise it or it didn’t.

The paper makes this concrete. If 20% of birthday facts appear exactly once in the training data, you should expect a base model to hallucinate on at least 20% of birthday questions. So hallucination isn’t a mysterious glitch. It’s a predictable statistical error rate.

2. The scoreboards reward bluffing

This is the part people usually miss. Think back to multiple-choice exams with no negative marking. If you don’t know the answer, you guess, because a blank is guaranteed zero and a guess sometimes pays off.

Models get graded the same way. Most benchmarks score plain accuracy, so a model that says “I don’t know” scores the same as one that’s wrong. Over thousands of questions, the model that always guesses climbs the leaderboard. Train against that leaderboard and you’re effectively teaching the model to bluff.

OpenAI’s own numbers show this clearly. On the SimpleQA benchmark, the older o4-mini edges out the newer model on accuracy, 24% vs 22%. But it answers almost everything, so it’s wrong 75% of the time. The newer model says “I don’t know” on about half the questions and cuts its error rate to 26%.

An accuracy-only leaderboard would rank o4-mini higher. As a user, I’d much rather have the other one.

The math behind this is simple. Suppose a correct answer scores 1, an abstention scores 0, and a wrong answer costs λ points. If the model’s confidence is p, answering is worth taking only when:

p \cdot 1 - (1 - p) \cdot \lambda > 0 \quad \Longleftrightarrow \quad p > \frac{\lambda}{1 + \lambda}

With plain accuracy scoring, λ = 0 and the threshold is zero, so guessing always pays. Set λ = 1 and the model should answer only when it’s more than 50% sure. Set λ = 3 and the bar is 75%. That’s the fix Kalai et al. argue for: change how the main benchmarks are scored, rather than adding yet another hallucination benchmark that nobody optimises for.

3. Fine-tuning on facts the model doesn’t know

Gekhman et al. (EMNLP 2024) ran a controlled experiment and found that fine-tuning examples with new facts are learned much more slowly than examples the model already agrees with. Once those new facts are learned, the model’s tendency to hallucinate goes up roughly linearly.

Their explanation is that models get most of their factual knowledge in pretraining. Fine-tuning mostly teaches them how to use what they already know. If you push brand-new facts in through SFT, what the model really learns is “answer confidently even when you don’t actually know.”

4. Decoding and context

The last push comes at inference time. Sampling with temperature above zero will sometimes pick a lower-probability token, and every token after that is conditioned on it. Zhang et al. call this hallucination snowballing. They found GPT-4 could recognise 87% of its own mistakes when asked separately, yet it still defended them once they were in the answer.

Context causes problems too. If the retrieved documents are missing the answer, or the answer sits in the middle of a long prompt, the model falls back on memory. Liu et al. showed that accuracy drops noticeably when the relevant passage is in the middle of a long context rather than at the start or end.

Mitigations that actually help

Nothing gets you to zero. What works in practice is layering: give the model the facts, make abstaining a legitimate option, then check the output before a user sees it. Here’s the pipeline I’d build today:

Each step deals with one of the causes above. Here they are in order of how much they bought me for the effort.

1. Ground the model in real sources (RAG, done properly)

If pretraining can’t reliably store rare facts, don’t ask the model to recall them. Retrieve them. The original RAG paper by Lewis et al. showed that pairing a generator with a retrieval index gives “more specific, diverse and factual language” than a model working from memory alone.

What the diagrams on vendor slides leave out is that retrieval quality is the ceiling. If the right chunk isn’t retrieved, the model will still produce an answer, just a made-up one. The details that made the biggest difference for me:

• Measure retrieval separately. Track recall@k on a labelled set before you look at a single generated answer.
• Put the best evidence at the start or end of the context, not buried in the middle.
• Say it explicitly: “Answer only using the sources below. If they don’t contain the answer, say so.”
• Require citations per claim, so every sentence points back to a chunk you can check.

2. Make “I don’t know” a first-class answer

This follows straight from the scoring argument. If your prompt, your eval, and your product all treat abstaining as failure, you’ll get bluffing. So:

• Tell the model in the system prompt that abstaining is acceptable, and show an example of it doing so.
• Score your internal evals with a penalty for wrong answers (the λ from earlier), not plain accuracy.
• Design the UI so “I couldn’t find this in our docs, here’s who to ask” is a normal outcome, not an error screen.

3. Detect uncertainty by sampling

This is my favourite technique because it’s so simple. Ask the same question several times at a non-zero temperature. If the model knows the answer, the samples agree. If it’s guessing, they scatter.

Farquhar et al. (Nature, 2024) formalised this as semantic entropy. You sample several answers, cluster them by meaning rather than exact wording (“Paris” and “It’s Paris, France” count as the same answer), and compute entropy over the clusters. High entropy flags a likely confabulation. It costs N times the inference, so I only run it on high-stakes questions.

4. Verify after generating, independently

Chain-of-Verification (CoVe) from Meta works in four steps: draft an answer, plan verification questions, answer those questions independently, then write a revised final answer.

The word “independently” is doing the heavy lifting. Remember the snowballing result: once a model has committed to a claim, it tends to defend it. Answering the checks in a fresh context, without the draft visible, breaks that loop. In RAG systems I do a lighter version: a second, cheaper call checks each cited claim against its source chunk and drops the claims that aren’t supported.

5. Hand deterministic work to tools

Dates, arithmetic, ID lookups, current prices, and user permissions shouldn’t be generated from memory. Give the model a tool and have it call the tool. A function call either returns the right value or fails with an error. It never invents a plausible number.

6. Fine-tune for behaviour, not facts

Given the Gekhman result, I’ve stopped trying to put knowledge into weights through SFT. Fine-tuning is great for format, tone, and teaching the model when to abstain. Facts belong in retrieval, where you can update them, cite them, and audit them.

Mitigations that are overrated

Some fixes get recommended a lot and don’t do much.

“Do not hallucinate” in the system prompt. The model has no switch to turn off. Explicit permission to abstain helps, as covered above. A bare instruction not to make things up does almost nothing.

Temperature 0 as a fix. Greedy decoding makes the output repeatable, not correct. A model that’s confidently wrong at temperature 0.7 is usually confidently wrong at 0 too, just the same way every time. You also lose the variation between samples

“Just wait for a bigger model.” Bigger models know more, so they hallucinate less on common facts. But OpenAI’s paper makes a point that stuck with me: some questions can’t be answered, so accuracy will never reach 100%. And knowing your own limits can actually be easier for a small model. Calibration comes from training incentives, not parameter count.

Asking “Are you sure?” in the same chat. Because of snowballing, the model tends to double down, or swing the other way and apologise for a correct answer. Verification works when it runs in a fresh context, not as a follow-up turn.

Treating RAG as done. Retrieval addresses factuality. It does nothing for faithfulness, where the model has the right document and still paraphrases it wrongly. You need the verification step for that.

Checklist

If you’re putting an LLM feature in front of users, run through this list. It’s short on purpose.

• Build an eval set of 100 to 200 real questions, including ones your sources can’t answer
• Score it three ways (correct, wrong, abstained), with a penalty for wrong answers
• Measure retrieval recall on its own, before measuring generation
• Tell the model it’s allowed to abstain, and give it an example
• Require a citation for every factual claim, and check the citations automatically
• Route dates, math, lookups, and permissions through tools, not generation
• Add a verification pass (CoVe-style or a claim-vs-source check) for high-stakes answers
• Use sampling consistency or semantic entropy where the extra cost is worth it
• Log every abstention and flagged answer, since they show you where your knowledge base has gaps
• Design the “I don’t know” state in the UI as carefully as the answer state

References

1. Kalai, Nachum, Vempala & Zhang, Why Language Models Hallucinate (OpenAI, 2025). Also see OpenAI’s summary post, which has the SimpleQA numbers.
2. Huang et al., A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (ACM TOIS, 2024)
3. Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (NeurIPS 2020)
4. Farquhar, Kossen, Kuhn & Gal, Detecting hallucinations in large language models using semantic entropy (Nature, 2024)
5. Dhuliawala et al., Chain-of-Verification Reduces Hallucination in Large Language Models (Meta AI, 2023)

Why LLMs Hallucinate (and What Actually Helps) was originally published in Tutorial Savvy on Medium, where people are continuing the conversation by highlighting and responding to this story.