This game is running inside a transformer.
There is no emulator on this page. Every pixel, every collision, every point on the scoreboard is a transformer predicting its next token. Its 139,229 weights were placed by hand. It never saw a single training example.
The neural network is the computer. Go play Tetris on it.
- Layers
- 18
- Attention heads
- 42
- Hand-set weights
- 139,229
- Gradient steps
- 0
Play
CHIP-8 · 64 × 32 · 60 Hzloading…
The transformer, live
HeadReLU- L00
- L01
- L02
- L03
- L04
- L05
- L06
- L07
- L08
- L09
- L10
- L11
- L12
- L13
- L14
- L15
- L16
- L17
A write to the machine’s memory. The screen is drawn from these.
Writing text and running a program are the same loop.
Read the context, predict a token, append it, repeat. Set every weight exactly right and that loop runs Tetris. Each token passes through all four stations below.
- 01 / PromptBOOT@M[200]=6A@M[201]=02…@PC=200STEP
The ROM is the prompt
The game loads the way any prompt does: as tokens, one pair per byte of memory. From then on every token comes from the model, except your keypresses and the clock tick.
- 02 / Attention
Attention is memory
Need register V3? A head searches back through everything the model has ever written and finds the last time it wrote V3. That lookup is a memory read.
- 03 / ReLU
ReLU is the ALU
Adding, comparing, XOR, decoding opcodes: all of it is 22,276 ReLU units, each placed by hand. One identity, step(x) = ReLU(x+1) − ReLU(x), does most of the heavy lifting.
- 04 / Output@SCREEN[12:3]=3C→
Every token is a write
The output is a stream of tiny edits: this cell, this value, STEP. Paint enough of them onto a 64×32 grid and you are playing Tetris.
A completely ordinary transformer.
That’s the unsettling part. No special layers, no custom ops, no external memory. A 40-line NumPy forward pass with standard softmax attention reproduces it bit for bit. The entire computer lives in the weights.
- Architecture
- Decoder-only transformer: token and position embeddings, causal softmax attention, ReGLU feed-forward, linear unembedding
- Layers
- 18
- Attention heads
- 42, each with 2-dimensional queries and keys
- Residual width
- 382
- Vocabulary
- 8,708 tokens: values, memory addresses, STEP / TICK / BOOT
- Weights
- 32,328,660 parameters, 139,229 nonzero, all set by hand
- Training
- None
- Correctness
- 118 of 119 games match a reference CHIP-8 emulator token for token
Games
14 cartridges · one set of weightsThe exact same 139,229 weights run every one of these. Swapping games just means a different prompt; each game’s quirks ride along in it, like a system prompt for a CPU.