ReplayHouse

01 — memory

Why replay buffers exist

The spotlight sweeps across the photo once. Two identical networks try to learn it. One can only train on what's under the spotlight right now. The other writes every pixel it sees into a ClickHouse table — a replay buffer — and trains on samples from the whole table. As the spotlight moves on, the first network starts losing regions it had already learned.

both networks train identically; the only difference is what data they can reach. the buffer is a real MergeTree table that grows as pixels are revealed, and the replay batches are weighted samples from it.

02 — priorities

Why sample by priority

Two identical networks learn to paint the same photo with the same number of training batches. One samples pixels uniformly at random. The other keeps a per-pixel error score in a ClickHouse table and samples the pixels it currently gets most wrong. First to get every patch of the image sharp wins.

you can interfere: click or drag on the photo (or the score panel) to bump those pixels' priorities — a real write to the table. the next batches fill up with them and that patch sharpens first.

nothing is simulated: every batch is a weighted draw from a MergeTree table in this tab, every error is written back as a priority, and the score panel is a live query over that table. this is the same loop RL and LLM post-training use, with pixels standing in for experiences.