Glossary

One complete game, played during training. It is the unit everything on this page is measured in: a checkpoint is named by how many episodes had been played when it was saved, and the agent that ships had played 400,000 of them.

Episodes are not games against you. They are simulated — the agent against a mix of opponents, including itself — and all of them happened before the page was built. Nothing you play here trains anything.

Learn what a move is worth by playing whole games and looking at how they ended, rather than by reasoning forward from the position.

The loop is three steps. Play a game to the finish. Credit every move the agent made in that game with the result — +1 for a win, 0 for a draw, −1 for a loss. Keep a running average of that credit for each move, and play whichever move has the best average.

“Monte Carlo” is the sampling: the value of a move is estimated from the games that actually went through it, not derived. “Control” means the estimates are used to choose moves, so the policy improves as the numbers do.

The number on each square: the average result of every game the agent played after moving there. +1 is a square it has only ever won from, −1 one it has only ever lost from, and 0 is an even split.

It is an average over experience, not a probability and not a score for the position. A move rated +0.10 is not “10% good” — it means games through that square came out ahead about a tenth of a game more often than behind.

The interesting reading is the gaps rather than the values. Late in training the safe moves all sit just above zero, because they all draw against a perfect player and what is left in the number is how often each one makes a fallible opponent go wrong. The losing moves sit near −0.8. The distance between those two groups is what the agent knows.

Assumed to draw, and so rated 0.

Zero rather than −1 is a real decision and it decides most of what an early checkpoint does, because most positions an early checkpoint meets are ones it has never seen. Assuming the worst would make it avoid everything unfamiliar; assuming the best would make it walk into anything. A draw is the honest middle: better than a move it knows loses, worse than one it knows wins.

When the agent meets a whole position it has never seen, the tool says so — every square looks identical to it and the move is arbitrary.

Beginning a training game from a random legal position and a random first move, rather than always from an empty board.

Without it a table has a hole exactly where its current policy never goes: a move the agent has decided is bad never gets played, so it never gets re-tested, so it stays bad forever whether or not it really is. Three in four training games start this way, which is what fills in the corners of the state space that greedy play would leave permanently blank.

Who the agent trained against, and in what proportion: 50% a perfect player, 30% itself, 10% a bot that takes the win and blocks the loss, 10% random.

The weighting is the reason it ends up safe. A running average learns what a move is worth against whoever it met, so against careless opponents a losing move still averages out positive — nobody ever punishes it — and the agent stays quietly beatable however long it trains. Half the games are against an opponent that punishes everything, which is what turns “usually fine” into “never loses”.

The weak opponents are not filler. Training against the perfect player alone still reached unbeatable, but cost more than ten percentage points of win rate against the fallible bot. Weak opponents are what teach it which of several equally safe moves is most likely to draw a mistake.

Every position is stored from the point of view of whoever is about to move — own marks one way, the opponent’s the other — so the same entry serves X and O.

It halves what has to be learned and, more usefully, transfers it: a lesson learned while playing X is available immediately when playing O, because as far as the table is concerned they are the same position. This is why you can switch sides here and meet the same strength of opponent.

The table saved to a file at one moment in training, named by how many episodes it had played. Twenty were kept, from 0 to 400,000.

They are not difficulty settings someone dialled in. They are one agent at twenty moments, and they get better in the order they were saved — which is why the useful question about a checkpoint is not “how hard is it” but “can it still be beaten”.

What a checkpoint actually is: a list of positions, each with a value for every move available in it. Nothing else. There is no model and no search — playing a move is a lookup.

A complete table holds 4,520 positions, which is every position the agent can ever be asked about. The untrained checkpoint holds 0; it reaches full coverage at 80,000 episodes, and keeps improving for 320,000 more without adding a single new entry. Coverage is not skill.

How the agent picks: look up the position, take the highest-valued move. No lookahead, no search, no consideration of what happens next — all of that already happened during training and is baked into the numbers.

“Greedy” is the technical word for taking the best-rated option every time rather than occasionally trying something else. During training it does explore; here it never does.

What happens when several moves share the top value. By default the agent picks among them at random, so the same opening does not always get the same reply.

It is why the same opening does not always get the same reply, and why no fixed sequence of moves beats a checkpoint reliably: a line that works depends on the tie falling one way, and next game it may not.

Ties are also why the proof is stated the way it is: the audit lets the agent take any tied move, so “unbeatable” holds however the tie falls rather than only for one way of breaking it.

A walk of the entire game tree under two rules: the agent may play any move tied for best under its own table, and the opponent may play anything at all.

If no branch of that tree ends in a loss, then no opponent can ever beat the table — not a strong one, not a lucky one, not one that has read the table. That is a proof about every possible game, which is only affordable because the whole of tic-tac-toe is 5,478 positions.

It is the reason nothing on this page has an error bar. The audit is re-run against the exact files this site serves, so the verdicts describe what you are playing rather than what was measured somewhere else.

No opponent can win against it, ever — proved by the audit, not observed over a number of games. The best anyone gets is a draw.

It arrives at 100,000 episodes. The checkpoint before it, 80,000, loses only 0.3% of games to a random player and looks finished from every angle except the one that counts: there is a fixed four-move sequence that beats it every time.

Note what this does not claim. An unbeatable table can still miss a win you handed it — that is a separate and later bar.

A position where the agent’s choice is not one a perfect player would make. Counting them is what the chart plots.

It starts at 3,191 — the untrained table, which has an opinion about nothing and so plays badly nearly everywhere — and reaches 0 at 150,000 episodes.

The shape is the point. By 12,000 episodes it is down to 126, and by 40,000 to 21. Both of those look like a finished agent on any chart with a linear axis, and both are still beatable. The last handful of bad moves take longer to find than the first three thousand, and they are the ones that decide whether the table is safe.

The best possible move in a position, worked out by looking at every continuation to the end of the game. Tic-tac-toe is small enough to solve this way completely, so “perfect” here is exact rather than aspirational.

It is used twice and learned never: as one of the training opponents, and as the reference the audit measures against. The agent has no access to it — that is the whole experiment. A minimax player is written in an afternoon and knows everything; the point is what a player that knows nothing can reach by playing.

Two perfect players always draw, which is why the ceiling on this game is a draw and not a win.

The stricter bar above unbeatable: every move the agent makes is one a perfect player would make, so it never lets a won position slip to a draw either.

It arrives at 150,000 episodes — 50,000 games after it stopped losing. Not losing is easy to reach and hard to notice; not wasting a win takes another third of the training run.

A fixed sequence of moves that beats a particular checkpoint every time — found by the same exhaustive search that produces the verdicts, and replayable rather than described.

It only works against that checkpoint. The four moves that beat the 80,000-episode table lose against the final one, which is twenty thousand games and about seven positions later. That difference is small enough to be invisible in every summary statistic and large enough to decide the game.

These are not offered as a way to play. A line assumes the opponent breaks every tie one particular way, and here it breaks them at random — so a published line is a proof that a checkpoint can be beaten, not a recipe that beats it every time. The site’s tests replay them against pinned tie-breaks, which is where that assumption is honest.

The fixed players every checkpoint is scored against, so twenty tables can be compared on the same terms.

Random plays any legal move. Take the win, block the loss never misses a one-move tactic but has no plan, so it walks into forks — the closest thing here to a casual human. Perfect play cannot be beaten.

The three answer different questions, and the middle one is the most informative: the final agent draws every game against perfect play and wins 93.3% against random, but its score against the fallible bot is where the difference between “safe” and “dangerous” actually shows.

Every win, draw and loss share on this page is computed by summing over the whole game tree weighted by move probabilities — not by playing some number of games and counting.

So the figures carry no sampling noise and no confidence interval. “93.3% against a random player” is the exact share over every game that could be played, and running it again returns the same number.