Tool
Beat it before it learns
An agent that began knowing nothing about tic-tac-toe and played 400,000 games to work it out. Twenty snapshots were kept along the way, and all of them are here. The early ones lose to almost anything. Past 100,000 games nobody has ever beaten one — and that is a proof rather than a win rate.
There is no search behind its move. It looks the position up in a table of values it learned by playing and takes the highest one. Every term on this page is defined in the glossary, and any dotted term explains itself on hover.
You
Opponent
can be beaten
Loading its table…
Its ratings of its own moves — not hints for you. It plays the highest.
It looks finished long before it is
Every dot is one of the saved tables in the list above. The line is how many moves it still plays that a perfect player would not — 3,191 of them before it had played a single game, and none by the end. The fall is almost over by 12,000 games; the last seven bad moves take another 20,000 to find.
Games trained on
The agent is playing 80,000 episodes, with 7 sub-optimal moves left of 3,191 before training. No checkpoint past 100,000 episodes can be beaten by any opponent.
80,000 episodes
- Positions it has an opinion about
- 4,520 of 4,520
- Sub-optimal moves left — its height on the curve
- 7
How it does against three fixed opponents — win, draw, loss, averaged over both sides.
The method
How it learns
It plays a whole game, and when the game ends it credits every move it made with the result: +1 for a win, 0 for a draw, −1 for a loss. Each move keeps a running mean of that credit, and the mean is what you see on the square. Nothing is looked ahead — positions are stored from the mover’s own point of view, which is what lets one table play both X and O.
Why the opponents it trained against matter
A running mean learns what a move is worth on average against whoever it met. Against careless opponents a losing move still averages out positive, because nobody ever punishes it, and the agent stays quietly beatable forever. Half its games were against a perfect player for that reason. The other half were not: weak opponents are what teach it which of several equally safe moves is most likely to make a fallible player go wrong.
What “unbeatable” means here
Not a win rate over sampled games. The whole game tree is walked with the agent allowed to take any move tied for best and the opponent allowed to play anything at all. If no leaf of that tree is a loss, no opponent can ever beat it. That happens at 100,000 episodes. Playing every move a perfect player would takes another 50,000.
Written for CMPT 310 at SFU in the summer of 2020, as the final project. The Python behind this page is a rewrite of it — same method, same result — and the twenty checkpoints came out of that run. The code is on GitHub, training included.