Libratus: Poker's hidden cards fall to game theory

Twenty days at Rivers Casino

On 30 January 2017, the last of 120,000 hands of heads-up no-limit Texas hold'em was dealt at Rivers Casino in Pittsburgh, closing a 20-day match billed as "Brains Vs. Artificial Intelligence: Upping the Ante." Four heads-up specialists, Dong Kim, Jason Les, Jimmy Chou and Daniel McAulay, faced Libratus, a program built by Tuomas Sandholm, a Carnegie Mellon computer science professor, and his doctoral student Noam Brown. Carnegie Mellon's announcement of the result reported that the program finished ahead by $1,766,250 in chips, and that its creators judged the outcome statistically significant rather than a run of good cards.

The chip figure is play money, and not the measure the researchers leaned on. In the peer-reviewed account Brown and Sandholm published in Science, the margin is given as 147 milli-big-blinds per game: on average the program won 0.147 of a big blind per hand, or 14.7 big blinds per hundred hands. The paper reports a p-value of 0.0002 if the hands are treated as independent, and states that Libratus beat each of the four humans individually as well as the team. IEEE Spectrum's report on the final day noted that Dong Kim, the most successful of the four against the machine, still finished $85,649 behind it.

The narrow, documented claim is that one program, under one set of rules, beat four top specialists over a sample large enough to make luck implausible. The broader claim, that game-theoretic computation had overtaken human expertise in a game of hidden information and bluffing, is defensible, but it rests on the method and on what followed, not on one month's chip count.

Why hidden cards were the hard case

The best-known earlier milestones in game-playing AI, in chess and Go, came in games where both players see the whole board, so a program can search ahead from the exact position. The Science paper sets out why poker breaks that model. A player must decide how often to take each action, not merely which action is best, because an opponent who spots a pattern can infer the hidden cards behind it. A strategy that never bluffs is easy to read and one that always bluffs is easy to punish, so the value of a bet depends on how often it is made. Worse, the right play in one situation can depend on how the program would have played in situations that never arose, so no part of the game can be analyzed in isolation.

The field's main tool for this problem was counterfactual regret minimization, introduced by Martin Zinkevich and colleagues at the University of Alberta in a 2007 NIPS paper on regret minimization in games with incomplete information. The algorithm plays the game against itself many times, keeps track of how much it regrets not having taken each action at each decision point, and shifts probability toward the actions it regrets skipping. In two-player zero-sum games the average strategy converges toward a Nash equilibrium. The 2007 paper solved simplified versions of limit hold'em with about 10^12 states. By January 2015, the same Alberta group reported in Science that heads-up limit hold'em was solved for practical purposes, with a program named Cepheus; the University of Alberta's announcement called it a world first in a challenge problem more than 40 years old.

Limit hold'em fixes the size of every bet; no-limit lets a player wager any amount up to their stack. According to the Libratus paper, heads-up limit hold'em has about 10^13 decision points, while the no-limit variant used in the Annual Computer Poker Competition has 10^161. No program could traverse that tree even once.

The 2015 loss that shaped the design

Carnegie Mellon had tried this before. From 24 April to 8 May 2015, its program Claudico, an improved version of the competition winner Tartanian7, played 80,000 hands against four professionals at the same casino. Carnegie Mellon's report at the close of that match said the humans finished $732,713 ahead collectively, a lead that fell just short of statistical significance. IEEE Spectrum summarized it as a statistical draw, with three of the four pros finishing ahead of the program.

Accounts of that match have shifted. In 2015 the university framed it as a tie. In a TED talk recorded in October 2024, Brown said the bot had lost by a wide margin and looked outmatched from the first day. He recalled that Claudico, trained for months, acted in about ten milliseconds however hard the decision, while the humans slowed down on difficult spots. Brown told the audience that later experiments showed that letting a poker bot think for 20 seconds per hand gave the same boost as scaling its model and training by 100,000 times, a result so large that he "literally thought it was a bug." That is a retrospective claim by an interested party, made in a talk, but it fits the design of Libratus, whose central novelty was computation spent during play.

The rematch was also built to settle the statistics that 2015 left open. Carnegie Mellon's announcement of the opening day explained that the pros agreed to 120,000 hands, half again as many as in 2015, to improve the chance of a significant result either way. Play was duplicate. The four were split into pairs, and each player received the cards that the computer held against his partner in a separate room, so a lucky run for the machine in one room was offset by the same cards going to a human in the other. According to the Science paper, each human was guaranteed $20,000 of a $200,000 prize pool, with the rest divided by performance.

Three modules: a blueprint, a solver at the table, and a nightly repair

The Science paper describes Libratus as three components, none specific to poker and none trained on records of human play.

The first is the blueprint, a precomputed strategy for a simplified game. Action abstraction treats similar bets as identical: there is little difference between a bet of $100 and one of $101, so the program considers a menu of sizes, mostly fractions or multiples of the pot. Card abstraction groups similar hands: the 55 million possible hand situations on the third betting round were clustered into 2.5 million buckets, and the 2.4 billion on the fourth round into 1.25 million. The first two rounds, where situations are fewer, used no card abstraction at all. The blueprint was computed with an improved Monte Carlo version of counterfactual regret minimization that probabilistically skipped actions with very negative regret, which the authors say made it about three times faster. Carnegie Mellon reported that building Libratus's strategies took about 15 million core hours on the Bridges supercomputer at the Pittsburgh Supercomputing Center.

The second module is what made the difference. From the third betting round onward, Libratus set the coarse blueprint aside and solved the rest of the hand in real time, at a finer granularity and with no card abstraction. The danger is that a strategy computed for one part of the game in isolation can be exploited by an opponent who changes how they arrived there. Libratus used safe subgame solving everywhere except a single unsafe solve on first reaching the third round, which the paper says works well in practice there. It built an augmented subgame in which the opponent could either enter the newly detailed situation or take a fallback payoff equal to what the blueprint estimated that particular hand was worth. Any solution therefore had to do roughly as well as the blueprint against every hand the opponent might hold, and the paper proves a bound on how much worse it can be when those estimates are off. A refinement let the solver worry less about hands, such as seven-deuce offsuit, that an opponent would only hold at that point if they had already made a costly mistake. The ideas are developed in Brown and Sandholm's paper on safe and nested subgame solving, which received a best paper award at NIPS 2017.

Nested refers to bets not on the program's menu. Earlier programs rounded an unexpected bet to the nearest known size, so a bet of $150 might be treated as $100 or $200. Libratus instead solved a fresh subgame after each such move. In a small test game where rounding had been the state of the art, the paper reports that nested safe solving cut exploitability, the loss against a worst-case opponent, from 1,465 to 119 milli-big-blinds per game. Libratus also shifted all of its own bet sizes by a random amount between 0 and 8 percent at the first solved subgame of each hand, so the humans never faced a fixed target.

The third module, the self-improver, worked overnight. Each night it took the first-round bet sizes the humans used most that lay far from the blueprint's menu, chose three (about what the available computing could solve well in 24 hours), and by morning had added them to the blueprint. This was not opponent modeling: Libratus did not try to exploit particular players, which would have exposed it to counter-exploitation, and its patches applied against anyone.

Tests against Baby Tartanian8, Carnegie Mellon's earlier bot and the 2016 Annual Computer Poker Competition winner, show where the strength came from. The blueprint alone lost to it by 8 milli-big-blinds per game, within the margin of error. With full nested subgame solving, Libratus won by 63 milli-big-blinds per game. The precomputed strategy was no stronger than its predecessor; the computation at the table was.

Inside the match

The contest did not run one way. IEEE Spectrum's day-by-day account records Libratus up $193,000 after day three, the humans winning chips back on days four and six, and the program's lead reduced to $50,513 before it pulled away with a $180,816 seventh day. Online betting sites, Spectrum reported, had made Libratus a four-to-one underdog before play began. In his 2024 talk, Brown recalled that by the eighth day bettors could no longer wager on which side would win, only on which human would lose least.

Carnegie Mellon's account of the result credited part of the program's improvement to its nightly patching of holes the pros had found. About 600 of Bridges' 846 compute nodes were in use during the competition, according to the university, and a later Carnegie Mellon release put Libratus's live play at 1,400 CPU cores. Brown told Spectrum the hardware demands made a flood of unbeatable online bots unlikely soon. Jason Les, quoted by Science News, said the program was much tougher than any human he had played.

DeepStack and the question of who was first

Five days before the Pittsburgh match began, a team from the University of Alberta and two Czech universities posted a preprint describing DeepStack, a program that had already beaten professional players at the same game; Science published it online on 2 March 2017, while the Libratus paper was still months away. Amii, the Alberta institute behind the work, reported that 33 professionals from 17 countries, recruited through the International Federation of Poker, played 44,852 games in a study completed in December 2016. Eleven completed the requested 3,000 games, and DeepStack beat all but one of them by a statistically significant margin, winning 49 big blinds per 100 hands overall. During play it ran on a gaming laptop with a single GPU.

DeepStack also re-solved the remainder of the game during play, which its authors called continual re-solving, but it limited how far ahead it looked and estimated values beyond that horizon with neural networks trained on random poker situations. The Libratus paper acknowledges that DeepStack's authors developed an algorithm similar to nested subgame solving independently and concurrently.

DeepStack's win rate was far higher, and Science News quoted Murray Campbell, a co-creator of Deep Blue, praising its huge margin of victory. Libratus faced fewer opponents but each was a heads-up specialist playing 30,000 hands, against the 3,000 requested of each DeepStack player. Brown and Sandholm's paper describes DeepStack's opponents as non-specialists and notes that DeepStack was never shown to beat the leading earlier programs, as Libratus beat Baby Tartanian8. These are a rival team's claims. The fair summary is that two independent teams cracked heads-up no-limit hold'em within weeks of each other by re-solving the remaining game during play, and that Libratus produced the stronger evidence against elite specialists.

The honest limits

The result is narrower than the word superhuman suggests. It concerns two-player zero-sum poker, where an equilibrium strategy carries a guarantee: played precisely enough, it cannot lose in expectation whatever the opponent does. The 2019 paper on multiplayer poker points out that this guarantee does not carry over to games with more than two players. The game was also standardized, with every hand starting from fresh $20,000 stacks under the competition rules the paper uses.

The statistics need care too. The p-value of 0.0002 assumes the hands are independent, and the authors themselves note that both sides adapted over the 20 days, offering their confidence intervals as intuition about variance rather than a strict test. Four humans, however strong, are a small sample of the world's players. The size of the margin also depends on who sits across the table. In April 2017 a version of the program called Lengpudashi beat a six-person team led by Alan Du, a venture capitalist and World Series of Poker bracelet winner, by 220 milli-big-blinds per game over 36,000 hands in Hainan, China, according to Carnegie Mellon, a larger margin than against the Pittsburgh specialists. MIT Technology Review described the opposing team as poker-playing AI researchers rather than professionals.

Libratus depended on a supercomputer before and during play, and Sandholm and Brown soon treated that as a problem to solve; Wired reported that they presented a smaller bot, Modicum, that could run on a single server. Libratus also did not learn to read people. It played toward an equilibrium, which made it robust, but the match says little about punishing the particular errors of weaker players. And the chips were not money: no one at the table risked a stake.

From Libratus to Pluribus

Recognition and commercialization followed. The Science paper appeared online on 17 December 2017. Sandholm's company Strategic Machine exclusively licensed Libratus and other technology from his lab and was set to collect a pot of about $290,000 from the Hainan event. Early in 2018 he founded a second company, Strategy Robot, to adapt the techniques for government use. Wired reported in January 2019 that public records showed a two-year Army contract worth up to $10 million, in support of the Pentagon's Defense Innovation Unit, aimed at war games and simulations; Greg Allen of the Center for a New American Security told Wired that such tools were a better proxy for the real world while remaining far from it. IJCAI gave the Libratus team its Marvin Minsky Medal for Outstanding Achievements in AI, presented at its 2019 conference in Macao, with the awards chair Michael Wooldridge crediting fundamentally new techniques for incomplete information.

The research line continued with Brown, who joined Facebook AI in 2018. In July 2019 he and Sandholm reported Pluribus, which beat elite professionals at six-player no-limit hold'em, the format most people actually play. The Pluribus paper explains that Libratus could afford real-time search only because it waited until the remaining game was short enough to search to the end, which becomes unaffordable with more players; Pluribus instead looked a limited distance ahead. Its blueprint took 12,400 CPU core hours over eight days on a single 64-core server, which the authors put at about $144 at cloud spot prices, and Carnegie Mellon noted that it used 28 cores in live play against Libratus's 1,400. Over 10,000 hands against five professionals at a time, Pluribus won 48 milli-big-blinds per game, with a standard error of 25.

Others generalized the recipe. In 2020 Brown and colleagues at Facebook described ReBeL, which combines self-play reinforcement learning with search, reaches superhuman heads-up no-limit play with far less poker-specific knowledge, and reduces to something like AlphaZero in perfect-information games. A team led by Martin Schmid and Matej Moravčík, two of DeepStack's authors, published Student of Games, one algorithm for perfect and imperfect information games. That line descends at least as much from DeepStack, so later progress cannot be credited to Libratus alone.

How to place Libratus in AI history

Libratus belongs with Deep Blue and AlphaGo among the public matches in which a program beat elite humans at a long-standing test of machine intelligence. What distinguishes it is the kind of game: Deep Blue and AlphaGo searched positions both sides could see. Libratus showed that a program could reason at a superhuman level about what it could not see, balancing bluffs and value bets so that its own cards stayed hidden, by computation grounded in game theory rather than imitation of human play. The Science paper's authors framed that capacity as relevant to negotiation, cybersecurity, finance and military settings; those applications remained largely prospective, though the Strategy Robot contract shows they attracted funding.

Its more durable legacy may be methodological, though the evidence is still accumulating. Libratus made a strong case that computation spent at decision time, solving the situation actually in front of the program, could beat further scaling of precomputation. Brown carried that idea forward. After Pluribus and work on the negotiation game Diplomacy at Meta, listed in his curriculum vitae, he moved to OpenAI in 2023. His own site describes him as a foundational contributor to the o1 reasoning models, and he is among the authors of the o1 system card. In his 2024 TED talk he tied his poker results on thinking time directly to o1, but that connection is his own account. Reasoning in language models has many other roots, and it is too early to say how much of it traces back to a poker table in Pittsburgh. What is established is narrower and still significant: in January 2017, hidden information stopped being a reliable refuge for human expertise in heads-up no-limit hold'em, the leading two-player benchmark for imperfect-information games.

References

• Noam Brown and Tuomas Sandholm, Science (2017). Superhuman AI for heads-up no-limit poker: Libratus beats top professionals.
• Carnegie Mellon University (2017). Carnegie Mellon Artificial Intelligence Beats Top Poker Pros.
• Carnegie Mellon University (2017). Poker Play Begins in Brains Vs. AI: Upping the Ante.
• Jeremy Hsu, IEEE Spectrum (2017). AI Decisively Defeats Human Poker Players.
• Martin Zinkevich, Michael Johanson, Michael Bowling and Carmelo Piccione, NIPS (2007). Regret Minimization in Games with Incomplete Information.
• Michael Bowling, Neil Burch, Michael Johanson and Oskari Tammelin, Science (2015). Heads-up limit hold'em poker is solved.
• University of Alberta (2015). University of Alberta Researchers Solve Heads-up Limit Texas Hold 'em Poker.
• Carnegie Mellon University (2015). Pros Rake In More Chips Than Computer Program During Poker Contest.
• Jeremy Hsu, IEEE Spectrum (2015). Poker Pros Battle Artificial Intelligence to a Statistical Draw.
• Noam Brown, TED (2024). AI Won't Plateau if We Give It Time to Think.
• Noam Brown and Tuomas Sandholm, NIPS (2017). Safe and Nested Subgame Solving for Imperfect-Information Games.
• Neural Information Processing Systems Foundation (2017). NIPS 2017 Awards.
• Carnegie Mellon University (2019). Carnegie Mellon and Facebook AI Beats Professionals in Six-Player Poker.
• Emily Conover, Science News (2017). Winning against a computer isn't in the cards for poker pros.
• Matej Moravčík and colleagues, Science (2017). DeepStack: Expert-level artificial intelligence in heads-up no-limit poker.
• Matej Moravčík and colleagues, arXiv (2017). DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker.
• Amii (2017). DeepStack: First AI to Outplay Human Poker Pros.
• Noam Brown and Tuomas Sandholm, Science (2019). Superhuman AI for multiplayer poker.
• Carnegie Mellon University (2017). Carnegie Mellon Artificial Intelligence Beats Chinese Poker Players.
• Will Knight, MIT Technology Review (2017). A Top Poker-Playing Algorithm Is Cleaning Up in China.
• Tom Simonite, Wired (2019). A Poker-Playing Robot Goes to Work for the Pentagon.
• IJCAI (2019). Marvin Minsky Medal for Outstanding Achievements in AI.
• Noam Brown, Anton Bakhtin, Adam Lerer and Qucheng Gong, arXiv (2020). Combining Deep Reinforcement Learning and Search for Imperfect-Information Games.
• Martin Schmid and colleagues, arXiv (2021). Student of Games: A unified learning algorithm for both perfect and imperfect information games.
• Noam Brown (2026). Curriculum Vitae.
• Noam Brown (2026). Noam Brown, personal website.
• OpenAI (2024). OpenAI o1 System Card.