An unbeatable poker bot that offers insight into the future of video game AI

Whether playing a diversion in Red Dead Redemption 2 or a complete game in itself, poker fans are routinely frustrated by a carefree AI for Kenny Rogers' timeless advice on holding, folding, etc. Some robots at the table can be bluffed by any hand; others will never be blown away. Some go to bed at the slightest provocation, while others follow with cards even worse than the ones you have. Players have as much visibility on the behavior of their opponents on the processor as their cards, that is to say none.

It's for this reason that research published by high-level problem-solving specialists from Facebook and Carnegie Mellon University caught my attention earlier this week. Do not expect this to appear in a video game any time soon. But their artificial intelligence in Pluribus poker is significant in that, through a game, computer engineers have once again imitated a behavior previously accepted as being human in nature. And it's bluffing.

"This is true for many innovations in artificial intelligence," said Thursday Noam Brown, researcher at Facebook and co-creator of the bot. "Many of the things we assume are limited to human capabilities are really possible with AI.

"In the '50s, people thought that playing chess was a very human thing that computers can not do," Brown said. "Then people thought that playing Go at grand master level is a very human thing that an AI could not do, and then people thought that bluffing is a very human thing that an AI could not do, and we see that, in fact, an AI can bluff better than any living human being. "

Facebook

The first scientist represented by Brown's research is accompanied by some qualifiers. Scientists have already used poker to study AI behavior and learning. In 2015, researchers at the University of Alberta built a pokerbot that was fundamentally unbeatable in the two-player Texas Hold'em limit. And, of course, applications as common as video games have put many AI participants at a poker table, especially at the height of poker madness at the turn of the century.

AIs that people like me know better are not as analytical as the frequency of a type of behavior applied to a given situation, whether it is the overall strength of the hand or the first on the flop. For years, poker simulators have introduced artificial intelligence sliders for aggressive and conservative games, the utility of which is to train a human to play disciplined hands, regardless of what the other player is doing.

This is before we come to bluffing, which is considered a human art form because of the tendency or tendency of other players to give up their trust or lack of confidence in their hands. The Coresoft World Championship Poker Series for PlayStation 2 even featured a bluffing mini-game that attempted to make it a more viable tactic. But most often, you get races where the opponents call everything, lift inexplicably or stand in empty hands as if it were a pair of valets. These games have not been entertaining in a sustainable way, as most players would end up bored or impatient.

Pluribus is different because it analyzes, more or less, the effect of bluffing – that is, wagering with a weak hand – rather than selling competitors to the strength of what it holds. "The bot does not consider him a deceiver or liar, he just considers it as" It 's the action that will bring me the most money in this situation, "Brown said.

Pluribus, created by Brown and his colleague CMU, Tuomas Sandholm, looks like an artificial intelligence chess that would represent a theoretical and hypothetical progress of many steps. The difference is that Brown and Sandholm's bot look only two or three strokes ahead. This short-term goal has made his stunning trends completely opaque to the five human professionals that Pluribus has relentlessly defeated over 10,000 hands.

This raises somehow an existential question: what defines the bluff more? The behavior or the result?

Brown did not want to answer that. His interest in poker, as a research community, dates back to his undergraduate studies at Rutgers University, about 15 years ago. "All this idea that there is this mathematical strategy in the game, you know, this perfect strategy that, if you can play it, no one will beat you," Brown fascinated.

Professional gamers have touted the systems for different games, with different levels of intellectual rigor and honesty, for years. Poker seems to be immune from system criticism because it depends on incomplete or imperfect information, as opposed to blackjack, go or chess, where the information is known to all participants (when the blackjack croupier can not act independently).

But in a way, Brown has proven that a strategy can be developed to win poker steadily ($ 1,000 an hour) – it's just a human who is not capable of the instant calculation needed to play it.

"This is one of the interesting things about this AI, it does not fit its opponent," said Brown. "He has his strategy. It's fixed, it does not change what it plays based on how humans play. All this idea that there might be such a strategy in the game, I found really fascinating and that's what really pushed me to study it more. It was a bit mystical, in a sense, there is this strategy that we know to exist, but we can not find it. "

A press release regarding Pluribus extolled the near-hardware nature of the hardware that powered it – a 64-core server with less than 512 GB of RAM, running on eight days, developed artificial intelligence. The researchers estimated that using cloud servers to train the program would cost only $ 150.

But do not expect Pluribus to come into the virtual poker rooms and start throwing everyone into the trash, or train a generation of great human players pocketing an hour. Brown said that it was not expected to turn Pluribus into a kind of commercial book. The AI ​​is just a proof of concept, whose lessons will help Brown and other researchers to tackle computer behavior in even more complex situations.

For example, autonomous cars. "One of the things we've mentioned to reporters is the ability to apply that to something like navigating in traffic with an autonomous car," Brown said.

It also comes down to another obvious video game application, and another artificial intelligence familiar to many video game fans: race car drivers, whose processor counterparts are not much more sophisticated than speed, a optimal line and the space they will leave to other drivers.

"Motorsports games are a great example of how this work can be applied in the future, because it's a multi-agent interaction, there are multiple players, and there's also a certain level of hidden information, "Brown said. "A lot of artificial intelligence, from what I understand, do not use very principled techniques, they are more hard-coded, more specific to the type of game they are. This of course makes debugging and understanding of what is happening easier.

"But as we develop these fundamental AI techniques, I think we'll start seeing them penetrate the computer gaming industry and become more important," he said. added. "I would not be surprised. This is one of the first places it really enters industrial applications. "

List file is Polygon's column on the intersection of sports and video games.