DeepMind's AI Agents Exceed "Human" Gameplay in Quake III

AI agents continue to accumulate victories in the video game world. Last week, the bots of OpenAI were playing Dota 2; This week, it's Quake III with a team of researchers from Google's DeepMind subsidiary that successfully train agents capable of beating humans during a flag-catching party.

As we have seen with previous examples of AI playing video games, the challenge here is to train an agent capable of navigating in a complex 3D environment with imperfect information. DeepMind researchers have used an AI training method that is also becoming standard: reinforcement learning, which essentially consists of large scale trial and error training.

Agents receive no instruction on how to play the game, but simply fight against themselves until They develop the strategies necessary to win. This usually means that a version of the AI ​​agent is playing against an identical clone. DeepMind has added depth to this formula by forming an entire cohort of 30 agents to introduce a "diversity" of play styles. How many games does it take to train an AI this way? Nearly half a million, each lasting five minutes.

As always, it is impressive to see how such a conceptually simple technique can generate complex behavior on the part of robots. DeepMind agents have not only learned the basic rules for capturing the flag (catch the flag of your opponents from their base and return it to yours before they do the same thing to you), but also strategies like keep your own flag. and following his teammates so you can hang on to the enemy.

To make the challenge more difficult for agents, each game was played on a brand new procedurally generated card. This allowed the bots to learn strategies that only worked on one card.

Unlike the OpenAI bots Dota 2 DeepMind's agents also did not have access to raw numeric data on sets of numbers representing information like the distance between them. opponents and health bars. Instead, they learned to play just by looking at the visual input of the screen, the same as a human. However, this does not necessarily mean that the DeepMind bots have faced a bigger challenge; Dota 2 is a much more complex game than the bare-bones version of Quake III that was used in this search.

To test the capabilities of AI agents, DeepMind held a tournament, with only two-player bots teams, only humans, and a mix of bots and humans standing up against others. The bot-only teams were the most successful, with a 74% chance of winning. This compares to 43 prior probabilities for average human players, and 52% probability for strong human players. So: clearly AI agents are the best players.

Credit: DeepMind

However, it is worth noting that the higher the number of DeepMind bots in a team. high, worse it is. A team of four DeepMind robots had a 65% probability of gain, suggesting that if the AI ​​agents of the researchers learned some elements of cooperative play, they are not necessarily up to the task. a more complex team dynamics.

As always with research like this, the goal is not to beat humans to video games, but to find new ways to teach agents to navigate complex environments while pursuing a common goal. In other words, it is about teaching collective intelligence – something that has (despite many evidence to the contrary) is integral to the success of the team. humanity as a species. Capture the flag is just a proxy for the biggest games to come.