In March 2023, the Game Developers Conference (GDC), one of the biggest events for video game developers, was held in San Francisco. Last year, we showcased a one-on-one boss battle featuring a knight character controlled by an ML-based game AI. That was a single-agent system, but this year, we have expanded the scope to multi-agent systems, presenting a talk and demo called "Candy Clash." Using the Unity ML-Agents Toolkit, we developed the multi-agent system where dozens of rabbit characters work as a team, aiming to crack their opponent's egg. In this blog series, I'll explain how we developed this game demo. Part 1 provides a general overview of the demo. We hope that this blog series will interest many game developers in machine learning technology.
In the Candy Clash demo, numerous rabbit characters split into two teams and act according to the situation, aiming to crack each other's eggs. The rabbit characters' actions are selected by their assigned neural network (NN) models, and the game was developed to demonstrate how agents behave as a group. Below is a screenshot of the game.
Figure 1. Candy Clash demo
The objective of this demo is to either crack the opponent's egg or defeat all the opponent’s rabbits. The gauge at the top of the game screen shows the eggs' hit points (HP). Below that, a gauge shows the remaining number of rabbits for each team. Also, cannons fire towards areas with a high concentration of rabbits, either at regular intervals or when the user presses a button. The cannons are controlled by human-programmed logic rather than ML. This programming indirectly affects the game's outcome. At GDC, we ran this demo on a Pixel 7 Pro and achieved a performance of 60 fps with 100 ML agents.
ML-Agents Toolkit enables game developers to train intelligent game AI agents within games and simulations. You can create more realistic and engaging gameplay experiences because non-player characters (NPCs) can learn from their surroundings and react more naturally to player inputs. Agents are trained through Reinforcement Learning (RL) using data gathered from their environment. This learning enables users to improve over time and make decisions based on what they have learned. Our previous blog post introduces the basic mechanism used for ML-Agents. Unity’s official documentation gives more details.
Multi-agent systems involve multiple ML agents working together to achieve a common goal. This scenario can potentially solve problems that are difficult for single-agent systems. This approach can lead to more complex and dynamic gameplay experiences, because different agents can take on different roles and cooperate or compete with each other. For example:
By incorporating multi-agent systems, developers can create games that keep players engaged with novel experiences.
When developing multi-agent systems, there are several approaches. Your approach depends on:
When the game setting is simple, you might create multiple instances of a single agent. In more complex settings, you might want a centralized agent to control all the characters. In recent years, an approach called Centralized Training, Decentralized Execution framework has emerged. In this framework, agents share data during training to learn optimal actions. During inference, agents act independently without sharing data with each other. Another blog post explains how multi-agent works in Unity.
Figure 2. Various possible design approaches for multi-agents
In Candy Clash, there are 3 roles for the rabbit characters:
A planner agent dynamically selects the roles assigned to each rabbit character. Through this combination of planner and roles, the rabbits' behaviors change and adapt in real-time to various game situations.
Figure 3. Our approach for Candy Clash
In part 2, I explore agents’ design in more details.