Aquarium
Reference
- Kölle, M., Erpelding, Y., Ritz, F., Phan, T., Illium, S., and Linnhoff-Popien, C. 2024. Aquarium: A Comprehensive Framework for Exploring Predator-Prey Dynamics through Multi-Agent Reinforcement Learning Algorithms. arXiv preprint arXiv:2401.07056.
Recent advances in multi-agent reinforcement learning have enabled the modeling of complex interactions between agents in simulated environments. In particular, predator-prey dynamics have garnered significant interest, and various simulations have been adapted to meet unique requirements. To avoid further time-intensive development efforts, we introduce Aquarium, a versatile multi-agent reinforcement learning environment designed for studying predator-prey interactions and emergent behavior. Aquarium is open-source and seamlessly integrates with the PettingZoo framework, allowing for a quick start using established algorithm implementations. It features physics-based agent movement on a two-dimensional, edge-wrapping plane. Both the agent-environment interactions (observations, actions, rewards) and environmental parameters (agent speed, prey reproduction, predator starvation, and more) are fully customizable. In addition to providing a resource-efficient visualization, Aquarium supports video recording, facilitating a visual understanding of agent behavior.
To showcase the environment’s capabilities, we conducted preliminary studies using proximal policy optimization (PPO) to train multiple prey agents to evade a predator. Consistent with existing literature, we found that individual learning leads to worse performance, while parameter sharing significantly improves coordination and sample efficiency. [Kölle et al. 2024]
