MAS Emergence
Reference
- Altmann, P., Schönberger, J., Illium, S., et al. 2024. Emergence in Multi-agent Systems: A Safety Perspective. International Symposium on Leveraging Applications of Formal Methods, Springer Nature Switzerland Cham, 104–120.
Emergent effects can occur in multi-agent systems (MAS), where decision-making is decentralized and based on local information. These effects may range from minor deviations in behavior to catastrophic system failures. To formally define these phenomena, we identify misalignments between the global inherent specification (the true specification) and its local approximation (e.g., the configuration of distinct reward components or observations). Leveraging established safety concepts, we develop a framework for understanding these emergent effects. To demonstrate the resulting implications, we examine two highly configurable gridworld scenarios, where inadequate specifications lead to unintended behavior deviations when derived independently. Acknowledging that a global solution may not always be practical, we propose adjusting the underlying parameterizations to mitigate these issues, thereby improving system alignment and reducing the risk of emergent failures.
[Altmann et al. 2024]
