Steffen Illium

PhD. in Comp. Science
AI Consultant & Researcher


Project Resources

logo

Robot Arm Icon In collaboration with Fraunhofer IKS, the AI-Fusion project addressed the critical challenge of understanding and ensuring safety in multi-agent reinforcement learning (MARL) systems. Emergence, defined as the arising of complex, often unpredictable, system-level dynamics from local interactions between agents and their environment, was a central focus due to its implications for system safety and reliability.

Project: AI-Fusion
Partner: Fraunhofer Institute for Cognitive Systems (IKS)
Duration: 2022 - 2023
Objective: To investigate the detection and mitigation of potentially unsafe emergent behaviors in complex systems composed of multiple interacting AI agents, particularly in scenarios involving heterogeneous agents (e.g., mixed-vendor autonomous systems).

To facilitate research into these phenomena, key contributions included the development of specialized simulation tools:

1. High-Performance MARL Simulation Environment:

  • A flexible and efficient simulation environment was developed in Python, adhering to the Gymnasium (formerly Gym) API specification.
  • Purpose: Designed specifically for training and evaluating reinforcement learning algorithms in multi-agent contexts prone to emergent behaviors.
  • Features:
    • Modularity: Supports diverse scenarios through configurable modules and configurations.
    • Observation/Action Spaces: Handles complex agent interactions, including per-agent observations and sequential/multi-agent action coordination.
    • Performance: Optimized for efficient simulation runs, enabling extensive experimentation.

2. Unity-Based Demonstrator Unit:

  • A complementary visualization tool was created using the Unity engine.
  • Purpose: Allows for the replay, inspection, and detailed analysis of specific simulation scenarios and agent interactions.
  • Utility: Aids researchers in identifying and understanding the mechanisms behind observed emergent dynamics.
  • View Demonstrator on GitHub
Diagram illustrating the concept of emergence from interactions between agents and environment
Conceptual relationship defining emergence in multi-agent systems.

This project involved close collaboration with industry-focused researchers, software development adhering to modern standards, and deep investigation into the theoretical underpinnings of emergence and safety in MARL systems. The developed tools provide a valuable platform for continued research in this critical area.

Reference

  1. Altmann, P., Schönberger, J., Illium, S., et al. 2024. Emergence in Multi-agent Systems: A Safety Perspective. International Symposium on Leveraging Applications of Formal Methods, Springer Nature Switzerland Cham, 104–120.

Related posts