The third edition of the Reinforcement Learning for Autonomous Accelerators (RL4AA) workshop, hosted by our partner DESY, brought together experts in control systems, accelerator physics, and machine learning.
The workshop focused on the application of reinforcement learning (RL) in operating complex research infrastructures, including particle accelerators and fusion devices.
The RF2.0 project was represented by colleagues from the Karlsruhe Institute of Technology (KIT), with contributions from Evangelos Matzoukas and Malik Muhammad Abdullah, who are working on machine learning–based control systems at the Karlsruhe Accelerator (KARA) and the Energy Lab. The event provided a valuable platform for technical exchange and interdisciplinary collaboration, with a strong emphasis on autonomy, reliability, and sustainability in future facility operation
Evangelos Matzoukas presented a poster outlining his current work at KARA under RF2.0. His project focuses on the use of Bayesian optimization and machine learning for accelerator tuning — from microtron injection to booster — aiming to enable more efficient and autonomous beam control. This work naturally connects with reinforcement learning, exploring how decision-making agents can be trained to perform complex tuning tasks in real time, adapt to changing beam conditions, and continuously optimize system performance.
The research aims to reduce manual intervention and support the long-term goal of autonomous control in high-performance scientific infrastructure. The use of data-driven approaches, in conjunction with physics-based modeling, continues to be a central pillar of the RF2.0 strategy.
Malik Muhammad Abdullah also took part in a hands-on coding challenge, where participants were divided into groups and were tasked with training a reinforcement learning agent to focus the beam in the ARES accelerator. The challenge emphasized the importance of hyperparameter tuning and reward engineering in achieving effective control. Malik, together with his teammates, developed the solution that was awarded 1st prize, marking a highlight of the workshop experience.
The RL4AA workshop underscored the importance of combining domain knowledge with modern machine learning approaches to build:
- Robust, interpretable, and safe control architectures
- Scalable solutions deployable across various experimental infrastructures
- Interdisciplinary workflows that bridge accelerator operations and AI research
Participants also had the opportunity to tour DESY’s accelerator halls, providing tangible context to the complexity of the systems under discussion. Hosting the workshop at DESY this year offered participants direct exposure to one of Europe’s leading accelerator centers, fostering a deeper understanding of real-world challenges and opportunities. The next RL4AA workshop is planned to be held at University of Liverpool in 2026, continuing the tradition of rotating venues to strengthen collaboration across institutions.
