Particle physics

Particle physics is the field which studies high energy sub-atomic particles in order to understand some of the open mysteries of the universe. Our group's activities in particle physics are focussed at the CERN accelerator complex in Geneva, which houses huge experiments underground which exploit the most energetic particle physics beam in the world. 

In the coming decades, new CERN detectors will be built  that will push the frontier of particle physics research forward. For example, the SHiP experiment will harness the extreme intensity from the CERN beam complex to search for so-called 'hidden' particles that would only interact with ordinary matter very weakly. Another example is the FCC

Designing a particle physics detector is a very complex multi-objective optimisation task. A typical particle physics detector is over 10 meters tall, consists of several sub-detectors and millions of electronic channels. The decision space of determining the specifications and locations of these components is vast, meaning that a brute force optimisation method is impractical.

To complicate issues further, a real detector must respect a similarly complex set of engineering constraints. For example, an experiment must be housed inside a finite volume, space must be allocated for cooling and easy access must be afforded for maintanance and repairs of key components. Optimising the performance of such a detector while obeying these constraints is very difficult and is currently therefore done in a manual iterative procedure.

Our approach is to use a branch of machine learning known as reinforcement learning. Reinforcement learning is an unsupervised learning technique in order to optimise behavior. The scientist specifies what the agent should do via a reward function, which determines what the agent outcomes should or should not achieve. The agent is then trained to maximise the total reward it obtains, which is done by optimising a policy. The policy is defined inside the framework of a Markov Decision Process, which determines which actions to take based on the current information (state). Reinforcement learning is a powerful tool for automatic decision making in complex systems. It is particularly advantageous at dealing with systems with a vast space of possible actions, where brute force methods would be unfeasible. 

In detector design, the actions employed by a reinforcement learning agent are to step-by-step build up a detector, with the reward given by the performance of a detector once the agent has decided to stop. Below is an illustration of this with layers representing detector planes that must be placed optimally to detect particles with the best performance. During the virtual detector construction, the agent makes observations about the current state and uses that to define where to place the next detector layer. Reinforcement learning has several advantages over other detector design methods with machine learning: it can take non-differentiable actions and therefore can interact with the state of the art simulation rather than needing to use differentiable surrogates.  It also can deal with complex boundary conditions that arise in real world systems such as engineering constraints which are critical to take into account in order to propose designs that can be used in real world use cases.

Designing a particle physics detector is done years (sometimes decades) in advance of the eventual construction, which means that the optimal design must be decided under a large level of uncertainty from various different sources. Even fundamental constraints such as the monetary budget and the space for the detector can be unknown, as well as the ever present complication that the real data will never be the same as simulation that is used to determine the performance (and therefore agent reward). Reoptimising a design when new information on these uncertain aspects is highly inefficient and time consuming, meaning that often the final implementation is out of date and therefore sub-optimal. 

Our approach to deal with this to directly feed in uncertain aspects of the design system into the optimisation itself. This can be done in several ways. One is to train the agent on an ensemble of environments which are varied based on the current uncertainty, meaning that when new information arises, a new optimal design can be chosen without retraining the agent, vastly speeding up the process. The other way to is to learn the distribution of the total reward during training and optimise for a certain quantile of that distribution. This is known as distributional reinforcement learning as is currently being pursued actively as a potential method to overcome this issue.