Training and Optimisation of Large Transformer Models - An ATLAS and CERN Use Case

Explore a conference talk detailing the implementation of a transformer model for Jet Flavour Tagging at the ATLAS Experiment using Kubeflow CERN machine learning cloud resources. Delve into the challenges of processing petabytes of data generated annually by the CERN Large Hadron Collider (LHC) experiments. Learn about the critical role of heavy flavour jets in CERN's physics program, including Higgs boson decay analyses and new particle searches. Discover how machine learning algorithms are instrumental in identifying jets originating from b- and c-quarks. Focus on the hyperparameter optimization process using Katib to enhance model performance on complex LHC datasets. Gain insights into streamlining workflows from notebook creation to distributed training, while effectively managing resource allocation and security in a large-scale scientific computing environment.