A Generalist Model for Particle Tracking
Date:
The application of deep learning models in particle tracking is pervasive. Graph Neural Networks are applied in track finding, Deep learning models in resolving merged tracks, Transformers in jet flavor tagging, and GravNet or its variations in one-short track finding. The current practice is to design one deep learning model for one task. However, these tasks are so deeply intertwined that factorizing them will inevitably lose information and hurt overall performance. We propose to design an intermediate generalist model that offers learned detector encodings for various particle tracking tasks.
Inspired by the BERT model, which is the pre-training of deep bidirectional transformers for language understanding, we propose to train deep bidirectional transformers to encode the detector modules for particle tracking. Similarly, we define two surrogate tasks for the training. One task is to predict masked hits in a particle track, and the other is to predict if track A has higher momentum than track B. The goal is to obtain novel representations of detector modules and to use those representations for various downstream tasks, including outlier/hole detection and track generation.
In this talk, we will present the preliminary results of training the BERT model for particle tracking and show the first application of the novel detector module representations for hole detection and track extrapolation. This study can be potentially extended to encode the whole particle detectors, including calorimeters and muon spectrometers, for more downstream particle reconstruction tasks.