Skip to main content

Language Models for the Prediction of SARS-CoV-2 Inhibitors...

Publication Type
Conference Paper
Journal Name
The International Journal of High Performance Computing Applications
Publication Date
Page Numbers
1 to 16
Conference Name
SC 21: The International Conference for High Performance Computing, Networking, Storage, and Analysis
Conference Location
St. Louis, Missouri, United States of America
Conference Sponsor
Conference Date

The COVID-19 pandemic highlights the need for computational tools to automate and accelerate drug design for novel protein targets. We leverage deep learning language models to generate and score drug candidates based on predicted protein binding affinity. We pre-trained a deep learning language model (BERT) on ∼9.6 billion molecules and achieved peak performance of 603 petaflops in mixed precision. Our work reduces pre-training time from days to hours, compared to previous efforts with this architecture, while also increasing the dataset size by nearly an order of magnitude. For scoring, we fine-tuned the language model using an assembled set of thousands of protein targets with binding affinity data and searched for inhibitors of specific protein targets, SARS-CoV-2 Mpro and PLpro. We utilized a genetic algorithm approach for finding optimal candidates using the generation and scoring capabilities of the language model. Our generalizable models accelerate the identification of inhibitors for emerging therapeutic targets.