Protein language models learn evolutionary statistics of interacting sequence motifs

• 成果类型: Article
• 署名作者: Zhang, Zhidian; Wayment-Steele, Hannah K.; Brixi, Garyk; Wang, Haobo; Kern, Dorothee; Ovchinnikov, Sergey
• 署名单位: Harvard University; Massachusetts Institute of Technology (MIT); Swiss School of Public Health (SSPH+); Swiss Federal Institutes of Technology Domain; Ecole Polytechnique Federale de Lausanne; Brandeis University; Howard Hughes Medical Institute; Brandeis University; Harvard University; Harvard University
• 刊物名称: PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
• ISSN/ISSBN: 0027-12052
• DOI: 10.1073/pnas.2406285121/-/DCSupplemental
• 发表日期: 2024-11-05
• 关键词: recognition; DESIGN

摘要

Protein language models (pLMs) have emerged as potent tools for predicting and designing protein structure and function, and the degree to which these models fundamentally understand the inherent biophysics of protein structure stands as an open question. Motivated by a finding that pLM-based structure predictors erroneously predict nonphysical structures for protein isoforms, we investigated the nature of sequence context needed for contact predictions in the pLM Evolutionary Scale Modeling (ESM-2). We demonstrate by use of a categorical Jacobian calculation that ESM-2 stores statistics of coevolving residues, analogously to simpler modeling approaches like Markov Random Fields and Multivariate Gaussian models. We further investigated how ESM-2 stores information needed to predict contacts by comparing sequence masking strategies, and found that providing local windows of sequence information allowed ESM-2 to best recover predicted contacts. This suggests that pLMs predict contacts by storing motifs of pairwise contacts. Our investigation highlights the limitations of current pLMs and underscores the importance of understanding the underlying mechanisms of these models.