A whole-slide foundation model for digital pathology from real-world data
成果类型：
Article
署名作者：
Xu, Hanwen; Usuyama, Naoto; Bagga, Jaspreet; Zhang, Sheng; Rao, Rajesh; Naumann, Tristan; Wong, Cliff; Gero, Zelalem; Gonzalez, Javier; Gu, Yu; Xu, Yanbo; Wei, Mu; Wang, Wenhui; Ma, Shuming; Wei, Furu; Yang, Jianwei; Li, Chunyuan; Gao, Jianfeng; Rosemon, Jaylen; Bower, Tucker; Lee, Soohee; Weerasinghe, Roshanthi; Wright, Bill J.; Robicsek, Ari; Piening, Brian; Bifulco, Carlo; Wang, Sheng; Poon, Hoifung
署名单位：
Microsoft; University of Washington; University of Washington Seattle; University of Washington; University of Washington Seattle
刊物名称：
Nature
ISSN/ISSBN：
0028-4399
DOI：
10.1038/s41586-024-07441-w
发表日期：
2024-06-06
关键词：
摘要：
Digital pathology poses unique computational challenges, as a standard gigapixel slide may comprise tens of thousands of image tiles(1-3). Prior models have often resorted to subsampling a small portion of tiles for each slide, thus missing the important slide-level context(4). Here we present Prov-GigaPath, a whole-slide pathology foundation model pretrained on 1.3 billion 256x256 pathology image tiles in 171,189 whole slides from Providence, a large US health network comprising 28 cancer centres. The slides originated from more than 30,000 patients covering 31 major tissue types. To pretrain Prov-GigaPath, we propose GigaPath, a novel vision transformer architecture for pretraining gigapixel pathology slides. To scale GigaPath for slide-level learning with tens of thousands of image tiles, GigaPath adapts the newly developed LongNet(5) method to digital pathology. To evaluate Prov-GigaPath, we construct a digital pathology benchmark comprising 9 cancer subtyping tasks and 17 pathomics tasks, using both Providence and TCGA data(6). With large-scale pretraining and ultra-large-context modelling, Prov-GigaPath attains state-of-the-art performance on 25 out of 26 tasks, with significant improvement over the second-best method on 18 tasks. We further demonstrate the potential of Prov-GigaPath on vision-language pretraining for pathology(7,8) by incorporating the pathology reports. In sum, Prov-GigaPath is an open-weight foundation model that achieves state-of-the-art performance on various digital pathology tasks, demonstrating the importance of real-world data and whole-slide modelling.