Training of physical neural networks

• 成果类型: Review
• 署名作者: Momeni, Ali; Rahmani, Babak; Scellier, Benjamin; Wright, Logan G.; Mcmahon, Peter L.; Wanjura, Clara C.; Li, Yuhang; Skalli, Anas; Berloff, Natalia G.; Onodera, Tatsuhiro; Oguz, Ilker; Morichetti, Francesco; del Hougne, Philipp; Le Gallo, Manuel; Sebastian, Abu; Mirhoseini, Azalia; Zhang, Cheng; Markovic, Danijela; Brunner, Daniel; Moser, Christophe; Gigan, Sylvain; Marquardt, Florian; Ozcan, Aydogan; Grollier, Julie; Liu, Andrea J.; Psaltis, Demetri; Alu, Andrea; Fleury, Romain
• 署名单位: Swiss Federal Institutes of Technology Domain; Ecole Polytechnique Federale de Lausanne; Microsoft; Microsoft United Kingdom; Yale University; Cornell University; Max Planck Society; University of California System; University of California Los Angeles; Universite Marie et Louis Pasteur; Universite de Technologie de Belfort-Montbeliard (UTBM); Centre National de la Recherche Scientifique (CNRS); University of Cambridge; Swiss Federal Institutes of Technology Domain; Ecole Polytechnique Federale de Lausanne; Polytechnic University of Milan; Centre National de la Recherche Scientifique (CNRS); CNRS - Institute for Engineering & Systems Sciences (INSIS); Universite de Rennes; Stanford University; Alphabet Inc.; DeepMind; Universite Paris Saclay; Centre National de la Recherche Scientifique (CNRS); Institut Polytechnique de Paris; Ecole Polytechnique; Thales Group; Universite PSL; College de France; Ecole Normale Superieure (ENS); Centre National de la Recherche Scientifique (CNRS); Sorbonne Universite; Thales Group; Centre National de la Recherche Scientifique (CNRS); Universite Paris Saclay; University of Pennsylvania; Swiss Federal Institutes of Technology Domain; Ecole Polytechnique Federale de Lausanne; City University of New York (CUNY) System; City University of New York (CUNY) System
• 刊物名称: Nature
• ISSN/ISSBN: 0028-3168
• DOI: 10.1038/s41586-025-09384-2
• 发表日期: 2025-09-04
• 页码: 53-61
• 关键词: backpropagation; algorithm

摘要

Physical neural networks (PNNs) are a class of neural-like networks that make use of analogue physical systems to perform computations. Although at present confined to small-scale laboratory demonstrations, PNNs could one day transform how artificial intelligence (AI) calculations are performed. Could we train AI models many orders of magnitude larger than present ones? Could we perform model inference locally and privately on edge devices? Research over the past few years has shown that the answer to these questions is probably yes, with enough research. Because PNNs can make use of analogue physical computations more directly, flexibly and opportunistically than traditional computing hardware, they could change what is possible and practical for AI systems. To do this, however, will require notable progress, rethinking both how AI models work and how they are trained-primarily by considering the problems through the constraints of the underlying hardware physics. To train PNNs, backpropagation-based and backpropagation-free approaches are now being explored. These methods have various trade-offs and, so far, no method has been shown to scale to large models with the same performance as the backpropagation algorithm widely used in deep learning today. However, this challenge has been rapidly changing and a diverse ecosystem of training techniques provides clues for how PNNs may one day be used to create both more efficient and larger-scale realizations of present-scale AI models.