Identification of movie encoding neurons enables movie recognition AI

• 成果类型: Article
• 署名作者: Hiramoto, Masaki; Cline, Hollis T.
• 署名单位: Scripps Research Institute
• 刊物名称: PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
• ISSN/ISSBN: 0027-14846
• DOI: 10.1073/pnas.2412260121
• 发表日期: 2024-11-26
• 关键词: wagon-wheel-illusion; receptive-fields; functional architecture; midbrain network; perception; neocognitron; plasticity; zebrafish; mechanism; MODEL

摘要

Natural visual scenes are dominated by spatiotemporal image dynamics, but how the visual system integrates movie information over time is unclear. We characterized optic tectal neuronal receptive fields using sparse noise stimuli and reverse correlation analysis. Neurons recognized movies of similar to 200- 600 ms durations with defined start and stop stimuli. Movie durations from start to stop responses were tuned by sensory experience though a hierarchical algorithm. Neurons encoded families of image sequences following trigonometric functions. Spike sequence and information flow suggest that repetitive circuit motifs underlie movie detection. Principles of frog topographic retinotectal plasticity and cortical simple cells are employed in machine learning networks for static image recognition, suggesting that discoveries of principles of movie encoding in the brain, such as how image sequences and duration are encoded, may benefit movie recognition technology. We built and trained a machine learning network that mimicked neural principles of visual system movie encoders. The network, named MovieNet, outperformed current machine learning image recognition networks in classifying natural movie scenes, while reducing data size and steps to complete the classification task. This study reveals how movie sequences and time are encoded in the brain and demonstrates that brain- based movie processing principles enable efficient machine learning.