Deep Discrete Encoders: Identifiable Deep Generative Models for Rich Data with Discrete Latent Layers

• 成果类型: Article
• 署名作者: Lee, Seunghyun; Gu, Yuqi
• 署名单位: Columbia University
• 刊物名称: JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
• ISSN/ISSBN: 0162-1459; 1537-274X
• DOI: 10.1080/01621459.2025.2587922
• 发表日期: 2026-01-02
• 页码: 194-208
• 关键词: Deep belief network; Directed graphical model; Identifiability; Interpretable artificial intelligence; Representation learning; COGNITIVE DIAGNOSTIC MODELS; likelihood; selection

摘要

In the era of generative AI, deep generative models (DGMs) with latent representations have gained tremendous popularity. Despite their impressive empirical performance, the statistical properties of these models remain underexplored. DGMs are often overparameterized, non-identifiable, and uninterpretable black boxes, raising serious concerns when deploying them in high-stakes applications. Motivated by this, we propose interpretable deep generative models for rich data types with discrete latent layers, called Deep Discrete Encoders (DDEs). A DDE is a directed graphical model with multiple binary latent layers. Theoretically, we propose transparent identifiability conditions for DDEs, which imply progressively smaller sizes of the latent layers as they go deeper. Identifiability ensures consistent parameter estimation and inspires an interpretable design of the deep architecture. Computationally, we propose a scalable estimation pipeline of a layerwise nonlinear spectral initialization followed by a penalized stochastic approximation EM algorithm. This procedure can efficiently estimate models with exponentially many latent components. Extensive simulation studies for high-dimensional data and deep architectures validate our theoretical results and demonstrate the excellent performance of our algorithms. We apply DDEs to three diverse real datasets with different data types to perform hierarchical topic modeling, image representation learning, and response time modeling in educational testing. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.