Tightening convex relaxations of trained neural networks

• 成果类型: Article; Early Access
• 署名作者: Carrasco, Pablo; Munoz, Gonzalo
• 署名单位: Universidad de Chile
• 刊物名称: MATHEMATICAL PROGRAMMING
• ISSN/ISSBN: 0025-5610; 1436-4646
• DOI: 10.1007/s10107-026-02363-z
• 发表日期: 2026-05-26
• 关键词: Neural networks; Convex envelopes; non-convex optimization; ENVELOPES

摘要

The non-convex nature of trained neural networks has created significant obstacles in their incorporation into optimization models. In this context, Anderson et al. (2020) provided a framework to obtain the convex hull of the graph of a piecewise linear convex activation function composed with an affine function; this effectively convexifies activations such as the ReLU together with the affine transformation that precedes it. In this article, we contribute to this line of work by developing a recursive formula that yields a tight convexification for the composition of an activation with an affine function for a wide scope of activation functions, namely, convex or S-shaped. Our approach can be used to efficiently compute separating hyperplanes or determine that none exists in various settings, including non-polyhedral cases.