Likelihood ratio of unidentifiable models and multilayer neural networks
成果类型：
Article
署名作者：
Fukumizu, K
署名单位：
Research Organization of Information & Systems (ROIS); Institute of Statistical Mathematics (ISM) - Japan
刊物名称：
ANNALS OF STATISTICS
ISSN/ISSBN：
0090-5364
DOI：
10.1214/aos/1056562464
发表日期：
2003
页码：
833-851
关键词：
摘要：
This paper discusses the behavior of the maximum likelihood estimator (MLE), in the case that the true parameter cannot be identified uniquely. Among many statistical models with unidentitiability, neural network models are the main concern of this paper. It is known in some models with unidentifiability that the asymptotics of the likelihood ratio of the MLE has an unusually larger order. Using the framework of locally conic models put forth by Dacunha-Castelle and Gassiat as a generalization of Hartigan's idea, a useful sufficient condition of such larger orders is derived. This result is applied to neural network models, and a larger order is proved if the true function is given by a smaller model. Also, under the condition that the model has at least two redundant hidden units, a log n lower bound for the likelihood ratio is derived.