Missing Values in Hierarchical Nonlinear Factor Analysis

The properties of hierarchical nonlinear factor analysis (HNFA) recently introduced by Valpola and others [3] are studied by reconstructing values. The variational Bayesian learning algorithm for HNFA has linear computational complexity and is able to infer the structure of the model in addition to estimating the parameters. To compare HNFA with other methods, we continued the experiments with speech spectrograms in [1] comparing nonlinear factor analysis (NFA) with linear factor analysis (FA) and with the self-organising map. Experiments suggest that HNFA lies between FA and NFA in handling nonlinear problems. Furthermore, HNFA gives better reconstructions than FA and it is more reliable than NFA. Introduction •Assume data X to be a set of real valued vectors •Missing values are components of the data that are not observed –Example: supervised learning can be seen as reconstructing missing values Training data data Test Supervised learning Desired outputs Data Unsupervised learning with missing values