Error Bounds for Functional Approximation and Estimation Using Mixtures of Experts

We examine some mathematical aspects of learning unknown mappings with the Mixture of Experts Model MEM Speci cally we observe that the MEM is at least as powerful as a class of neural networks in a sense that will be made precise Upper bounds on the approximation error are established for a wide class of target functions The general theorem states that kf fnkp c n r d for f W r p L a Sobolev class over d and fn belongs to an n dimensional manifold of normalized ridge functions The same bound holds for the MEM as a special case of the above The stochastic error in the context of learning from i i d examples is also examined An asymptotic analysis establishes the limiting behavior of this error in terms of certain pseudo information matrices These results substantiate the intuition behind the MEM and motivate applications