We present two novel bounds for the classification error that, at the same time, can be used as practical training criteria. Unlike the bounds reported in the literature so far, these novel bounds are based on a strict distinction between the true but unknown distribution and the model distribution, which is used in the decision rule. The two bounds we derive are the squared distance and the Ku...

In this paper we prove the optimality of an aggregation procedure. We prove lower bounds for aggregation of model selection type of M density estimators for the Kullback-Leibler divergence (KL), the Hellinger’s distance and the L1-distance. The lower bound, with respect to the KL distance, can be achieved by the on-line type estimate suggested, among others, by Yang (2000a). Combining these res...

Under external load excitation, damage such as breathing cracks and bolt loosening will cause structural time domain acceleration to have nonlinear features. To solve the problem of identification, a identification method based on Kullback–Leibler (KL) distance model residuals is proposed in this paper. First, an autoregressive (AR) order was selected using autocorrelation function (ACF) Akaike...

We develop two new multivariate statistical dependence measures. The first, based on the Kullback-Leibler distance, results in a single value that indicates the general level of dependence among the random variables. The second, based on an orthonormal series expansion of joint probability density functions, provides more detail about the nature of the dependence. We apply these dependence meas...