This paper details the differential and numeric properties of two measures of entropy, Shannon entropy and Kullback-Leibler distance, applicable for the unit root hypothesis. It is found that they are differentiable functions of the degree of trending in any included deterministic component and of the correlation of the underlying innovations. Moreover, Shannon entropy is concave in these, and ...

By subjecting a dynamical system to a series of short pulses and varying several time delays, we can obtain multidimensional characteristic measures of the system. Multidimensional Kullback-Leibler response function (KLRF), which are based on the Kullback-Leibler distance between the initial and final states, are defined. We compare the KLRF, which are nonlinear in the probability density, with...

In this paper, we develop a general classification framework called Kullback-Leibler Boosting, or KLBoosting. KLBoosting has following properties. First, classification is based on the sum of histogram divergences along corresponding global and discriminating linear features. Second, these linear features, called KL features, are iteratively learnt by maximizing the projected Kullback-Leibler d...

In music similarity and in the related task of genre classification, a distance measure between Gaussian mixture models is frequently needed. We present a comparison of the KullbackLeibler distance, the earth movers distance and the normalized L2 distance for this application. Although the normalized L2 distance was slightly inferior to the Kullback-Leibler distance with respect to classificati...

A class of statistical distance measures and their spectral counterparts are presented. They have strong physical foundations since they are based on the combinatorial law leading to Bose-Einstein statistics in statistical physics. It is shown that these distance measures are very closely related to the recently introduced Jensen-Shannon divergence measure. The Kullback-Leibler number is found ...

Window profiles of amino acids in protein sequences are used to describe the amino acid environment. The relative entropy or Kullback-Leibler distance derived from these profiles is used as a measure of dissimilarity for comparison of amino acids and secondary structure conformations. Distance matrices of amino acid pairs at different conformations are obtained, which display a non-negligible d...

The concept of information distance in non-commutative setting is re-considered. Additive information, such as Kullback-Leibler divergence, is defined using convex functional with gradient having the property of homomorphism between multiplicative and additive subgroups. We review several geometric properties, such as the logarithmic law of cosines, Pythagorean theorem and a lower bound given b...

In this article, a method based on a non-parametric estimation of the Kullback–Leibler divergence using a local feature space is proposed for synthetic aperture radar (SAR) image change detection. First, local features based on a set of Gabor filters are extracted from both preand post-event images. The distribution of these local features from a local neighbourhood is considered as a statistic...

We provide background information to allow a heuristic understanding of two types of criteria used in selecting a model for making inferences from ringing data. The first type of criteria (e.g., AIC, AIC, QAIC and TIC) are estimates of (relative) Kullback-Leibler information or-distance and attempt to select a good approximating model for inference, based on the Principle of Parsimony. The seco...