We propose an information complexity-based regularization parameter selection method for solution of ill-conditioned inverse problems. The regularization parameter is selected to be the minimizer of the Kullback-Leibler (KL) distance between the unknown data-generating distribution and the fitted distribution. The KL distance is approximated by an information complexity (ICOMP) criterion develo...

We propose a stochastic framework to analyze and compare differences in human motions for applications in injury prevention, rehabilitation, sports training and performance research. Human motions are modeled as Hidden Markov Models and the differences between the motions are measured using the Kullback-Leibler distance metric. The distance metric is recomputed with degrees of freedom excluded ...

In this paper, we describe two approaches to the problem of encoding cortical waves of turtle visual cortex. The first approach relies on representing the response of individual pyramidal cells using various temporal scales (multiresolution analysis). In the second approach, we consider decomposing the visual cortex into various spatial grids. Each of these spatial grid contains several neurons...

An extension of the switching-state models (SSSM) that allows arbitrary number of components is presented. We introduce a Dirichlet process prior over the mixture components of the linear models. This prior allows the inference on the number of linear models to be put into the mixture. We develop a distance measure in the space of linear Kalman filters with the use of the Kullback-Leibler diver...

In this paper we investigate distance functions on finite state Markov processes that measure the behavioural similarity of non-bisimilar processes. We consider both probabilistic bisimilarity metrics, and tracebased distances derived from standard Lp and Kullback-Leibler distances. Two desirable continuity properties for such distances are identified. We then establish a number of results that...

Regression problems under Poisson variability arise in many different scientific areas such as, for examples, astrophysics and medical imaging. This article considers the problem of bandwidth selection for kernel smoothing of Poisson data. Its first contribution is the proposal of a new bandwidth selectionmethod that aims to choose the bandwidth thatminimizes theKullback–Leibler (KL)distance be...

In this paper identification of laryngeal disorders using cepstral parameters of human voice is researched. Mel-frequency cepstral coefficients (MFCCs), extracted from audio recordings of patient’s voice, are further approximated, using various strategies (sampling, averaging, and clustering by Gaussian mixture model). The effectiveness of similarity-based classification techniques in categoriz...

Bregman divergences are generalizations of the well known Kullback Leibler divergence. They are based on convex functions and have recently received great attention. We present a class of “squared root metrics” based on Bregman divergences. They can be regarded as natural generalization of Euclidean distance. We provide necessary and sufficient conditions for a convex function so that the squar...

In this paper we propose a Bayesian, information theoretic approach to dimensionality reduction. The approach is formulated as a variational principle on mutual information, and seamlessly addresses the notions of sufficiency, relevance, and representation. Maximally informative statistics are shown to minimize a Kullback-Leibler distance between posterior distributions. Illustrating the approa...