In this paper, we introduce the notion of relative probability measure spaces by the concept of observer. The notions of relative entropy and relative conditional entropy of countable σ-algebras are defined and studied. Using the suggested concept of relative entropy we define the relative entropy of a dynamical system and we prove some of its ergodic properties. Finally it is proved that, the ...

In this paper, we present a new image thresholding technique which uses the relative entropy (also known as the Kullback-Leiber discrimination distance function) as a criterion of thresholding an image. As a result, a gray level minimizing the relative entropy will be the desired threshold. The proposed relative entropy approach is different from two known entropy-based thresholding techniques,...

This paper concentrates on a new procedure which experimentally recognises gears and bearings faults of a typical gearbox system using a least square support vector machine (LSSVM). Two wavelet selection criteria Maximum Energy to Shannon Entropy ratio and Maximum Relative Wavelet Energy are used and compared to select an appropriate wavelet for feature extraction. The fault diagnosis method co...

In the previous lecture, we saw that PAC learning is not enough to model real learning problems. We may not desire or even may not be able to find a hypothesis that is consistent with the training data. So we introduced a more general model in which the data is generated as a pair (x, y) from an unknown distribution D. And we defined the generalization error to be errD(h) = Pr(x,y)∼D[h(x) 6= y]...

The above inequality implies a logarithmic Sobolev inequality for q. We get an explicit lower bound for the logarithmic Sobolev constant of q under the assumptions that: (i) the local specifications of q satisfy logarithmic Sobolev inequalities with constants ρi, and (ii) they also satisfy some condition expressing that the mixed partial derivatives of the Hamiltonian of q are not too large rel...