In this paper, the concept of the classical f -divergence for a pair of measures is extended to the mixed f -divergence for multiple pairs of measures. The mixed f -divergence provides a way to measure the difference between multiple pairs of (probability) measures. Properties for the mixed f -divergence are established, such as permutation invariance and symmetry in distributions. An Alexandro...

Rényi divergence is related to Rényi entropy much like information divergence (also called Kullback-Leibler divergence or relative entropy) is related to Shannon’s entropy, and comes up in many settings. It was introduced by Rényi as a measure of information that satisfies almost the same axioms as information divergence. We review the most important properties of Rényi divergence, including it...

Streaming data are dynamic in nature with frequent changes. To detect such changes, most methods measure the difference between the data distributions in a current time window and a reference window. Divergence metrics and density estimation are required to measure the difference between the data distributions. Our study shows that the Kullback-Leibler (KL) divergence, the most popular metric f...

K e y w o r d s P a r a m e t r i c measure, Nonparametric measure, Csiszg~r's f-divergence, Information measure. 1. I N T R O D U C T I O N There are several types of information divergence measures s tudied in l i te ra ture which compare two probabi l i ty d is t r ibut ions and have appl icat ions in informat ion theory, s tat is t ics and engineering. A convenient classification to differe...

In image and signal processing, the beta-divergence is well known as a similarity measure between two positive objects. However, it is unclear whether or not the distance-like structure of beta-divergence is preserved, if we extend the domain of the beta-divergence to the negative region. In this article, we study the domain of the beta-divergence and its connection to the Bregman-divergence as...

An accurate estimate of the divergence time between Native Americans is important for understanding the initial entry and early dispersion of human beings in the New World. Current methods for estimating the genetic divergence time of populations could seriously depart from a linear relationship with the true divergence for multiple populations of a different population size and significant pop...