A Note on Gradient Based Learning in Vector Quantization Using Differentiable Kernels for Hilbert and Banach Spaces

Supervised and unsupervised prototype based vector quantization frequently are proceeded in the Euclidean space. In the last years, also non-standard metrics became popular. For classification by support vector machines, Hilbert or Banach space representations are very successful based on so-called kernel metrics. In this paper we give the mathematical justification that gradient based learning in prototype-based vector quantization is possible by means of kernel metrics instead of the standard Euclidean distance. We will show that an appropriate handling requires differentiable universal kernels defining the kernel metric. This allows an prototype adaptation in the original data space but equipped with a metric determined by the kernel. This approach avoids the Hilbert space representation as known for support vector machines. Moreover, we give prominent examples for differentiable universal kernels based on information theoretic concepts. Machine Learning Reports http://www.techfak.uni-bielefeld.de/∼fschleif/mlr/mlr.html 2 A Note on Gradient Based Learning in Vector Quantization Using Di erentiable Kernels for Hilbert and Banach Spaces Thomas Villmann ∗ and Sven Haase Computational Intelligence Group, University of Applied Sciences Mittweida, Technikumplatz 17, 09648 Mittweida, Germany,