In this paper we present a new approach for clustering data. The clustering metric used is the normalized crosscorrelation, also known as similarity, instead of the traditionally used Euclidean distance. The main advantage of this metric is that it depends on the signal shape rather than its amplitude. Under an assumption of an exponential probability model that has several desirable properties...

Estimating parameters of mixture models is a typical application of the expectation maximization (EM) algorithm. For the family of multivariate exponential power (MEP) distributions, which is a generalization of the well known multivariate Gaussian distribution, we introduce an approximative EM algorithm, and a probabilistic variant called stochastic EM algorithm, which provides a significant s...

Aggelos K, Katsaggelos, MEMBERSPIE Northwestern University McCormick School of Engineering and Applied Science Department of Electrical Engineering and Computer Science Evanston, Illinois 60208-3118 E-mail: aggk@eecs,nwu.edu Abstract. Previous work has demonstrated the effectiveness of the expectation-maximization algorithm to restore noisy and blurred singlechannel images and simultaneously id...

Last week, we saw how we could represent clustering with a probabilistic model. In this model, called a Gaussian mixture model, we model each datapoint x i as originating from some cluster, with a corresponding cluster label y i distributed according to p(y), and the corresponding distribution for that cluster given by a multivariate Gaussian: p(x|y = k) =

Common-sense physical reasoning is an essential ingredient for any intelligent agent operating in the real-world. For example, it can be used to simulate the environment, or to infer the state of parts of the world that are currently unobserved. In order to match real-world conditions this causal knowledge must be learned without access to supervised data. To solve this problem, we present a no...

We introduce a novel framework for clustering that combines generalized EM with neural networks and can be implemented as an end-to-end differentiable recurrent neural network. It learns its statistical model directly from the data and can represent complex non-linear dependencies between inputs. We apply our framework to a perceptual grouping task and empirically verify that it yields the inte...

We present Fitness Expectation Maximization (FEM), a novel method for performing ‘black box’ function optimization. FEM searches the fitness landscape of an objective function using an instantiation of the well-known Expectation Maximization algorithm, producing search points to match the sample distribution weighted according to higher expected fitness. FEM updates both candidate solution para...

So far, we discussed clustering algorithms that involve a hard assignment of each datapoint to a cluster, typically based on its proximity to other points in that cluster. However, simply assigning points to the nearest cluster is not always adequate to capture more complex structure. For example, the lecture slides show an example where one cluster is much larger and less dense than another. W...

Scalable cluster analysis addresses the problem of processing large data sets with limited resources, e.g., memory and computation time. A data summarization or sampling procedure is an essential step of most scalable algorithms. It forms a compact representation of the data. Based on it, traditional clustering algorithms can process large data sets efficiently. However, there is little work on...

We present an Expectation-Maximization algorithm for the fractal inverse problem: the problem of fitting a fractal model to data. In our setting the fractals are Iterated Function Systems (IFS), with similitudes as the family of transformations. The data is a point cloud in R with arbitrary dimension H. Each IFS defines a probability distribution on R , so that the fractal inverse problem can b...