A robust family of algorithms generalizing the EM-algorithm for fitting parametric deterministic multi-trajectories observed in Gaussian noise and clutter is proposed. It is based on the M-estimation generalizing the Maximum Likelihood estimation in the M-step of the EM-algorithm. Simulation results of comparative performance of our and traditional EM-algorithm in noise and clutter are described.

Estimating the mean and the covariance matrix of an incomplete dataset and filling in missing values with imputed values is generally a nonlinear problem, which must be solved iteratively. The expectation maximization (EM) algorithm for Gaussian data, an iterative method both for the estimation of mean values and covariance matrices from incomplete datasets and for the imputation of missing val...

The celebrated expectation-maximization (EM) algorithm is one of the most widely used optimization methods in statistics. In recent years it has been realized that EM algorithm is a special case of the more general minorization-maximization (MM) principle. Both algorithms creates a surrogate function in the first (E or M) step that is maximized in the second M step. This two step process always...

This paper introduces variational expectation-maximization (VEM) algorithm for training Gaussian networks. Hyperparameters model distributions of parameters characterizing Gaussian mixture densities. The proposed algorithm employs a hierarchical learning strategy for estimating a set of hyperparameters and the number of Gaussian mixture components. A dual EM algorithm is employed as the initial...

3 Clustering algorithms 5 3.1 ML/MAP based clustering . . . . . . . . . . . . . . 5 3.1.1 Parameter estimation based on ML . . . . . . 5 3.1.2 Parameter estimation based on MAP . . . . . 5 3.1.3 Membership distribution . . . . . . . . . . . 6 3.1.4 Dissimilarity . . . . . . . . . . . . . . . . . 7 3.1.5 Clustering . . . . . . . . . . . . . . . . . . . 7 3.1.6 Relevance analysis . . . . . . . . ...

In the previous class we already mentioned that many of the most powerful probabilistic models contain hidden variables. We will denote these variables with y. It is usually also the case that these models are most easily written in terms of their joint density, p(d,y,θ) = p(d|y,θ) p(y|θ) p(θ) (1) Remember also that the objective function we want to maximize is the log-likelihood (possibly incl...

The EM algorithm is a very powerful optimization method and has reached popularity in many fields. Unfortunately, EM is only a local optimization method and can get stuck in suboptimal solutions. While more and more contemporary data/model combinations yield more than one optimum, there have been only very few attempts at making EM suitable for global optimization. In this paper we review the b...

This paper presented a hybrid optimal estimation algorithm for solving multi-level thresholding problems in image segmentation. The distribution of image intensity is modeled as a random variable, which is approximated by a mixture Gaussian model. The Gaussian’s parameter estimates are iteratively computed by using the proposed PSO + EM algorithm, which consists of two main components: (i) glob...

Learning with hidden variables is a central challenge in probabilistic graphical models that has important implications for many real-life problems. The classical approach is using the Expectation Maximization (EM) algorithm. This algorithm, however, can get trapped in local maxima. In this paper we explore a new approach that is based on the Information Bottleneck principle. In this approach, ...

• Σ is a set of output symbols, for example Σ = {a, b} • Θ is a vector of parameters. It contains three types of parameters: – πj for j = 1 . . . N is the probability of choosing state j as an initial state. Note that ∑N j=1 πj = 1. – aj,k for j = 1 . . . (N − 1), k = 1 . . . N , is the probability of transitioning from state j to state k. Note that for all j, ∑N k=1 aj,k = 1. – bj(o) for j = 1...