3 The Expectation-Maximization algorithm 7 3.1 Jointly-non-concave incomplete log-likelihood . . . . . . . . . . . 7 3.2 (Possibly) Concave complete data log-likelihood . . . . . . . . . . 8 3.3 The general EM derivation . . . . . . . . . . . . . . . . . . . . . 10 3.4 The E& M-steps . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.5 The EM algorithm . . . . . . . . . . . . . . . . . . ...

This introduction to the expectation–maximization (EM) algorithm provides an intuitive and mathematically rigorous understanding of EM. Two of the most popular applications of EM are described in detail: estimating Gaussian mixture models (GMMs), and estimating hidden Markov models (HMMs). EM solutions are also derived for learning an optimal mixture of fixed models, for estimating the paramete...

The Expectation-Maximization (EM) algorithm is a hill-climbing approach to finding a local maximum of a likelihood function [7, 8]. The EM algorithm alternates between finding a greatest lower bound to the likelihood function (the “E Step”), and then maximizing this bound (the “M Step”). The EM algorithm belongs to a broader class of alternating minimization algorithms [6], which includes the A...

In lifetime analysis of electric transformers, the maximum likelihood estimation has been proposed with the EM algorithm. However, it is not clear whether the EM algorithm offers a better solution compared to the simpler Newton-Raphson algorithm. In this paper, the first objective is a systematic comparison of the EM algorithm with the Newton-Raphson algorithm in terms of convergence performanc...

We investigate in this research report the rigid registration of a set of points with a surface for computer-guided oral implants surgery. We rst formulate the Iterative Closest Point (ICP) algorithm as a Maximum Likelihood (ML) estimation of the transformation and the matches. Then, considering matches as a hidden random variable, we show that the ML estimation of the transformation alone lead...

We present a noise-injected version of the Expectation-Maximization (EM) algorithm: the Noisy Expectation Maximization (NEM) algorithm. The NEM algorithm uses noise to speed up the convergence of the EM algorithm. The NEM theorem shows that injected noise speeds up the average convergence of the EM algorithm to a local maximum of the likelihood surface if a positivity condition holds. The gener...

Tony Robinson Cambridge University Engineering Department Cambridge CB2 IPZ England [email protected] The speech waveform can be modelled as a piecewise-stationary linear stochastic state space system, and its parameters can be estimated using an expectation-maximisation (EM) algorithm. One problem is the initialisation of the EM algorithm. Standard initialisation schemes can lead to poor forma...

In order to realize an input-output relation given by noise-contaminated examples, it is e ective to use a stochastic model of neural networks. A model network includes hidden units whose activation values are not speci ed nor observed. It is useful to estimate the hidden variables from the observed or speci ed input-output data based on the stochastic model. Two algorithms, the EM and em-algor...

Decentralized supply chain management is found to be significantly relevant in today’s competitive markets. Production and distribution planning is posed as an important optimization problem in supply chain networks. Here, we propose a multi-period decentralized supply chain network model with uncertainty. The imprecision related to uncertain parameters like demand and price of the final produc...

The Expectation-Maximization (EM) algorithm is a popular tool for determining maximum likelihood estimates (MLE) when a closed form solution does not exist. It is often used for parametric density estimation, that is, to estimate the parameters of a density function when knowledge of the parameters is equivalent to knowledge of the density. The most famous case is the Gaussian distribution, whi...