Expectation-Maximization is a general algorithm to find maximum likelihood solutions for probabilistic models that include latent variables. Usually the assumption is that a joint distribution defined over observed and latent variables is more tractable than the marginal distribution over the observed variables and therefore easier to work with when modeling complex marginal distributions.
With observed variables , latent variables and a joint distribution parametrized by the data log-likelihood (which we want to optimize wrt ) is given by
The typical assumption is that the joint distribution takes a simple form that is easy to optimize whereas the data log-likelihood is difficult to optimize directly. However, this strategy still ends up with a complicated expression for the data log-likelihood because of the summation inside the logarithm necessitating indirect optimization strategies.
If the true values of were known, the complete-data log-likelihood would be straightforward to compute but that misses the point of latent variables as assumed unobservables. The best we can do is to compute the expected complete-data log-likelihood under a posterior belief (current knowledge) about the latent variables given current parameter values. This expected log-likelihood (which we later optimize) is given by
EM alternates the above expectation step with a maximization step that updates the parameters in turn, with the combined steps guaranteed to increase the (incomplete/marginal) data log likelihood.
This two-step process is repeated until convergence. This entire procedure is optimal and corresponds to maximizing a lower bound on the data likelihood.