For one recorded time $y_i$, the missing route could be any component. The probability of observing that time therefore adds the mutually exclusive route explanations. Route $j$ contributes its usage probability $\pi_j$ multiplied by the probability density its travel-time pattern assigns to $y_i$:

Independent delivery records multiply these summed densities. Taking a logarithm converts the product across deliveries into a sum, but the addition across possible routes remains inside each logarithm. The parameters for all components are consequently coupled inside one observed-data objective:

The easier form becomes visible by writing a route indicator. Let $z_{ij}=1$ when delivery $i$ used component $j$, and let it be zero otherwise. If these indicators were recorded alongside the times, $(\mathbf y,\mathbf z)$ would be the complete data. Its log-likelihood separates the route contributions:

The missing indicators cannot be replaced by unsupported hard labels. Stage 1 of EM instead supplies provisional component parameters. Those parameters make it possible to calculate a probability for every missing indicator—the expectation required by stage 2.

The GMM laboratory reports that its observed-data log-likelihood rises after each complete cycle, even though the M-step maximizes $Q$ rather than the observed log-likelihood directly. To see why, consider any proposed probability distribution $q(\mathbf c)$ over the missing assignments. Two terms combine into an objective that sits at or below the observed log-likelihood: the average complete-data log-likelihood under $q$ and the uncertainty of $q$ itself.

The uncertainty term is the entropy $H(q)=-\mathbb E_q[\log q(\mathbf c)]$. Their sum is the evidence lower bound, abbreviated ELBO and written $\mathcal F(q,\Psi)$:

The gap between the observed log-likelihood and this lower objective measures how differently $q$ and the model's current assignment posterior distribute probability across the same possible completions. This nonnegative mismatch is the Kullback–Leibler divergence, or KL divergence:

In the E-step, set $q^{(t)}(\mathbf c)=p(\mathbf c\mid\mathbf y,\Psi^{(t)})$. The two assignment distributions then match, their KL divergence is zero, and the ELBO touches the observed log-likelihood at the current model. In the M-step, keep $q^{(t)}$ fixed and raise the ELBO by changing $\Psi$. Because the observed log-likelihood equals that raised ELBO plus another nonnegative gap, it cannot finish below the previous likelihood.

Component labels are only one kind of latent variable. In another model, the observations may be noisy measurements of an unknown function, so its values at the observed inputs are latent. In a third model, the unknown object may be a probability distribution whose atoms supply a growing collection of component parameters. EM applies when these hidden quantities produce a useful expected complete-data objective and the resulting expectation and maximization calculations can be performed exactly or approximated honestly.

Finite mixtures and distributions over functions randomize different objects

A GMM is one probability distribution for an observation such as a delivery time. It first chooses one of finitely many components and then draws the observation from that component's Gaussian distribution. A Gaussian process, or GP, assigns probability to a complete function $x\mapsto f(x)$ so that the function values at every finite input list have a multivariate Gaussian joint distribution. The word “Gaussian” therefore operates at different levels.

A finite list of GP values gives EM a finite doorway

Let $f\sim\mathcal{GP}(m,k_\phi)$ and observe noisy outputs $y_i=f(x_i)+\varepsilon_i$. Although the random function spans a complete input domain, the likelihood of the measured dataset depends on the finite latent vector $\mathbf f_X=(f(x_1),\ldots,f(x_n))^{\mathsf T}$. Kernel and noise hyperparameters collected in $\phi$ can be fitted by integrating over this latent vector:

For ordinary GP regression with Gaussian observation noise, both the latent posterior and marginal likelihood are available in closed form, so directly optimizing the log marginal likelihood is usually the standard route. EM is valid but not automatically the most efficient organization of the same calculation. With non-Gaussian observations, such as class labels, the latent function posterior is generally intractable; an approximate E-step based on expectation propagation, Laplace approximation, or variational inference produces an approximate EM-type method.

A growing mixture changes exact EM into approximate posterior inference

A Dirichlet process, or DP, assigns probability to complete probability measures $G$ over component parameters. In a Dirichlet-process mixture, draw one such measure, draw a component parameter for each observation from it, and then draw the observation from that component:

Repeated values of $\theta_i$ form clusters, while the number of occupied clusters can grow with the data. The hidden structure now includes assignments and a countably unbounded collection of possible atoms and weights. An exact finite responsibility table is therefore unavailable without another representation or approximation.

One deterministic route truncates a stick-breaking representation at a large finite level and chooses a tractable probability distribution $q$ over assignments, stick weights, and component parameters. Coordinate updates alternate between expected assignment probabilities and updated variational factors. This is often described as variational EM, but its target is an ELBO that approximates the posterior evidence, not the exact observed-data likelihood used by ordinary finite-mixture EM. Markov chain Monte Carlo supplies a different route by sampling the latent partitions and component parameters instead of optimizing a variational approximation.