The approximating posterior distribution

As with the HMM, the approximating posterior distribution is chosen to have a factorial form

$$q(\boldsymbol{S}, \boldsymbol{\theta}) = q(\boldsymbol{S}) q(\boldsymbol{\theta}) = q(\boldsymbol{S}) \prod_i q(\theta_i).$$ (5.41)

The independent distributions for the parameters $\theta_i$ are all Gaussian with

$$q(\theta_i) = N(\theta_i;\; \overline{\theta}_i, \widetilde{\theta}_i)$$ (5.42)

where $\overline{\theta}_i$ and $\widetilde{\theta}_i$ are the variational parameters whose values must be optimised to minimise the cost function.

Because of the strong temporal correlations between the source values at consecutive time instants, the same approach cannot be applied to form $q(\boldsymbol{S})$. Therefore the approximation is chosen to be of the form

$$q(\boldsymbol{S}) = \prod_{k=1}^n \left[ q(s_k(1)) \prod_{t=1}^{T-1} q(s_k(t+1) \vert s_k(t)) \right]$$ (5.43)

where the factors are again Gaussian.

The distributions for $s_k(1)$ can be handled as before with

$$q(s_k(1)) = N(s_k(1);\; \overline{s}_k(1), \widetilde{s}_k(1)).$$ (5.44)

The conditional distribution $q(s_k(t+1) \vert s_k(t))$ must be modified slightly to include the contribution of the previous state value. Saving the notation $\widetilde{s}_k(t)$ for the marginal variance of $q(s_k(t))$, the variance of the conditional distribution is denoted with $\ensuremath{\overset{\raisebox{-0.3ex}[0.5ex][0ex]{\ensuremath{\scriptscriptstyle \,\circ}}}{s}}_k(t+1)$. The mean of the distribution,

$$\mu_{si}(t+1) = \overline{s}_k(t+1) + \breve{s}_k(t, t+1) [ s_k(t) - \overline{s}_k(t) ],$$ (5.45)

depends linearly on the previous state value $s_k(t)$. This yields

$$q(s_k(t+1) \vert s_k(t)) = N(s_k(t+1);\; \mu_{si}(t+1), \ensurema... ...ebox{-0.3ex}[0.5ex][0ex]{\ensuremath{\scriptscriptstyle \,\circ}}}{s}}_k(t+1)).$$ (5.46)

The variational parameters of the distribution are thus the mean $\overline{s}_k(t+1)$, the linear dependence $\breve{s}_k(t, t+1)$ and the variance $\ensuremath{\overset{\raisebox{-0.3ex}[0.5ex][0ex]{\ensuremath{\scriptscriptstyle \,\circ}}}{s}}_k(t+1)$. It should be noted that this dependence is only to the same component of the previous state value. The posterior dependence between the different components is neglected.

The marginal distribution of the states at time instant $t$ may now be evaluated inductively starting from the beginning. Assuming $q(s_k(t-1)) = N(s_k(t-1);\; \overline{s}_k(t-1), \widetilde{s}_k(t-1))$, this yields

$$\begin{split}q(s_k(t)) &= \int q(s_k(t) \vert s_k(t-1)) q(s_k... ...k(t) + \breve{s}_k^2(t-1, t) \widetilde{s}_k(t-1)). \end{split}$$ (5.47)

Thus the marginal mean is the same as the conditional mean and the marginal variances can be computed using the recursion

$$\widetilde{s}_k(t) = \ensuremath{\overset{\raisebox{-0.3ex}[0.5ex... ...ptscriptstyle \,\circ}}}{s}}_k(t) + \breve{s}_k^2(t-1, t) \widetilde{s}_k(t-1).$$ (5.48)