Let and be independent zero-mean real Gaussian vectors of respective length and with respective invertible covariance matrices and . Let be a full-rank matrix and define by

If we are given the vector we know that will be Gaussian with mean and covariance .

However suppose we are given and wish to find the conditional distribution of . Here we may think of as a hidden variable and as the observed variable. In this case the result is a little more involved. If we recover the results of this earlier blog post, is jointly Gaussian and so is Gaussian with mean

and covariance

(Here is the cross-covariance of and , while .)

Using the fact that , , ,

and , (2) and (3) become

In this post we also derive the following alternative expressions (also described here) which the covariances appear as inverse matrices.

Note that in the scalar case with variances and (5) and (7) become the identity

In the case where is a scalar and is a diagonal matrix, is a weighted sum of the elements of vector and (8) becomes the inverse of a sum of inverses of variances (inverse-variance weighting).

One can check algebraically that the expressions (4),(6) and (5),(7) are equivalent in the matrix case, or we may proceed as follows.

Let be the covariance matrix of the joint vector . Then since we have and

Then as the Gaussian vector is zero-mean with covariance , the joint pdf of and is proportional to .

The key step now is to make use of the following identity (also see this explanation) based on completion of squares:

where and are matrices, defined similarly to and in the scalar equation

We will show that and ( is the Schur complement of in also discussed in this previous blog post).

The second term in the right side of (11) is proportional to the pdf of (being Gaussian) , so the first term must be proportional to the conditional pdf . We are left to find and .

From (11), is the top-left block of while is the top-right block of . To find these blocks, consider the block matrix equation

This is the same as the system of equations

Multiplying (13) by gives

Subtracting this from (14) gives

Since the coefficients of and in (15) are the bottom-left and bottom-right blocks of respectively. We may then write and so from (15)

Also we have from (13)

Using (16) this becomes

Note that analogous to (16) we could have written

Comparing coefficients of , of (18) and (19) gives

and

In the same way that , , which for is simply as we saw before.

Hence from (20),

and from the -coefficient of (18),

Hence as desired, and equations (6)-(8) have been verified.

## Leave a Reply