The cancellation of the last term in Eq. (15) for independent
r.v.'s motivates
the concept of the *covariance*.

If and are independent, then .
However, it is possible to have
even if and are not independent.
It should be noted that the
covariance can be negative. A related quantity that arises often in
statistical analysis is the *correlation coefficient*,
which is a convenient measure of the degree
to which two r.v.'s are correlated (or anti-correlated).

It is easily shown that .