The cancellation of the last term in Eq. (15) for independent r.v.'s motivates the concept of the covariance.
If and are independent, then . However, it is possible to have even if and are not independent. It should be noted that the covariance can be negative. A related quantity that arises often in statistical analysis is the correlation coefficient, which is a convenient measure of the degree to which two r.v.'s are correlated (or anti-correlated).
It is easily shown that .