next up previous

Exercise 2; Effects of regularization.

As an exercise in regularization, calculate the singular values for the matrix A, where A is . Look at how the values are distributed and notice that there is a large jump between the first few values and the other values. Recalculate the solution of the system, this time using the SVD program, and zero out the entry that correspond to the smallest singular value.

You will find, perhaps to your surprise, that the solution now gives you reasonable results (at least more reasonable than your first, unregularized solution). What you have just done is a form of regularization known as truncated singular value decomposition (TSVD). In effect, you have applied a filter to the system, which filters out the high frequency noise. At the same time, you are also filtering out useful information. Basically, you are giving up some information for an added degree of smoothness of the solution. Unfortunately, this method doesn't work well for all ill-conditioned matrices (you might try to figure out for which cases the truncated SVD method would work well and for which cases it would not). Other methods have been developed which try to regularize the system, not by completely removing the singular values, but by modifying them so that they can contribute to the information content. Such methods were developed by the Russian mathematician Tikhonov [16].