next up previous

2.5.1 Newton Methods Overview     continued...

In the ``classic'' Newton method, the Newton direction is used to update each previous iterate by the formula , until convergence. The reader may recognize the one-dimensional version of Newton's method for solving a nonlinear equation : . The analogous iteration process for minimizing is: . Note that the one-dimensional search vector is replaced by the Newton direction in the multivariate case. This direction is defined for nonsingular but its solution may be unstable. When is sufficiently close to a solution , quadratic convergence can be proven for Newton's method [16,23,45]. In practice, this means that the number of digits of accuracy in the solution is approximately doubled at every step! This rapid convergence can be seen from the program output for a simple one-dimensional application of Newton's method to finding the root of a (equivalently, solving or minimizing ) (see Table 3). See the linear algebra chaptergif for related details. Note in the double precision version the round-off in the last steps.