In the ``classic'' Newton method, the Newton direction is
used to update each previous iterate by the formula , until convergence. The reader
may recognize the one-dimensional version of Newton's method for
solving a nonlinear equation
: .
The analogous iteration process for minimizing
is: .
Note that the one-dimensional search vector is
replaced by the Newton direction in the multivariate
case. This direction is defined for nonsingular but its solution
may be unstable.
When is sufficiently close to a solution ,
quadratic convergence can be proven for Newton's method
[16,23,45].
In practice, this means that the number of digits of accuracy in the
solution is approximately doubled at every step! This rapid
convergence can be seen
from the program output for a simple one-dimensional application
of Newton's method to finding the root of **a** (equivalently,
solving or minimizing )
(see Table 3).
See the linear algebra chapter for related details.
Note in the double
precision version the round-off in the last steps.