next up previous

2.4.5 Nonlinear Conjugate Gradient

Extensions of the linear CG method to nonquadratic problems have been developed and extensively researched [23,52,61]. In the common variants, the basic idea is to avoid matrix operations altogether and simply express the search directions recursively as

for , with . The new iterates for the minimum point can then be set to

where is the step length. In comparing this iterative procedure to the linear CG of Algorithm 2.2, we note that for a quadratic function. The parameter above is chosen so that if f were a convex quadratic and is the exact one-dimensional minimizer of f along , the nonlinear CG reduces to the linear CG method and terminates in at most n steps in exact arithmetic.