Extensions of the linear CG method to nonquadratic problems have been developed and extensively researched [23,52,61]. In the common variants, the basic idea is to avoid matrix operations altogether and simply express the search directions recursively as
for , with . The new iterates for the minimum point can then be set to
where is the step length. In comparing this iterative procedure to the linear CG of Algorithm 2.2, we note that for a quadratic function. The parameter above is chosen so that if f were a convex quadratic and is the exact one-dimensional minimizer of f along , the nonlinear CG reduces to the linear CG method and terminates in at most n steps in exact arithmetic.