next up previous

2.4.5 Nonlinear Conjugate Gradient     continued...

Three of the best known formulas for are titled Fletcher-Reeves (FR), Polak-Ribière (PR), and Hestenes-Stiefel (HS) after their developers. They are given by the following formulas:

Interestingly, the last two formulas are generally preferred in practice, though the first has better theoretical global convergence properties. In fact, very recent research has focused on combining these practical and theoretical properties for construction of more efficient schemes. The simple modification of

for example, can be used to prove global convergence of this nonlinear CG method, even with inexact line searches [26]. A more general condition on , including relaxation of its nonnegativity, has also been derived.