2.2.1 Descent Directions

next up previous
Next: 2.2.2 Line Search and Up: 2.2 Basic Structure Of Previous: 2.2 Basic Structure Of

2.2.1 Descent Directions


It is reasonable to choose a search vector that will be a descent direction; that is, a direction leading to function reduction. A descent direction is defined as one along which the directional derivative is negative:


When we write the approximation


we see that the negativity of the right-hand side guarantees that a lower function value can be found along for a sufficiently small .

Different methods are distinguished by their choice of search directions. Algorithms can be classified into nonderivative, gradient, and second-derivative (Newton) methods depending on the technique used to determine in Algorithm 2.1. These classes will be discussed in turn beginning in Section 2.3.