It is reasonable to choose a search vector that will be a
* descent* direction; that is, a direction leading to function reduction.
A descent direction is defined as one along which the directional
derivative is negative:

When we write the approximation

we see that the negativity of the right-hand side guarantees that a lower function value can be found along for a sufficiently small .

Different methods are distinguished by their choice of search directions. Algorithms can be classified into nonderivative, gradient, and second-derivative (Newton) methods depending on the technique used to determine in Algorithm 2.1. These classes will be discussed in turn beginning in Section 2.3.