The fundamental structure of local iterative techniques for solving unconstrained minimization problems is simple. A starting point is chosen; a direction of movement is prescribed according to some algorithm, and a line search or trust region approach is performed to determine an appropriate next step. The process is repeated at the new point and the algorithm continues until a local minimum is found (see Figure 5). Schematically, a model local minimizer method can be sketched as follows:
Figure 5: The Structure of a Line-Search Based Local Minimization Algorithm.
(See exercise 6.)