In this section we outline some basic techniques
involving *deterministic* algorithms
for finding *local* minima of
multivariate functions whose arguments are continuous and on which
no restrictions are imposed. For constrained problems, techniques
are based on those for unconstrained problems, and we mention only
general approaches to them at the end of this section. It should be
emphasized that finding the *global* minimum is an entirely different,
and more challenging, problem which will not be
addressed here.
Basically, stochastic methods
are better suited at this time for
large-scale global optimization (see Figure 4) and some appropriate
algorithms will be outlined in Section 5.

Figure 4 The Structure of Local and Global Minimization Algorithms. View Figure

For comprehensive presentations on deterministic optimization techniques for multivariate functions, we refer the reader to excellent textbooks [45][29][22][16][9][4][1], some recent volumes and reviews [59][48][23]. The outline in this section is not intended to provide full algorithmic details of all available methods; rather it exposes key algorithmic concepts and modules, so that the reader could consult specialized literature for further details. Further background details can also be obtained from the linear algebra chapter.