`What's new?' is an interesting and broadening eternal question, but one which, if pursued exclusively, results only in an endless parade of trivia and fashion, the silt of tomorrow. I would like, instead, to be concerned with the question `What is best?', a question which cuts deeply rather than broadly, a question whose answers tend to move the silt downstream.Robert M. Pirsig
Mathematical optimization is the formal title given to the branch of computational science that seeks to answer the question `What is best?' for problems in which the quality of any answer can be expressed as a numerical value. Such problems arise in all areas of mathematics, the physical, chemical and biological sciences, engineering, architecture, economics, and management, and the range of techniques available to solve them is nearly as wide.
The purpose of this chapter is not to make the reader an expert on all aspects of mathematical optimization but to provide a broad overview of the field. The beginning sections introduce the terminology of optimization and the ways in which problems and their solutions are formulated and classified. Subsequent sections consider the most appropriate methods for several classes of optimization problems, with emphasis placed on powerful, versatile algorithms well suited to optimizing functions of many variables on high performance computational platforms. High-performance computational issues, such as vectorization and parallelization of optimization codes, are beyond the scope of this chapter. This field is still in its infancy at this time, with general strategies adopted from numerical linear algebra codes (see chapter on Linear Algebra ). However, the last section contains a brief overview of possible approaches.