There is a special class of problems, examples of which are particularly common in the fields of operations research and engineering design, in which the task is to find the optimum permutation of some control variables. These are known as combinatorial optimization problems. The most famous example is the traveling salesman problem (TSP) in which the shortest cyclical itinerary visiting N cities is sought. The solutions to such problems are usually represented as ordered lists of integers (indicating, for example in the TSP, the cities in the order they are to be visited), and they are, of course, constrained, since not all integer lists represent valid solutions. These and other classifications are summarized in Table 1. Table 2 lists application examples from the wide range of fields where optimization is employed and gives their classifications under this taxonomy.
Section 2 details methods appropriate to unconstrained continuous univariate/multivariate problems, and Section 3 mentions methods appropriate to constrained continuous multivariate problems. Section considers methods appropriate to (mixed) integer multivariate problems, and Section 4 discusses what to do if none of these methods succeed. In general, the optimization of the complex functions that occur in many practical applications is difficult. However, with persistence and resourcefulness solutions can often be obtained.