For computational efficiency, the products in step (2) can generally be computed satisfactorily by the following finite-difference design of gradients at the expense of only one additional gradient evaluation per inner iteration:
where h is a suitably chosen small number, such as
The paths of TN minimization by the package TNPACK of Schlick & Fogelson [56,57] for Rosenbrock's and Beale's functions are shown in Figure 12 and Figure 16. Note again how efficiently the paths trace the valley toward the minimum and appear even more direct than QN.
For discussions on practical ways of choosing , factoring when the problem does not guarantee that is positive definite by the modified Cholesky factorization [29,58,60], and performing efficient Hessian/vector products, see [55,56,57].
(See exercise 6.)