next up previous

2.4.1 Derivative Programming     continued...

Derivatives are tested using a Taylor expansion of f around a given point . The following Taylor series is formulated at where is a scalar:

where and are the gradient and Hessian, respectively, evaluated at . If only the gradient routines are tested, the second-order Taylor term YHY is set to zero, and the truncation error is . Our test is performed by computing this Taylor approximation at smaller and smaller values of and checking whether the truncation errors are as expected: and if the approximation is correct up to the gradient and Hessian terms, respectively. At every step we half and test if indeed the truncation errors decrease as they should (i.e., if the error corresponding to is , the error for should be if the gradient is correct, and if the Hessian is also correct.)