User Guide

14 Understandin

Optimization Principles and Options

The optimizer uses gradient-based optimization algorithms that

use a finite difference method to approximate the gradients

(gradients are not known analytically). To implement finite

differencing, the optimizer:

Perturbs each parameter in turn from its current value by an

amount h.

Evaluates the function at the perturbed value.

Subtracts the old function value from the new.

Divides the result by h.

Note

There is a tradeoff. If h is too small, the difference

in function values is unreliable due to numerical

inaccuracies. However if h is too lar

e, the result is

a poor approximation to the true

radient.