Last modification: 27/04/99 Minimization methods in MLPfit

 

Training a Multi-Layer Perceptron consist in minimizing the error considered as a function of the weights. The minimization methods available in MLPfit are:

• Stochastic minimization. This is the method which is usually (wrongly) called "online backpropagation"
• Conjugate Gradient methods. Both the Polak-Ribiere and Fletcher-Reeves updating formulae are available
• The Broyden, Fletcher, Goldfarb, Shanno (BFGS) method.
• An hybrid linear - BFGS method.

The figure below compares the learning curves for the various methods on the problem of fitting the function x**2*sin(5xy) on [-1,1]*[-1,1] from 1681 examples. All curves stop after the same CPU time.