New Page 2

RBF Networks Trainer

Download (1028k) - RBF_Trainer written in C++, which works in Windows (unpack to a directory and run RBF_Trainer.exe)

Download (1064k) instruction fro RBF_Trainer

NNT - Neural Networks Trainer

Neural network training software for networks with arbitrarily connected neurons

You are free to download and to use the software:

(1) Download (45k) - MATLAB version, which works only in MATLAB environment

(unpack to a directory, open this directory from MATLAB, and run NNT.m)

(2) Download (177,099k) - MATLAB version, which works independently form MATLAB

(unpack to a directory and run NNT.exe)

(3) Download (1470k) - NBN-2.08 written in C++, which works in Windows (unpack to a directory and run NBN 2.08.exe)

(4) Download (1330k) - NBN-2.10 written in C++, which works in Windows (unpack to a directory and run NBN 2.10.exe)

(5) Download (397k) - NBN vanilla version in MATLAB (can be used and modified as long the original algorithm is cited - see reference page)

The following algorithms are currently implemented in MATLAB versions (1) and (2):

EBP - Traditional Error Back Propagation

NBN - Neuron by Neuron algorithm which is a modification of the Levenberg Marquet algorithm for arbitrarily connected neurons ACN.

SA - Self Aware algorithm which is modification of NBN. It evaluates the progression of the algorithms training and determines if the algorithm is failing to converge. If the algorithm begins to fail the weights are reset and another trial is attempted. In this situation the program displays its progress to the user as dotted red line on the display and begins again. The algorithm continues to attempt to solve the problem until either it is successful or the user cancels the process.

ESA - Enhanced Self Aware algorithm which is also modification of NBN algorithm and is used in order to increase chances for convergence. The modification was made to the Jacobian matrix in order to allow the algorithm to be much more successful in solving very difficult problems with deep local minima. It also is aware of its current solving status and will reset when necessary.

F- ESA – is another modification of NBN algorithm where an alternative method for calculating the Jacobian matrix is used. The calculation of Jacobian is unique in the sense that only feed-forward calculations are needed. This approach is then paired with ESA algorithm.

Evolutionary Gradient – newly developed algorithm, which evaluates gradients form randomly generated weight sets and use gradient information to generate new population of weights. This is a hybrid algorithm which combines the use of random populations with an approximated gradient approach. Like standard methods of evolutionary computation, the algorithm is better suited for avoiding local minima when compared to common gradient methods such as EBP. What sets the method apart is the use of an approximated gradient which is calculated with each population. By generating successive populations in the gradient direction, the algorithm is able to converge much faster than other forms of evolutionary computation. This combination of gradient and evolutionary methods essentially offers the best of both worlds.

The following algorithms are currently implemented in the C++ version (3):

EBP: This is EBP algorithm with traditional forward-backward computation; for EBP algorithm, it may work a little bit faster than forward-only computation. Now it is only used for standard MLP networks. EBP algorithm converges slowly, but it can be used for huge patterns training.

LM: This is LM algorithm with traditional forward-backward computation; for LM (and NBN) algorithm, the improved forward-only computation performs faster training than forward-backward computation for networks with multiple outputs. Now it is also only used for standard MLP networks. LM (and NBN) algorithm converges much faster than EBP algorithm for small and media sized patterns training.

NBN: This is NBN algorithm with forward-backward computation. NBN algorithm is developed based on LM algorithm, but it can handle arbitrarily connected neuron (ACN) networks, also, the convergence is improved [1][2].

EBP, forward-only: This is EBP algorithm with forward-only computation [3]. It can work on arbitrarily connected neuron networks.

NBN, forward-only: This is NBN algorithm with forward-only computation. It can handle arbitrarily connected neuron networks and, as mentioned above, it works faster than “NBN” algorithm, especially for networks with multiple outputs.

NBN, improved: This is a newly developed second order algorithm, implemented with forward-only computation, so it can handle arbitrarily connected neuron networks. In this algorithm, Hessian matrix is inverted only one time per iteration, so this algorithm is supposed to compute faster than LM (and NBN) algorithm which may have several times Hessian matrix inversion per iteration. The train ability (convergence) is also improved in this algorithm.

The first two algorithms only work for limited networks, while the rest of algorithms can handle arbitrarily connected neuron (ACN) networks. In spite of “NBN, improved” (need further improvement), “EBP, forward-only” and “NBN, forward-only” are recommended for training. The former one could be used for huge patterns and networks training, while the latter one is suitable for small and media sized patterns training.

The following algorithms are currently implemented in the C++ version (4):

References

B. M. Wilamowski, H. Yu, “Improved Computation for Levenberg Marquardt Training,” IEEE Trans. on Neural Networks, vol. 21, no. 6, pp. 930-937, June 2010. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5451114
B. M. Wilamowski and H. Yu, “Neural Network Learning Without Backpropagation," IEEE Trans. on Neural Networks, 2010 (Early Access) http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5580116
B. M. Wilamowski, ” Neural Network Architectures and Learning algorithms”, IEEE Industrial Electronics Magazine, vol 3, no 4, pp.56-63, Nov 2009. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5352485
B. M. Wilamowski, N. J. Cotton, O. Kaynak, G. Dundar, "Computing Gradient Vector and Jacobian Matrix in Arbitrarily Connected Neural Networks," IEEE Trans. on Industrial Electronics, vol. 55, no. 10, pp. 3784-3790, Oct 2008 http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4602720
J. D. Hewlett, B. M. Wilamowski, G. Dundar, "Optimization Using a Modified Second-Order Approach With Evolutionary Enhancement," IEEE Trans. on Industrial Electronics, vol. 55, no. 9, pp. 3374-3380, Sept 2008 http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4559389
H. Yu and B. M. Wilamowski, “Fast and efficient and training of neural networks,” in Proc. 3nd IEEE Human System Interaction Conf. HSI 2010, Rzeszow, Poland, May 13-15, 2010, pp. 175-181. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5514571
B. M. Wilamowski, " Challenges in Applications of Computational Intelligence in Industrial Electronics ISIE10 - International Symposium on Industrial Electronics, Bari, Italy, July 4-7, 2010, pp. 15-22.
N. J. Cotton and Bogdan M. Wilamowski “Compensation of Sensors Nonlinearity with Neural Networks”, 24th IEEE International Conference on Advanced Information Networking and Applications 2010, pp. 1210-1217, April 2010. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5474850
H. Yu and B.M. Wilamowski “Efficient and Reliable Training of Neural Networks” IEEE Human System Interaction Conference, HSI 2009, Catania. Italy, May 21-23, 2009, pp. 109 – 115. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5090963

H. Yu and B.M. Wilamowski “C++ Implementation of Neural Networks Trainer” 13-th IEEE Intelligent Engineering Systems Conference, INES 2009, Barbados, April 16-18,2009, pp. 257-262.  http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4924772