Documentation: class simpleNet v0.8



General purposes

The C++ class simpleNet is (better: should be) an easy to use implementation of a simple artificial neural net. It provides functions to build up, train and run any 3-layer (input, hidden, output layer) feed-forward-net under use of the back-propagation algorithm with an impuls term for faster learning. With this a user of the class is able to solve any linear problem as pattern recognition (like OCR...) or even simple weather forecast.

Performance

simpleNet works intern with native float**-matrices instead of a self-made matrix class. This is not the best style indeed, but it's the fastest method for training the net (because it can be well optimized by good compilers) - and performance is a main criteria in this case.

Files


Structures

ERROR_ANN::file_exception

This structure is part of the namespace ERROR_ANN. You only have to use it if you want to load the weight matrices with constructor simpleNet(const string& filename) or save them with saveWeightsToFile(const string& filename). In these cases if something goes wrong an exception is thrown with type of file_exception. The structure has one member variable: const string p; if you catch an exception you can check p for the message why the exception occured.

For instance:
...
try{	//try to load weights from file
	simpleNet *ANN=new simpleNet(filename);
}
catch(ANN_ERROR::file_exception e){
	cout << "Exception: " << e.p << endl;
}
...

...
try{	//try to save weights to file
	ANN->SaveWeightsToFile(filename);
}
catch(ANN_ERROR::file_exception e){
	cout << "Exception: " << e.p << endl;
}
...

Public member functions

simpleNet(const size_t dimI, const size_t dimH, const size_t dimO)

The standard constructor of the class returns you a neural net with dimI neurons in the input, dimH neurons in the hidden and dimO neurons in the output layer. The weight matrices between the layers are calculated randomly.

simpleNet(const size_t dimI, const size_t dimH, const size_t dimO, const float **ihW, const float **hoW)

With this second constructor you can set user defined weight matrices. Be sure that they have the right dimensions: ihW[dimI+1][dimH], hoW[dimH+1][dimO].

simpleNet(const string& filename)

With this constructor you can set up a neural net with weight matrices saved in the given file filename. Normally this file should have resulted from a run of SaveWeightsToFile(filename).
The advantage is that you can have a train phase where you let the net your patterns learn. This phase can take long so it's not recommended to strain a user with it. Afterwards you can save the learned weights with SaveWeightsToFile() and then if needed you easily make a new instance from simpleNet with the constructor simpleNet(string filename).
Be aware that the constructor throws an exception from type ANN_ERROR::file_exception if something goes wrong loading the weight matrices from the given file (see description of file_exception above).

~simpleNet()

The destructor frees allocated space for an instance of class simpleNet. You should not call this explicitly!

void SaveWeightsToFile(const string& filename)

You can save the weights of a e.g. learned net to file with this function. Afterwards you can load it again while creating a new instance from simpleNet with the constructor simpleNet(string filename).
If something goes wrong saving the weights to file an exception from type ANN_ERROR::file_exception is thrown (see description of file_exception above).
The function returns "false" if something went wrong and "true" else.

float GetE()

This function calcs and returns the quadratic error of the last training run made. If there was no training yet it will return 0. Note that you can calc the error only if you have a target output - so a GetE() after a Run() will not return a valid result.

float *GetOutput()

After a Run() or Train() of the neural net, you can fetch the calculated output with this function explicitly. The returned vector has the dimension dimO (amount of output neurons).

float *GetODiff()

With this function you can get the difference between expected and actual output. The returned vector has the dimension dimO (amount of output neurons).

size_t GetInputNeurons()

This function returns the number of input neurons.

size_t GetHiddenNeurons()

This function returns the number of hidden neurons.

size_t GetOutputNeurons()

This function returns the number of output neurons.

void SetErrTolerance(const float tol)

Here you can set the error tolerance explicitly to parameter tol. The per default in the constructor assigned tolerance is tol=0.1. It gives the upper limit for the output error sum.

void SetLearnRate(const float rate)

The default learning rate is rate=2.0, which works well for many problems. Setting the rate higher can make your net train faster, but also can make it error susceptible (perhaps it can't find a minimum for the quadratic error any more and is pending between some values). You've not to call this function explicitly, if you want to use the default learning rate.
NOTE: If you choose a too big learning rate then your net will normally not find a good minimal error value, by which it can't compute valid outputs after training phase. To validate your net's correctness it's recommended to call GetE() after training phase for all your net's patterns and look if the quadratic error is little enough for your matters.

void SetImpuls(const float imp)

With the impuls term you can make the net learning slower for matters of robustness or faster for matters of speed. The default rate imp=0.4 should be good for most problems, so you don't need to call this function anytime.
NOTE: If you choose a too big impuls term then your net will normally not find a good minimal error value, by which it can't compute valid outputs after training phase. To validate your net's correctness it's recommended to call GetE() after training phase for all your net's patterns and look if the quadratic error is little enough for your matters.

float *Run(const float *input)

The first of the two most important functions makes a complete run through your net. It calculates the input and output vectors of all layers and returns you the output of the output layer with dimension dimO (amount of output neurons).
Calculation of the inputs does a summation input_sum=sum(weight*input), the outputs are got with the sigmoide function output=1/(1+exp(-input_sum)). With parameter input[dimI] you have to give the input data, for which you want to have calculated the output.
You need to call this function anytime you need to get the output for a given input, especially in the testing and employment phase after you have trained the network properly.

bool Train(const float *input, const float *pOutput)

Before you can test and employ your net you have to present it all correct input/output-combinations you want to train it for. Not only one time, but many times, depending on the problem you want to solve.
Train() first makes a forward Run() with your given input vector and calculates the output it would produce with the present weight matrices. With your given correct output vector it calculates the output difference and error vectors and makes a backward run to adjust the weight matrices. Remember that Train() only handles one adjustment of one input/output-combination, which does not say that the difference of the output is in the limits afterwards. Train() returns true if the error sum has been within the limits to the tolerance before the training run and false else. For getting a new error sum you either can train the net again or call Run() and GetE() for the error.
Notice, that one Train() run isn't enough. To adjust the weight matrices correctly, that means to let the neural net learn the correct input/output-combinations you first have to present it one combination after another and then present the several combinations many times (depending on the problem and the impuls/learnRate paramteres, but 1000 times or more are normal). To do that you have to create a loop within your main program, where you call Train() for the different combinations again and again. If you fetch the boolean return value of Train() you can let the net learn until it returns true for all choices. But notice that this makes not sure that the net will work well. Because it's only trained that the output difference is shortly within the tolerance, but it's very possible that it does not return the correct values for falsified inputs.

Remarks

I think the algorithm is implemented correctly. The biggest problem for you is to use the class properly. You first have to create a model for your problem, which is linear and works with the backpropagation algorithm. Then you have to train it correctly. Be sure that the training values are different to the test and employ values. This way you can ensure that the net will work properly for data different to the training vectors. After training you have to give practical input and show if the output is correct. Then you can employ the net, but it's not given that it returns the correct output everytime. This depends on the data, the training and testing phase, so make sure to do your best there.
For questions and remarks you are welcome to mail me. Have fun...