Testing

Correctness checking

For forward propagation:
- We slot in some numbers, calculate the results manually, and compare with results actually obtained
- We also forward propagate pictures/photos, and check the results look approximately like what we would expect
For backward propagation:
- We use numerical validation, since the sum of the square of the weight changes, divided by the learning rate, approximately equals the change in loss. Or it should. We test this :-)
Standard test sets
- Checked using implementations for MNIST, and NORB is in progress

Network optimization is stochastic, and there are typically numerous local minima, into which the optimization can get stuck
For unit testing, this is not very suitable, since unit tests must run repeatably, reliably, quickly
Therefore, for unit-testing, the network weights are preset to a fixed set of values
- using a random number generator with a fixed seed
- or by explicitly giving a hard-coded set of weights
Then, the test checks that the network converges to better than an expected loss, and accuracy, within a preset number of epochs
We also have unit tests for forward-propagation, and backward propagation, as per section Correctness checking above.

Using googletest, which:
- compiles quickly
- gives awesome colored output
- lets you choose which tests to run using --gtest_filter= option
Dont need to install anything: it's included in the thirdparty directory, and added to the build automatically
To run the unit tests:

make unittests
./unittests

make unittests
./unittests --gtest_filter=testbackprop.*

./unittests --gtest_filter=-*SLOW*

Actually, by default, with no arguments, the argument --gtest_filter=-SLOW* will be appended automatically
Also, rather than having to type --gtest_filter=[something], you can just type tests=[something], and this will be converted into --gtest_filter=[something] automatically