Basic_DNN

An adapted implementation of the MATLAB code provided in [1]. The original code provided with the paper is available at: https://www.maths.ed.ac.uk/~dhigham/algfiles.html

[1] Higham, C.F. and Higham, D.J., 2019. Deep Learning: An Introduction for Applied Mathematicians. SIAM Review, 61(4), pp.860-891.

Notable Additional Functionality:

• arbitrary number of hidden layers with arbitrary number of nodes in each
• choice of activation function including linear, sigmoid, tanh, ReLU, leaky ReLU and ELU
• batch stochastic Gradient descent rather than (single sample) stochastic gradient descent
• parameters such as number of iterations and learning rate not fixed

To-Do Items:

• (potential:) vectorised version - less 'readable' but quicker to perform experiments
• adaptive learning rate
• arbitrary number of data points in training set

Python Version Available:

A Python implementation of this code has been written by Alex Hiles and is available at https://github.com/alexhiles/NN