Development

The project was initially started as part of my position as Machine Learning Research Assistant. I created it as a resource to help newer members of the team to better understand the implementation of the backpropagation algorithm.

Reference Material

The primary reference material used for the implementing the backpropagation algorithm was Neural Networks and Deep Learning - an online book that teaches about neural networks and how they are trained.

Future Plans

Although the program successfully trains a network consisting of 3 layers, there are still many issues and missing features. For one, trying to train a network with more than 3 layers results in terrible convergence, usually staying at around 10% accuracy - essentially random guessing.

In addition, the cost function as well as activation functions are not ideal for classification networks. It would probably be better to implement some other cost functions as well as letting the user pick and choose to experiment.

Finally, I'd like to create a web UI to make it easier to work with and manipulate the networks. I'm hoping that this could become a valuable tool for beginners to play around with the inner workings of DNNs and the backpropagation algorithm.