DISSERTAION: link
This experiment is divided into two sections:
- Dimensional Reduction
- Pruning
I used some of the well-recognized dimensional reduction techniques like Principal Component Analysis, Independent Component Analysis, and Isomap to apply them to the weight matrices as opposed to applying them straight onto raw data as part of preprocessing methods. This way, I test the potential path into a new optimization technique, where we perform matrix decomposition of the matrix and also avoid pretraining.
So far not all of the dimensionally reduction methods have been used to the project. This project is in the state-of-the-art state, hence means it requires further development for a better performance. The eventual objective is to achieve fully functional small library-like package that can be utilized for pretrained network optimization.