transformers-stuff

This repository contains codes, scripts, and notebooks on various aspects of transformer models. Implementation uses PyTorch.

GPT_Transformer.ipynb contains a detailed walk-through of the transformer implemenatation while explaining core concepts along the way.
GPT_Language_Model.ipynb builds and trains a language model from scratch using wikipedia data, included relevant pre-processing of the data.