Skip to content

Latest commit

 

History

History
16 lines (15 loc) · 631 Bytes

README.md

File metadata and controls

16 lines (15 loc) · 631 Bytes

TinyBERT - Demo

Based on https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/TinyBERT

Installation

Run command below to install the environment(using python3)

pip install -r requirements.txt

Useage

distillation_demo.py includes intermediate distillation on a proxy (random) dataset. To execute run:

python -u /home/shai.rozenberg/PycharmProjects/TinyBERT-demo/distillation_demo.py --teacher_model /home/shai.rozenberg/PycharmProjects/TinyBERT-demo/models/teacher --student_model /home/shai.rozenberg/PycharmProjects/TinyBERT-demo/models/student