EasyLLMFeaturePorter

1-Click is all you need.

Methodology

1. Get diff between base model and context expanded model.
2. Get diff between base model and finetuned model(model to expand context). Then use this diff to calculate activation ratio.
3. Add diff * (1 - activation ratio) to finetuned model.

Done !

Explanation

Korean

English

Special Thanks

Thanks to kuotient for letting me know issue on initial commit. Huggingface

Thanks to Sionic Ai for providing A100 cluster.