diff --git a/examples/llama2_7b_finetuning_4bit/README.md b/examples/llama2_7b_finetuning_4bit/README.md index 9af187fbcf0..0afe4116132 100644 --- a/examples/llama2_7b_finetuning_4bit/README.md +++ b/examples/llama2_7b_finetuning_4bit/README.md @@ -12,7 +12,18 @@ with GPU availability. - Access approval to [Llama2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) - GPU with at least 12 GiB of VRAM (in our tests, we used an Nvidia T4) -## Running the example +## Running + +### Command Line + +Set your token environment variable from the terminal, then run the API script: + +```bash +export HUGGING_FACE_HUB_TOKEN="" +./run_train.sh +``` + +### Python API Set your token environment variable from the terminal, then run the API script: @@ -20,3 +31,11 @@ Set your token environment variable from the terminal, then run the API script: export HUGGING_FACE_HUB_TOKEN="" python train_alpaca.py ``` + +## Upload to HuggingFace + +You can upload to the HuggingFace Hub from the command line: + +```bash +ludwig upload hf_hub -r / -m +``` diff --git a/examples/llama2_7b_finetuning_4bit/llama2_7b_4bit.yaml b/examples/llama2_7b_finetuning_4bit/llama2_7b_4bit.yaml new file mode 100644 index 00000000000..8bd613cc5d4 --- /dev/null +++ b/examples/llama2_7b_finetuning_4bit/llama2_7b_4bit.yaml @@ -0,0 +1,28 @@ +model_type: llm +base_model: meta-llama/Llama-2-7b-hf + +quantization: + bits: 4 + +adapter: + type: lora + +input_features: + - name: instruction + type: text + +output_features: + - name: output + type: text + +trainer: + type: finetune + learning_rate: 0.0003 + batch_size: 2 + gradient_accumulation_steps: 8 + epochs: 3 + learning_rate_scheduler: + warmup_fraction: 0.01 + +backend: + type: local diff --git a/examples/llama2_7b_finetuning_4bit/run_train.sh b/examples/llama2_7b_finetuning_4bit/run_train.sh new file mode 100755 index 00000000000..372cc22d4e5 --- /dev/null +++ b/examples/llama2_7b_finetuning_4bit/run_train.sh @@ -0,0 +1,10 @@ +#!/usr/bin/env bash + +# Fail fast if an error occurs +set -e + +# Get the directory of this script, which contains the config file +SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) + +# Train +ludwig train --config ${SCRIPT_DIR}/llama2_7b_4bit.yaml --dataset ludwig://alpaca