model_conversion_en

Model Merging and Conversion

Here are the steps for manually merging LoRA weights with the original Llama-3-8B to create a complete model. If you have sufficient network bandwidth, it is recommended to download the full model directly.

Preparation

1. Ensure you have pulled the latest version of the repository: git pull
2. Ensure your machine has enough memory to load the complete model for merging operations.
3. Install the required libraries (from the project root directory requirements.txt):

$ pip install -r requirements.txt

Step 1: Obtain the Original Llama-3-8B Model

Download the original Llama-3-8B model (not the Instruct version) from: Meta-Llama-3-8B on Hugging Face

Suggested files to download (only download safetensors format weights):

original/  <-- This folder is not needed, it contains pth format weights
config.json
generation_config.json
model-00001-of-00004.safetensors
model-00002-of-00004.safetensors
model-00003-of-00004.safetensors
model-00004-of-00004.safetensors
model.safetensors.index.json
special_tokens_map.json
tokenizer_config.json
tokenizer.json

Step 2: Merge LoRA Weights to Generate Full Model Weights

This step involves merging LoRA weights to create full model weights in safetensors format. Execute the following command:

$ python scripts/merge_llama3_with_chinese_lora_low_mem.py \
    --base_model path_to_original_llama3_dir \
    --lora_model path_to_lora \
    --output_dir path_to_output_dir 

Parameter explanations:

• --base_model: The directory containing the original Llama-3 model weights and configuration files.
• --lora_model: The directory where the Llama-3-Chinese or Llama-3-Chinese-Instruct LoRA extracted files are located, or use the model calling name from 🤗 Model Hub (which will automatically download the model).
• --output_dir: The directory to save the complete model weights, default is ./
• (Optional) --verbose: Display detailed information during the merging process.