本文將介紹兩種開源工具來微調(diào)LLAMA-2。

一、使用autotrain-advanced微調(diào)LLAMA-2

? ? ? ? AutoTrain是一種無代碼工具，用于為自然語(yǔ)言處理（NLP）任務(wù)、計(jì)算機(jī)視覺（CV）任務(wù)、語(yǔ)音任務(wù)甚至表格任務(wù)訓(xùn)練最先進(jìn)的模型。

1） 安裝相關(guān)庫(kù)，使用huggingface_hub下載微調(diào)數(shù)據(jù)

!pip install autotrain-advanced!pip install huggingface_hub

2） 更新autotrain-advanced所需要的包

# update torch!autotrain?setup?--update-torch

3） 登錄Huggingface

# Login to huggingface?from huggingface_hub import notebook_loginnotebook_login()

4）?開始微調(diào)LLAMA-2

! autotrain llm \--train \--model {MODEL_NAME} \--project-name {PROJECT_NAME} \--data-path data/ \--text-column text \--lr {LEARNING_RATE} \--batch-size {BATCH_SIZE} \--epochs {NUM_EPOCHS} \--block-size {BLOCK_SIZE} \--warmup-ratio {WARMUP_RATIO} \--lora-r {LORA_R} \--lora-alpha {LORA_ALPHA} \--lora-dropout {LORA_DROPOUT} \--weight-decay {WEIGHT_DECAY} \--gradient-accumulation {GRADIENT_ACCUMULATION}

核心參數(shù)含義：

llm: 微調(diào)模型的類型

— project_name:?項(xiàng)目名稱

— model:?需要微調(diào)的基礎(chǔ)模型

— data_path: 指定微調(diào)所需要的數(shù)據(jù)，可以使用huggingface上的數(shù)據(jù)集

— text_column: 如果數(shù)據(jù)是表格，需要指定instructions和responses對(duì)應(yīng)的列名

— use_peft: 指定peft某一種方法

— use_int4:?指定int 4量化

— learning_rate:?學(xué)習(xí)率

— train_batch_size: 訓(xùn)練批次大小

— num_train_epochs: 訓(xùn)練輪數(shù)大小

— trainer:?指定訓(xùn)練的方式

— model_max_length: 設(shè)置模型最大上下文窗口

— push_to_hub（可選）: 微調(diào)好的模型是否需要存儲(chǔ)到Hugging Face??

— repo_id: 如果要存儲(chǔ)微調(diào)好的模型到Hugging Face，需要指定repository ID

— block_size:?設(shè)置文本塊大小

下面看一個(gè)具體的示例：

!autotrain llm--train--project_name "llama2-autotrain-openassitant"--model TinyPixel/Llama-2-7B-bf16-sharded--data_path timdettmers/openassistant-guanaco--text_column text--use_peft--use_int4--learning_rate 0.4--train_batch_size 3--num_train_epochs 2--trainer sft--model_max_length 1048--push_to_hub--repo_id trojrobert/llama2-autotrain-openassistant--block_size 1048 > training.log

二、使用TRL微調(diào)LLAMA-2

? ? ? ?TRL是一個(gè)全棧庫(kù)，提供了通過強(qiáng)化學(xué)習(xí)來訓(xùn)練transformer語(yǔ)言模型一系列工具，包括從監(jiān)督微調(diào)步驟（SFT）、獎(jiǎng)勵(lì)建模步驟（RM）到近端策略優(yōu)化（PPO）步驟。

1）安裝相關(guān)的庫(kù)

!pip install -q -U trl peft transformers  datasets bitsandbytes wandb

2）從Huggingface導(dǎo)入數(shù)據(jù)集

from datasets import load_dataset?dataset_name = "timdettmers/openassistant-guanaco"?dataset = load_dataset(dataset_name, split="train")

3）量化配置，從Huggingface下載模型

import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig??# quantizition configurationbnb_config = BitsAndBytesConfig(    load_in_4bit=True,    bnb_4bit_quant_type="nf4",    bnb_4bit_compute_dtype=torch.float16,)??# download model?model_name = "TinyPixel/Llama-2-7B-bf16-sharded"model = AutoModelForCausalLM.from_pretrained(    model_name,    quantization_config=bnb_config,    trust_remote_code=True)model.config.use_cache = False

4）下載Tokenizer

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)tokenizer.pad_token = tokenizer.eos_token

5）創(chuàng)建PEFT配置

from peft import LoraConfig, get_peft_model?lora_alpha = 16lora_dropout = 0.1lora_r = 64?peft_config = LoraConfig(    lora_alpha=lora_alpha,    lora_dropout=lora_dropout,    r=lora_r,    bias="none",    task_type="CAUSAL_LM")

6）創(chuàng)建微調(diào)和訓(xùn)練配置

from transformers import TrainingArguments?output_dir = "./results"per_device_train_batch_size = 4gradient_accumulation_steps = 4optim = "paged_adamw_32bit"save_steps = 100logging_steps = 10learning_rate = 2e-4max_grad_norm = 0.3max_steps = 100warmup_ratio = 0.03lr_scheduler_type = "constant"?training_arguments = TrainingArguments(    output_dir=output_dir,    per_device_train_batch_size=per_device_train_batch_size,    gradient_accumulation_steps=gradient_accumulation_steps,    optim=optim,    save_steps=save_steps,    logging_steps=logging_steps,    learning_rate=learning_rate,    fp16=True,    max_grad_norm=max_grad_norm,    max_steps=max_steps,    warmup_ratio=warmup_ratio,    group_by_length=True,    lr_scheduler_type=lr_scheduler_type,)

7）創(chuàng)建SFTTrainer配置

from trl import SFTTrainer?max_seq_length = 512?trainer = SFTTrainer(    model=model,    train_dataset=dataset,    peft_config=peft_config,    dataset_text_field="text",    max_seq_length=max_seq_length,    tokenizer=tokenizer,    args=training_arguments,)

8）在微調(diào)的時(shí)候，對(duì)LN層使用float 32訓(xùn)練更穩(wěn)定

for name, module in trainer.model.named_modules():    if "norm" in name:        module = module.to(torch.float32)

9）開始微調(diào)

trainer.train()

10）保存微調(diào)好的模型

model_to_save = trainer.model.module if hasattr(trainer.model, 'module') else trainer.model  # Take care of distributed/parallel trainingmodel_to_save.save_pretrained("outputs")

11）加載微調(diào)好的模型

lora_config = LoraConfig.from_pretrained('outputs')tuned_model = get_peft_model(model, lora_config)

12）測(cè)試微調(diào)好的模型效果

?text = "What is a large language model?"device = "cuda:0"?inputs = tokenizer(text, return_tensors="pt").to(device)outputs = tuned_model.generate(**inputs, max_new_tokens=50)print(tokenizer.decode(outputs[0], skip_special_tokens=True))

參考文獻(xiàn)：

[1]?https://trojrobert.medium.com/4-easier-ways-for-fine-tuning-llama-2-and-other-open-source-llms-eb3218657f6e

[2]?https://colab.research.google.com/drive/1JMEi2VMNGMOTyfEcQZyp23EISUrWg5cg?usp=sharing

[3]?https://colab.research.google.com/drive/1ctevXhrE60s7o9RzsxpIqq37EjyU9tBn?usp=sharing#scrollTo=bsbdrb5p2ONa文章來源地址http://www.zghlxwxcb.cn/news/detail-756823.html

到了這里，關(guān)于LLM微調(diào)（二）| 微調(diào)LLAMA-2和其他開源LLM的兩種簡(jiǎn)單方法的文章就介紹完了。如果您還想了解更多內(nèi)容，請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！