目錄
一.引言
二.模型下載
三.快速測試
四.訓(xùn)練數(shù)據(jù)
五.總結(jié)
一.引言
自打 LLama-2 發(fā)布后就一直在等大佬們發(fā)布 LLama-2 的適配中文版，也是這幾天蹲到了一版由 LinkSoul 發(fā)布的?Chinese-Llama-2-7b，其共發(fā)布了一個常規(guī)版本和一個 4-bit 的量化版本，今天我們主要體驗下 Llama-2 的中文邏輯順便看下其訓(xùn)練樣本的樣式，后續(xù)有機會把訓(xùn)練和微調(diào)跑起來。
二.模型下載
HuggingFace:?https://huggingface.co/LinkSoul/Chinese-Llama-2-7b
4bit 量化版本:?https://huggingface.co/LinkSoul/Chinese-Llama-2-7b-4bit
這里我們先整一版量化版本：
?省事且網(wǎng)絡(luò)好的同學(xué)可以直接用 Hugging Face 的 API 下載，網(wǎng)不好就半夜慢慢下載吧。
from huggingface_hub import hf_hub_download, snapshot_download
snapshot_download(repo_id="LinkSoul/Chinese-Llama-2-7b-4bit", local_dir='./models')三.快速測試
Tips?測試用到的基本庫的版本，運行顯卡為 Tesla-V100 32G：
python 3.9.11
numpy==1.23.5
torch==2.0.1
transformers==4.29.1
測試代碼：
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, TextStreamer
# Original version
# model_path = "LinkSoul/Chinese-Llama-2-7b"
# 4 bit version
model_path = "/models/LLama2_4bit"
tokenizer = AutoTokenizer.from_pretrained(model_path, use_fast=False)
if model_path.endswith("4bit"):
model = AutoModelForCausalLM.from_pretrained(
model_path,
torch_dtype=torch.float16,
device_map='auto'
)
else:
model = AutoModelForCausalLM.from_pretrained(model_path).half().cuda()
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
instruction = """[INST] <<SYS>>\nYou are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.
If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\n<</SYS>>\n\n{} [/INST]"""
while True:
text = input("請輸入 prompt\n")
if text == "q":
break
prompt = instruction.format(text)
generate_ids = model.generate(tokenizer(prompt, return_tensors='pt').input_ids.cuda(), max_new_tokens=4096, streamer=streamer)★ 常規(guī)測試
知識：
?推理：
★?一些 Bad Case
知識錯亂：
重復(fù)：
這里由于是 4-bit 的量化版本，模型的效果可能也會受影響，可以看到圖中原始 LLama2 的知識能力相對還算不錯。
四.訓(xùn)練數(shù)據(jù)
LinkSoul 在 LLama2 的基礎(chǔ)上使用了中英文 SFT 數(shù)據(jù)集，數(shù)據(jù)量 1000 萬：
LinkSoul/instruction_merge_set · Datasets at Hugging Face
?給定的數(shù)據(jù)是 parquet 格式的：
{"instruction":"","id":"chatalpaca_data_10k_0","conversations":"WrappedArray([human,Find the product of the numbers: 5 and 8], [gpt,The product of 5 and 8 is 40.], [human,What is the sum of the numbers 6 and 12?], [gpt,The sum of the numbers 6 and 12 is 18.], [human,Can you tell me the quotient of 20 and 5?], [gpt,Yes, the quotient of 20 and 5 is 4.], [human,What is the difference between 25 and 13?], [gpt,The difference between 25 and 13 is 12.], [human,What is the square of 9?], [gpt,The square of 9 is 81.], [human,What is the cube of 6?], [gpt,The cube of 6 is 216.])"}
{"instruction":"","id":"chatalpaca_data_10k_1","conversations":"WrappedArray([human,Provide five tips for effectively using tape measures], [gpt,1. Tighten the tape measure so that it stays in place when measuring.\n2. Make sure that the hook of the tape measure hits the starting point precisely.\n3. Allow for a bit of slack when measuring around curves or corners.\n4. Measure from a stable surface, such as a table or workbench.\n5. Mark the beginning and end of each measurement with a pen or marker.], [human,Can you give me an example of when to use tip number three, \"Allow for a bit of slack when measuring around curves or corners\"?], [gpt,Sure! Let's say you're measuring the circumference of a circular object, like a pipe. If you pull the tape measure too tightly, you won't get an accurate measurement because the tape measure will be stretched out of its original shape, making the measurement longer than it should be. Instead, you should wrap the tape measure around the curve or corner of the object without pulling it too tightly. This way, you'll get a more accurate measurement that takes into account the shape of the object.], [human,Can you tell me more about the markers or pens that should be used to mark the measurements? Do you recommend a certain type of pen?], [gpt,As an AI language model, I do not have personal preferences as a human, but there are certain types of markers or pens that are better suited for marking measurements than others. For example, a fine-tipped marker or mechanical pencil is great for making precise marks on a tape measure, because they produce a precise line that is easy to see. You'll want to avoid using a ballpoint pen or a marker with a thick tip, since these types of pens can produce marks that are too thick or blurry, making it difficult to accurately read the measurement. It's also a good idea to choose a marker or pen that is a different color than the color of the tape measure, so that you can easily distinguish your marks from the printed markings on the tape measure.], [human,Are there any other tips you recommend for maintaining tape measures in good condition? \n\nassistant: \nYes! One tip is to keep your tape measure clean and free from debris, so that it moves smoothly and doesn't get jammed or stuck. You can use a soft cloth or a brush to wipe down the tape measure after each use, and make sure to store it in a dry place to prevent rust or corrosion. Another tip is to avoid dropping or mishandling the tape measure, since this can damage the internal components and make it less accurate. It's also a good idea to periodically check your tape measure against a known length or standard to make sure that it is still measuring accurately.], [gpt,Finally, when retracting the tape measure, avoid letting it snap back quickly, as this can cause the tape to twist or become tangled. Instead, hold it firmly and retract it slowly and steadily to avoid damaging the internal components. With proper care and maintenance, your tape measure should last for years and provide accurate measurements every time.], [human,Goodbye.])"}instruction 指令可選，剩下 conversations 里是 human 和? gpt 的多輪對話。
def tokenize(item, tokenizer):
roles = {"human": "user", "gpt": "assistant"}
input_ids = []
labels = []
if "instruction" in item and len(item["instruction"]) > 0:
system = item["instruction"]
else:
system = dummy_message["system"]
system = B_SYS + system + E_SYS
# add system before the first content in conversations
item["conversations"][0]['value'] = system + item["conversations"][0]['value']
for i, turn in enumerate(item["conversations"]):
role = turn['from']
content = turn['value']
content = content.strip()
if role == 'human':
content = f"{B_INST} {content} {E_INST} "
content_ids = tokenizer.encode(content)
labels += [IGNORE_TOKEN_ID] * (len(content_ids))
else:
# assert role == "gpt"
content = f"{content} "
content_ids = tokenizer.encode(content, add_special_tokens=False) + [tokenizer.eos_token_id] # add_special_tokens=False remove bos token, and add eos at the end
labels += content_ids
input_ids += content_ids
input_ids = input_ids[:tokenizer.model_max_length]
labels = labels[:tokenizer.model_max_length]
trunc_id = last_index(labels, IGNORE_TOKEN_ID) + 1
input_ids = input_ids[:trunc_id]
labels = labels[:trunc_id]
if len(labels) == 0:
return tokenize(dummy_message, tokenizer)
input_ids = safe_ids(input_ids, tokenizer.vocab_size, tokenizer.pad_token_id)
labels = safe_ids(labels, tokenizer.vocab_size, IGNORE_TOKEN_ID)
return input_ids, labels訓(xùn)練代碼：https://github.com/LinkSoul-AI/Chinese-Llama-2-7b/blob/main/train.py
中展示了 tokenizer 原始樣本的流程：
◆? 根據(jù)指令生成 system
◆? 根據(jù) from 和 value 的多輪對話生成 input_ids 和 labels
Tips: 這里會把前面生成的 system 綴到第一個 value 前面，labels 會在 human 部分用?IGNORE_TOKEN_ID 的掩碼進(jìn)行 Mask
◆? 最后 safe_ids 用于限制 id < max_value 超過使用 pad_id 進(jìn)行填充
def safe_ids(ids, max_value, pad_id):
return [i if i < max_value else pad_id for i in ids]這里輸入格式嚴(yán)格遵循 llama-2-chat 格式，兼容適配所有針對原版 llama-2-chat 模型的優(yōu)化。
五.總結(jié)
這里簡單介紹了 LLama-2 7B Chinese 的推理和數(shù)據(jù)樣式，后續(xù)有機會訓(xùn)練和微調(diào)該模型。
參考：?
Chinese Llama 2 7B:?https://github.com/LinkSoul-AI/Chinese-Llama-2-7b
Model:?https://huggingface.co/LinkSoul/Chinese-Llama-2-7b
Instruction_merge_set:?https://huggingface.co/datasets/LinkSoul/instruction_merge_set/文章來源：http://www.zghlxwxcb.cn/news/detail-608454.html
Download Files:?https://huggingface.co/docs/huggingface_hub/v0.16.3/guides/download文章來源地址http://www.zghlxwxcb.cn/news/detail-608454.html
到了這里，關(guān)于LLM - Chinese-Llama-2-7b 初體驗的文章就介紹完了。如果您還想了解更多內(nèi)容，請在右上角搜索TOY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！
