目錄

1. 資源下載

2. 部署啟動

1. 資源下載

• HuggingFace 模型權(quán)重下載

# install git-lfs
git lfs install  

# download checkpoint
# clone the repo
git clone https://huggingface.co/THUDM/chatglm-6b

• 手動模型權(quán)重下載

# download checkpoint 
# clone the repo, skip large files, large files need to download separately
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/THUDM/chatglm-6b
# download large files from thu cloud
git clone https://github.com/chenyifanthu/THU-Cloud-Downloader.git
cd THU-Cloud-Downloader/
python main.py \
--link https://cloud.tsinghua.edu.cn/d/fb9f16d6dc8f482596c2/ \
--save ../chatglm-6b

• 源碼下載

# clone source code
git clone https://github.com/THUDM/ChatGLM-6B.git
cd ChatGLM2-6B
pip install -r requirements.txt

2. 部署啟動

• 源碼修改

對于 `cli_demo.py` 和 `web_demo.py` 中模型加載部分需要從 HuggingFace 下載

tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True).half().cuda()

改為從本地加載

tokenizer = AutoTokenizer.from_pretrained("../chatglm-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("../chatglm-6b", trust_remote_code=True).half().cuda()

• 命令行啟動

# command line (specify model path)
python cli_demo.py

• 網(wǎng)頁啟動

# web by Gradio (specify model path)
python web_demo.py

啟動后的效果：

文章來源地址http://www.zghlxwxcb.cn/news/detail-632181.html

到了這里，關(guān)于【ChatGLM】大模型之 ChatGLM 部署的文章就介紹完了。如果您還想了解更多內(nèi)容，請在右上角搜索TOY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！