項(xiàng)目地址

chatglm2-6b

下載模型

cd ~/Downloads
git clone --depth=1 https://www.modelscope.cn/ZhipuAI/chatglm2-6b.git

創(chuàng)建測試項(xiàng)目

cd ~/code
mkdir ai/chatglm2-6b -p
cd ai/chatglm2-6b
mkdir ZhipuAI
ln -s ~/Downloads/chatglm2-6b ZhipuAI/

創(chuàng)建虛擬環(huán)境安裝依賴

python3 -m venv venv
source venv/bin/activate

pip install transformers modelscope cpm_kernels -U
pip install torch
pip install accelerate
pip install sentencepiece

完整的venv環(huán)境

(venv) yeqiang@yeqiang-MS-7B23:~/code/ai/chatglm2-6b$ pip list
Package                  Version
------------------------ ------------
accelerate               0.26.1
addict                   2.4.0
aiohttp                  3.9.1
aiosignal                1.3.1
aliyun-python-sdk-core   2.14.0
aliyun-python-sdk-kms    2.16.2
async-timeout            4.0.3
attrs                    23.2.0
certifi                  2023.11.17
cffi                     1.16.0
charset-normalizer       3.3.2
cpm-kernels              1.0.11
crcmod                   1.7
cryptography             41.0.7
datasets                 2.16.1
dill                     0.3.7
einops                   0.7.0
filelock                 3.13.1
frozenlist               1.4.1
fsspec                   2023.10.0
gast                     0.5.4
huggingface-hub          0.20.2
idna                     3.6
importlib-metadata       7.0.1
Jinja2                   3.1.3
jmespath                 0.10.0
MarkupSafe               2.1.3
modelscope               1.11.0
mpmath                   1.3.0
multidict                6.0.4
multiprocess             0.70.15
networkx                 3.2.1
numpy                    1.26.3
nvidia-cublas-cu12       12.1.3.1
nvidia-cuda-cupti-cu12   12.1.105
nvidia-cuda-nvrtc-cu12   12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12        8.9.2.26
nvidia-cufft-cu12        11.0.2.54
nvidia-curand-cu12       10.3.2.106
nvidia-cusolver-cu12     11.4.5.107
nvidia-cusparse-cu12     12.1.0.106
nvidia-nccl-cu12         2.18.1
nvidia-nvjitlink-cu12    12.3.101
nvidia-nvtx-cu12         12.1.105
oss2                     2.18.4
packaging                23.2
pandas                   2.1.4
pillow                   10.2.0
pip                      22.0.2
platformdirs             4.1.0
psutil                   5.9.7
pyarrow                  14.0.2
pyarrow-hotfix           0.6
pycparser                2.21
pycryptodome             3.20.0
python-dateutil          2.8.2
pytz                     2023.3.post1
PyYAML                   6.0.1
regex                    2023.12.25
requests                 2.31.0
safetensors              0.4.1
scipy                    1.11.4
sentencepiece            0.1.99
setuptools               59.6.0
simplejson               3.19.2
six                      1.16.0
sortedcontainers         2.4.0
sympy                    1.12
tokenizers               0.15.0
tomli                    2.0.1
torch                    2.1.2
torchflow                0.0.3
tqdm                     4.66.1
transformers             4.36.2
triton                   2.1.0
typing_extensions        4.9.0
tzdata                   2023.4
urllib3                  2.1.0
xxhash                   3.4.1
yapf                     0.40.2
yarl                     1.9.4
zipp                     3.17.0

main.py

# 備注：最新模型版本要求modelscope >= 1.9.0
# pip install modelscope -U 

from modelscope.utils.constant import Tasks
from modelscope import Model
from modelscope.pipelines import pipeline
model = Model.from_pretrained('ZhipuAI/chatglm2-6b', device_map='auto', revision='v1.0.12')
pipe = pipeline(task=Tasks.chat, model=model)
inputs = {'text':'你好', 'history': []}
result = pipe(inputs)
inputs = {'text':'介紹下清華大學(xué)', 'history': result['history']}
result = pipe(inputs)

執(zhí)行

python進(jìn)程cpu使用率100.3%持續(xù)了很長時(shí)間，沒有啟用多線程？

文章來源地址http://www.zghlxwxcb.cn/news/detail-808631.html

到了這里，關(guān)于【AI】RTX2060 6G Ubuntu 22.04.1 LTS (Jammy Jellyfish) 部署chatglm2-6b 開源中英雙語對話模型的文章就介紹完了。如果您還想了解更多內(nèi)容，請?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！