開源大模型ChatGLM2-6B 2. 跟著LangChain參考文檔搭建LLM+知識庫問答系統(tǒng)

這篇具有很好參考價值的文章主要介紹了開源大模型ChatGLM2-6B 2. 跟著LangChain參考文檔搭建LLM+知識庫問答系統(tǒng)。希望對大家有所幫助。如果存在錯誤或未考慮完全的地方，請大家不吝賜教，您也可以點擊"舉報違法"按鈕提交疑問。

0. 環(huán)境

租用了1臺GPU服務器，系統(tǒng) ubuntu20，Tesla V100-16GB

（GPU服務器已經關機結束租賃了）
SSH地址：*
端口：17520

SSH賬戶：root
密碼：Jaere7pa

內網： 3389 ，外網：17518

VNC地址：*
端口：17519

VNC用戶名：root
密碼：Jaere7pa

硬件需求，ChatGLM-6B和ChatGLM2-6B相當。
量化等級?? ?最低 GPU 顯存
FP16（無量化）?? ?13 GB
INT8?? ?10 GB
INT4?? ?6 GB

1. 基本環(huán)境

1.1 測試gpu

nvidia-smi
(base) root@ubuntuserver:~# nvidia-smi
Tue Sep 12 02:06:45 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.54 ? ? ? Driver Version: 510.54 ? ? ? CUDA Version: 11.6 ? ? |
|-------------------------------+----------------------+----------------------+
| GPU ?Name ? ? ? ?Persistence-M| Bus-Id ? ? ? ?Disp.A | Volatile Uncorr. ECC |
| Fan ?Temp ?Perf ?Pwr:Usage/Cap| ? ? ? ? Memory-Usage | GPU-Util ?Compute M. |
| ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ?| ? ? ? ? ? ? ? MIG M. |
|===============================+======================+======================|
| ? 0 ?Tesla V100-PCIE... ?Off ?| 00000000:00:07.0 Off | ? ? ? ? ? ? ? ? ? ?0 |
| N/A ? 42C ? ?P0 ? ?38W / 250W | ? ? ?0MiB / 16384MiB | ? ? ?0% ? ? ?Default |
| ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? | ? ? ? ? ? ? ? ? ? ? ?| ? ? ? ? ? ? ? ? ?N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?|
| ?GPU ? GI ? CI ? ? ? ?PID ? Type ? Process name ? ? ? ? ? ? ? ? ?GPU Memory |
| ? ? ? ?ID ? ID ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Usage ? ? ?|
|=============================================================================|
| ?No running processes found ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? |
+-----------------------------------------------------------------------------+

(base) root@ubuntuserver:~#

1.2 python

當前LangChain安裝說明，需要Python 3.8 - 3.10 版本
執(zhí)行python3
可以看到python3.9

# 如果低于這個版本，可使用conda安裝環(huán)境
$ conda create -p /root/work/conda_py310_chatglm2 python=3.10

# 激活環(huán)境
$ source activate /root/work/conda_py310_chatglm2

# 更新py庫
$ pip3 install --upgrade pip

# 關閉環(huán)境
$ source deactivate /root/work/conda_py310_chatglm2

# 刪除環(huán)境
$ conda env remove -p ?/root/work/conda_py310_chatglm2

1.3 pip

pip3 install --upgrade pip -i https://pypi.tuna.tsinghua.edu.cn/simple

1.4 準備倉庫

git clone https://github.com/chatchat-space/Langchain-Chatchat.git
cd Langchain-Chatchat

1.5 升級cuda

查看顯卡驅動版本要求：
https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html

發(fā)現cuda 11.8需要 >=450.80.02。已經滿足。

執(zhí)行指令更新cuda

wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run
sh cuda_11.8.0_520.61.05_linux.run

-> 輸入 accept
-> 取消勾選 Driver
-> 點擊 install

export PATH=$PATH:/usr/local/cuda-11.8/bin
nvcc --version

準備switch-cuda.sh腳本

#!/usr/bin/env bash
# Copyright (c) 2018 Patrick Hohenecker
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

# author:   Patrick Hohenecker <mail@paho.at>
# version:  2018.1
# date:     May 15, 2018


set -e


# ensure that the script has been sourced rather than just executed
if [[ "${BASH_SOURCE[0]}" = "${0}" ]]; then
    echo "Please use 'source' to execute switch-cuda.sh!"
    exit 1
fi

INSTALL_FOLDER="/usr/local"  # the location to look for CUDA installations at
TARGET_VERSION=${1}          # the target CUDA version to switch to (if provided)

# if no version to switch to has been provided, then just print all available CUDA installations
if [[ -z ${TARGET_VERSION} ]]; then
    echo "The following CUDA installations have been found (in '${INSTALL_FOLDER}'):"
    ls -l "${INSTALL_FOLDER}" | egrep -o "cuda-[0-9]+\\.[0-9]+$" | while read -r line; do
        echo "* ${line}"
    done
    set +e
    return
# otherwise, check whether there is an installation of the requested CUDA version
elif [[ ! -d "${INSTALL_FOLDER}/cuda-${TARGET_VERSION}" ]]; then
    echo "No installation of CUDA ${TARGET_VERSION} has been found!"
    set +e
    return
fi

# the path of the installation to use
cuda_path="${INSTALL_FOLDER}/cuda-${TARGET_VERSION}"

# filter out those CUDA entries from the PATH that are not needed anymore
path_elements=(${PATH//:/ })
new_path="${cuda_path}/bin"
for p in "${path_elements[@]}"; do
    if [[ ! ${p} =~ ^${INSTALL_FOLDER}/cuda ]]; then
        new_path="${new_path}:${p}"
    fi
done

# filter out those CUDA entries from the LD_LIBRARY_PATH that are not needed anymore
ld_path_elements=(${LD_LIBRARY_PATH//:/ })
new_ld_path="${cuda_path}/lib64:${cuda_path}/extras/CUPTI/lib64"
for p in "${ld_path_elements[@]}"; do
    if [[ ! ${p} =~ ^${INSTALL_FOLDER}/cuda ]]; then
        new_ld_path="${new_ld_path}:${p}"
    fi
done

# update environment variables
export CUDA_HOME="${cuda_path}"
export CUDA_ROOT="${cuda_path}"
export LD_LIBRARY_PATH="${new_ld_path}"
export PATH="${new_path}"

echo "Switched to CUDA ${TARGET_VERSION}."

set +e
return

用法

source switch-cuda.sh 11.8

1.6 單獨安裝torch-gpu版本

$ pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

1.7 安裝全部依賴

$ pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

驗證torch是否帶有cuda

import torch
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
print(device)

2. 下載模型

2.1 chatglm2-6b

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/THUDM/chatglm2-6b
下載ChatGLM2作者上傳到清華網盤的模型文件
https://cloud.tsinghua.edu.cn/d/674208019e314311ab5c/?p=%2Fchatglm2-6b&mode=list
并覆蓋到chatglm2-6b

先前以為用wget可以下載，結果下來的文件是一樣大的，造成推理失敗。
win10 逐一校驗文件SHA256，需要和https://huggingface.co/THUDM/chatglm2-6b中Git LFS Details的匹配。

C:\Users\qjfen\Downloads\chatglm2-6b>certutil -hashfile pytorch_model-00001-of-00007.bin SHA256
pytorch_model-00001-of-00007.bin ?? ??? ?cdf1bf57d519abe11043e9121314e76bc0934993e649a9e438a4b0894f4e6ee8
pytorch_model-00002-of-00007.bin?? ??? ?1cd596bd15905248b20b755daf12a02a8fa963da09b59da7fdc896e17bfa518c
pytorch_model-00003-of-00007.bin ?? ??? ?812edc55c969d2ef82dcda8c275e379ef689761b13860da8ea7c1f3a475975c8
pytorch_model-00004-of-00007.bin ?? ??? ?555c17fac2d80e38ba332546dc759b6b7e07aee21e5d0d7826375b998e5aada3
pytorch_model-00005-of-00007.bin ?? ??? ?cb85560ccfa77a9e4dd67a838c8d1eeb0071427fd8708e18be9c77224969ef48
pytorch_model-00006-of-00007.bin ?? ??? ?09ebd811227d992350b92b2c3491f677ae1f3c586b38abe95784fd2f7d23d5f2
pytorch_model-00007-of-00007.bin ?? ??? ?316e007bc727f3cbba432d29e1d3e35ac8ef8eb52df4db9f0609d091a43c69cb

這里需要推到服務器中。并在ubuntu下用sha256sum <filename> 校驗下大文件。

2.2 text2vec

GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese text2vec
下載這兩份文件，并放到 text2vec 內：

model.safetensors?? ??? ??? ??? ??? ??? ?eaf5cb71c0eeab7db3c5171da504e5867b3f67a78e07bdba9b52d334ae35adb3
pytorch_model.bin?? ??? ??? ??? ??? ??? ?5883cb940ac5509b75e9fe23a9aea62694045849dc8c8c2da2894861a045d7f5

3. 參數配置

cd configs
cp configs/model_config.py.example configs/model_config.py
cp configs/server_config.py.example configs/server_config.py

修改configs/model_config.py·

embedding_model_dict = {
? ? "text2vec": "/root/work/Langchain-Chatchat/text2vec",
}
# 選用的 Embedding 名稱
EMBEDDING_MODEL = "text2vec"

llm_model_dict = {
? ? "chatglm2-6b": {
? ? ? ? "local_model_path": "/root/work/Langchain-Chatchat/chatglm2-6b",
? ? },
}
# LLM 名稱
LLM_MODEL = "chatglm2-6b"

4. 知識庫初始化與遷移

初始化知識庫：

$ python init_database.py --recreate-vs

5. 一鍵啟動API 服務或 Web UI

5.1 啟動命令

一鍵啟動腳本 startup.py,一鍵啟動所有 Fastchat 服務、API 服務、WebUI 服務，示例代碼：

$ python startup.py -a

5.2 運行測試

瀏覽器打開 127.0.0.1:8501。

對話模式支持LLM對話，知識庫問答，搜索引擎問答。

大模型外掛文檔問答系統(tǒng),深度學習,Python,人工智能

知識庫問答看起來是本倉庫作者制作的，根據分析、數據檢索生成的問答結果。

大模型外掛文檔問答系統(tǒng),深度學習,Python,人工智能文章來源地址http://www.zghlxwxcb.cn/news/detail-773387.html

參考：

[1]https://github.com/THUDM/ChatGLM2-6B
[2]ChatGLM-6B (介紹以及本地部署)，https://blog.csdn.net/qq128252/article/details/129625046
[3]ChatGLM2-6B｜開源本地化語言模型，https://openai.wiki/chatglm2-6b.html
[3]免費部署一個開源大模型 MOSS，https://zhuanlan.zhihu.com/p/624490276
[4]LangChain + ChatGLM2-6B 搭建個人專屬知識庫，https://zhuanlan.zhihu.com/p/643531454
[5]https://pytorch.org/get-started/locally/

到了這里，關于開源大模型ChatGLM2-6B 2. 跟著LangChain參考文檔搭建LLM+知識庫問答系統(tǒng)的文章就介紹完了。如果您還想了解更多內容，請在右上角搜索TOY模板網以前的文章或繼續(xù)瀏覽下面的相關文章，希望大家以后多多支持TOY模板網！