1.簡介：

組織機構(gòu)：Meta（Facebook）

代碼倉：https://github.com/facebookresearch/llama

模型：llama-2-7b

下載：使用download.sh下載

硬件環(huán)境：暗影精靈7Plus

Windows版本：Windows 11家庭中文版 Insider Preview 22H2

內(nèi)存 32G

GPU顯卡：Nvidia GTX 3080 Laptop （16G）

2.代碼和模型下載：

下載llama.cpp的代碼倉：

git clone https://github.com/ggerganov/llama.cpp

需要獲取原始LLaMA的模型文件，放到 models目錄下，現(xiàn)在models目錄下是這樣的：

參考 https://blog.csdn.net/snmper/article/details/133578456

將上次在Jetson AGX Orin上的成功運行的7B模型文件傳到 models目錄下：

3.llama.cpp環(huán)境安裝：

查看readme，找到llamp.cpp在Windows上的安裝方式

打開 https://github.com/skeeto/w64devkit/releases

找到最新fortran版本的 w64devkit：

下載完成后系統(tǒng)彈出：

往前找一個版本v.19.0試試：https://github.com/skeeto/w64devkit/releases/tag/v1.19.0

解壓到 D:\w64devkit

運行 w64devkit.exe

切換到 d: 盤

cd llama.cpp

python -V

這里python是3.7.5版本。

查看下make,cmake,gcc,g++的版本：

編譯試試：

make

耐心等待編譯結(jié)束（或者編譯出錯）

這個到底算不算惡意軟件呢？

張小白感覺不像，于是到llama.cpp的官方去提了個issue確認一下：https://github.com/ggerganov/llama.cpp/issues/3463

官方回答如下：https://github.com/ggerganov/llama.cpp/discussions/3464

張小白還是決定使用 w64devkit，而且是最新版。在編譯期間關(guān)掉 360殺毒軟件?。。。ㄆ鋵嵾€得關(guān)閉360安全衛(wèi)士）

重新打開 https://github.com/skeeto/w64devkit/releases

下載 w64devkit-fortran-1.20.0.zip

解壓到D盤:

雙擊運行 w64devkit.exe

cd d:/

cd llama.cpp

make

耐心等待編譯結(jié)束：

編譯成功。

其中exe就是生成好的windows可執(zhí)行文件。

退出 w64devkit.編譯環(huán)境。

4.安裝依賴

創(chuàng)建conda環(huán)境

conda create -n llama python=3.10

conda activate llama

cd llama.cpp

pip install -r requirements.txt

5.部署驗證

閱讀下面這段內(nèi)容：

將7B模型（14G左右）轉(zhuǎn)換成 ggml FP16模型

python convert.py models/7B/

模型寫到了 models\7B\ggml-model-f16.gguf 文件中：也是14G左右。

將剛才轉(zhuǎn)換好的FP16模型進行4-bit量化：

./quantize ./models/7B/ggml-model-f16.gguf ./models/7B/ggml-model-q4_0.gguf q4_0

量化后的文件為：./models/7B/ggml-model-q4_0.gguf

大小只有3.8G了。

進行推理：

./main -m ./models/7B/ggml-model-q4_0.gguf -n 128

運行結(jié)果如下：

Refresh your summer look with our stylish new range of women's swimwear. Shop the latest styles in bikinis, tankinis and one pieces online at Simply Beach today! Our collection offers a wide selection of flattering designs from classic cuts to eye-catching prints that will turn heads on your next day by the pool. [end of text]

這個貌似是隨機生成的一段話。

換個提示詞： ./main -m ./models/7B/ggml-model-q4_0.gguf --prompt "Once upon a time"

補齊的文字如下：

Once upon a time, there was no such thing as a "social network". The idea of connecting with someone else on the Internet simply by clicking on their name and seeing who they were connected to and what we might have in common is a relatively new concept. But this kind of connection has become so much a part of our lives that we don't even think twice about it, right?
But once upon a time there was only one way to connect with someone: you either knew them or you didn't. And if you met somebody and became friends, the way you maintained your relationship was to stay in touch by phone, letter, or in person. It wasn't that easy before e-mail, cell phones, Facebook, Twitter, texting, and all the other ways we keep in touch today.
So I say once upon a time because social networking is not quite as new as it seems to be. In fact, I think the first true social network was formed back in 1594 when Shakespeare's "Hamlet" premiered at London's Globe Theatre and his performance was greeted by thunderous applause and a standing ovation by the entire audience.
At that time there were no movie theatre chains to advertise, no TV shows, no radio stations or even newspapers with paid reviews to promote "Hamlet" in advance of its opening night. Shakespeare's only way to get the word out about his latest production was through a series of "word-of-mouth" conversations between the people who had gone to see it and all those they encountered afterwards.
This was, by far, the most advanced social network that existed up until that time! And yet this type of social networking is probably still used today in the modern theatre world where actors and producers meet with audience members after their show to get feedback on how well (or not) it went over for them.
What we now call "social networking" is nothing more than the latest iteration of a centuries-old system that's already proven itself to be effective, but only when used by those who choose to engage in it voluntarily and without coercion. And yes, I realize that this particular definition of social networking has changed over time as well: from Shakespeare's "word of mouth" all the way up to the first online bulletin board systems (BBS) with 300-baud modems.
And yet, the latest innovation in social networking, Web 2.0 and its accompanying sites like Facebook, Twitter and LinkedIn still have yet to surpass these earlier methods in the minds of those who prefer not to use them (and they exist by virtue of an ever-growing user base).
So why is it that so many people are afraid of social networking? After all, there's no reason for anyone to feel compelled or coerced into joining these sites. And yet, despite this fact, a growing number of people seem more than willing to give up their personal information and privacy on the Internet. Why is that?
The answer is simple: most people don't have an accurate picture of what social networking really means. What they imagine it looks like bears little resemblance to how these sites actually work, let alone what's actually going on behind the scenes.
In a nutshell, those who believe that Web 2.0 is nothing but another attempt at getting us all "connected" are missing out on something very important: social networking isn't really about connecting with other people (much like Facebook and LinkedIn) or exchanging information (like Twitter). It's actually about the things we do when we connect, exchange information and interact.
So what does this mean? Simply put, all of these sites are ineffective at helping us get to know each other better. They have very little influence on how we choose who to trust or not to trust among our personal networks. What they're actually good for is gathering data (or information) about us as a way to sell us things that we don't really need and might not even want.
This isn't an attack on social networking, it's just the truth. Facebook may have started out as a site where students can connect with each other but it has now evolved into something much more sinister: a database of personal information about every one of its users that can be sold to anyone at any time without your consent (or even knowledge).
In effect, sites like LinkedIn and Facebook are nothing but the modern version of the old fashioned "spammers" who used to send us junk email. In addition to their obvious privacy concerns and their inability to help us connect with each other or exchange information, these social networking sites should also be regarded as a direct threat to our personal safety.
Why? Well, for one thing the information that they collect about us (and sell to others) can also be used by criminals to commit fraud against us and even extort money from us. This is why it's so important that we take control of this information and use it wisely instead of letting these sites control our private lives for their own selfish reasons.
The reality is that these sites cannot be trusted with the kind of personal information that they require about each one of us. Sites like LinkedIn or Facebook are nothing more than a threat to our privacy and should be regarded as such by every single person who uses them. In fact, sites like this (and any others) are in effect "spammers" who use the same tactics that spammers used to use in order to scam us into using their services.
I don't have a Facebook account and I don't plan on ever creating one either. This site is actually nothing more than a direct threat to my privacy because it uses the same old trick of collecting personal information about me (without my permission) in order to spam me with ads that will help them get rich at my expense. They have even resorted to using psychological tricks and sophisticated surveys in order to manipulate our feelings into believing that they are something important to us.
The truth is that sites like Facebook (or LinkedIn) can only be trusted if we're the ones who control them instead of letting others control them so that they can profit from it. In fact, a site like this can never even hope to become our friend because it doesn't respect the privacy rights of its users at all. This is why I am against these sites and their invasive surveys but if you want to know more about how these sites work then check out the link that we have below in order to learn a bit more about these sites. [end of text]

由于llama原始模型都是英文回答（后面會考慮試驗改進后的中文），有請詞霸翻譯一下：

先試驗到這里吧！文章來源地址http://www.zghlxwxcb.cn/news/detail-773860.html

到了這里，關(guān)于大模型部署手記（8）LLaMa2+Windows+llama.cpp+英文文本補齊的文章就介紹完了。如果您還想了解更多內(nèi)容，請在右上角搜索TOY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！