最近在學(xué)習(xí)docker的實(shí)現(xiàn)時(shí)看到這么一個(gè)概念:Union File System,先讓我們來(lái)介紹介紹它。
Union File System
定義:聯(lián)合文件系統(tǒng)(UnionFS)是一種分層、輕量級(jí)并且高性能的文件系統(tǒng),它支持對(duì)文件系統(tǒng)的修改作為一次提交來(lái)一層層的疊加,同時(shí)可以將不同目錄掛載到同一個(gè)虛擬文件系統(tǒng)下(unite several directories into a single virtual filesystem)。
主要有兩個(gè)細(xì)節(jié):
- 可以將不同目錄掛載到同一個(gè)虛擬文件系統(tǒng)下:
這就意味著一個(gè)文件系統(tǒng)被掛載時(shí)不再只能有一個(gè)目錄下的內(nèi)容,而是多個(gè)。 - 支持對(duì)文件系統(tǒng)的修改作為一次提交來(lái)一層層的疊加:
這點(diǎn)其實(shí)有一點(diǎn)像git的工作方式,每次的commit就相當(dāng)于一次增量,上層的commit由下層的一層層commit,以增量的方式組織在一起。
那么這么做有什么好處?
我們知道,鏡像可以理解為容器的模板,一套鏡像可以衍生出多個(gè)容器。
那么多個(gè)容器勢(shì)必有相同的鏡像層,如果我們每次拷貝一份,那么對(duì)存儲(chǔ)空間的要求的相當(dāng)大的,并且也不利于后續(xù)的整合和發(fā)布,因?yàn)檫@樣意味著當(dāng)別人需要你的容器時(shí),你commit的就是一整份鏡像,這好比使用git時(shí),當(dāng)你需要獲取origin的更新資源時(shí),需要pull一整份代碼下來(lái)。
因此,通過(guò)鏡像層增量的方式來(lái)組織,使不同的docker容器可以共享相同的鏡像層,再加上自己的改動(dòng)進(jìn)行發(fā)布。
而docker就支持多種UnionFS,如aufs、overlay2等等。
overlay2運(yùn)作方式
overlay2主要由merged、lowerdir、upperdir、workdir組成。
其中,lowerdir對(duì)應(yīng)底層文件系統(tǒng),也就是那一層層“commit”的內(nèi)容,它是能被上層文件系統(tǒng)upperdir所共享的只讀層。workdir則可以理解為overlay2運(yùn)作的一個(gè)工作目錄,用于完成copy-on-write等操作。copy-on-write這點(diǎn)我們會(huì)在實(shí)操中談到它。
overlay2運(yùn)作時(shí),會(huì)將lowerdir、upperdir和workdir聯(lián)合掛載到merged目錄,為使用者提供一個(gè)“統(tǒng)一視圖”。
這張其實(shí)是docker官網(wǎng)中的overlay的原理示意圖,不過(guò)也能幫助我們理解overlay2的運(yùn)作方式。這里要注意的是,overlay2中l(wèi)owerdir可以有很多層,這里只畫(huà)了一層,我們可以腦補(bǔ)它為很多層。overlay和overlay2的區(qū)別可以參考下面。
那么當(dāng)我們想刪除或者修改lowerdir中內(nèi)容時(shí),overlay2是如何處理的?不是說(shuō)它是只讀層嗎,只讀意味著無(wú)法修改,但我們又確實(shí)可能需要修改它,那lowerdir的出現(xiàn)都無(wú)法滿(mǎn)足我們的使用需求了嗎?
下面一起實(shí)操來(lái)看看overlay2是怎么解決這些問(wèn)題的吧。
overlay2實(shí)操
下面是man mount中對(duì)overlay2的描述。
Mount options for overlay
Since Linux 3.18 the overlay pseudo filesystem implements a union mount for other filesystems.
An overlay filesystem combines two filesystems - an upper filesystem and a lower filesystem. When a name exists in both filesystems, the
object in the upper filesystem is visible while the object in the lower filesystem is either hidden or, in the case of directories,
merged with the upper object.
The lower filesystem can be any filesystem supported by Linux and does not need to be writable. The lower filesystem can even be another
overlayfs. The upper filesystem will normally be writable and if it is it must support the creation of trusted.* extended attributes,
and must provide a valid d_type in readdir responses, so NFS is not suitable.
A read-only overlay of two read-only filesystems may use any filesystem type. The options lowerdir and upperdir are combined into a
merged directory by using:
mount -t overlay overlay \
-olowerdir=/lower,upperdir=/upper,workdir=/work /merged
lowerdir=directory
Any filesystem, does not need to be on a writable filesystem.
upperdir=directory
The upperdir is normally on a writable filesystem.
workdir=directory
The workdir needs to be an empty directory on the same filesystem as upperdir.
我們照葫蘆畫(huà)瓢,創(chuàng)建好對(duì)應(yīng)的目錄和文件。
nigo@DESKTOP-95TV8LK ~/overlay2> tree
.
├── lower1
├── lower2
├── merged
├── upper
└── work
5 directories, 0 file
nigo@DESKTOP-95TV8LK ~/overlay2> echo 'I\'m file1, belong to lower1' > lower1/file1.txt
nigo@DESKTOP-95TV8LK ~/overlay2> echo 'I\'m file2, belong to lower2' > lower2/file2.txt
nigo@DESKTOP-95TV8LK ~/overlay2> echo 'I\'m file3, belong to upper' > upper/file3.txt
現(xiàn)在lowerdir和upperdir都有它們各自的文件。
nigo@DESKTOP-95TV8LK ~/overlay2> tree
.
├── lower1
│ └── file1.txt
├── lower2
│ └── file2.txt
├── merged
├── upper
│ └── file3.txt
└── work
5 directories, 3 files
可以看到,我們成功掛載了這些目錄。
nigo@DESKTOP-95TV8LK ~/overlay2 [1]> mount | grep overlay
nigo@DESKTOP-95TV8LK ~/overlay2 [0|1]> sudo mount -t overlay overlay -olowerdir=lower1:lower2,upperdir=upper,workdir=work merged/
[sudo] password for nigo:
nigo@DESKTOP-95TV8LK ~/overlay2> mount | grep overlay
overlay on /home/nigo/overlay2/merged type overlay (rw,relatime,lowerdir=lower1:lower2,upperdir=upper,workdir=work)
而merged中也出現(xiàn)了我們期待的內(nèi)容。
nigo@DESKTOP-95TV8LK ~/overlay2> sudo tree
.
├── lower1
│ └── file1.txt
├── lower2
│ └── file2.txt
├── merged
│ ├── file1.txt
│ ├── file2.txt
│ └── file3.txt
├── upper
│ └── file3.txt
└── work
└── work
6 directories, 6 files
nigo@DESKTOP-95TV8LK ~/overlay2> cat merged/*
I'm file1, belong to lower1
I'm file2, belong to lower2
I'm file3, belong to upper
改動(dòng)屬于upperdir的內(nèi)容肯定會(huì)映射到upperdir中,畢竟upperdir是屬于當(dāng)前層的可讀寫(xiě)層。而上面提到的lowerdir呢?讓我們驗(yàn)證一下它吧。
修改lowerdir的文件
修改lowerdir1中的file1。
nigo@DESKTOP-95TV8LK ~/overlay2> echo 'I\'m file1, belong to lower1' > lower1/file1.txt
nigo@DESKTOP-95TV8LK ~/overlay2> echo 'file1 has been changed' > merged/file1.txt
nigo@DESKTOP-95TV8LK ~/overlay2> cat merged/*
file1 has been changed
I'm file2, belong to lower2
I'm file3, belong to upper
nigo@DESKTOP-95TV8LK ~/overlay2> cat lower1/file1.txt
I'm file1, belong to lower1
nigo@DESKTOP-95TV8LK ~/overlay2> sudo tree
.
├── lower1
│ └── file1.txt
├── lower2
│ └── file2.txt
├── merged
│ ├── file1.txt
│ ├── file2.txt
│ └── file3.txt
├── upper
│ ├── file1.txt
│ └── file3.txt
└── work
└── work
6 directories, 7 files
可以看到,merged中的file1.txt確實(shí)被我們修改了,但lowerdir中的內(nèi)容仍然不變,而是在upperdir中生成了一個(gè)file1.txt,這就是copy-on-write。這也驗(yàn)證了lowerdir是只讀層這一點(diǎn)。
copy-on-write,即寫(xiě)時(shí)復(fù)制,概念類(lèi)似于linux fork后嘗試對(duì)父子進(jìn)程共享的頁(yè)表進(jìn)行修改時(shí),內(nèi)核為子進(jìn)程重新復(fù)制一份頁(yè)表。在這里,overlay2在upperdir中生成了一份file1.txt。
刪除lowerdir的文件
試著刪除file2.txt。
nigo@DESKTOP-95TV8LK ~/overlay2> rm merged/file2.txt
nigo@DESKTOP-95TV8LK ~/overlay2> sudo tree
.
├── lower1
│ └── file1.txt
├── lower2
│ └── file2.txt
├── merged
│ ├── file1.txt
│ └── file3.txt
├── upper
│ ├── file1.txt
│ ├── file2.txt
│ └── file3.txt
└── work
└── work
6 directories, 7 files
nigo@DESKTOP-95TV8LK ~/overlay2> cat merged/*
file1 has been changed
I'm file3, belong to upper
nigo@DESKTOP-95TV8LK ~/overlay2> cat lower2/file2.txt
I'm file2, belong to lower2
nigo@DESKTOP-95TV8LK ~/overlay2> ll upper/file2.txt
c--------- 1 root root 0, 0 Apr 22 14:44 upper/file2.txt
從結(jié)果來(lái)看,file2.txt確實(shí)被我們“刪掉”了。但在upperdir中,我們看到生成了一個(gè)file2.txt的特殊的字符設(shè)備文件。
而overlay2在聯(lián)合掛載時(shí),看到這個(gè)特殊的字符設(shè)備文件,會(huì)選擇性的忽略lowerdir中對(duì)應(yīng)的內(nèi)容。
而在aufs(另一種UnionFS)中表現(xiàn)為whiteout文件(可以查一下這個(gè)詞,意為臨時(shí)性失明,挺形象的emm)。感興趣的同學(xué)可以搜一下,目前網(wǎng)上的大多數(shù)實(shí)驗(yàn)也是關(guān)于aufs的。
overlay2與Docker
行實(shí)驗(yàn)前,請(qǐng)確保你的docker使用overlay2的驅(qū)動(dòng)方式。
可以使用docker info,或者查看/etc/docker/daemon.json中的內(nèi)容。
具體請(qǐng)參考:Use the OverlayFS storage driver | Docker Documentation
nigo@DESKTOP-95TV8LK ~> docker info | grep Storage
Storage Driver: overlay2
我們以u(píng)buntu為例子,pull下來(lái)三層鏡像層。
nigo@DESKTOP-95TV8LK ~> docker pull ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
a70d879fa598: Pull complete
c4394a92d1f8: Pull complete
10e6159c56c0: Pull complete
Digest: sha256:3c9c713e0979e9bd6061ed52ac1e9e1f246c9495aa063619d9d695fb8039aa1f
Status: Downloaded newer image for ubuntu:latest
docker.io/library/ubuntu:latest
在/var/lib/docker/overlay2中我們可以成功的找到它們,由于一些目錄比較深,我們可以通過(guò)-L參數(shù)來(lái)指定tree訪(fǎng)問(wèn)的深度。
root@DESKTOP-95TV8LK:/var/lib/docker/overlay2# tree -L 2
.
├── 809e4cfaa089d57ba81faea4570d6689cf6fe9a424b982ba6859b094340eef04
│ ├── committed
│ ├── diff
│ └── link
├── ab0963faec278aa7c9c40c79642774451b2a5ecd9142706d7b6165864d55ad59
│ ├── committed
│ ├── diff
│ ├── link
│ ├── lower
│ └── work
├── adfcb936bd0fac351f71721610abbec97b7309c1ae8323ebc6795c5c96ac0278
│ ├── diff
│ ├── link
│ ├── lower
│ └── work
└── l
├── 2FMRPFC5X2PFHGEZII4KC47JF4 -> ../ab0963faec278aa7c9c40c79642774451b2a5ecd9142706d7b6165864d55ad59/diff
├── EVRLLRGLJ5K5374Z6B32BREDLT -> ../809e4cfaa089d57ba81faea4570d6689cf6fe9a424b982ba6859b094340eef04/diff
└── UXEF4DBCSIKECRM2J2IPAD4WY5 -> ../adfcb936bd0fac351f71721610abbec97b7309c1ae8323ebc6795c5c96ac0278/diff
12 directories, 7 files
在/var/lib/docker/overlay2中我們可以成功的找到它們,由于一些目錄比較深,我們可以通過(guò)-L參數(shù)來(lái)指定tree訪(fǎng)問(wèn)的深度。
root@DESKTOP-95TV8LK:/var/lib/docker/overlay2# tree -L 2
.
├── 809e4cfaa089d57ba81faea4570d6689cf6fe9a424b982ba6859b094340eef04
│ ├── committed
│ ├── diff
│ └── link
├── ab0963faec278aa7c9c40c79642774451b2a5ecd9142706d7b6165864d55ad59
│ ├── committed
│ ├── diff
│ ├── link
│ ├── lower
│ └── work
├── adfcb936bd0fac351f71721610abbec97b7309c1ae8323ebc6795c5c96ac0278
│ ├── diff
│ ├── link
│ ├── lower
│ └── work
└── l
├── 2FMRPFC5X2PFHGEZII4KC47JF4 -> ../ab0963faec278aa7c9c40c79642774451b2a5ecd9142706d7b6165864d55ad59/diff
├── EVRLLRGLJ5K5374Z6B32BREDLT -> ../809e4cfaa089d57ba81faea4570d6689cf6fe9a424b982ba6859b094340eef04/diff
└── UXEF4DBCSIKECRM2J2IPAD4WY5 -> ../adfcb936bd0fac351f71721610abbec97b7309c1ae8323ebc6795c5c96ac0278/diff
12 directories, 7 files
但在這里多出了一個(gè)l目錄,里面存放的指向各層的軟鏈接,而且軟鏈接的名字顯然被縮短過(guò)。根據(jù)docker官網(wǎng)的說(shuō)明,這些軟鏈接用于避免達(dá)到 mount命令對(duì)頁(yè)面參數(shù)的頁(yè)面大小限制。
在各層目錄下還存在著link文件,這些目錄中放的就是l目錄中那些被縮短過(guò)的軟鏈接名稱(chēng)。
root@DESKTOP-95TV8LK:/var/lib/docker/overlay2# cat adfcb936bd0fac351f71721610abbec97b7309c1ae8323ebc6795c5c96ac0278/link
UXEF4DBCSIKECRM2J2IPAD4WY5
使用docker inspect,在GraphDriver這里找到關(guān)于該ubuntu鏡像的信息。
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/ab0963faec278aa7c9c40c79642774451b2a5ecd9142706d7b6165864d55ad59/diff:/var/lib/docker/overlay2/809e4cfaa089d57ba81faea4570d6689cf6fe9a424b982ba6859b094340eef04/diff",
"MergedDir": "/var/lib/docker/overlay2/adfcb936bd0fac351f71721610abbec97b7309c1ae8323ebc6795c5c96ac0278/merged",
"UpperDir": "/var/lib/docker/overlay2/adfcb936bd0fac351f71721610abbec97b7309c1ae8323ebc6795c5c96ac0278/diff",
"WorkDir": "/var/lib/docker/overlay2/adfcb936bd0fac351f71721610abbec97b7309c1ae8323ebc6795c5c96ac0278/work"
},
"Name": "overlay2"
}
從inspect返回的信息中可以看到,各層的diff目錄就是該層“different”于下層的內(nèi)容,對(duì)于下層鏡像,它是只讀層(lowerdir),而對(duì)于上層,它是可讀寫(xiě)層(upperdir),它們也是和workdir被聯(lián)合掛載到mergeddir的。
上面沒(méi)有提到的commit文件則是記錄這每個(gè)層的相關(guān)commit信息。
最后讓我們來(lái)嘗試一下創(chuàng)建容器,以及commit對(duì)該目錄的影響。
nigo@DESKTOP-95TV8LK ~> docker run -itd --name myubuntu ubuntu
d6adc07566d205f1554b2db9534c76713f830a7705e9f41ab30f9c0f4d118b1d
root@DESKTOP-95TV8LK:/var/lib/docker/overlay2# tree -L 2
.
├── 6c287e2696d9a1593ae358045b511d95e6dc5f1bbe021a4f72a7892f3a8c5778
│ ├── diff
│ ├── link
│ ├── lower
│ ├── merged
│ └── work
├── 6c287e2696d9a1593ae358045b511d95e6dc5f1bbe021a4f72a7892f3a8c5778-init
│ ├── committed
│ ├── diff
│ ├── link
│ ├── lower
│ └── work
├── 809e4cfaa089d57ba81faea4570d6689cf6fe9a424b982ba6859b094340eef04
│ ├── committed
│ ├── diff
│ └── link
├── ab0963faec278aa7c9c40c79642774451b2a5ecd9142706d7b6165864d55ad59
│ ├── committed
│ ├── diff
│ ├── link
│ ├── lower
│ └── work
├── adfcb936bd0fac351f71721610abbec97b7309c1ae8323ebc6795c5c96ac0278
│ ├── committed
│ ├── diff
│ ├── link
│ ├── lower
│ └── work
└── l
├── 2FMRPFC5X2PFHGEZII4KC47JF4 -> ../ab0963faec278aa7c9c40c79642774451b2a5ecd9142706d7b6165864d55ad59/diff
├── EVRLLRGLJ5K5374Z6B32BREDLT -> ../809e4cfaa089d57ba81faea4570d6689cf6fe9a424b982ba6859b094340eef04/diff
├── M32S6F25IVIE4YRIR2BYQX5S4H -> ../6c287e2696d9a1593ae358045b511d95e6dc5f1bbe021a4f72a7892f3a8c5778-init/diff
├── RPNEL4XVFQMLG5Q4BF7BXUYY5C -> ../6c287e2696d9a1593ae358045b511d95e6dc5f1bbe021a4f72a7892f3a8c5778/diff
└── UXEF4DBCSIKECRM2J2IPAD4WY5 -> ../adfcb936bd0fac351f71721610abbec97b7309c1ae8323ebc6795c5c96ac0278/diff
運(yùn)行容器后生成了兩個(gè)新的層,其中一個(gè)為init層,這是用來(lái)存儲(chǔ)和容器環(huán)境相關(guān)內(nèi)容的只讀層,由于這些環(huán)境在每臺(tái)機(jī)器上都可能不同,docker的策略是放在init層,每個(gè)鏡像生成容器時(shí)去生成環(huán)境相關(guān)的配置。我們?cè)赿ocker commit時(shí),不提交init層的內(nèi)容。
寫(xiě)入新的文件,執(zhí)行docker commit來(lái)觀(guān)察結(jié)果。
nigo@DESKTOP-95TV8LK ~> docker start myubuntu
myubuntu
nigo@DESKTOP-95TV8LK ~> docker exec -it myubuntu /bin/bash
root@d6adc07566d2:/# touch hello-overlay2.txt
root@d6adc07566d2:/# exit
root@DESKTOP-95TV8LK:/var/lib/docker/overlay2# tree -L 2
.
├── 0991cf894ea2ed9bb2b8313331ac9eb72c3678c26dc0152241e6228105df25f2
│ ├── diff
│ ├── link
│ ├── lower
│ └── work
├── 6c287e2696d9a1593ae358045b511d95e6dc5f1bbe021a4f72a7892f3a8c5778
│ ├── diff
│ ├── link
│ ├── lower
│ └── work
├── 6c287e2696d9a1593ae358045b511d95e6dc5f1bbe021a4f72a7892f3a8c5778-init
│ ├── committed
│ ├── diff
│ ├── link
│ ├── lower
│ └── work
├── 809e4cfaa089d57ba81faea4570d6689cf6fe9a424b982ba6859b094340eef04
│ ├── committed
│ ├── diff
│ └── link
├── ab0963faec278aa7c9c40c79642774451b2a5ecd9142706d7b6165864d55ad59
│ ├── committed
│ ├── diff
│ ├── link
│ ├── lower
│ └── work
├── adfcb936bd0fac351f71721610abbec97b7309c1ae8323ebc6795c5c96ac0278
│ ├── committed
│ ├── diff
│ ├── link
│ ├── lower
│ └── work
└── l
├── 2FMRPFC5X2PFHGEZII4KC47JF4 -> ../ab0963faec278aa7c9c40c79642774451b2a5ecd9142706d7b6165864d55ad59/diff
├── DD5NLJQIUJRFR45OMBJIF26CQ3 -> ../0991cf894ea2ed9bb2b8313331ac9eb72c3678c26dc0152241e6228105df25f2/diff
├── EVRLLRGLJ5K5374Z6B32BREDLT -> ../809e4cfaa089d57ba81faea4570d6689cf6fe9a424b982ba6859b094340eef04/diff
├── M32S6F25IVIE4YRIR2BYQX5S4H -> ../6c287e2696d9a1593ae358045b511d95e6dc5f1bbe021a4f72a7892f3a8c5778-init/diff
├── RPNEL4XVFQMLG5Q4BF7BXUYY5C -> ../6c287e2696d9a1593ae358045b511d95e6dc5f1bbe021a4f72a7892f3a8c5778/diff
└── UXEF4DBCSIKECRM2J2IPAD4WY5 -> ../adfcb936bd0fac351f71721610abbec97b7309c1ae8323ebc6795c5c96ac0278/diff
24 directories, 15 files
果然,新的鏡像層生成了!
nigo@DESKTOP-95TV8LK ~> mount | grep overlay
overlay on /var/lib/docker/overlay2/6c287e2696d9a1593ae358045b511d95e6dc5f1bbe021a4f72a7892f3a8c5778/merged type overlay (rw,relatime,lowerdir=/var/lib/docker/overlay2/l/M32S6F25IVIE4YRIR2BYQX5S4H:/var/lib/docker/overlay2/l/UXEF4DBCSIKECRM2J2IPAD4WY5:/var/lib/docker/overlay2/l/2FMRPFC5X2PFHGEZII4KC47JF4:/var/lib/docker/overlay2/l/EVRLLRGLJ5K5374Z6B32BREDLT,upperdir=/var/lib/docker/overlay2/6c287e2696d9a1593ae358045b511d95e6dc5f1bbe021a4f72a7892f3a8c5778/diff,workdir=/var/lib/docker/overlay2/6c287e2696d9a1593ae358045b511d95e6dc5f1bbe021a4f72a7892f3a8c5778/work)
而這些被聯(lián)合掛載的目錄,也是l目錄中那些被縮短過(guò)的軟鏈接。
overlay和overlay2
由于內(nèi)核支持的原因,這點(diǎn)我并沒(méi)有做過(guò)具體的實(shí)驗(yàn),下面是道聽(tīng)途說(shuō)的結(jié)論:
不同點(diǎn):
overlay的lowdir只有一層,每層只讀層都通過(guò)硬鏈接共享文件,因此每層只讀層都有一套完整的增量。
overlay2的只讀層是獨(dú)立的個(gè)體,容器啟動(dòng)時(shí)統(tǒng)一掛載到merged。
相同點(diǎn):
都有work目錄用于完成copy-on-write等工作,啟動(dòng)時(shí)掛載到merged目錄。
都支持頁(yè)緩存,同一鏡像的容器有機(jī)會(huì)使用同一文件,內(nèi)存消耗更小。
具體請(qǐng)參考:overlay和overlay2的區(qū)別 - ElNinoT - 博客園
實(shí)驗(yàn)之前也請(qǐng)同樣的修改docker的Storage Driver。
ref:https://blog.csdn.net/qq_45858169/article/details/115918469文章來(lái)源:http://www.zghlxwxcb.cn/news/detail-723483.html
overlay2 的文件過(guò)多
可通過(guò)執(zhí)行docker system prune 命令可用于清理磁盤(pán),刪除關(guān)閉的容器、無(wú)用的數(shù)據(jù)卷和網(wǎng)絡(luò),以及dangling鏡像(即無(wú)tag的鏡像)文章來(lái)源地址http://www.zghlxwxcb.cn/news/detail-723483.html
到了這里,關(guān)于Docker:overlay2淺析以及解決overlay2 文件過(guò)大的問(wèn)題的文章就介紹完了。如果您還想了解更多內(nèi)容,請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章,希望大家以后多多支持TOY模板網(wǎng)!