本文分享自華為云社區(qū)《K8s 鏡像緩存管理 kube-fledged 認(rèn)知》，作者： 山河已無(wú)恙。
我們知道?k8s?上的容器調(diào)度需要在調(diào)度的節(jié)點(diǎn)行拉取當(dāng)前容器的鏡像，在一些特殊場(chǎng)景中，
-
需要
快速啟動(dòng)和/或擴(kuò)展的應(yīng)用程序。例如，由于數(shù)據(jù)量激增，執(zhí)行實(shí)時(shí)數(shù)據(jù)處理的應(yīng)用程序需要快速擴(kuò)展。 -
鏡像比較龐大，涉及多個(gè)版本，節(jié)點(diǎn)存儲(chǔ)有限，需要?jiǎng)討B(tài)清理不需要的鏡像
-
無(wú)服務(wù)器函數(shù)通常需要在幾分之一秒內(nèi)立即對(duì)傳入事件和啟動(dòng)容器做出反應(yīng)。 -
在邊緣設(shè)備上運(yùn)行的?
IoT 應(yīng)用程序，需要容忍邊緣設(shè)備和鏡像鏡像倉(cāng)庫(kù)之間的間歇性網(wǎng)絡(luò)連接。 -
如果需要從
專用倉(cāng)庫(kù)中拉取鏡像，并且無(wú)法授予每個(gè)人從此鏡像倉(cāng)庫(kù)拉取鏡像的訪問(wèn)權(quán)限，則可以在群集的節(jié)點(diǎn)上提供鏡像。 -
如果集群管理員或操作員需要對(duì)應(yīng)用程序進(jìn)行升級(jí)，并希望事先驗(yàn)證是否可以成功拉取新鏡像。
kube-fledged?是一個(gè)?kubernetes operator，用于直接在 Kubernetes 集群的?worker?節(jié)點(diǎn)上創(chuàng)建和管理容器鏡像緩存。它允許用戶定義鏡像列表以及這些鏡像應(yīng)緩存到哪些工作節(jié)點(diǎn)上（即拉?。?。因此，應(yīng)用程序 Pod 幾乎可以立即啟動(dòng)，因?yàn)椴恍枰獜溺R像倉(cāng)庫(kù)中提取鏡像。
kube-fledged?提供了 CRUD API 來(lái)管理鏡像緩存的生命周期，并支持多個(gè)可配置的參數(shù)，可以根據(jù)自己的需要自定義功能。
Kubernetes 具有內(nèi)置的鏡像垃圾回收機(jī)制。節(jié)點(diǎn)中的 kubelet 會(huì)定期檢查磁盤使用率是否達(dá)到特定閾值（可通過(guò)標(biāo)志進(jìn)行配置）。一旦達(dá)到這個(gè)閾值，kubelet 會(huì)自動(dòng)刪除節(jié)點(diǎn)中所有未使用的鏡像。
需要在建議的解決方案中實(shí)現(xiàn)自動(dòng)和定期刷新機(jī)制。如果鏡像緩存中的鏡像被 kubelet 的 gc 刪除，下一個(gè)刷新周期會(huì)將已刪除的鏡像拉入鏡像緩存中。這可確保鏡像緩存是最新的。
設(shè)計(jì)流程
https://github.com/senthilrch/kube-fledged/blob/master/docs/kubefledged-architecture.png
部署 kube-fledged
Helm 方式部署
──[root@vms100.liruilongs.github.io]-[~/ansible] └─$mkdir kube-fledged ┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$cd kube-fledged ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged] └─$export KUBEFLEDGED_NAMESPACE=kube-fledged ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged] └─$kubectl create namespace ${KUBEFLEDGED_NAMESPACE} namespace/kube-fledged created ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged] └─$helm repo add kubefledged-charts https://senthilrch.github.io/kubefledged-charts/ "kubefledged-charts" has been added to your repositories ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged] └─$helm repo update Hang tight while we grab the latest from your chart repositories... ...Successfully got an update from the "kubefledged-charts" chart repository ...Successfully got an update from the "kubescape" chart repository ...Successfully got an update from the "rancher-stable" chart repository ...Successfully got an update from the "skm" chart repository ...Successfully got an update from the "openkruise" chart repository ...Successfully got an update from the "awx-operator" chart repository ...Successfully got an update from the "botkube" chart repository Update Complete. ?Happy Helming!? ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged] └─$helm install --verify kube-fledged kubefledged-charts/kube-fledged -n ${KUBEFLEDGED_NAMESPACE} --wait
實(shí)際部署中發(fā)現(xiàn)，由于網(wǎng)絡(luò)問(wèn)題，chart?無(wú)法下載，所以通過(guò)?make deploy-using-yaml?使用 yaml 方式部署
Yaml 文件部署
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged] └─$git clone https://github.com/senthilrch/kube-fledged.git 正克隆到 'kube-fledged'... remote: Enumerating objects: 10613, done. remote: Counting objects: 100% (1501/1501), done. remote: Compressing objects: 100% (629/629), done. remote: Total 10613 (delta 845), reused 1357 (delta 766), pack-reused 9112 接收對(duì)象中: 100% (10613/10613), 34.58 MiB | 7.33 MiB/s, done. 處理 delta 中: 100% (4431/4431), done. ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged] └─$ls kube-fledged ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged] └─$cd kube-fledged/ ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged] └─$make deploy-using-yaml kubectl apply -f deploy/kubefledged-namespace.yaml
第一次部署，發(fā)現(xiàn)鏡像拉不下來(lái)
┌──[root@vms100.liruilongs.github.io]-[~] └─$kubectl get all -n kube-fledged NAME READY STATUS RESTARTS AGE pod/kube-fledged-controller-df69f6565-drrqg 0/1 CrashLoopBackOff 35 (5h59m ago) 21h pod/kube-fledged-webhook-server-7bcd589bc4-b7kg2 0/1 Init:CrashLoopBackOff 35 (5h58m ago) 21h pod/kubefledged-controller-55f848cc67-7f4rl 1/1 Running 0 21h pod/kubefledged-webhook-server-597dbf4ff5-l8fbh 0/1 Init:CrashLoopBackOff 34 (6h ago) 21h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kube-fledged-webhook-server ClusterIP 10.100.194.199 <none> 3443/TCP 21h service/kubefledged-webhook-server ClusterIP 10.101.191.206 <none> 3443/TCP 21h NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/kube-fledged-controller 0/1 1 0 21h deployment.apps/kube-fledged-webhook-server 0/1 1 0 21h deployment.apps/kubefledged-controller 0/1 1 0 21h deployment.apps/kubefledged-webhook-server 0/1 1 0 21h NAME DESIRED CURRENT READY AGE replicaset.apps/kube-fledged-controller-df69f6565 1 1 0 21h replicaset.apps/kube-fledged-webhook-server-7bcd589bc4 1 1 0 21h replicaset.apps/kubefledged-controller-55f848cc67 1 1 0 21h replicaset.apps/kubefledged-webhook-server-597dbf4ff5 1 1 0 21h ┌──[root@vms100.liruilongs.github.io]-[~] └─$
這里我們找一下要拉取的鏡像
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$cat *.yaml | grep image: - image: senthilrch/kubefledged-controller:v0.10.0 - image: senthilrch/kubefledged-webhook-server:v0.10.0 - image: senthilrch/kubefledged-webhook-server:v0.10.0
單獨(dú)拉取一些，當(dāng)前使用?ansible?在所有工作節(jié)點(diǎn)批量操作
┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$ansible k8s_node -m shell -a "docker pull docker.io/senthilrch/kubefledged-cri-client:v0.10.0" -i host.yaml
其他相關(guān)的鏡像都拉取一下
操作完成之后容器狀態(tài)全部正常
┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$kubectl -n kube-fledged get all NAME READY STATUS RESTARTS AGE pod/kube-fledged-controller-df69f6565-wdb4g 1/1 Running 0 13h pod/kube-fledged-webhook-server-7bcd589bc4-j8xxp 1/1 Running 0 13h pod/kubefledged-controller-55f848cc67-klxlm 1/1 Running 0 13h pod/kubefledged-webhook-server-597dbf4ff5-ktbsh 1/1 Running 0 13h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kube-fledged-webhook-server ClusterIP 10.100.194.199 <none> 3443/TCP 36h service/kubefledged-webhook-server ClusterIP 10.101.191.206 <none> 3443/TCP 36h NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/kube-fledged-controller 1/1 1 1 36h deployment.apps/kube-fledged-webhook-server 1/1 1 1 36h deployment.apps/kubefledged-controller 1/1 1 1 36h deployment.apps/kubefledged-webhook-server 1/1 1 1 36h NAME DESIRED CURRENT READY AGE replicaset.apps/kube-fledged-controller-df69f6565 1 1 1 36h replicaset.apps/kube-fledged-webhook-server-7bcd589bc4 1 1 1 36h replicaset.apps/kubefledged-controller-55f848cc67 1 1 1 36h replicaset.apps/kubefledged-webhook-server-597dbf4ff5 1 1 1 36h
驗(yàn)證是否安裝成功
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged] └─$kubectl get pods -n kube-fledged -l app=kubefledged NAME READY STATUS RESTARTS AGE kubefledged-controller-55f848cc67-klxlm 1/1 Running 0 16h kubefledged-webhook-server-597dbf4ff5-ktbsh 1/1 Running 0 16h ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged] └─$kubectl get imagecaches -n kube-fledged No resources found in kube-fledged namespace.
使用 kubefledged
創(chuàng)建鏡像緩存對(duì)象
根據(jù)?Demo?文件，創(chuàng)建鏡像緩存對(duì)象
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged] └─$cd deploy/ ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$cat kubefledged-imagecache.yaml --- apiVersion: kubefledged.io/v1alpha2 kind: ImageCache metadata: # Name of the image cache. A cluster can have multiple image cache objects name: imagecache1 namespace: kube-fledged # The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preference labels: app: kubefledged kubefledged: imagecache spec: # The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled). cacheSpec: # Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster - images: - ghcr.io/jitesoft/nginx:1.23.1 # Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector - images: - us.gcr.io/k8s-artifacts-prod/cassandra:v7 - us.gcr.io/k8s-artifacts-prod/etcd:3.5.4-0 nodeSelector: tier: backend # Specifies a list of image pull secrets to pull images from private repositories into the cache imagePullSecrets: - name: myregistrykey
官方的 Demo 中對(duì)應(yīng)的 鏡像拉取不下來(lái)，所以換一下
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$docker pull us.gcr.io/k8s-artifacts-prod/cassandra:v7 Error response from daemon: Get "https://us.gcr.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$
為了測(cè)試選擇器標(biāo)簽的使用,我們找一個(gè)節(jié)點(diǎn)的標(biāo)簽單獨(dú)做鏡像緩存
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$kubectl get nodes --show-labels
同時(shí)我們直接從公有倉(cāng)庫(kù)拉取鏡像，所以不需要?imagePullSecrets?對(duì)象
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy]
└─$vim kubefledged-imagecache.yaml
修改后的?yaml?文件
-
添加了一個(gè)所有節(jié)點(diǎn)的 liruilong/my-busybox:latest 鏡像緩存
-
添加了一個(gè)?
kubernetes.io/hostname: vms105.liruilongs.github.io?對(duì)應(yīng)標(biāo)簽選擇器的?liruilong/hikvision-sdk-config-ftp:latest?鏡像緩存
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$cat kubefledged-imagecache.yaml --- apiVersion: kubefledged.io/v1alpha2 kind: ImageCache metadata: # Name of the image cache. A cluster can have multiple image cache objects name: imagecache1 namespace: kube-fledged # The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preference labels: app: kubefledged kubefledged: imagecache spec: # The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled). cacheSpec: # Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster - images: - liruilong/my-busybox:latest # Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector - images: - liruilong/hikvision-sdk-config-ftp:latest nodeSelector: kubernetes.io/hostname: vms105.liruilongs.github.io # Specifies a list of image pull secrets to pull images from private repositories into the cache #imagePullSecrets: #- name: myregistrykey ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$
直接創(chuàng)建報(bào)錯(cuò)了
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$kubectl create -f kubefledged-imagecache.yaml Error from server (InternalError): error when creating "kubefledged-imagecache.yaml": Internal error occurred: failed calling webhook "validate-image-cache.kubefledged.io": failed to call webhook: Post "https://kubefledged-webhook-server.kube-fledged.svc:3443/validate-image-cache?timeout=1s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubefledged.io") ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$kubectl get imagecaches -n kube-fledged No resources found in kube-fledged namespace. ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$
解決辦法，刪除對(duì)應(yīng)的對(duì)象，重新創(chuàng)建
我在當(dāng)前項(xiàng)目的一個(gè)?issues?下面找到了解決辦法?https://github.com/senthilrch/kube-fledged/issues/76
看起來(lái)這是因?yàn)?Webhook CA?是硬編碼的，但是當(dāng)?webhook?服務(wù)器啟動(dòng)時(shí)，會(huì)生成一個(gè)新的 CA 捆綁包并更新 webhook 配置。當(dāng)發(fā)生另一個(gè)部署時(shí)，將重新應(yīng)用原始 CA 捆綁包，并且 Webhook 請(qǐng)求開(kāi)始失敗，直到再次重新啟動(dòng) Webhook 組件以修補(bǔ)捆綁包init-server
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged] └─$make remove-kubefledged-and-operator # Remove kubefledged kubectl delete -f deploy/kubefledged-operator/deploy/crds/charts.helm.kubefledged.io_v1alpha2_kubefledged_cr.yaml error: resource mapping not found for name: "kube-fledged" namespace: "kube-fledged" from "deploy/kubefledged-operator/deploy/crds/charts.helm.kubefledged.io_v1alpha2_kubefledged_cr.yaml": no matches for kind "KubeFledged" in version "charts.helm.kubefledged.io/v1alpha2" ensure CRDs are installed first ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged] └─$make deploy-using-yaml kubectl apply -f deploy/kubefledged-namespace.yaml namespace/kube-fledged created kubectl apply -f deploy/kubefledged-crd.yaml customresourcedefinition.apiextensions.k8s.io/imagecaches.kubefledged.io unchanged .................... kubectl rollout status deployment kubefledged-webhook-server -n kube-fledged --watch Waiting for deployment "kubefledged-webhook-server" rollout to finish: 0 of 1 updated replicas are available... deployment "kubefledged-webhook-server" successfully rolled out kubectl get pods -n kube-fledged NAME READY STATUS RESTARTS AGE kubefledged-controller-55f848cc67-76c4v 1/1 Running 0 112s kubefledged-webhook-server-597dbf4ff5-56h6z 1/1 Running 0 66s
重新創(chuàng)建緩存對(duì)象，創(chuàng)建成功
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$kubectl create -f kubefledged-imagecache.yaml imagecache.kubefledged.io/imagecache1 created ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$kubectl get imagecaches -n kube-fledged NAME AGE imagecache1 10s ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$
查看當(dāng)前被納管的鏡像緩存
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged] └─$kubectl get imagecaches imagecache1 -n kube-fledged -o json { "apiVersion": "kubefledged.io/v1alpha2", "kind": "ImageCache", "metadata": { "creationTimestamp": "2024-03-01T15:08:42Z", "generation": 83, "labels": { "app": "kubefledged", "kubefledged": "imagecache" }, "name": "imagecache1", "namespace": "kube-fledged", "resourceVersion": "20169836", "uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72" }, "spec": { "cacheSpec": [ { "images": [ "liruilong/my-busybox:latest" ] }, { "images": [ "liruilong/hikvision-sdk-config-ftp:latest" ], "nodeSelector": { "kubernetes.io/hostname": "vms105.liruilongs.github.io" } } ] }, "status": { "completionTime": "2024-03-02T01:06:47Z", "message": "All requested images pulled succesfully to respective nodes", "reason": "ImageCacheRefresh", "startTime": "2024-03-02T01:05:33Z", "status": "Succeeded" } } ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged] └─$
通過(guò) ansible 來(lái)驗(yàn)證
┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml 192.168.26.102 | CHANGED | rc=0 >> liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB 192.168.26.101 | CHANGED | rc=0 >> liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB 192.168.26.103 | CHANGED | rc=0 >> liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB 192.168.26.105 | CHANGED | rc=0 >> liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB 192.168.26.100 | CHANGED | rc=0 >> liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB 192.168.26.106 | CHANGED | rc=0 >> liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB ┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$ ┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$ansible all -m shell -a "docker images | grep liruilong/hikvision-sdk-config-ftp" -i host.yaml 192.168.26.102 | FAILED | rc=1 >> non-zero return code 192.168.26.100 | FAILED | rc=1 >> non-zero return code 192.168.26.103 | FAILED | rc=1 >> non-zero return code 192.168.26.105 | CHANGED | rc=0 >> liruilong/hikvision-sdk-config-ftp latest a02cd03b4342 4 months ago 830MB 192.168.26.101 | FAILED | rc=1 >> non-zero return code 192.168.26.106 | FAILED | rc=1 >> non-zero return code ┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$
開(kāi)啟自動(dòng)刷新
┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$kubectl annotate imagecaches imagecache1 -n kube-fledged kubefledged.io/refresh-imagecache= imagecache.kubefledged.io/imagecache1 annotated ┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$
添加鏡像緩存
添加一個(gè)新的鏡像緩存
┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$kubectl get imagecaches.kubefledged.io -n kube-fledged imagecache1 -o json { "apiVersion": "kubefledged.io/v1alpha2", "kind": "ImageCache", "metadata": { "creationTimestamp": "2024-03-01T15:08:42Z", "generation": 92, "labels": { "app": "kubefledged", "kubefledged": "imagecache" }, "name": "imagecache1", "namespace": "kube-fledged", "resourceVersion": "20175233", "uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72" }, "spec": { "cacheSpec": [ { "images": [ "liruilong/my-busybox:latest", "liruilong/jdk1.8_191:latest" ] }, { "images": [ "liruilong/hikvision-sdk-config-ftp:latest" ], "nodeSelector": { "kubernetes.io/hostname": "vms105.liruilongs.github.io" } } ] }, "status": { "completionTime": "2024-03-02T01:43:32Z", "message": "All requested images pulled succesfully to respective nodes", "reason": "ImageCacheUpdate", "startTime": "2024-03-02T01:40:34Z", "status": "Succeeded" } } ┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$
通過(guò) ansible 確認(rèn)
┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml 192.168.26.101 | FAILED | rc=1 >> non-zero return code 192.168.26.100 | FAILED | rc=1 >> non-zero return code 192.168.26.102 | FAILED | rc=1 >> non-zero return code 192.168.26.103 | FAILED | rc=1 >> non-zero return code 192.168.26.105 | FAILED | rc=1 >> non-zero return code 192.168.26.106 | FAILED | rc=1 >> non-zero return code ┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml 192.168.26.101 | CHANGED | rc=0 >> liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB 192.168.26.102 | CHANGED | rc=0 >> liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB 192.168.26.100 | CHANGED | rc=0 >> liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB 192.168.26.103 | CHANGED | rc=0 >> liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB 192.168.26.105 | CHANGED | rc=0 >> liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB 192.168.26.106 | CHANGED | rc=0 >> liruilong/jdk1.8_191 latest 17dbd4002a8c 5 years ago 170MB ┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$
刪除鏡像緩存
┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$kubectl edit imagecaches imagecache1 -n kube-fledged imagecache.kubefledged.io/imagecache1 edited ┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$kubectl get imagecaches.kubefledged.io -n kube-fledged imagecache1 -o json { "apiVersion": "kubefledged.io/v1alpha2", "kind": "ImageCache", "metadata": { "creationTimestamp": "2024-03-01T15:08:42Z", "generation": 94, "labels": { "app": "kubefledged", "kubefledged": "imagecache" }, "name": "imagecache1", "namespace": "kube-fledged", "resourceVersion": "20175766", "uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72" }, "spec": { "cacheSpec": [ { "images": [ "liruilong/jdk1.8_191:latest" ] }, { "images": [ "liruilong/hikvision-sdk-config-ftp:latest" ], "nodeSelector": { "kubernetes.io/hostname": "vms105.liruilongs.github.io" } } ] }, "status": { "message": "Image cache is being updated. Please view the status after some time", "reason": "ImageCacheUpdate", "startTime": "2024-03-02T01:48:03Z", "status": "Processing" } }
通過(guò) Ansible 確認(rèn)，可以看到無(wú)論是 mastere 上的節(jié)點(diǎn)還是 work 的節(jié)點(diǎn)，對(duì)應(yīng)的鏡像緩存都被清理
┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml 192.168.26.102 | CHANGED | rc=0 >> liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB 192.168.26.101 | CHANGED | rc=0 >> liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB 192.168.26.105 | FAILED | rc=1 >> non-zero return code 192.168.26.100 | CHANGED | rc=0 >> liruilong/my-busybox latest 497b83a63aad 11 months ago 1.24MB 192.168.26.103 | FAILED | rc=1 >> non-zero return code 192.168.26.106 | FAILED | rc=1 >> non-zero return code ┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$ansible all -m shell -a "docker images | grep liruilong/my-busybox" -i host.yaml 192.168.26.105 | FAILED | rc=1 >> non-zero return code 192.168.26.102 | FAILED | rc=1 >> non-zero return code 192.168.26.103 | FAILED | rc=1 >> non-zero return code 192.168.26.101 | FAILED | rc=1 >> non-zero return code 192.168.26.100 | FAILED | rc=1 >> non-zero return code 192.168.26.106 | FAILED | rc=1 >> non-zero return code ┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$
這里需要注意如果清除所有的鏡像緩存，那么需要把?images?下的數(shù)組 寫成 "".
┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$kubectl edit imagecaches imagecache1 -n kube-fledged imagecache.kubefledged.io/imagecache1 edited ┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$ansible all -m shell -a "docker images | grep liruilong/jdk1.8_191" -i host.yaml 192.168.26.102 | FAILED | rc=1 >> non-zero return code 192.168.26.101 | FAILED | rc=1 >> non-zero return code 192.168.26.100 | FAILED | rc=1 >> non-zero return code 192.168.26.105 | FAILED | rc=1 >> non-zero return code 192.168.26.103 | FAILED | rc=1 >> non-zero return code 192.168.26.106 | FAILED | rc=1 >> non-zero return code ┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$kubectl get imagecaches.kubefledged.io -n kube-fledged imagecache1 -o json { "apiVersion": "kubefledged.io/v1alpha2", "kind": "ImageCache", "metadata": { "creationTimestamp": "2024-03-01T15:08:42Z", "generation": 98, "labels": { "app": "kubefledged", "kubefledged": "imagecache" }, "name": "imagecache1", "namespace": "kube-fledged", "resourceVersion": "20176849", "uid": "3a680a57-d8ab-444f-b9c9-4382459c5c72" }, "spec": { "cacheSpec": [ { "images": [ "" ] }, { "images": [ "liruilong/hikvision-sdk-config-ftp:latest" ], "nodeSelector": { "kubernetes.io/hostname": "vms105.liruilongs.github.io" } } ] }, "status": { "completionTime": "2024-03-02T01:52:16Z", "message": "All cached images succesfully deleted from respective nodes", "reason": "ImageCacheUpdate", "startTime": "2024-03-02T01:51:47Z", "status": "Succeeded" } } ┌──[root@vms100.liruilongs.github.io]-[~/ansible] └─$
如果通過(guò)下面的方式刪除，直接注釋調(diào)對(duì)應(yīng)的標(biāo)簽
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$cat kubefledged-imagecache.yaml --- apiVersion: kubefledged.io/v1alpha2 kind: ImageCache metadata: # Name of the image cache. A cluster can have multiple image cache objects name: imagecache1 namespace: kube-fledged # The kubernetes namespace to be used for this image cache. You can choose a different namepace as per your preference labels: app: kubefledged kubefledged: imagecache spec: # The "cacheSpec" field allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pre-pulled). cacheSpec: # Specifies a list of images (nginx:1.23.1) with no node selector, hence these images will be cached in all the nodes in the cluster #- images: #- liruilong/my-busybox:latest # Specifies a list of images (cassandra:v7 and etcd:3.5.4-0) with a node selector, hence these images will be cached only on the nodes selected by the node selector - images: - liruilong/hikvision-sdk-config-ftp:latest nodeSelector: kubernetes.io/hostname: vms105.liruilongs.github.io # Specifies a list of image pull secrets to pull images from private repositories into the cache #imagePullSecrets: #- name: myregistrykey ┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$
那么會(huì)報(bào)下面的錯(cuò)
┌──[root@vms100.liruilongs.github.io]-[~/ansible/kube-fledged/kube-fledged/deploy] └─$kubectl edit imagecaches imagecache1 -n kube-fledged error: imagecaches.kubefledged.io "imagecache1" could not be patched: admission webhook "validate-image-cache.kubefledged.io" denied the request: Mismatch in no. of image lists You can run `kubectl replace -f /tmp/kubectl-edit-4113815075.yaml` to try this update again.
博文部分內(nèi)容參考
? 文中涉及參考鏈接內(nèi)容版權(quán)歸原作者所有，如有侵權(quán)請(qǐng)告知,如果你認(rèn)可它不要吝嗇星星哦 :)
https://github.com/senthilrch/kube-fledged
?文章來(lái)源地址http://www.zghlxwxcb.cn/news/detail-852062.html
點(diǎn)擊關(guān)注，第一時(shí)間了解華為云新鮮技術(shù)~文章來(lái)源：http://www.zghlxwxcb.cn/news/detail-852062.html
?
到了這里，關(guān)于詳解K8s 鏡像緩存管理kube-fledged的文章就介紹完了。如果您還想了解更多內(nèi)容，請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！
