一、nfs-client-provisioner簡介

Kubernetes集群中NFS類型的存儲(chǔ)沒有內(nèi)置 Provisioner。但是你可以在集群中為NFS配置外部Provisioner。

Nfs-client-provisioner是一個(gè)開源的NFS 外部Provisioner，利用NFS Server為Kubernetes集群提供持久化存儲(chǔ)，并且支持動(dòng)態(tài)購買PV。但是nfs-client-provisioner本身不提供NFS，需要現(xiàn)有的NFS服務(wù)器提供存儲(chǔ)。持久卷目錄的命名規(guī)則為:$namespace?${pvcName}-${pvName}。

K8S的外部NFS驅(qū)動(dòng)可以按照其工作方式（是作為NFS server還是NFS client）分為兩類：

nfs-client:

它通過K8S內(nèi)置的NFS驅(qū)動(dòng)掛載遠(yuǎn)端的NFS服務(wù)器到本地目錄；然后將自身作為storage provider關(guān)聯(lián)storage class。當(dāng)用戶創(chuàng)建對(duì)應(yīng)的PVC來申請(qǐng)PV時(shí)，該provider就將PVC的要求與自身的屬性比較，一旦滿足就在本地掛載好的NFS目錄中創(chuàng)建PV所屬的子目錄，為Pod提供動(dòng)態(tài)的存儲(chǔ)服務(wù)。

nfs-server:

與nfs-client不同，該驅(qū)動(dòng)并不使用k8s的NFS驅(qū)動(dòng)來掛載遠(yuǎn)端的NFS到本地再分配，而是直接將本地文件映射到容器內(nèi)部，然后在容器內(nèi)使用ganesha.nfsd來對(duì)外提供NFS服務(wù)；在每次創(chuàng)建PV的時(shí)候，直接在本地的NFS根目錄中創(chuàng)建對(duì)應(yīng)文件夾，并export出該子目錄。

本文將介紹使用nfs-client-provisioner這個(gè)應(yīng)用，利用NFS Server給Kubernetes作為持久存儲(chǔ)的后端，并且動(dòng)態(tài)提供PV。前提條件是有已經(jīng)安裝好的NFS服務(wù)器，并且NFS服務(wù)器與Kubernetes的Slave節(jié)點(diǎn)網(wǎng)絡(luò)能夠連通。將nfs-client驅(qū)動(dòng)做為一個(gè)deployment部署到K8S集群中，然后對(duì)外提供存儲(chǔ)服務(wù)文章來源地址http://www.zghlxwxcb.cn/news/detail-664860.html

二、準(zhǔn)備NFS服務(wù)端

2.0 當(dāng)前環(huán)境信息

[root@master1 ~]# kubectl get nodes -owide
NAME               STATUS   ROLES    AGE     VERSION    INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION              CONTAINER-RUNTIME
master1.k8s.test   Ready    <none>   5d21h   v1.22.17   10.140.20.141   <none>        CentOS Linux 7 (Core)   6.3.2-1.el7.elrepo.x86_64   docker://19.3.15
master2.k8s.test   Ready    <none>   5d21h   v1.22.17   10.140.20.142   <none>        CentOS Linux 7 (Core)   6.3.2-1.el7.elrepo.x86_64   docker://19.3.15
master3.k8s.test   Ready    <none>   5d21h   v1.22.17   10.140.20.143   <none>        CentOS Linux 7 (Core)   6.3.2-1.el7.elrepo.x86_64   docker://19.3.15
node1.k8s.test     Ready    <none>   5d21h   v1.22.17   10.140.20.156   <none>        CentOS Linux 7 (Core)   6.3.2-1.el7.elrepo.x86_64   docker://19.3.15

2.1 通過yum安裝nfs server端

[root@master1 ~]# rpm -qa|egrep "nfs|rpc"
[root@master1 ~]# yum -y install nfs-utils rpcbind

2.2 啟動(dòng)服務(wù)和設(shè)置開機(jī)啟動(dòng)

#啟動(dòng)nfs-server，并加入開機(jī)啟動(dòng)
[root@master1 ~]# systemctl start rpcbind.service
[root@master1 ~]# systemctl enable rpcbind.service
[root@master1 ~]# systemctl start nfs
[root@master1 ~]# systemctl enable nfs-server --now
#查看nfs server是否已經(jīng)正常啟動(dòng)
[root@master1 ~]# systemctl status nfs-server

2.3 編輯配置文件，設(shè)置共享目錄

[root@master1 ~]# mkdir /data/nfs-provisioner -p
[root@master1 ~]# cat > /etc/exports <<EOF
/data/nfs_provisioner   10.140.20.0/24(rw,no_root_squash)
EOF

#不用重啟nfs服務(wù),配置文件就會(huì)生效

[root@master1 ~]# exportfs -arv 
exporting 10.140.20.0/24:/data/nfs_provisioner

用于配置NFS服務(wù)程序配置文件的參數(shù)：

2.4 客戶端測試

客戶端需要安裝nfs-utils

[root@master1 ~]# yum -y install nfs-utils
[root@master1 ~]# systemctl enable nfs --now
[root@master1 ~]# systemctl status nfs

客戶端驗(yàn)證

[root@master2 ~]# showmount -e 10.140.20.141
Export list for 10.140.20.141:
/data/nfs_provisioner 10.140.20.0/24

三、部署nfs-provisioner

3.1.0 創(chuàng)建namespace、工作目錄

[root@master1 ~]# kubectl create namespace test
[root@master1 ~]# mkdir  nfs-provisioner
[root@master1 ~]# cd  nfs-provisioner

3.1 創(chuàng)建ServiceAccount

[root@master1 ~]# cat > nfs-sa.yaml << EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: test
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: test
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: test
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: test
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: test
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io
EOF

應(yīng)用生效

[root@master1 nfs-provisioner]# kubectl apply -f nfs-sa.yaml
serviceaccount/nfs-client-provisioner created
clusterrole.rbac.authorization.k8s.io/nfs-client-provisioner-runner created
clusterrolebinding.rbac.authorization.k8s.io/run-nfs-client-provisioner created
role.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created
rolebinding.rbac.authorization.k8s.io/leader-locking-nfs-client-provisioner created

3.2 創(chuàng)建Deployment

[root@master1 ~]# cat > nfs-deployment.yaml << EOF
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nfs-client-provisioner
  namespace: test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nfs-client-provisioner
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: registry.cn-beijing.aliyuncs.com/mydlq/nfs-subdir-external-provisioner:v4.0.0
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: nfs-provisioner # 和3.Storage中provisioner保持一致便可
            - name: NFS_SERVER
              value: 10.140.20.141
            - name: NFS_PATH
              value: /data/nfs_provisioner
      volumes:
        - name: nfs-client-root
          nfs:
            server: 10.140.20.141
            path: /data/nfs_provisioner
EOF

應(yīng)用生效

[root@master1 nfs-provisioner]# kubectl apply -f nfs-deployment.yaml
deployment.apps/nfs-client-provisioner created

3.3 創(chuàng)建storageclass

[root@master1 ~]# cat > nfs-sc.yaml << EOF
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  annotations:
    storageclass.kubernetes.io/is-test-class: "true"
  name: nfs-storage
  namespace: test
provisioner: nfs-provisioner
volumeBindingMode: Immediate
reclaimPolicy: Delete
EOF

應(yīng)用生效

[root@master1 nfs-provisioner]# kubectl apply -f nfs-sc.yaml
storageclass.storage.k8s.io/nfs-storage created

3.4 創(chuàng)建pvc

[root@master yaml]# cat > nfs-pvc.yaml << EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-pvc
  namespace: test
  labels:
    app: nfs-pvc
spec:
  accessModes: #指定訪問類型
  - ReadWriteOnce
  volumeMode: Filesystem #指定卷類型
  resources:
    requests:
      storage: 2Gi
  storageClassName: nfs-storage #指定創(chuàng)建的存儲(chǔ)類的名字
EOF

#創(chuàng)建pvc
[root@master1 nfs-provisioner]# kubectl apply -n test -f nfs-pvc.yaml 
persistentvolumeclaim/nfs-pvc created
#查看pvc
[root@master1 nfs-provisioner]# kubectl get pvc -n test
NAME        STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
nfs-pvc     Bound    pvc-791dc175-c068-4977-a14b-02f8cb153bc3   2Gi        RWO            nfs-storage    8s
www-web-0   Bound    pvc-b666f81e-9723-4e88-8e81-157b9e081577   10Mi       RWO            nfs-storage    17m
www-web-1   Bound    pvc-a900806b-47e0-432e-81fd-865c5ff6e3ba   10Mi       RWO            nfs-storage    16m
#查看nfs共享目錄
[root@master1 nfs-provisioner]# ls /data/nfs_provisioner/
test-nfs-pvc-pvc-791dc175-c068-4977-a14b-02f8cb153bc3
test-www-web-0-pvc-b666f81e-9723-4e88-8e81-157b9e081577
test-www-web-1-pvc-a900806b-47e0-432e-81fd-865c5ff6e3ba


#總結(jié)：創(chuàng)建pvc使使用storageclass，那么將會(huì)自動(dòng)創(chuàng)建pv并進(jìn)行綁定

四、創(chuàng)建應(yīng)用測試動(dòng)態(tài)添加PV

4.1 創(chuàng)建一個(gè)nginx應(yīng)用

cat > nginx_sts_pvc.yaml << EOF
apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: test
  labels:
    app: nginx
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
  namespace: test
spec:
  serviceName: "nginx"
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
          name: web
        volumeMounts:
        - name: www
          mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
  - metadata:
      name: www
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "nfs-storage"  #使用新建的sc
      resources:
        requests:
          storage: 10Mi
EOF

應(yīng)用生效

[root@master1 nfs-provisioner]# kubectl apply -f nginx_sts_pvc.yaml
service/nginx created
statefulset.apps/web created

4.2 檢查結(jié)果

檢查deployment、statefulset狀態(tài)

[root@master1 nfs-provisioner]# kubectl get sts,deploy -n test
NAME                   READY   AGE
statefulset.apps/web   2/2     4m49s

NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nfs-client-provisioner   1/1     1            1           15m

檢查pod狀態(tài)

[root@master1 nfs-provisioner]# kubectl get pods -n test
NAME                                     READY   STATUS    RESTARTS   AGE
nfs-client-provisioner-fb55999fb-pcrqt   1/1     Running   0          9m30s
web-0                                    1/1     Running   0          3m40s
web-1                                    1/1     Running   0          3m15s

檢查nfs-server服務(wù)器是否創(chuàng)建pv持久卷：

[root@master1 nfs-provisioner]# kubectl get pvc,pv -n test
NAME                              STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/nfs-pvc     Bound    pvc-791dc175-c068-4977-a14b-02f8cb153bc3   2Gi        RWO            nfs-storage    2m45s
persistentvolumeclaim/www-web-0   Bound    pvc-b666f81e-9723-4e88-8e81-157b9e081577   10Mi       RWO            nfs-storage    19m
persistentvolumeclaim/www-web-1   Bound    pvc-a900806b-47e0-432e-81fd-865c5ff6e3ba   10Mi       RWO            nfs-storage    19m

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM            STORAGECLASS   REASON   AGE
persistentvolume/pvc-791dc175-c068-4977-a14b-02f8cb153bc3   2Gi        RWO            Delete           Bound    test/nfs-pvc     nfs-storage             2m45s
persistentvolume/pvc-a900806b-47e0-432e-81fd-865c5ff6e3ba   10Mi       RWO            Delete           Bound    test/www-web-1   nfs-storage             19m
persistentvolume/pvc-b666f81e-9723-4e88-8e81-157b9e081577   10Mi       RWO            Delete           Bound    test/www-web-0   nfs-storage             19m


[root@master1 nfs-provisioner]# kubectl exec -it -n test web-0 bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
root@web-0:/# echo 1 >  /usr/share/nginx/html/1.txt
root@web-0:/# exit
[root@master1 nfs-provisioner]# ls /data/nfs_provisioner/
test-nfs-pvc-pvc-791dc175-c068-4977-a14b-02f8cb153bc3
test-www-web-0-pvc-b666f81e-9723-4e88-8e81-157b9e081577
test-www-web-1-pvc-a900806b-47e0-432e-81fd-865c5ff6e3ba
[root@master1 nfs-provisioner]# cat  /data/nfs_provisioner/test-www-web-0-pvc-b666f81e-9723-4e88-8e81-157b9e081577/1.txt 
1

到了這里，關(guān)于k8s通過nfs-provisioner配置持久化存儲(chǔ)的文章就介紹完了。如果您還想了解更多內(nèi)容，請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！