一、環(huán)境介紹
默認情況下，Harbor 不附帶證書。可以在沒有安全保護的情況下部署 Harbor，以便您可以通過 HTTP 連接到它。在生產環(huán)境中，推薦始終使用 HTTPS。要配置 HTTPS，必須創(chuàng)建 SSL 證書?？梢允褂糜墒苄湃蔚牡谌?CA 簽名的證書，也可以使用自簽名證書。本文以自簽名證書為例。
使用到的各個軟件版本
操作系統(tǒng)版本：ubuntu 20.04
harbor版本：v2.5.3-797c3536
docker版本：20.10.8
kubernetets版本：1.22.2
harbor地址：192.168.10.112 域名：harbor.snow.com
master01地址：192.168.10.100
二、倉庫部署
配置主機名與hosts文件
root@harbor:~# cat /etc/hosts
127.0.0.1 localhost
192.168.10.112 harbor.snow.com
修改主機名
root@barbor:~# hostnamectl set-hostname harbor
root@harbor:~# bash
root@harbor:~# hostname
harbor
下載harbor安裝包
root@harbor:~#wget https://github.com/goharbor/harbor/releases/download/v2.5.3/harbor-offline-installer-v2.5.3.tgz
解壓harbor安裝包
root@harbor:~# tar xf harbor-offline-installer-v2.5.3.tgz -C /usr/local/src/
安裝docker-compose
root@harbor:~# curl -SL https://github.com/docker/compose/releases/download/v2.7.0/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose
生成證書頒發(fā)機構證書及私鑰
root@harbor:/usr/local/src/harbor/certs# openssl genrsa -out ca.key 4096
Generating RSA private key, 4096 bit long modulus (2 primes)
............................++++
............................................++++
e is 65537 (0x010001)
root@harbor:/usr/local/src/harbor/certs# openssl req -x509 -new -nodes -sha512 -days 3650 \
> -subj "/C=CN/ST=Shanghai/L=Shanghai/O=SmartX/OU=Lab/CN=harbor.snow.com" \
> -key ca.key \
> -out ca.crt
root@harbor:/usr/local/src/harbor/certs# ls
ca.crt ca.key
生成服務器私鑰及證書簽名請求（CSR）
root@harbor:/usr/local/src/harbor/certs# openssl genrsa -out harbor.snow.com.key 4096
Generating RSA private key, 4096 bit long modulus (2 primes)
...........++++
.....................................................................++++
e is 65537 (0x010001)
生成證書簽名請求
root@harbor:/usr/local/src/harbor/certs# openssl req -sha512 -new \
> -subj "/C=CN/ST=Shanghai/L=Shanghai/O=SmartX/OU=Lab/CN=harbor.snow.com" \
> -key harbor.snow.com.key \
> -out harbor.snow.com.csr
root@harbor:/usr/local/src/harbor/certs# ls
ca.crt ca.key harbor.snow.com.csr harbor.snow.com.key
生成 x509 v3 擴展文件。
無論使用 FQDN 還是 IP 地址連接到Harbor服務器，都必須創(chuàng)建此文件，以便keyi 為的 Harbor 主機生成符合主題備用名稱 (SAN) 和 x509 v3 的證書擴展要求。
root@harbor:/usr/local/src/harbor/certs# cat v3.ext
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1=harbor.snow.top
DNS.2=snow.top
DNS.3=harbor
使用該v3.ext文件為 Harbor 服務器生成證書。
root@harbor:/usr/local/src/harbor/certs# openssl x509 -req -sha512 -days 3650 \
> -extfile v3.ext \
> -CA ca.crt -CAkey ca.key -CAcreateserial \
> -in harbor.snow.com.csr \
> -out harbor.snow.com.crt
Signature ok
subject=C = CN, ST = Shanghai, L = Shanghai, O = SmartX, OU = Lab, CN = harbor.snow.com
Getting CA Private Key
將 harbor.snow.com.crt 轉換為 harbor.snow.com.cert , 供 Docker 使用。Docker 守護進程將.crt文件解釋為 CA 證書，.cert將文件解釋為客戶端證書。
root@harbor:/usr/local/src/harbor/certs# openssl x509 -inform PEM -in harbor.snow.com.crt -out harbor.snow.com.cert
配置 harbor yml文件，注意需要修改hostname與指定證書位置。
root@harbor:/usr/local/src/harbor# cat harbor.yml |more
# Configuration file of Harbor
# The IP address or hostname to access admin UI and registry service.
# DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
hostname: harbor.snow.com
# http related config
http:
# port for http, default is 80. If https enabled, this port will redirect to https port
port: 80
# https related config
https:
# https port for harbor, default is 443
port: 443
# The path of cert and key files for nginx
certificate: /usr/local/src/harbor/certs/harbor.snow.com.cert
private_key: /usr/local/src/harbor/certs/harbor.snow.com.key
安裝harbor
root@harbor:/usr/local/src/harbor# ./install.sh
[Step 0]: checking if docker is installed ...
Note: docker version: 20.10.12
[Step 1]: checking docker-compose is installed ...
Note: docker-compose version: 1.25.0
[Step 2]: loading Harbor images ...
Loaded image: goharbor/harbor-portal:v2.5.3
Loaded image: goharbor/harbor-core:v2.5.3
Loaded image: goharbor/redis-photon:v2.5.3
Loaded image: goharbor/prepare:v2.5.3
Loaded image: goharbor/harbor-db:v2.5.3
Loaded image: goharbor/chartmuseum-photon:v2.5.3
Loaded image: goharbor/harbor-jobservice:v2.5.3
Loaded image: goharbor/harbor-registryctl:v2.5.3
Loaded image: goharbor/nginx-photon:v2.5.3
Loaded image: goharbor/notary-signer-photon:v2.5.3
Loaded image: goharbor/harbor-log:v2.5.3
Loaded image: goharbor/harbor-exporter:v2.5.3
Loaded image: goharbor/registry-photon:v2.5.3
Loaded image: goharbor/notary-server-photon:v2.5.3
Loaded image: goharbor/trivy-adapter-photon:v2.5.3
[Step 3]: preparing environment ...
[Step 4]: preparing harbor configs ...
prepare base dir is set to /usr/local/src/harbor
Clearing the configuration file: /config/registryctl/env
Clearing the configuration file: /config/registryctl/config.yml
Clearing the configuration file: /config/jobservice/env
Clearing the configuration file: /config/jobservice/config.yml
Clearing the configuration file: /config/portal/nginx.conf
Clearing the configuration file: /config/nginx/nginx.conf
Clearing the configuration file: /config/log/rsyslog_docker.conf
Clearing the configuration file: /config/log/logrotate.conf
Clearing the configuration file: /config/registry/passwd
Clearing the configuration file: /config/registry/config.yml
Clearing the configuration file: /config/core/env
Clearing the configuration file: /config/core/app.conf
Clearing the configuration file: /config/db/env
Generated configuration file: /config/portal/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
loaded secret from file: /data/secret/keys/secretkey
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
[Step 5]: starting Harbor ...
Creating network "harbor_harbor" with the default driver
Creating harbor-log ... done
Creating harbor-portal ... done
Creating registryctl ... done
Creating registry ... done
Creating redis ... done
Creating harbor-db ... done
Creating harbor-core ... done
Creating nginx ... done
Creating harbor-jobservice ... done
? ----Harbor has been installed and started successfully.----
安裝完harbor后驗證
root@harbor:/usr/local/src/harbor# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0304ebf0cddc goharbor/nginx-photon:v2.5.3 "nginx -g 'daemon of…" 38 seconds ago Up 35 seconds (healthy) 0.0.0.0:80->8080/tcp, :::80->8080/tcp, 0.0.0.0:443->8443/tcp, :::443->8443/tcp nginx
10d1ff19120b goharbor/harbor-jobservice:v2.5.3 "/harbor/entrypoint.…" 38 seconds ago Up 36 seconds (healthy) harbor-jobservice
2d5e3d8877f8 goharbor/harbor-core:v2.5.3 "/harbor/entrypoint.…" 39 seconds ago Up 38 seconds (healthy) harbor-core
ee6c12da89d9 goharbor/harbor-db:v2.5.3 "/docker-entrypoint.…" 43 seconds ago Up 39 seconds (healthy) harbor-db
5b7455e56733 goharbor/redis-photon:v2.5.3 "redis-server /etc/r…" 43 seconds ago Up 39 seconds (healthy) redis
a0c279ca0206 goharbor/harbor-registryctl:v2.5.3 "/home/harbor/start.…" 43 seconds ago Up 41 seconds (healthy) registryctl
478f35433f69 goharbor/registry-photon:v2.5.3 "/home/harbor/entryp…" 43 seconds ago Up 39 seconds (healthy) registry
eb33121f4126 goharbor/harbor-portal:v2.5.3 "nginx -g 'daemon of…" 43 seconds ago Up 39 seconds (healthy) harbor-portal
b0c7ab3a7133 goharbor/harbor-log:v2.5.3 "/bin/sh -c /usr/loc…" 44 seconds ago Up 43 seconds (healthy) 127.0.0.1:1514->10514/tcp harbor-log
修改宿主機hosts文件，做好harbor主機的域名解析，在瀏覽器使用域名訪問harbor服務器發(fā)現(xiàn)提示（忽略域名，域名是第一次測試的時候用的）
在當前頁面用鍵盤輸入thisisunsafe，不是在地址欄輸入，就直接敲鍵盤就行了，頁面即會自動刷新進入網頁
使用默認用戶名admin，密碼Harbor12345即可登錄。
三、配置docker客戶端
將證書發(fā)送到master01服務器,注意在操作之前要先在master01創(chuàng)建好存放證書的文件夾/etc/docker/certs.d/harbor.snow.com
root@master01:~# mkdir -p /etc/docker/certs.d/harbor.snow.com
root@harbor:/usr/local/src/harbor/certs# scp harbor.snow.com.cert 192.168.10.100:/etc/docker/certs.d/harbor.snow.com/
root@192.168.10.100's password:
harbor.snow.com.cert 100% 2102 2.3MB/s 00:00
root@harbor:/usr/local/src/harbor/certs# scp harbor.snow.com.key 192.168.10.100:/etc/docker/certs.d/harbor.snow.com/
root@192.168.10.100's password:
harbor.snow.com.key 100% 2045 2.0MB/s 00:00
root@harbor:/usr/local/src/harbor/certs# scp ca.crt 192.168.10.100:/etc/docker/certs.d/harbor.snow.com
root@192.168.10.100's password:
ca.crt 100% 2045 2.0MB/s 00:00
在master01節(jié)點看到證書已經成功發(fā)送過來
root@master01:/etc/docker/certs.d/harbor.snow.com# ls
ca.crt harbor.snow.com.cert harbor.snow.com.key
修改docker的daemon.json文件，使其信任harbor.snow.com倉庫
root@master01:/etc/docker# cat daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"],
"insecure-registries":["harbor.snow.com"],
"registry-mirrors": [
"https://docker.mirrors.ustc.edu.cn",
"http://hub-mirror.c.163.com"
],
"max-concurrent-downloads": 10,
"log-driver": "json-file",
"log-level": "warn",
"log-opts": {
"max-size": "10m",
"max-file": "3"
},
"data-root": "/var/lib/docker"
}
重啟docker服務并修改其hosts文件使master01節(jié)點能夠通過域名找到對應的harbor倉庫
root@master01:~# systemctl restart docker
root@master01:~# cat /etc/hosts
127.0.0.1 localhost
192.168.10.112 harbor.snow.com
登錄倉庫
root@master01:~# docker login harbor.snow.com
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
上傳鏡像進行測試，注意要提前在web頁面創(chuàng)建好新的項目。
root@master01:~# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
calico/cni v3.19.2 05bf027c9836 17 months ago 146MB
為鏡像改名字
root@master01:~# docker tag calico/cni:v3.19.2 harbor.snow.com/calico/cni:v3.19.2
推送到鏡像倉庫
root@master01:~# docker push harbor.snow.com/calico/cni:v3.19.2
The push refers to repository [harbor.snow.com/calico/cni]
758850cabe72: Pushed
e88fd392bb0c: Pushed
fb44224b4823: Pushed
v3.19.2: digest: sha256:00a619424a20d1b31c0f65d1194484aebe9f90dfc1d4d6396ce706941004860b size: 946
推送到鏡像倉庫后即可在web頁面看到剛剛推送上去鏡像
鏡像拉取測試
root@master01:~# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
kubernetesui/dashboard v2.3.1 e1482a24335a 18 months ago 220MB
root@master01:~# docker pull harbor.snow.com/calico/cni:v3.19.2
v3.19.2: Pulling from calico/cni
Digest: sha256:00a619424a20d1b31c0f65d1194484aebe9f90dfc1d4d6396ce706941004860b
Status: Downloaded newer image for harbor.snow.com/calico/cni:v3.19.2
harbor.snow.com/calico/cni:v3.19.2
root@master01:~# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
kubernetesui/dashboard v2.3.1 e1482a24335a 18 months ago 220MB
harbor.snow.com/calico/cni v3.19.2 05bf027c9836 17 months ago 146MB
四、K8S集群配置使用私有鏡像倉庫Harbor
在配置k8s使用harbor之前需要先完成本文的第三步，先讓docker可以正常使用harbor。
在docker成功登錄harbor后會在用戶家目錄生成一個秘鑰文件
root@master01:~# cat /root/.docker/config.json
{
"auths": {
"harbor.snow.com": {
"auth": "YWRtaW46SGFyYm9yMTIzNDU="
}
}
}
將秘鑰進行base64加密
root@master01:~# cat /root/.docker/config.json | base64 -w 0
ewoJImF1dGhzIjogewoJCSJoYXJib3Iuc25vdy5jb20iOiB7CgkJCSJhdXRoIjogIllXUnRhVzQ2U0dGeVltOXlNVEl6TkRVPSIKCQl9Cgl9Cn0=
創(chuàng)建用戶harbor認證的secret的yaml文件
root@master01:~/secret# cat registry-pull-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: registry-pull-secret
namespace: kube-system
data:
.dockerconfigjson: ewoJImF1dGhzIjogewoJCSJoYXJib3Iuc25vdy5jb20iOiB7CgkJCSJhdXRoIjogIllXUnRhVzQ2U0dGeVltOXlNVEl6TkRVPSIKCQl9Cgl9Cn0=
type: kubernetes.io/dockerconfigjson
創(chuàng)建secret，注意secret為名稱空間級別的資源，調用這個secret的yaml文件要與此secret在同一個名稱空間下。
root@master01:~/secret# kubectl delete -f registry-pull-secret.yaml
secret "registry-pull-secret" deleted
root@master01:~/secret# kubectl apply -f registry-pull-secret.yaml
secret/registry-pull-secret created
root@master01:~/secret# kubectl get secret
NAME TYPE DATA AGE
default-token-rxd89 kubernetes.io/service-account-token 3 5h2m
registry-pull-secret kubernetes.io/dockerconfigjson 1 3h
最后只需要在需要去harbor拉取鏡像的pod的yaml文件中添加如下配置即可（只展示出了完成yaml文件的部分內容）注意其是與containers語句是平級的。
template:
metadata:
labels:
k8s-app: metrics-server
spec:
imagePullSecrets:
- name: registry-pull-secret
containers:
image: harbor.snow.com/base/metrics-server:v0.5.0
imagePullPolicy: Always
配置完成后k8s集群即可成功拉取harbor中的鏡像。如果在harbor中將要訪問的鏡像級別設置為公開，則不需要secret認證即可正常拉取。文章來源：http://www.zghlxwxcb.cn/news/detail-771471.html
ps：以上內容在本人實現(xiàn)環(huán)境中已試驗成功，如發(fā)現(xiàn)有問題或表述不清的地方歡迎指正。文章來源地址http://www.zghlxwxcb.cn/news/detail-771471.html
到了這里，關于Harbor私有倉庫搭建并配置https對接docker與kubernetes的文章就介紹完了。如果您還想了解更多內容，請在右上角搜索TOY模板網以前的文章或繼續(xù)瀏覽下面的相關文章，希望大家以后多多支持TOY模板網！
