K8S離線部署的方案
離線包，自己可以跟著下面步驟自己下載。
https://download.csdn.net/download/u010952056/86748944
萬字長(zhǎng)文詳解 PaaS toB 場(chǎng)景下 K8s 離線部署方案
| Item |
Language |
離線部署支持情況 |
| kops |
Golang |
不支持 |
| kubespray |
Ansible |
支持，需自行構(gòu)建安裝包 |
| kubeasz |
Ansible |
支持，需自行構(gòu)建安裝包 |
| sealos |
Golang |
支持，需付費(fèi)充值會(huì)員 |
| RKE |
Golang |
不支持，需自行安裝 docker |
| sealer |
Golang |
支持，源自 sealos |
| kubekey |
Golang |
部分支持，僅鏡像可離線 |
機(jī)器最小配置 3G ,2核，過低安裝不成功，磁盤空間>130G,練手在虛擬機(jī)
機(jī)器能聯(lián)網(wǎng) ?就裝個(gè) ?
apt-get install -y vim //vim
apt-get install -y net-tools //ifconfig
apt-get install -y openssh-server //ssh 自己機(jī)器的ip規(guī)劃 ?僅參考 ????????????
192.168.1.xx1? master01
192.168.1.xx2? node01
系統(tǒng)ubuntu22.04 ???k8s v1.20.10 ??docker 19.03
1、環(huán)境初始化
這些設(shè)置每個(gè)機(jī)器都要設(shè)置
1.1、關(guān)閉swap
swapoff -a
sed -ri 's/.*swap.*/#&/' /etc/fstab
rm -f /swap.img
swapoff -a && sed -i '/swap/d' /etc/fstab
編輯下面的文件swap.img 這一行給注釋掉,持久生效
vim /etc/fstab
# /swap.img //注釋掉這行 如果有1.2、關(guān)閉防火墻
systemctl disable ufw && systemctl stop ufw
# 關(guān)閉selinux # 永久
sed -i 's/enforcing/disabled/' /etc/selinux/config1.3、開啟ip轉(zhuǎn)發(fā)
cat > /etc/sysctl.d/k8s.conf << EOF
net.ipv4.ip_forward=1
EOF
# 將橋接的IPv4流量傳遞到iptables的鏈
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-ip6tables=1
net.bridge.bridge-nf-call-iptables=1
EOF
sysctl --system // 生效1.4、時(shí)間同步
這個(gè)是聯(lián)網(wǎng)同步時(shí)間
yum install ntpdate -y
ntpdate time.windows.com
沒網(wǎng)絡(luò)的 使用date設(shè)置時(shí)間(需要集群的機(jī)器時(shí)間一致，)
date -s 'yyyy-mmm-ddd hh:mm:ss'
做完時(shí)間同步后，進(jìn)行時(shí)間設(shè)置
證書問題，先將時(shí)間設(shè)置到95年之后，安裝完集群后，再將時(shí)間改回來。
Date -s 'yyyy-mmm-dd' //只改日起就行
設(shè)置時(shí)區(qū)
timedatectl set-timezone Asia/Shanghai
根據(jù)提示順次輸入Asia--> Chongqing的編號(hào)
修改時(shí)間為24小時(shí)
vim /etc/default/locale
增加一行
LC_TIME=en_DK.UTF-8
修改后，如果想使得系統(tǒng)日志的時(shí)間戳也立即生效
systemctl restart rsyslog1.5、設(shè)置ssh root賬號(hào)登陸
passwd root
設(shè)置root的密碼
vim /etc/ssh/sshd_config
PermitRootLogin yes # 添加
sudo systemctl restart sshd.service1.6、設(shè)置主機(jī)名修改hosts
//根據(jù)自己的master節(jié)點(diǎn)和node節(jié)點(diǎn)有幾個(gè)決定
hostnamectl set-hostname master01
hostnamectl set-hostname master0n
hostnamectl set-hostname node01
hostnamectl set-hostname node01n
cat >> /etc/hosts << EOF
節(jié)點(diǎn)機(jī)器的IP 機(jī)器的名稱 // 主機(jī)名和ip根據(jù)實(shí)際情況修改
EOF
舉例：#結(jié)合自己的ip
cat >> /etc/hosts << EOF
172.16.106.38 master01
172.16.106.39 node01
172.16.106.50 node02
172.16.106.51 master02
EOF上述設(shè)置操作完后， reboot 重啟機(jī)器
a. 檢查防火墻是否關(guān)閉 systemctl status ufw
b. 檢查機(jī)器名稱是否改變 hostnamectl
c. 查看hosts cat /etc/hosts
d. 檢查時(shí)間 date
e. 查看ip ip addr
f. ssh root賬號(hào)登陸 ssh@ip?回車 輸入密碼
找個(gè)有網(wǎng)絡(luò)的機(jī)器下載資料 ??root賬號(hào)下操作（為了方便）
2、docker 環(huán)境安裝
這個(gè)需要網(wǎng)絡(luò)下載資料，下載完后離線同樣安裝，離線省略下載步驟, 每個(gè)機(jī)器都安裝。
2.1、下載docker安裝包
版本：19.03
安裝包下載地址：
Index of linux/ubuntu/dists/bionic/pool/stable/amd64/
使用wget下載
wget -P /home/deploy/deb/docker/ https://download.docker.com/linux/ubuntu/dists/bionic/pool/stable/amd64/docker-ce_19.03.13~3-0~ubuntu-bionic_amd64.deb
wget -P /home/deploy/deb/docker/ https://download.docker.com/linux/ubuntu/dists/bionic/pool/stable/amd64/containerd.io_1.3.7-1_amd64.deb
wget -P /home/deploy/deb/docker/ https://download.docker.com/linux/ubuntu/dists/bionic/pool/stable/amd64/docker-ce-cli_19.03.13~3-0~ubuntu-bionic_amd64.deb下載后的文件，在/home/deploy/deb/docker下
containerd.io_1.3.7-1_amd64.deb
docker-ce_19.03.13~3-0~ubuntu-bionic_amd64.deb
docker-ce-cli_19.03.13~3-0~ubuntu-bionic_amd64.deb
2.2、安裝docker
cd /home/deploy/deb/docker
dpkg -i ./*.deb安裝后默認(rèn)cgroups驅(qū)動(dòng)使用cgroupfs ，需要調(diào)整為systemd，編輯docker配置文件
vi /etc/docker/daemon.json
添加如下內(nèi)容：
{
"exec-opts": ["native.cgroupdriver=systemd"]
}重啟docker，執(zhí)行：
systemctl daemon-reload && sudo systemctl restart docker2.3、下載k8s
機(jī)器沒有安裝curl命令就執(zhí)行下面的命令
apt-get update //可以不執(zhí)行
apt-get install curl -y
安裝GPG證書
curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -
添加apt源
cat > /etc/apt/sources.list.d/kubernetes.list << ERIC
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
ERIC
apt-get update
查看可安裝版本
apt-cache madison kubeadm
一大堆版本 自己選擇（我選擇的1.20.10-00）
安裝指定版本
VERSION=1.20.10-00
mkdir -p /home/deploy/deb/k8s/partial
apt-get autoclean
上面兩條命令解決2: No such file or dire這個(gè)錯(cuò)誤
將包下載到本地
apt-get install -y --download-only -o dir::cache::archives=/home/deploy/deb/k8s kubelet=$VERSION kubeadm=$VERSION kubectl=$VERSION
網(wǎng)速?zèng)Q定快慢。。。。。。下載后在/home/deploy/deb/k8s這個(gè)路徑下,這些文件用有
conntrack_1%3a1.4.5-2_amd64.deb
cri-tools_1.25.0-00_amd64.deb
ebtables_2.0.11-3build1_amd64.deb
kubeadm_1.20.10-00_amd64.deb
kubectl_1.20.10-00_amd64.deb
kubelet_1.20.10-00_amd64.deb
kubernetes-cni_1.1.1-00_amd64.deb
socat_1.7.3.3-2_amd64.deb
2.4、安裝 k8s
cd /home/deploy/deb/k8s
dpkg -i ./*.deb
設(shè)置開機(jī)啟動(dòng)
systemctl enable kubelet && sudo systemctl start kubelet2.5、查看k8s初始化需要的鏡像
將自己電腦斷網(wǎng)?執(zhí)行下面命令，否則顯示最新版的
kubeadm config images list --kubernetes-version=v1.20.10打印結(jié)果如下 需要7個(gè)鏡像
k8s.gcr.io/kube-apiserver:v1.20.10
k8s.gcr.io/kube-controller-manager:v1.20.10
k8s.gcr.io/kube-scheduler:v1.20.10
k8s.gcr.io/kube-proxy:v1.20.10
k8s.gcr.io/pause:3.2
k8s.gcr.io/etcd:3.4.13-0
k8s.gcr.io/coredns:1.7.0
開啟自己的電腦網(wǎng)絡(luò)
2.6、使用國(guó)內(nèi)的鏡像加速器下載這些鏡像
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.20.10
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.20.10
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.20.10
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.20.10
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.4.13-0
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.7.0將pull下來的鏡像 docker tag
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.20.10 k8s.gcr.io/kube-proxy:v1.20.10
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.20.10 k8s.gcr.io/kube-controller-manager:v1.20.10
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.20.10 k8s.gcr.io/kube-apiserver:v1.20.10
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.20.10 k8s.gcr.io/kube-scheduler:v1.20.10
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.4.13-0 k8s.gcr.io/etcd:3.4.13-0
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:1.7.0 k8s.gcr.io/coredns:1.7.0
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.2 k8s.gcr.io/pause:3.2查看tag后的7個(gè)鏡像
docker images | grep k8s.k8s.gcr.io/kube-proxy v1.20.10
k8s.gcr.io/kube-apiserver v1.20.10
k8s.gcr.io/kube-controller-manager v1.20.10
k8s.gcr.io/kube-scheduler v1.20.10
k8s.gcr.io/etcd 3.4.13-0
k8s.gcr.io/coredns 1.7.0
k8s.gcr.io/pause 3.2
網(wǎng)絡(luò)插件flannel需要一個(gè)鏡像 順便下載下來，
為啥沒用Calico，太難裝了，自己可以試試
https://projectcalico.docs.tigera.io/getting-started/kubernetes/quickstart
docker pull quay.io/coreos/flannel:v0.14.0一共8個(gè)鏡像
k8s.gcr.io/kube-proxy:v1.20.10
k8s.gcr.io/kube-apiserver:v1.20.10
k8s.gcr.io/kube-controller-manager:v1.20.10
k8s.gcr.io/kube-scheduler:v1.20.10
k8s.gcr.io/etcd:3.4.13-0
k8s.gcr.io/coredns:1.7.0
k8s.gcr.io/pause:3.2
quay.io/coreos/flannel:v0.14.0
鏡像保存出來
cd /home/deploy/deb (下面命令是一行）
docker save -o k8simages.tar k8s.gcr.io/kube-proxy:v1.20.10 k8s.gcr.io/kube-apiserver:v1.20.10 k8s.gcr.io/kube-controller-manager:v1.20.10 k8s.gcr.io/kube-scheduler:v1.20.10 k8s.gcr.io/etcd:3.4.13-0 k8s.gcr.io/coredns:1.7.0 k8s.gcr.io/pause:3.2 quay.io/coreos/flannel:v0.14.02.7、找一個(gè)網(wǎng)絡(luò)插件的配置文件
自學(xué)k8s-安裝過程為下載flannel.ym配置文件 - iXiAo9 - 博客園
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp.flannel.unprivileged
annotations:
seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
spec:
privileged: false
volumes:
- configMap
- secret
- emptyDir
- hostPath
allowedHostPaths:
- pathPrefix: "/etc/cni/net.d"
- pathPrefix: "/etc/kube-flannel"
- pathPrefix: "/run/flannel"
readOnlyRootFilesystem: false
# Users and groups
runAsUser:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
fsGroup:
rule: RunAsAny
# Privilege Escalation
allowPrivilegeEscalation: false
defaultAllowPrivilegeEscalation: false
# Capabilities
allowedCapabilities: ['NET_ADMIN', 'NET_RAW']
defaultAddCapabilities: []
requiredDropCapabilities: []
# Host namespaces
hostPID: false
hostIPC: false
hostNetwork: true
hostPorts:
- min: 0
max: 65535
# SELinux
seLinux:
# SELinux is unused in CaaSP
rule: 'RunAsAny'
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: flannel
rules:
- apiGroups: ['extensions']
resources: ['podsecuritypolicies']
verbs: ['use']
resourceNames: ['psp.flannel.unprivileged']
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- apiGroups:
- ""
resources:
- nodes
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes/status
verbs:
- patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: flannel
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: flannel
subjects:
- kind: ServiceAccount
name: flannel
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: flannel
namespace: kube-system
---
kind: ConfigMap
apiVersion: v1
metadata:
name: kube-flannel-cfg
namespace: kube-system
labels:
tier: node
app: flannel
data:
cni-conf.json: |
{
"name": "cbr0",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "flannel",
"delegate": {
"hairpinMode": true,
"isDefaultGateway": true
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
net-conf.json: |
{
"Network": "10.244.0.0/16",
"Backend": {
"Type": "vxlan"
}
}
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: kube-flannel-ds
namespace: kube-system
labels:
tier: node
app: flannel
spec:
selector:
matchLabels:
app: flannel
template:
metadata:
labels:
tier: node
app: flannel
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/os
operator: In
values:
- linux
hostNetwork: true
priorityClassName: system-node-critical
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: quay.io/coreos/flannel:v0.14.0
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: quay.io/coreos/flannel:v0.14.0
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: false
capabilities:
add: ["NET_ADMIN", "NET_RAW"]
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run/flannel
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run/flannel
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg將網(wǎng)頁中的文本復(fù)制 保存為kube-flannel.yml 文件
到此文件下載基本完成，全部文件如下
a. docker/
containerd.io_1.3.7-1_amd64.deb
docker-ce_19.03.13~3-0~ubuntu-bionic_amd64.deb
docker-ce-cli_19.03.13~3-0~ubuntu-bionic_amd64.deb
b. k8s/
conntrack_1%3a1.4.5-2_amd64.deb
cri-tools_1.25.0-00_amd64.deb
ebtables_2.0.11-3build1_amd64.deb
kubeadm_1.20.10-00_amd64.deb
kubectl_1.20.10-00_amd64.deb
kubelet_1.20.10-00_amd64.deb
kubernetes-cni_1.1.1-00_amd64.deb
socat_1.7.3.3-2_amd64.deb
c. k8simages.tar
將/home/deploy/deb
deb這個(gè)文件夾拷貝到主節(jié)點(diǎn)和從節(jié)點(diǎn)。
3、離線安裝
在自己的集群節(jié)點(diǎn)操作，下面三步每個(gè)機(jī)器都要操作
3.1、離線Docker安裝
參照【2.2】
3.2、離線K8s安裝
參照【2.4】
3.3、離線加載鏡像文件
docker load < k8simages.tar3.4、主節(jié)點(diǎn)初始化
kubeadm init --kubernetes-version=v1.20.10 --pod-network-cidr=10.244.0.0/16 --service-cidr=10.96.0.0/12 --ignore-preflight-errors=Swap初始化成功信息 有顏色的需要再次執(zhí)行
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/configAlternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.confYou should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
??https://kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 172.16.245.133:6443 --token 9c11at.6ebo4g8lfcnqujwr \
--discovery-token-ca-cert-hash sha256:382e46547d0242164825d6799895f8a263e896f409ab417a03ae1464b3e7f7fa
這個(gè)用你自己的 3.5、主節(jié)點(diǎn)安裝網(wǎng)絡(luò)插件 Flannel
kubectl apply -f kube-flannel.yml只在`master`節(jié)點(diǎn)執(zhí)行即可，插件使用的是DaemonSet的控制器
3.6、查看主節(jié)點(diǎn)狀態(tài)
kubectl get pod -n kube-system3.7、加入worker節(jié)點(diǎn)
需要執(zhí)行上面的紅色的命令
在主節(jié)點(diǎn)上執(zhí)行
kubectl label node 【node_name你自己的從節(jié)點(diǎn)機(jī)器名】 node-role.kubernetes.io/worker=worker在主節(jié)點(diǎn)上查看節(jié)點(diǎn)狀態(tài)
kubectl get nodes注意:
kubectl命令需要使用kubernetes-admin來運(yùn)行，需要admin.conf文件。
如果需要在子結(jié)點(diǎn)執(zhí)行kubectl 命令，就把主節(jié)點(diǎn)的
$HOME/.kube/config???這個(gè)文件拷貝到子結(jié)點(diǎn)
子結(jié)點(diǎn)執(zhí)行
chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=/etc/kubernetes/admin.conf3.8、將時(shí)間改回來
查看證書時(shí)間
kubeadm alpha certs check-expiration生成新的證書之前最好備份一下數(shù)據(jù)：
cp -rp /etc/kubernetes /etc/kubernetes.bak
cp -rp /var/lib/etcd /var/lib/etcd.bak生成新的證書：
kubeadm alpha certs renew all然后將之前設(shè)置的時(shí)間改到現(xiàn)在的互聯(lián)網(wǎng)時(shí)間。(每個(gè)機(jī)器時(shí)間需要一樣)
date -s 'yyyy-mmm-dd' //只改日起就行改完后再查看一次證書時(shí)間
kubeadm alpha certs check-expiration至此基本安裝完成后續(xù)使用測(cè)試
4、參考
4.1、證書到期問題
Kubernetes kubeadm 證書到期，更新證書_大漠知秋的博客-CSDN博客_kubelet 證書更新
Kubernetes v1.25 編譯 kubeadm 修改證書有效期到 100 年 - sysin - 博客園
4.2、控制面板
https://kuboard.cn/install/maintain/certs.html
網(wǎng)絡(luò)組件下載
自學(xué)k8s-安裝過程為下載flannel.yml配置文件 - iXiAo9 - 博客園
4.3、安裝參考鏈接
Ubuntu20.04 離線部署 k8s1.20.10 - 知乎
ubuntu20.04安裝k8s_Professorboy的博客-CSDN博客
內(nèi)網(wǎng)ubuntu環(huán)境下離線部署K8s - MadLife
https://baijiahao.baidu.com/s?id=1724382781225457375&wfr=spider&for=pc
calico網(wǎng)絡(luò)組件(可選)
Quickstart for Calico on Kubernetes
ubuntu20.04安裝k8s(1.18.4 & 1.22.0)_還在下雨嗎的博客-CSDN博客_ubuntu20安裝k8s
4.4、這個(gè)沒成功
sealos安裝集群文章來源：http://www.zghlxwxcb.cn/news/detail-447621.html
https://huaweicloud.csdn.net/63311fb4d3efff3090b52d55.html?spm=1001.2101.3001.6650.1&utm_medium=distribute.pc_relevant.none-task-blog-2%7Edefault%7ECTRLIST%7Eactivity-1-119522721-blog-124778122.pcrelevantt0_20220926_downloadratepraise_v1&depth_1-utm_source=distribute.pc_relevant.none-task-blog-2%7Edefault%7ECTRLIST%7Eactivity-1-119522721-blog-124778122.pcrelevantt0_20220926_downloadratepraise_v1&utm_relevant_index=2文章來源地址http://www.zghlxwxcb.cn/news/detail-447621.html
到了這里，關(guān)于Ubuntu 使用Kubeadm 離線安裝k8s的文章就介紹完了。如果您還想了解更多內(nèi)容，請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！
