1.部署k8s前置条件
本次部署完全学习于华子的博客点击此处进入华子主页
K8S中文官网：https://kubernetes.io/zh-cn
笔者的k8s部署前置条件为
【1】.需要harbor仓库，存放镜像，拉取镜像，上传镜像。一台主节点，两台工作节点
【2】.由于笔者是k8s最小化部署，只用到四台虚拟机配置如下
系统： centos7
hostname |
配置 |
IP |
|---|---|---|
| master1 | 4G 2核 | 192.168.138.139 |
| node1 | 2G 2核 | 192.168.138.149 |
| node2 | 2G 2核 | 192.168.138.150 |
| harhar.com | 2G 2核 | 192.168.138.161 |
【3】.集群主机均需做如下操作
3.1hostname设置命令如下
[root@master1 ~]# hostnamectl set-hostname master1
3.2四台主机 /etc/hosts 解析如下
[root@master1 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.138.139 master1
192.168.138.149 node1
192.168.138.150 node2
192.168.138.161 harhar.com
3.3关闭firewalld,selinux，swap分区
进入/etc/fstab 注释掉swap这一行
[root@master1 ~]# cat /etc/fstab
#
# /etc/fstab
# Created by anaconda on Fri Aug 11 22:13:06 2023
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/centos-root / xfs defaults 0 0
UUID=0de2b180-2825-4ed9-8c2a-e59ab18d6cdd /boot xfs defaults 0 0
#/dev/mapper/centos-swap swap swap defaults 0 0
改写SELINUX=disabled
[root@master1 ~]# cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
#SELINUX=enforcing
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
[root@master1 ~]# systemctl disable firewalld --now
[root@master1 ~]# reboot # 重启一下让selinux，swap生效
3.4设置yum源
两个仓库均是网络源，读者需要做网络环境，即可以联通互联网
k8s仓库
[root@master1 ~]# cat /etc/yum.repos.d/k8s.repo
[k8s]
name=k8s
baseurl=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.30/rpm
gpgcheck=0
[root@master1 ~]#
docker仓库
[root@master1 ~]# cat /etc/yum.repos.d/docker-ce.repo
[docker-ce-stable]
name=Docker CE Stable - $basearch
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/stable
enabled=1
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
[docker-ce-stable-debuginfo]
name=Docker CE Stable - Debuginfo $basearch
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/debug-$basearch/stable
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
[docker-ce-stable-source]
name=Docker CE Stable - Sources
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/source/stable
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
[docker-ce-test]
name=Docker CE Test - $basearch
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/test
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
[docker-ce-test-debuginfo]
name=Docker CE Test - Debuginfo $basearch
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/debug-$basearch/test
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
[docker-ce-test-source]
name=Docker CE Test - Sources
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/source/test
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
[docker-ce-nightly]
name=Docker CE Nightly - $basearch
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/nightly
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
[docker-ce-nightly-debuginfo]
name=Docker CE Nightly - Debuginfo $basearch
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/debug-$basearch/nightly
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
[docker-ce-nightly-source]
name=Docker CE Nightly - Sources
baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/source/nightly
enabled=0
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
3.5.安装docker,设置docker开机自启，安装命令补全工具.
[root@master1 ~]# yum install bash-completion -y
[root@master1 ~]# yum install yum-utils -y
[root@master1 ~]# yum install docker-ce -y
[root@master1 ~]# systemctl enabel --now docker
3.6设置docker镜像源,即harbor仓库地址
[root@master1 ~]# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://harhar.com"]
}
[root@master1 ~]# systemctl daemon-reload
[root@master1 ~]# systemctl restart docker
3.7 登录harbor
[root@master1 ~]# docker login -u admin -p 123456 harhar.com
2.正式部署kubernetes
部署k8s方式有很多种，本次采用的是cri-docker。kubeadm方式部署
2.1.列出所有需要的工具，选择版本安装，本次部署是1.30版本
[root@master1 ~]# yum list kubelet --showduplicates
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* extras: mirrors.aliyun.com
* updates: mirrors.aliyun.com
Installed Packages
kubelet.x86_64 1.30.0-150500.1.1 @k8s
Available Packages
kubelet.x86_64 1.30.0-150500.1.1 k8s
kubelet.x86_64 1.30.1-150500.1.1 k8s
kubelet.x86_64 1.30.2-150500.1.1 k8s
kubelet.x86_64 1.30.3-150500.1.1 k8s
kubelet.x86_64 1.30.4-150500.1.1 k8s
kubelet.x86_64 1.30.5-150500.1.1 k8s
kubelet.x86_64 1.30.6-150500.1.1 k8s
kubelet.x86_64 1.30.7-150500.1.1 k8s
kubelet.x86_64 1.30.8-150500.1.1 k8s
[root@master1 ~]# yum list kubeadm --showduplicates
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* extras: mirrors.aliyun.com
* updates: mirrors.aliyun.com
Installed Packages
kubeadm.x86_64 1.30.0-150500.1.1 @k8s
Available Packages
kubeadm.x86_64 1.30.0-150500.1.1 k8s
kubeadm.x86_64 1.30.1-150500.1.1 k8s
kubeadm.x86_64 1.30.2-150500.1.1 k8s
kubeadm.x86_64 1.30.3-150500.1.1 k8s
kubeadm.x86_64 1.30.4-150500.1.1 k8s
kubeadm.x86_64 1.30.5-150500.1.1 k8s
kubeadm.x86_64 1.30.6-150500.1.1 k8s
kubeadm.x86_64 1.30.7-150500.1.1 k8s
kubeadm.x86_64 1.30.8-150500.1.1 k8s
[root@master1 ~]# yum list kubectl --showduplicates
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* extras: mirrors.aliyun.com
* updates: mirrors.aliyun.com
Installed Packages
kubectl.x86_64 1.30.0-150500.1.1 @k8s
Available Packages
kubectl.x86_64 1.30.0-150500.1.1 k8s
kubectl.x86_64 1.30.1-150500.1.1 k8s
kubectl.x86_64 1.30.2-150500.1.1 k8s
kubectl.x86_64 1.30.3-150500.1.1 k8s
kubectl.x86_64 1.30.4-150500.1.1 k8s
kubectl.x86_64 1.30.5-150500.1.1 k8s
kubectl.x86_64 1.30.6-150500.1.1 k8s
kubectl.x86_64 1.30.7-150500.1.1 k8s
kubectl.x86_64 1.30.8-150500.1.1 k8s
[root@master1 ~]#
2.2master节点需要都安装，node1,node2只安装kubeadm，kubelet即可
[root@master1 ~]# yum install \
> kubelet-1.30.0-150500.1.1 \
> kubeadm-1.30.0-150500.1.1 \
> kubectl-1.30.0-150500.1.1 -y
[root@node1 ~]# yum install \
> kubelet-1.30.0-150500.1.1 \
> kubeadm-1.30.0-150500.1.1 \ -y
[root@node2 ~]# yum install \
> kubelet-1.30.0-150500.1.1 \
> kubeadm-1.30.0-150500.1.1 \ -y
2.3安装cri-docker,注意自身版本。笔者提前下载好了放到主机里
cri-docker下载地址
https://github.com/Mirantis/cri-dockerd/tags
[root@master1 packags]# ls
cri-dockerd-0.3.6.20231018204925.877dc6a4-0.el7.x86_64.rpm libcgroup-0.41-19.el8.x86_64.rpm
[root@master1 ~]# yum install cri-dockerd-0.3.6.20231018204925.877dc6a4-0.el7.x86_64.rpm libcgroup-0.41-19.el8.x86_64.rpm -y
[root@master1 ~]# systemctl enable --now cri-docker
[root@node1 ~]# yum install cri-dockerd-0.3.6.20231018204925.877dc6a4-0.el7.x86_64.rpm libcgroup-0.41-19.el8.x86_64.rpm -y
[root@node1 ~]# systemctl enable --now cri-docker
```bash
[root@node2 ~]# yum install cri-dockerd-0.3.6.20231018204925.877dc6a4-0.el7.x86_64.rpm libcgroup-0.41-19.el8.x86_64.rpm -y
[root@node2 ~]# systemctl enable --now cri-docker
2.4.阿里云拉取k8s镜像
```bash
[rootmaster1 ~]# kubeadm config images pull \
> --image-repository registry.aliyuncs.com/google_containers \
> --kubernetes-version v1.30.0 \
> --cri-socket=unix:///var/run/cri-dockerd.sock
# 查看镜像是否成功拉取
[rootmaster1 ~]# docker images
2.5 上传镜像到harbor仓库，提前在harbor仓库创建公开仓库
批量打标签上传
[root@k8s-master ~]# docker images | awk '/google/{ print $1":"$2}' \
> | awk -F "/" '{system("docker tag "$0" harhar.com/k8s/"$3)}'
[root@k8s-master ~]# docker images | awk '/k8s/{system("docker push "$1":"$2)}'
注意每台主机都需要改，除了harbor机器
2.6修改cri-docker.service文件,指定harbor仓库
[root@master1 packags]# cat /lib/systemd/system/cri-docker.service
[Unit]
Description=CRI Interface for Docker Application Container Engine
Documentation=https://docs.mirantis.com
After=network-online.target firewalld.service docker.service
Wants=network-online.target
Requires=cri-docker.socket
[Service]
Type=notify
ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --network-plugin=cni --pod-infra-container-image=harhar.com/k8s/pause:3.9
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
# Both the old, and new location are accepted by systemd 229 and up, so using the old location
# to make them work for either version of systemd.
StartLimitBurst=3
# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
# this option work for either version of systemd.
StartLimitInterval=60s
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Comment TasksMax if your systemd version does not support it.
# Only systemd 226 and above support this option.
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target
修改这行指定好你的harbor仓库地址
ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --network-plugin=cni --pod-infra-container-image=harhar.com/k8s/pause:3.9
[root@master1 ~]# systemctl daemon-reload
[root@master1 ~]# systemctl restart docker
2.7集群启动kubelet
systemctl enable --now kubelet
2.8集群初始化
master做
[root@master1 ~]# kubeadm init --pod-network-cidr=10.244.0.0/16 \
> --image-repository harhar.com/k8s \
> --kubernetes-version v1.30.0 \
> --cri-socket=unix:///var/run/cri-dockerd.sock
注意他最后会输出一个kubeadm join将他保存下来，node要用
[root@master ~]# export KUBECONFIG=/etc/kubernetes/admin.conf
[root@master ~]# echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> ~/.bash_profile
[root@master ~]# source ~/.bash_profile
初始化失误可重置，后续node也可以重置。重置后再初始化就行
kubeadm reset --cri-docker=unix:///var/run/cri-dockerd.sock
2.9做集群中的网络，安装flannel插件，此处需要自己下载镜像和kube-flannel.yml，放到主机内。
下载地址如下
https://github.com/flannel-io/flannel/tags
[root@master ~]# docker tag flannel/flannel:v0.25.5 harhar.com/flannel/flannel:v0.25.5
[root@master ~]# docker tag flannel/flannel-cni-plugin:v1.5.1-flannel1 harhar.com/flannel/flannel-cni-plugin:v1.5.1-flannel1
[root@master ~]# docker push harhar.com/flannel/flannel:v0.25.5
[root@master ~]# docker images
[root@master ~]# docker push harhar.com/flannel/flannel-cni-plugin:v1.5.1-flannel1
编辑kube-flannel.yml
[root@master1 docker-images]# cat kube-flannel.yml | grep image
image: harhar.com/flannel/flannel:v0.25.5
image: harhar.com/flannel/flannel-cni-plugin:v1.5.1-flannel1
image: harhar.com/flannel/flannel:v0.25.5
启动flannel并查看
[root@master1 docker-images]# kubectl apply -f kube-flannel.yml
[root@master1 docker-images]# kubectl -n kube-flannel get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-flannel-ds-7zn6k 1/1 Running 0 3d4h 192.168.138.150 node2 <none> <none>
kube-flannel-ds-bxszc 1/1 Running 0 3d4h 192.168.138.139 master1 <none> <none>
kube-flannel-ds-c7fgn 1/1 Running 0 3d4h 192.168.138.149 node1 <none> <none>
2.10 node1，node2将节点加入集群
[root@node1]# kubeadm join 192.168.138.139:6443 --token mj7z17.fsb5iq1bab4sqv4l --discovery-token-ca-cert-hash sha256:3a0bb3de8c2d9b8aa404b7b888f26796bc2c7784044f8acc74fa652306b83865 --cri-socket=unix:///var/run/cri-dockerd.sock
[root@node2]# kubeadm join 192.168.138.139:6443 --token mj7z17.fsb5iq1bab4sqv4l --discovery-token-ca-cert-hash sha256:3a0bb3de8c2d9b8aa404b7b888f26796bc2c7784044f8acc74fa652306b83865 --cri-socket=unix:///var/run/cri-dockerd.sock
2.10 去master查看一下，我们发现集群部署完成
[root@master1 docker-images]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
lee-579c757b65-qg7mz 1/1 Running 0 3d4h 10.244.1.2 node1 <none> <none>
lee-579c757b65-s8qsv 1/1 Running 0 3d4h 10.244.2.2 node2 <none> <none>