【云原生 · Kubernetes】kubernetes v1.23.3 二进制部署（三）

5 部署 etcd 集群

etcd 是基于 Raft 的分布式 KV 存储系统，由 CoreOS 开发，常用于服务发现、共享配置以及并发控制（如 leader 选举、分布式锁等）。
kubernetes 使用 etcd 集群持久化存储所有 API 对象、运行数据。

etcd 集群节点名称和 IP 如下：

注意：

1. 如果没有特殊指明，本文档的所有操作均在qist 节点上执行；

5.1 下载和分发 etcd 二进制文件

cd /opt/k8s/work
wget https://github.com/etcd-io/etcd/releases/download/v3.5.2/etcd-v3.5.2-linuxamd64.tar.gz
tar -xvf tcd-v3.5.2-linux-amd64.tar.gz

分发二进制文件到集群所有节点：

cd /opt/k8s/work
scp -r etcd-v3.5.2-linux-amd64/etcd* root@192.168.2.175:/apps/etcd/bin
scp -r etcd-v3.5.2-linux-amd64/etcd* root@192.168.2.176:/apps/etcd/bin
scp -r etcd-v3.5.2-linux-amd64/etcd* root@192.168.2.177:/apps/etcd/bin

5.2 创建 etcd 证书和私钥

• 创建etcd服务证书
• 创建证书签名请求：

cat > /opt/k8s/cfssl/etcd/etcd-server.json << EOF
{
  "CN": "etcd",
  "hosts": [
    "127.0.0.1",
    "192.168.2.175","192.168.2.176","192.168.2.177",
    "k8s-master-1","k8s-master-2","k8s-master-3"
],
"key": {
  "algo": "rsa",
  "size": 2048
},
"names": [
  {
        "C": "CN",
        "ST": "GuangDong",
		"L": "GuangZhou",
		"O": "k8s",
		"OU": "Qist"
    }
  ]
}
EOF

生成证书和私钥：

cfssl gencert \
    -ca=/opt/k8s/cfssl/pki/etcd/etcd-ca.pem \
    -ca-key=/opt/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
    -config=/opt/k8s/cfssl/ca-config.json \
    -profile=kubernetes \
    /opt/k8s/cfssl/etcd/etcd-server.json | \
    cfssljson -bare /opt/k8s/cfssl/pki/etcd/etcd-server

• 创建etcd节点证书

192.168.2.175节点

cat > /opt/k8s/cfssl/etcd/k8s-master-1.json << EOF
{
  "CN": "etcd",
  "hosts": [
    "127.0.0.1",
    "192.168.2.175",
    "k8s-master-1"
],
"key": {
    "algo": "rsa",
    "size": 2048
},
"names": [
  {
        "C": "CN",
		"ST": "GuangDong",
		"L": "GuangZhou",
		"O": "k8s",
		"OU": "Qist"
    }
  ]
}
EOF

生成证书和私钥：

cfssl gencert \
    -ca=/opt/k8s/cfssl/pki/etcd/etcd-ca.pem \
	-ca-key=/opt/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
	-config=/opt/k8s/cfssl/ca-config.json \
	-profile=kubernetes \
	/opt/k8s/cfssl/etcd/k8s-master-1.json | \
	cfssljson -bare /opt/k8s/cfssl/pki/etcd/etcd-member-k8s-master-1

192.168.2.176节点

cat > /opt/k8s/cfssl/etcd/k8s-master-2.json << EOF
{
  "CN": "etcd",
  "hosts": [
	"127.0.0.1",
	"192.168.2.176",
	"k8s-master-2"
],
192.168.2.176节点
"key": {
  "algo": "rsa",
  "size": 2048
},
"names": [
  {
		"C": "CN",
		"ST": "GuangDong",
		"L": "GuangZhou",
		"O": "k8s",
		"OU": "Qist"
	}
  ]
}
EOF

生成证书和私钥：

cfssl gencert \
	-ca=/opt/k8s/cfssl/pki/etcd/etcd-ca.pem \
	-ca-key=/opt/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
	-config=/opt/k8s/cfssl/ca-config.json \
	-profile=kubernetes \
	/opt/k8s/cfssl/etcd/k8s-master-2.json | \
	cfssljson -bare /opt/k8s/cfssl/pki/etcd/etcd-member-k8s-master-2

192.168.2.177 节点

cat > /opt/k8s/cfssl/etcd/k8s-master-3.json << EOF
{
  "CN": "etcd",
  "hosts": [
    "127.0.0.1",
    "192.168.2.177",
"k8s-master-3"
],
"key": {
	"algo": "rsa",
	"size": 2048
},
"names": [
	{
		"C": "CN",
		"ST": "GuangDong",
		"L": "GuangZhou",
		"O": "k8s",
		"OU": "Qist"
	}
  ]
}
EOF

生成证书和私钥：

cfssl gencert \
		-ca=/opt/k8s/cfssl/pki/etcd/etcd-ca.pem \
		-ca-key=/opt/k8s/cfssl/pki/etcd/etcd-ca-key.pem \
		-config=/opt/k8s/cfssl/ca-config.json \
		-profile=kubernetes \
		/opt/k8s/cfssl/etcd/etcd-client.json | \
		cfssljson -bare /opt/k8s/cfssl/pki/etcd/etcd-client

分发生成的证书和私钥到各 etcd 节点：

# 分发server 证书
scp -r /opt/k8s/cfssl/pki/etcd/etcd-server* root@192.168.2.175:/apps/etcd/ssl
scp -r /opt/k8s/cfssl/pki/etcd/etcd-server* root@192.168.2.176:/apps/etcd/ssl
scp -r /opt/k8s/cfssl/pki/etcd/etcd-server* root@192.168.2.177:/apps/etcd/ssl
# 分发192.168.2.175 节点证书
scp -r /opt/k8s/cfssl/pki/etcd/etcd-member-k8s-master-1*
root@192.168.2.175:/apps/etcd/ssl
# 分发192.168.2.176 节点证书
scp -r /opt/k8s/cfssl/pki/etcd/etcd-member-k8s-master-2*
root@192.168.2.176:/apps/etcd/ssl
# 分发192.168.2.177 节点证书
scp -r /opt/k8s/cfssl/pki/etcd/etcd-member-k8s-master-3*
root@192.168.2.175:/apps/etcd/ssl
# 分发客户端证书到K8S master 节点 kube-apiserver 连接etcd 集群使用
scp -r /opt/k8s/cfssl/pki/etcd/etcd-client* root@192.168.2.175:/apps/k8s/ssl/etcd/
scp -r /opt/k8s/cfssl/pki/etcd/etcd-client* root@192.168.2.176:/apps/k8s/ssl/etcd/
scp -r /opt/k8s/cfssl/pki/etcd/etcd-client* root@192.168.2.177:/apps/k8s/ssl/etcd/

• WorkingDirectory 、 --data-dir ：指定工作目录和数据目录为${ETCD_DATA_DIR}，需在启动服务前创建这个目录；
• –wal-dir ：指定 wal 目录，为了提高性能，一般使用 SSD 或者和 --data-dir 不同的磁盘；
• –name ：指定节点名称，当 --initial-cluster-state 值为 new 时， --name 的参数值必须位于 --initial-cluster 列表中；
• –cert-file 、 --key-file ：etcd server 与 client 通信时使用的证书和私钥；
• –trusted-ca-file ：签名 client 证书的 CA 证书，用于验证 client 证书；
• –peer-cert-file 、 --peer-key-file ：etcd 与 peer 通信使用的证书和私钥；
• –peer-trusted-ca-file ：签名 peer 证书的 CA 证书，用于验证 peer 证书；

5.3 创建etcd 运行用户

k8s-master-1 k8s-master-2 k8s-master-3 节点上执行

• 创建etcd用户

useradd etcd -s /sbin/nologin -M

• etcd 目录给用户权限

chown -R etcd:etcd /apps/etcd
[root@k8s-master-3 ~]# ls -la /apps/etcd/
total 4
drwxr-xr-x 7 etcd etcd 64 Feb 10 20:32 .
drwxr-xr-x. 8 root root 85 Aug 26 18:54 ..
drwxr-xr-x 3 etcd etcd 117 Feb 10 20:28 bin
drwxr-xr-x 2 etcd etcd 18 Feb 10 20:33 conf
drwxr-xr-x 3 etcd etcd 26 Aug 26 12:57 data
drwxr-xr-x 2 etcd etcd 4096 Aug 26 12:58 ssl

5.4 启动 etcd 服务

k8s-master-1 k8s-master-2 k8s-master-3 节点上执行

# 全局刷新service
systemctl daemon-reload
# 设置etcd 开机启动
systemctl enable etcd
#重启etcd
systemctl restart etcd

• 必须先创建 etcd 数据目录和工作目录;
• etcd 进程首次启动时会等待其它节点的 etcd 加入集群，命令 systemctl start etcd 会卡住一段时 间，为正常现象；

5.5 检查启动结果

k8s-master-1 k8s-master-2 k8s-master-3 节点上执行

systemctl status etcd|grep Active
[root@k8s-master-1 conf]# systemctl status etcd|grep Active
Active: active (running) since Fri 2022-02-11 13:49:37 CST; 4h 5min ago
[root@k8s-master-2 ~]# systemctl status etcd|grep Active
Active: active (running) since Fri 2022-02-11 13:49:36 CST; 4h 4min ago
[root@k8s-master-3 ~]# systemctl status etcd|grep Active
Active: active (running) since Fri 2022-02-11 13:49:36 CST; 4h 5min ago

期待下次的分享，别忘了三连支持博主呀~
我是 念舒_C.ying ，期待你的关注~💪💪💪