《kubernetes》k8s实战之部署PHP/JAVA网站

1. 容器交付流程

2. k8s部署项目流程

3. 部署harbor

3.1 部署不加密harbor

3.1 安装helm

mkdir -p /opt/k8s/helm && cd /opt/k8s/helm
wget https://get.helm.sh/helm-v3.9.0-rc.1-linux-amd64.tar.gz
tar -xf helm-v3.9.0-rc.1-linux-amd64.tar.gz
ln -s /opt/k8s/helm/linux-amd64/helm /usr/bin/helm
helm version
helm help

3.2 部署harbor

下载软件包

#创建名称空间
kubectl create namespace harbor
#添加harbor的helm仓库
helm repo add harbor https://helm.goharbor.io
#下载软件包
helm pull harbor/harbor

编写配置文件

# cat conf-1.yml 
expose:
  type: nodePort
  tls:
    enabled: false
  nodePort:
    ports:
      http:
        port: 80
        nodePort: 30002
 
harborAdminPassword: "admin"
 
externalURL: http://172.16.90.111:30002
 
persistence:
  enabled: false

# 没有使用加密和持久化存储
登录账号为admin/admin

部署

[root@master1 stl]# helm install harbor ./harbor-1.9.3.tgz -f ./conf-1.yml -n harbor
NAME: harbor
LAST DEPLOYED: Wed Aug 24 09:18:40 2022
NAMESPACE: harbor
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Please wait for several minutes for Harbor deployment to complete.
Then you should be able to visit the Harbor portal at http://172.16.90.81:30002
For more details, please visit https://github.com/goharbor/harbor

访问

3.2 部署加密harbor

3.2.1 安装helm

mkdir -p /opt/k8s/helm && cd /opt/k8s/helm
wget https://get.helm.sh/helm-v3.9.0-rc.1-linux-amd64.tar.gz
tar -xf helm-v3.9.0-rc.1-linux-amd64.tar.gz
ln -s /opt/k8s/helm/linux-amd64/helm /usr/bin/helm
helm version
helm help

3.2.2 创建stl证书

mkdir -p /opt/k8s/helm/stl && cd /opt/k8s/helm/stl
# 生成 CA 证书私钥
openssl genrsa -out ca.key 4096
# 生成 CA 证书
openssl req -x509 -new -nodes -sha512 -days 3650 \
 -subj "/C=CN/ST=Guangdong/L=Shenzhen/O=harbor/OU=harbor/CN=myharbor.com" \
 -key ca.key \
 -out ca.crt
# 创建域名证书，生成私钥
openssl genrsa -out myharbor.com.key 4096
# 生成证书签名请求 CSR
openssl req -sha512 -new \
    -subj "/C=CN/ST=Guangdong/L=Shenzhen/O=harbor/OU=harbor/CN=myharbor.com" \
    -key myharbor.com.key \
    -out myharbor.com.csr
# 生成 x509 v3 扩展
cat > v3.ext <<-EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names

[alt_names]
DNS.1=myharbor.com
DNS.2=*.myharbor.com
DNS.3=hostname
EOF
#创建 Harbor 访问证书
openssl x509 -req -sha512 -days 3650 \
    -extfile v3.ext \
    -CA ca.crt -CAkey ca.key -CAcreateserial \
    -in myharbor.com.csr \
    -out myharbor.com.crt

3.2.3 安装nfs

1.所有节点安装nfs

yum -y install  nfs-utils rpcbind

2.在master节点创建共享目录并授权

mkdir /opt/nfsdata
# 授权共享目录
chmod 666 /opt/nfsdata

3. 配置exports文件

cat >> /etc/exports<<EOF
/opt/nfsdata *(rw,no_root_squash,no_all_squash,sync)
EOF
# 配置生效
exportfs -r

exportfs命令

常用选项

-a 全部挂载或者全部卸载

-r 重新挂载

-u 卸载某一个目录

-v 显示共享目录 以下操作在服务端上

4. 启动rpc和nfs

systemctl start rpcbind
systemctl start nfs-server
systemctl enable rpcbind
systemctl enable nfs-server

查看

showmount -e

showmount -e 172.16.90.111

-e 显示NFS服务器的共享列表

-a 显示本机挂载的文件资源的情况NFS资源的情况

-v 显示版本号

3.2.4 创建nfs provisioner和持久化存储SC

GitHub地址： https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner

helm部署nfs-subdir-external-provisioner

NFS Provisioner 是一个自动配置卷程序，它使用现有的和已配置的 NFS 服务器来支持通过持久卷声明动态配置 Kubernetes 持久卷。

• 持久卷被配置为：namespace {namespace}-namespace−{pvcName}-${pvName}。

1. 添加helm仓库

2. helm安装nfs provisioner

【温馨提示】默认镜像是无法访问的，这里使用dockerhub搜索到的镜像registry.cn-beijing.aliyuncs.com/mydlq/nfs-subdir-external-provisioner:v4.0.0，还有就是StorageClass不分命名空间，所有在所有命名空间下都可以使用。

helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
  --namespace=nfs-provisioner \
  --create-namespace \
  --set image.repository=registry.cn-beijing.aliyuncs.com/mydlq/nfs-subdir-external-provisioner \
  --set image.tag=v4.0.0 \
  --set replicaCount=1 \
  --set storageClass.name=nfs-client \
  --set storageClass.defaultClass=true \
  --set nfs.server=172.16.90.111 \
  --set nfs.path=/opt/nfsdata

3. 查看

kubectl get pods,deploy,sc -n nfs-provisioner
NAME                                                   READY   STATUS    RESTARTS   AGE
pod/nfs-subdir-external-provisioner-545cd85dbf-qlxn8   1/1     Running   0          8h

NAME                                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nfs-subdir-external-provisioner   1/1     1            1           8h

NAME                                               PROVISIONER                                     RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
storageclass.storage.k8s.io/nfs-client (default)   cluster.local/nfs-subdir-external-provisioner   Delete          Immediate           true                   8h

3.2.5 部署Harbor （Https方式）

1. 创建 Namespace

kubectl create ns harbor

2. 创建证书密钥

kubectl create secret tls myharbor.com --key myharbor.com.key --cert myharbor.com.crt -n harbor
kubectl get secret myharbor.com -n harbor

3. 添加 Chart 库

helm repo add harbor https://helm.goharbor.io

4. 通过helm安装harbor

helm install myharbor --namespace harbor harbor/harbor \
  --set expose.type=ingress \
  --set expose.ingress.hosts.core=myharbor.com \
  --set expose.ingress.hosts.notary=notary.myharbor.com \
  --set-string expose.ingress.annotations.'nginx\.org/client-max