文章目录
前言
Ingress 是 kubernetes API 中的标准资源类型之一,ingress 实现的功能是在应用层对客户端请求的 host 名称或请求的 URL 路径把请求转发到指定的 service 资源的规则,即用 于将 kubernetes 集群外部的请求资源转发之集群内部的 service,再被 service 转发之 pod 处理客户端的请求。
安装Ingress
Ingress 资源需要指定监听地址、请求的 host 和 URL 等配置,然后根据这些规则的匹 配机制将客户端的请求进行转发,这种能够为 ingress 配置资源监听并转发流量的组件称为 ingress 控制器(ingress controller),ingress controller 是 kubernetes 的一个附件,类似于 dashboard 或者 flannel 一样,需要单独部署。
yaml文件下载地址:
https://github.com/kubernetes/ingress-nginx/blob/cd151e3db8bd4f884511d229c2134e89f3c22a98/deploy/static/provider/baremetal/deploy.yaml
部署ingress-nginx,查看pod与service:
kubectl apply -f deploy.yaml
...
...
kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
ingress-nginx ingress-nginx-admission-create-nl8xw 0/1 Completed 0 8m10s
ingress-nginx ingress-nginx-admission-patch-fjjvt 0/1 Completed 0 8m10s
ingress-nginx ingress-nginx-controller-5bd986b4f9-b6znm 1/1 Running 0 8m10s
kubectl get svc -A
ingress-nginx ingress-nginx-controller NodePort 10.68.236.4 <none> 80:33646/TCP,443:32883/TCP 48m
ingress-nginx ingress-nginx-controller-admission ClusterIP 10.68.124.109 <none> 443/TCP 48m
Ingress 也起了NodePort服务,33646 对应80请求,32883 对应443请求。
部署应用服务
部署两个tomcat应用,对应的nodePort分别为30081和30082。并在各个webapps下创建目录与首页。
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: myserver-tomcat-app1-deployment-label
name: myserver-tomcat-app1-deployment
namespace: myserver
spec:
replicas: 1
selector:
matchLabels:
app: myserver-tomcat-app1-selector
template:
metadata:
labels:
app: myserver-tomcat-app1-selector
spec:
containers:
- name: myserver-tomcat-app1-container
image: tomcat:7.0.94-alpine
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
protocol: TCP
name: http
---
kind: Service
apiVersion: v1
metadata:
labels:
app: myserver-tomcat-app1-service-label
name: myserver-tomcat-app1-service
namespace: myserver
spec:
type: NodePort
ports:
- name: http
port: 80
protocol: TCP
targetPort: 8080
nodePort: 30081
selector:
app: myserver-tomcat-app1-selector
测试服务是否可用,IP可以是集群内任一节点IP:
curl http://192.168.100.164:30081/app/
tomcat-app1
curl http://192.168.100.164:30082/app/
tomcat-app2
Ingress-http案例
single-host
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-web
namespace: myserver
annotations:
kubernetes.io/ingress.class: "nginx" ##指定Ingress Controller的类型
nginx.ingress.kubernetes.io/use-regex: "true" ##指定后面rules定义的path可以使用正则表达式
nginx.ingress.kubernetes.io/proxy-connect-timeout: "600" ##连接超时时间,默认为5s
nginx.ingress.kubernetes.io/proxy-send-timeout: "600" ##后端服务器回转数据超时时间,默认为60s
nginx.ingress.kubernetes.io/proxy-read-timeout: "600" ##后端服务器响应超时时间,默认为60s
nginx.ingress.kubernetes.io/proxy-body-size: "50m" ##客户端上传文件,最大大小,默认为20m
#nginx.ingress.kubernetes.io/rewrite-target: / ##URL重写
nginx.ingress.kubernetes.io/app-root: /index.html
spec:
rules:
- host: www.app1.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: myserver-tomcat-app1-service
port:
number: 80
在集群外的机子进行测试:
# 修改hosts文件,ip可以为集群内任一节点ip。
echo '192.168.100.161 www.app1.com' >> /etc/hosts
# 33646为ingress的nodePort。
curl www.app1.com:33646/app/
tomcat-app1
访问域名www.app1.com时,ingress转发给了app1的service。
multi-host
当访问不同域名host时,Ingress会转发给不同的service。
...
#前半部与上一例子相同
...
spec:
rules:
- host: www.app1.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: myserver-tomcat-app1-service
port:
number: 80
- host: www.app2.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: myserver-tomcat-app2-service
port:
number: 80
访问域名www.app1.com时,ingress转发给了app1的service。
访问域名www.app2.com时,ingress转发给了app2的service。
echo '192.168.100.161 www.app1.com' >> /etc/hosts
curl www.app1.com:33646/app/
tomcat-app1
echo '192.168.100.162 www.app2.com' >> /etc/hosts
curl www.app2.com:33646/app/
tomcat-app2
ingress-url
访问不同url路径时,Ingress也可以根据规则转发到不同的service。
...
#前半部与上一例子相同
...
spec:
rules:
- host: www.myserver.com
http:
paths:
- pathType: Prefix
path: "/app1" # tomcat的webapps下必须有app1目录
backend:
service:
name: myserver-tomcat-app1-service
port:
number: 80
- pathType: Prefix
path: "/app2" # tomcat的webapps下必须有app2目录
backend:
service:
name: myserver-tomcat-app2-service
port:
number: 80
访问www.myserver.com/app1/时,ingress转发给了app1的service。
访问www.myserver.com/app2/时,ingress转发到了app2的service。
echo '192.168.100.163 www.myserver.com' >> /etc/hosts
curl www.myserver.com:33646/app1/
tomcat-app1
echo '192.168.100.163 www.myserver.com' >> /etc/hosts
curl www.myserver.com:33646/app2/
tomcat-app2
Ingress-https案例
外部IP连接Ingress使用https协作,证书签发给Ingress。而Ingress连接service使用http,不再需要证书。
https证书签发
在deploy部署服务器创建目录,为自己的域名 app1.myserver.com 签发证书:
# 制作10年有效期的ca证书与私钥
openssl req -x509 -sha256 -newkey rsa:4096 -keyout ca.key -out ca.crt -days 3560 -nodes -subj '/CN=app1.myserver.com'
# ca.crt ca.key
# 给服务器制作证书请求与私钥
openssl req -new -newkey rsa:4096 -keyout server.key -out server.csr -nodes -subj '/CN=app1.myserver.com'
# server.csr server.key
# 证书请求+ca证书=服务器证书
openssl x509 -req -sha256 -days 3650 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt
# server.crt
将证书上传k8s并保存成secret:
kubectl create secret tls tls-secret-app1 --cert=server.crt --key=server.key -n myserver
single-host
单个域名https转发,需要先指定上述的secret。
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-web
namespace: myserver
annotations:
kubernetes.io/ingress.class: "nginx" # 指定Ingress Controller的类型
nginx.ingress.kubernetes.io/ssl-redirect: 'true' # SSL重定向,即将http请求强制重定向至https,等于nginx中的全站https
spec:
tls:
- hosts:
- app1.myserver.com
secretName: tls-secret-app1
rules:
- host: app1.myserver.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: myserver-tomcat-app1-service
port:
number: 80
在集群外windows电脑做测试,先修改hosts文件:
"C:\Windows\System32\drivers\etc\hosts"
192.168.100.165 app1.myserver.com
使用chrome浏览器访问 https://app1.myserver.com:32883/
由于ca证书不是公认的,证书显示无效。但是可以选择继续前往。
https请求转发到了app1服务
multi-host
多个域名,每个域名都需要生成证书:
openssl req -x509 -sha256 -newkey rsa:4096 -keyout ca.key -out ca.crt -days 3560 -nodes -subj '/CN=app2.myserver.com'
openssl req -new -newkey rsa:4096 -keyout server.key -out server.csr -nodes -subj '/CN=app2.myserver.com'
openssl x509 -req -sha256 -days 3650 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt
将证书上传k8s并保存成另一个secret:
kubectl create secret tls tls-secret-app2 --cert=server.crt --key=server.key -n myserver
创建多个域名的Ingress请求,需指定多个host的证书文件。
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-web
namespace: myserver
annotations:
kubernetes.io/ingress.class: "nginx" # 指定Ingress Controller的类型
nginx.ingress.kubernetes.io/ssl-redirect: 'true'
spec:
tls:
- hosts:
- app1.myserver.com
secretName: tls-secret-app1
- hosts:
- app2.myserver.com
secretName: tls-secret-app2
rules:
- host: app1.myserver.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: myserver-tomcat-app1-service
port:
number: 80
- host: app2.myserver.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: myserver-tomcat-app2-service
port:
number: 80
在集群外windows电脑做测试,IP为集群内任一节点IP,app1与app2也可以指定相同IP:
"C:\Windows\System32\drivers\etc\hosts"
192.168.100.165 app1.myserver.com
192.168.100.166 app2.myserver.com
想显示如下首页文字,需要进pod添加首页文件。
kubectl exec -it -n myserver myserver-tomcat-app2-deployment-7c5f59bb99-f7nc6 -- bash
echo myserver-tomcat-app2 > /usr/local/tomcat/webapps/ROOT/index.html
Ingress证书更新
在上述例子中,如果只创建1年有效期的证书。很快证书会到期,需要平滑的不影响业务的更新证书。
基本思路:创建新证书,把证书crt和key经过base64加密,直接修改secret文件。
创建生成证书,域名一致,有效期延长至5年:
openssl req -x509 -sha256 -newkey rsa:4096 -keyout ca.key -out ca.crt -days 1825 -nodes -subj '/CN=app1.myserver.com'
openssl req -new -newkey rsa:4096 -keyout server.key -out server.csr -nodes -subj '/CN=app1.myserver.com'
openssl x509 -req -sha256 -days 1825 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt
将crt与key进行base64加密,-w 0 不自动换行,拷贝两tls文件的值。
# server.crt对应secret中tls.crt的值
base64 -w 0 server.crt > tls.crt
# server.key对应secret中tls.key的值
base64 -w 0 server.key > tls.key
修改tls的secret文件,插入上述拷贝的两个值。
kubectl edit secrets tls-secret-app1 -n myserver
# 修改好会提示已编辑
secret/tls-secret-app1 edited
重新在chrome访问页面,查看证书:
或者在linux查看证书 curl -kvl https://app1.myserver.com:32883
curl -kvl https://app1.myserver.com:32883
* About to connect() to app1.myserver.com port 32883 (#0)
* Trying 192.168.100.164...
* Connected to app1.myserver.com (192.168.100.164) port 32883 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* subject: CN=app1.myserver.com
* start date: 9月 27 13:09:12 2022 GMT
* expire date: 9月 26 13:09:12 2027 GMT
* common name: app1.myserver.com
* issuer: CN=app1.myserver.com
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: app1.myserver.com:32883
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 27 Sep 2022 13:52:27 GMT
< Content-Type: text/html
< Content-Length: 21
< Connection: keep-alive
< Accept-Ranges: bytes
< ETag: W/"21-1664279374000"
< Last-Modified: Tue, 27 Sep 2022 11:49:34 GMT
< Strict-Transport-Security: max-age=15724800; includeSubDomains
<
myserver-tomcat-app1
* Connection #0 to host app1.myserver.com left intact