k8s-ingress控制器
k8s-ingress控制器
- 一、概念
- 二、安装ingress controller
- 2.1 将镜像scp到所有的node节点服务器上
- 2.2 导入镜像,在所有的节点服务器上进行
- 2.3 使用ingress-controller-deploy.yaml 文件去启动ingress controller
- 2.4 查看ingress controller的相关信息
- 2.5 创建pod和暴露pod的服务
- 2.6 启用ingress 关联ingress controller 和service
- 2.7 查看ingress controller 里的nginx.conf 文件里是否有ingress对应的规则
- 2.8 进入ingress controller对应的pod里查看nginx.conf的配置
- 二、基于url的负载均衡的实现
官方文档:https://kubernetes.io/zh-cn/docs/concepts/services-networking/ingress/
一、概念
Ingress 控制器是实现外部流量(如来自互联网的 HTTP/HTTPS 请求)路由到集群内部服务的核心组件。它解决了传统 Service(如 NodePort、LoadBalancer)在多服务、多域名场景下的灵活性不足问题,提供了统一的入口管理、域名路由、SSL 终止等高级功能
Ingress只需要一个NodePort或者一个LB就可以满足暴露多个Service的需求
实际上,Ingress相当于一个7层的负载均衡器,是kubernetes对反向代理的一个抽象,它的工作原理类似于Nginx,可以理解成在Ingress里建立诸多映射规则,Ingress Controller通过监听这些配置规则并转化成Nginx的反向代理配置 , 然后对外部提供服务
- ingress:kubernetes中的一个对象,作用是定义请求如何转发到service的规则
- ingress controller:具体实现反向代理及负载均衡的程序,对ingress定义的规则进行解析,根据配置的规则来实现请求转发
二、安装ingress controller
2.1 将镜像scp到所有的node节点服务器上
[root@k8s-1 ingress]# scp ingress-nginx-controllerv1.1.0.tar.gz k8s-2:/root
[root@k8s-1 ingress]# scp ingress-nginx-controllerv1.1.0.tar.gz k8s-3:/root
[root@k8s-1 ingress]# scp kube-webhook-certgen-v1.1.0.tar.gz k8s-2:/root
[root@k8s-1 ingress]# scp kube-webhook-certgen-v1.1.0.tar.gz k8s-3:/root
2.2 导入镜像,在所有的节点服务器上进行
[root@k8s-2 ~]# docker load -i ingress-nginx-controllerv1.1.0.tar.gz
[root@k8s-2 ~]# docker load -i kube-webhook-certgen-v1.1.0.tar.gz
[root@k8s-2 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller v1.1.0 ae1a7201ec95 3 years ago 285MB
registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen v1.1.1 c41e9fcadf5a 3 years ago 47.7MB
2.3 使用ingress-controller-deploy.yaml 文件去启动ingress controller
[root@k8s-1 ingress]# kubectl apply -f ingress-controller-deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
2.4 查看ingress controller的相关信息
[root@k8s-1 ingress]# kubectl get ns|grep ingress
NAME STATUS AGE
ingress-nginx Active 32s
[root@k8s-1 ingress]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.105.212.102 <none> 80:31407/TCP,443:32222/TCP 46s
ingress-nginx-controller-admission ClusterIP 10.106.179.205 <none> 443/TCP 46s
[root@k8s-1 ingress]# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-rq5cg 0/1 Completed 0 58s
ingress-nginx-admission-patch-hchjm 0/1 Completed 0 58s
ingress-nginx-controller-7cd558c647-hmzmx 1/1 Running 0 58s
ingress-nginx-controller-7cd558c647-w867n 1/1 Running 0 58s
2.5 创建pod和暴露pod的服务
[root@k8s-1 ingress]# kubectl apply -f sc-nginx-svc-1.yaml
deployment.apps/sc-nginx-deploy created
service/sc-nginx-svc created
[root@k8s-1 ingress]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
sc-nginx-deploy 3/3 3 3 13s
[root@k8s-1 ingress]# kubectl get pod
NAME READY STATUS RESTARTS AGE
sc-nginx-deploy-55cc5ffddf-2tl7b 1/1 Running 0 16s
sc-nginx-deploy-55cc5ffddf-crdhm 1/1 Running 0 16s
sc-nginx-deploy-55cc5ffddf-xxkwt 1/1 Running 0 16s[root@k8s-1 ingress]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
sc-nginx-svc ClusterIP 10.100.3.0 <none> 80/TCP 63s
# 访问服务暴露的ip
[root@k8s-1 ingress]# curl 10.100.3.0
2.6 启用ingress 关联ingress controller 和service
[root@k8s-1 ingress]# cat sc-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: sc-ingressannotations:kubernets.io/ingress.class: nginx #注释 这个ingress 是关联ingress controller的
spec:ingressClassName: nginx #关联ingress controllerrules:- host: www.feng.comhttp:paths:- pathType: Prefixpath: /backend:service:name: sc-nginx-svcport:number: 80- host: www.zhang.comhttp:paths:- pathType: Prefixpath: /backend:service:name: sc-nginx-svc-2port:number: 80
# 执行文件
[root@k8s-1 ingress]# kubectl apply -f sc-ingress.yaml
ingress.networking.k8s.io/sc-ingress created
# 查看效果
[root@k8s-1 ingress]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
sc-ingress nginx www.feng.com,www.zhang.com 192.168.168.146,192.168.168.147 80 56s
2.7 查看ingress controller 里的nginx.conf 文件里是否有ingress对应的规则
[root@k8s-1 ingress]# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-rq5cg 0/1 Completed 0 46m
ingress-nginx-admission-patch-hchjm 0/1 Completed 0 46m
ingress-nginx-controller-7cd558c647-hmzmx 1/1 Running 0 46m
ingress-nginx-controller-7cd558c647-w867n 1/1 Running 0 46m
2.8 进入ingress controller对应的pod里查看nginx.conf的配置
[root@k8s-1 ingress]# kubectl exec -n ingress-nginx -it ingress-nginx-controller-7cd558c647-hmzmx -- bash
bash-5.1$ cd /etc/nginx/
bash-5.1$ cat nginx.conf|grep zhang.com## start server www.zhang.comserver_name www.zhang.com ;## end server www.zhang.com
bash-5.1$ cat nginx.conf|grep feng.com## start server www.feng.comserver_name www.feng.com ;## end server www.feng.com
bash-5.1$ cat nginx.conf|grep -C3 upstream_balancererror_log /var/log/nginx/error.log notice;upstream upstream_balancer {server 0.0.0.1:1234; # placeholderbalancer_by_lua_block {
获取ingress controller对应的service暴露宿主机的端口,访问宿主机和相关端口,就可以验证ingress controller是否能进行负载均衡
[root@k8s-1 ingress]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.105.212.102 <none> 80:31407/TCP,443:32222/TCP 51m
ingress-nginx-controller-admission ClusterIP 10.106.179.205 <none> 443/TCP 51m# 使用域名进行访问
[root@k8s-1 ingress]# vim /etc/hosts
192.168.168.146 www.feng.com
192.168.168.147 www.zhang.com
因为我们是基于域名做的负载均衡的配置,所有必须要在浏览器里使用域名去访问,不能使用ip地址
同时ingress controller做负载均衡的时候是基于http协议的,7层负载均衡
[root@k8s-1 ingress]# curl www.feng.com
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p><p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p><p><em>Thank you for using nginx.</em></p>
</body>
</html>
访问www.zhang.com出现异常,503错误,是nginx内部错误,因为没创建,创建后也可以正常访问
[root@k8s-1 ingress]# curl www.zhang.com
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx</center>
</body>
</html>
二、基于url的负载均衡的实现
路由规则定义
[root@k8s-1 ingress]# cat sc-ingress-url.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: simple-url-lb-exampleannotations:kubernets.io/ingress.class: nginx
spec:ingressClassName: nginxrules:- host: www.wen.comhttp:paths:- path: /foopathType: Prefixbackend:service:name: sc-nginx-svc-3port:number: 80- path: /barpathType: Prefixbackend:service:name: sc-nginx-svc-4port:number: 80
[root@k8s-1 ingress]# kubectl apply -f sc-ingress-url.yaml
ingress.networking.k8s.io/simple-fanout-example created
[root@k8s-1 ingress]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
sc-ingress nginx www.feng.com,www.zhang.com 192.168.168.146,192.168.168.147 80 18m
simple-fanout-example nginx www.wen.com 192.168.168.146,192.168.168.147 80 45s
后端服务配置
[root@master url]# cat sc-nginx-svc-3.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: sc-nginx-deploy-3labels:app: sc-nginx-feng-3
spec:replicas: 3selector:matchLabels:app: sc-nginx-feng-3template:metadata:labels:app: sc-nginx-feng-3spec:containers:- name: sc-nginx-feng-3image: nginximagePullPolicy: IfNotPresentports:- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:name: sc-nginx-svc-3labels:app: sc-nginx-svc-3
spec:selector:app: sc-nginx-feng-3ports:- name: name-of-service-portprotocol: TCPport: 80targetPort: 80[root@master url]# cat sc-nginx-svc-4.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: sc-nginx-deploy-4labels:app: sc-nginx-feng-4
spec:replicas: 3selector:matchLabels:app: sc-nginx-feng-4template:metadata:labels:app: sc-nginx-feng-4spec:containers:- name: sc-nginx-feng-4image: nginximagePullPolicy: IfNotPresentports:- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:name: sc-nginx-svc-4labels:app: sc-nginx-svc-4
spec:selector:app: sc-nginx-feng-4ports:- name: name-of-service-portprotocol: TCPport: 80targetPort: 80[root@k8s-1 ingress]# kubectl apply -f sc-nginx-svc-3.yaml
deployment.apps/sc-nginx-deploy-3 created
service/sc-nginx-svc-3 created
[root@k8s-1 ingress]# kubectl apply -f sc-nginx-svc-4.yaml
deployment.apps/sc-nginx-deploy-4 created
service/sc-nginx-svc-4 created
在/etc/hosts文件里添加域名解析记录
[root@k8s-1 ingress]# cat /etc/hosts
192.168.168.146 www.wen.com
192.168.168.147 www.wen.com
测试发现不能找到页面
/usr/share/nginx/html/bar 文件夹不存在,导致404错误
进入service4 对应的一个pod里,新建bar和foo文件夹以及index.html网页文件
echo “hello,bar” >bar/index.html
再次在nfs服务器上测试,多测试几次,因为service 背后的ipvs的调度算法是轮询的
[root@nfs-server ~]# curl www.wen.com/foo/index.html
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.21.5</center>
</body>
</html>
[root@nfs-server ~]# curl www.wen.com/bar/index.html
hello,bar
查看配置的负载均衡策略
[root@k8s-1 ingress]# kubectl describe ingress simple-fanout-example
Name: simple-fanout-example
Labels: <none>
Namespace: default
Address: 192.168.168.146,192.168.168.147
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:Host Path Backends---- ---- --------www.wen.com /foo sc-nginx-svc-3:80 (<none>)/bar sc-nginx-svc-4:80 (<none>)
Annotations: kubernets.io/ingress.class: nginx
Events:Type Reason Age From Message---- ------ ---- ---- -------Normal Sync 5h35m nginx-ingress-controller Scheduled for syncNormal Sync 5h34m nginx-ingress-controller Scheduled for sync