Traefik实现Ingress-IngressRoute-IngressRouteTCP-IngressRouteUDP及Traefik高级流量治理
🌟Ingress和Service的区别
- 1.Service是实现了四层代理,不需要部署额外组件,K8S原生支持;
- 2.Ingress实现的是七层代理,但需要部署附加组件(IngressClass)来解析Ingress资源清单;
Ingress是k8s集群的一个资源,用来映射七层代理到svc的关系。
🌟基于helm部署trafik实战
参考链接: https://doc.traefik.io/traefik/getting-started/install-traefik/#use-the-helm-chart
添加仓库
[root@master231 ~]# helm repo add traefik https://traefik.github.io/charts
"traefik" has been added to your repositories
[root@master231 ~]# helm repo list
NAME URL
traefik https://traefik.github.io/charts
更新仓库信息
[root@master231 ~]# helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "traefik" chart repository
Update Complete. ⎈Happy Helming!⎈
导入镜像(所有节点)
docker load -i traefik-v3.4.3.tar.gz
安装traefik
[root@master231 ~]# helm search repo traefik
[root@master231 ~]# helm pull traefik/traefik --version 36.3.0
[root@master231 ~]# tar xf traefik-36.3.0.tgz -C /zhu/manifests/add-ons/
[root@master231 ~]# helm install traefik-server /zhu/manifests/add-ons/traefik/ -n traefik --create-namespace
查看服务
[root@master231 ~]# kubectl get ingressclasses,deploy,svc,po -o wide -n traefik
访问Traefik的WebUI
http://10.0.0.151/
温馨提示: 返回页面应该是404,说明我们的traefik部署成功了。
🌟Ingress实现修仙业务代理实战
编写资源清单
[root@master231 ingresses]# cat 01-deploy-svc-xiuxian.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: deploy-xiuxian-v1
spec:replicas: 3selector:matchLabels:apps: v1template:metadata:labels:apps: v1spec:containers:- name: c1image: registry.cn-hangzhou.aliyuncs.com/yinzhengjie-k8s/apps:v1ports:- containerPort: 80---
apiVersion: apps/v1
kind: Deployment
metadata:name: deploy-xiuxian-v2
spec:replicas: 3selector:matchLabels:apps: v2template:metadata:labels:apps: v2spec:containers:- name: c1image: registry.cn-hangzhou.aliyuncs.com/yinzhengjie-k8s/apps:v2ports:- containerPort: 80---
apiVersion: apps/v1
kind: Deployment
metadata:name: deploy-xiuxian-v3
spec:replicas: 3selector:matchLabels:apps: v3template:metadata:labels:apps: v3spec:containers:- name: c1image: registry.cn-hangzhou.aliyuncs.com/yinzhengjie-k8s/apps:v3ports:- containerPort: 80---apiVersion: v1
kind: Service
metadata:name: svc-xiuxian-v1
spec:type: ClusterIPselector:apps: v1ports:- port: 80---apiVersion: v1
kind: Service
metadata:name: svc-xiuxian-v2
spec:type: ClusterIPselector:apps: v2ports:- port: 80---apiVersion: v1
kind: Service
metadata:name: svc-xiuxian-v3
spec:type: ClusterIPselector:apps: v3ports:- port: 80
[root@master231 ingresses]#
创建资源
[root@master231 ingresses]# kubectl apply -f 01-deploy-svc-xiuxian.yaml
deployment.apps/deploy-xiuxian-v1 created
deployment.apps/deploy-xiuxian-v2 created
deployment.apps/deploy-xiuxian-v3 created
service/svc-xiuxian-v1 created
service/svc-xiuxian-v2 created
service/svc-xiuxian-v3 created
编写Ingress规则
[root@master231 ingresses]# cat 02-ingress-xiuxian.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: ingress-xiuxian
spec:# 指定IngressClass的名称,不使用之前的Ingress-nginx,而是使用Traefik。# ingressClassName: nginxingressClassName: traefik-serverrules:- host: v1.zhubl.xyzhttp:paths:- pathType: Prefixpath: /backend:service:name: svc-xiuxian-v1port:number: 80- host: v2.zhubl.xyzhttp:paths:- pathType: Prefixbackend:service:name: svc-xiuxian-v2port:number: 80path: /- host: v3.zhubl.xyzhttp:paths:- pathType: Prefixbackend:service:name: svc-xiuxian-v3port:number: 80path: /
[root@master231 ingresses]#
创建ingress资源
[root@master231 ingresses]# kubectl apply -f 02-ingress-xiuxian.yaml
ingress.networking.k8s.io/ingress-xiuxian created
访问测试
[root@master231 ingresses]# curl -H "HOST: v1.zhubl.xyz" http://10.0.0.151/
[root@master231 ingresses]# curl -H "HOST: v2.zhubl.xyz" http://10.0.0.151/
[root@master231 ingresses]# curl -H "HOST: v3.zhubl.xyz" http://10.0.0.151/
🌟Ingress实现uri多路径匹配
编写资源清单
[root@master231 ingresses]# cat 03-ingress-xiuxian-uri.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: ingress-xiuxian-uri
spec:ingressClassName: traefik-serverrules:- host: xiuxian.zhubl.xyzhttp:paths:- pathType: Prefixpath: /v1backend:service:name: svc-xiuxian-v1port:number: 80- pathType: Prefixpath: /v2backend:service:name: svc-xiuxian-v2port:number: 80- pathType: Prefixpath: /v3backend:service:name: svc-xiuxian-v3port:number: 80
创建Ingress资源
[root@master231 ingresses]# kubectl apply -f 03-ingress-xiuxian-uri.yaml
ingress.networking.k8s.io/ingress-xiuxian-uri created[root@master231 ingresses]# kubectl get ing ingress-xiuxian-uri[root@master231 ingresses]# kubectl describe ing ingress-xiuxian-uri
修改Pod副本数量
[root@master231 ingresses]# kubectl scale deployment deploy-xiuxian-v1 --replicas=1
deployment.apps/deploy-xiuxian-v1 scaled
[root@master231 ingresses]# kubectl scale deployment deploy-xiuxian-v2 --replicas=1
deployment.apps/deploy-xiuxian-v2 scaled
[root@master231 ingresses]# kubectl scale deployment deploy-xiuxian-v3 --replicas=1
deployment.apps/deploy-xiuxian-v3 scaled
修改测试数据
[root@master231 ingresses]# kubectl exec -it deploy-xiuxian-v1-6bc556784f-x6t8k -- sh
mkdir /usr/share/nginx/html/v1
echo "<h1 style='color: red;'>www.zhubl.xyz</h1>" > /usr/share/nginx/html/v1/index.html
cat /usr/share/nginx/html/v1/index.html[root@master231 ingresses]# kubectl exec -it deploy-xiuxian-v2-64bb8c9785-vgrws -- sh
mkdir /usr/share/nginx/html/v2
echo "<h1 style='color: green;'>www.zhubl.xyz</h1>" > /usr/share/nginx/html/v2/index.html
cat /usr/share/nginx/html/v2/index.html[root@master231 ingresses]# kubectl exec -it deploy-xiuxian-v3-698c86cf85-dv8rf -- sh
mkdir /usr/share/nginx/html/v3
echo "<h1 style='color: pink;'>www.zhubl.xyz</h1>" > /usr/share/nginx/html/v3/index.html
cat /usr/share/nginx/html/v3/index.html
测试验证
http://xiuxian.zhubl.xyz/v1/
http://xiuxian.zhubl.xyz/v2/
http://xiuxian.zhubl.xyz/v3/
🌟ingress的映射https
生成证书文件
如果有生产环境的证书此步骤可以跳过
openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=www.zhubl.xyz"
将证书文件以secrets形式存储
[root@master231 ingresses]# kubectl create secret tls ca-secret --cert=tls.crt --key=tls.key
secret/ca-secret created
[root@master231 ingresses]# kubectl get secrets ca-secret
NAME TYPE DATA AGE
ca-secret kubernetes.io/tls 2 14s
部署测试服务
[root@master231 ingresses]# cat 04-deploy-apple.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: deployment-apple
spec:replicas: 3selector:matchLabels:apps: appletemplate:metadata:labels:apps: applespec:containers:- name: appleimage: registry.cn-hangzhou.aliyuncs.com/yinzhengjie-k8s/apps:v1ports:- containerPort: 80---apiVersion: v1
kind: Service
metadata:name: svc-apple
spec:selector:apps: appleports:- protocol: TCPport: 80targetPort: 80
创建资源
[root@master231 ingresses]# kubectl apply -f 04-deploy-apple.yaml
deployment.apps/deployment-apple created
service/svc-apple created
配置Ingress添加TLS证书
[root@master231 ingresses]# cat 05-ingress-tls.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: ingress-tls-https# 如果指定了"ingressClassName"参数,就不需要在这里重复声明啦。# 如果你的K8S 1.22- 版本,则使用注解的方式进行传参即可。#annotations:# kubernetes.io/ingress.class: "nginx"
spec:# 指定Ingress Class,要求你的K8S 1.22+ingressClassName: traefik-serverrules:- host: www.zhubl.xyzhttp:paths:- backend:service:name: svc-appleport:number: 80path: /pathType: ImplementationSpecific# 配置https证书tls:- hosts:- www.zhubl.xyzsecretName: ca-secret
创建资源
[root@master231 ingresses]# kubectl apply -f 05-ingress-tls.yaml
ingress.networking.k8s.io/ingress-tls-https created
访问测试
https://www.zhubl.xyz/
温馨提示:
如果google浏览器自建证书不认可,可以用鼠标在空白处单击左键,而后输入:“thisisunsafe”,就会自动跳转。
当然,如果不想打这个代码,可以使用火狐浏览器打开即可。
🌟Traefik开启Dashboard
开启Dashboard参数
[root@master231 ingresses]# vim /zhu/manifests/add-ons/traefik/values.yaml +192
...187 ingressRoute:188 dashboard:189 # -- Create an IngressRoute for the dashboard 190 # enabled: false191 enabled: true
重新安装traefik
[root@master231 ingresses]# helm -n traefik uninstall traefik-server
release "traefik-server" uninstalled[root@master231 ingresses]# helm -n traefik install traefik-server /zhu/manifests/add-ons/traefik/
NAME: traefik-server
LAST DEPLOYED: Thu Oct 2 12:18:12 2025
NAMESPACE: traefik
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
traefik-server with docker.io/traefik:v3.4.3 has been deployed successfully on traefik namespace !
创建svc关联Dashboard
[root@master231 ingresses]# cat 06-svc-ing-traefik-dashboard.yaml
apiVersion: v1
kind: Service
metadata:name: traefik-dashboardnamespace: traefik
spec:ports:- name: dashboardport: 8080selector:app.kubernetes.io/name: traefiktype: ClusterIP---apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: ingress-traefik-dashboardnamespace: traefik
spec:ingressClassName: traefik-serverrules:- host: traefik.zhubl.xyzhttp:paths:- pathType: Prefixpath: /backend:service:name: traefik-dashboardport:number: 8080
创建资源
[root@master231 ingresses]# kubectl apply -f 06-svc-ing-traefik-dashboard.yaml
service/traefik-dashboard created
ingress.networking.k8s.io/ingress-traefik-dashboard created
[root@master231 ingresses]# kubectl -n traefik get ingress ingress-traefik-dashboard
[root@master231 ingresses]# kubectl -n traefik describe ingress ingress-traefik-dashboard
访问traefik的WebUI
http://traefik.zhubl.xyz/dashboard/#/
🌟Traefik架构
Traefik核心概念
参考链接:https://docs.traefik.cn/basics
Traefik所示一个边缘路由器,它会拦截外部的请求并根据逻辑规则选择不同的操作方式,这些规则决定着这些请求到底该如何处理。
Traefik提供自动发现能力,会实时检测服务,并自动更新路由规则。
请求首先会链接到"entrypoint(入口点)","frontends(前端)“和"backends(后端)”
-
entrypoint(入口点):
请求在入口点处结束, 顾名思义, 它们是Træfɪk的网络入口(监听端口, SSL, 流量重定向…)。
Entrypoints是Traefik的网络入口,它定义接受请求的接口,以及是否监听TCP或者UDP。
-
frontends(前端):
之后流量会导向一个匹配的前端。 前端是定义入口点到后端之间的路由的地方。
路由是通过请求字段(Host, Path, Headers…) 来定义的,它可以匹配或否定一个请求。
-
backends(后端):
前端将会把请求发送到后端。后端可以由一台或一个通过负载均衡策略配置后的多台服务器组成。
最后, 服务器将转发请求到对应私有网络的微服务当中去。
重要核心组件
-
Providers
Providers是基础组件,Traefik的配置发现是通过它来实现的,它可以是协调器,容器引擎,云提供商或者键值存储。
Traefik通过查询Provides的API来查询路由的相关信息,一旦检测到变化,就会动态的更新路由。
-
Routers
Routess主要用于分析请求,并负责将这些请求连接到对应的服务上去。在这个过程中,Routers还可以使用Middlewares来更新请求,比如:在把请求发到服务之前添加一些Headers。
-
Middlewares
Middlewarees用来修改请求或者根据请求来做出一些判断(authentiacation,rate limiting,headers,…),
中间件被附加到路由上,是一种在请求发送到你的服务之前(或者在服务的响应发送到客户端之前)调整请求的一种方法。
-
Services
Servers负责配置如何到达最终将处理传入的实际服务。
Traefik支持的路由规则
Traefik提供了三种创建路由规则的方法:
-
原生Ingress
K8S原生支持的资源。
-
CRD IngressRoute
部署Traefik时安装的自定义资源。
-
Gateway API
基于Gateway的API来实现暴露。是k8s官方基于Ingress的一种扩展实现。
🌟基于IngressRoute暴露Traefik的Dashboard
查看K8S集群现有的Ingress资源
[root@master231 Prometheus]# kubectl get ingress -A
编写资源清单
[root@master231 ingressroutes]# cat 01-ingressroutes-traefik-dashboard.yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:name: ingressroute-traefiknamespace: traefik
spec:# 指定入口点entryPoints:- web# 配置路由routes:# 匹配访问的请求为: "traefik.zhubaolin.com/"- match: Host(`traefik.zhubaolin.com`) && PathPrefix(`/`)kind: Rule# 调度到后端的svcservices:# 指定svc- name: traefik-dashboard# 指定端口port: 8080
[root@master231 ingressroutes]#
创建资源
[root@master231 ingressroutes]# kubectl apply -f 01-ingressroutes-traefik-dashboard.yaml
ingressroute.traefik.io/ingressroute-traefik created
[root@master231 ingressroutes]#
[root@master231 ingressroutes]# kubectl get ingressroute -n traefik
NAME AGE
ingressroute-traefik 11s
traefik-server-dashboard 3h8m
访问测试
http://traefik.zhubaolin.com/dashboard/#/
🌟Traefik的测试环境部署IngressRoute
K8S所有节点导入镜像
traefik-whoamiudp-v0.2.tar.gz
traefik-whoamitcp-v0.3.tar.gz
traefik-whoami-v1.11.tar.gzfor i in `ls -1 traefik-whoami*` ; do docker load -i $i;done
编写资源清单
[root@master231 ingressroutes]# cat 02-deploy-svc-whoami.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: deploy-whoami
spec:replicas: 2selector:matchLabels:apps: whoamitemplate:metadata:labels:apps: whoamispec:containers:- name: whoamiimage: docker.io/traefik/whoami:v1.11imagePullPolicy: IfNotPresentports:- containerPort: 80---apiVersion: apps/v1
kind: Deployment
metadata:name: deploy-whoamitcp
spec:replicas: 2selector:matchLabels:apps: whoamitcptemplate:metadata:labels:apps: whoamitcpspec:containers:- name: whoamitcpimage: docker.io/traefik/whoamitcp:v0.3imagePullPolicy: IfNotPresentports:- containerPort: 8080protocol: TCP---apiVersion: apps/v1
kind: Deployment
metadata:name: deploy-whoamiudp
spec:replicas: 2selector:matchLabels:apps: whoamiudptemplate:metadata:labels:apps: whoamiudpspec:containers:- name: whoamiudpimage: docker.io/traefik/whoamiudp:v0.2imagePullPolicy: IfNotPresentports:- containerPort: 8080protocol: UDP---apiVersion: v1
kind: Service
metadata:name: svc-whoami
spec:ports:- name: httpport: 80selector:apps: whoami---apiVersion: v1
kind: Service
metadata:name: svc-whoamitcp
spec:ports:- name: tcpport: 8080selector:apps: whoamitcp---apiVersion: v1
kind: Service
metadata:name: svc-whoamiudp
spec:ports:- name: udpport: 8080protocol: UDPselector:apps: whoamiudp
[root@master231 ingressroutes]#
创建资源
[root@master231 ingressroutes]# kubectl apply -f 02-deploy-svc-whoami.yaml
deployment.apps/deploy-whoami created
deployment.apps/deploy-whoamitcp created
deployment.apps/deploy-whoamiudp created
service/svc-whoami created
service/svc-whoamitcp created
service/svc-whoamiudp created
创建IngressRoute路由规则代理whoami程序
[root@master231 ingressroutes]# cat 03-ingressroutes-whoami.yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:name: ingressroute-whoaminamespace: default
spec:entryPoints:- webroutes:- match: Host(`whoami.zhubaolin.com`) && PathPrefix(`/`)kind: Ruleservices:- name: svc-whoamiport: 80[root@master231 ingressroutes]# kubectl apply -f 03-ingressroutes-whoami.yaml
ingressroute.traefik.io/ingressroute-whoami created
[root@master231 ingressroutes]#
[root@master231 ingressroutes]# kubectl get ingressroute ingressroute-whoami
NAME AGE
ingressroute-whoami 13s
测试验证
[root@master231 ingressroutes]# kubectl get svc traefik-server -n traefik
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik-server LoadBalancer 10.200.215.150 10.0.0.151 80:12847/TCP,443:30897/TCP 3h22m
[root@master231 ingressroutes]# curl -H 'HOST: whoami.zhubaolin.com' http://10.0.0.151
浏览器访问
https://whoamissl.zhubaolin.com/
🌟配置https路由规则之whoami
配置https路由规则注意事项
如果我们需要使用https来访问我们这个应用的话,就需要监听websecure这个入口点,也就是通过443端口来访问。
用HTTPS访问应用必然就需要证书,这个证书可以是自签证书,也可以是权威机构颁发的证书。
创建证书并封装为secret资源
使用openssl自建证书
[root@master231 ingressroutes]# openssl req -x509 -nodes -days 365 --newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=whoamissl.zhubaolin.com"[root@master231 ingressroutes]# ll tls.*
-rw-r--r-- 1 root root 1151 Oct 2 15:41 tls.crt
-rw------- 1 root root 1704 Oct 2 15:41 tls.key
将证书封装为secrets资源
[root@master231 ingressroutes]# kubectl create secret tls whoami-tls --cert=tls.crt --key=tls.key
secret/whoami-tls created
[root@master231 ingressroutes]# kubectl get secrets whoami-tls
NAME TYPE DATA AGE
whoami-tls kubernetes.io/tls 2 7s
创建https应用路由规则并访问测试
[root@master231 ingressroutes]# cat 04-ingressroutes-whoami-https.yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:name: ingressroute-whoami-httpsnamespace: default
spec:tls:secretName: whoami-tlsentryPoints:- websecureroutes:- match: Host(`whoamissl.zhubaolin.com`)kind: Ruleservices:- name: svc-whoamiport: 80[root@master231 ingressroutes]# kubectl apply -f 04-ingressroutes-whoami-https.yaml
ingressroute.traefik.io/ingressroute-whoami-https created
[root@master231 ingressroutes]# kubectl get ingressroute ingressroute-whoami-https
NAME AGE
ingressroute-whoami-https 9s
访问测试
https://whoamissl.zhubaolin.com/
🌟配置tcp路由规则之MySQL
配置tcp路由规则注意事项
SNI为服务名称标识,是TLS协议的扩展,因此,只有TLS路由才能使用该规则指定域名。
但是,非TLS路由必须带有"*"(所有域)的规则来声明每个非TLS请求都将由路由进行处理
重新部署Traefik
修改values.yaml配置文件
目的是为了添加’entryPoints’
[root@master231 ingressroutes]# vim /zhu/manifests/add-ons/traefik/values.yaml +635
...
ports:# 添加如下2行,不要做任何修改原有的配置。mysql:port: 3306
卸载Traefik服务
[root@master231 ingressroutes]# helm -n traefik uninstall traefik-server
release "traefik-server" uninstalled
安装Traefik服务
[root@master231 ingressroutes]# helm -n traefik install traefik-server /zhu/manifests/add-ons/traefik/
NAME: traefik-server
LAST DEPLOYED: Thu Oct 2 15:45:31 2025
NAMESPACE: traefik
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
traefik-server with docker.io/traefik:v3.4.3 has been deployed successfully on traefik namespace !
访问webUI
http://traefik.zhubaolin.com/dashboard/#/
部署MySQL
编写资源清单
[root@master231 ingressroutes]# cat 05-deploy-mysql.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: deploy-mysql
spec:replicas: 1selector:matchLabels:apps: mysqltemplate:metadata:labels:apps: mysqlspec:containers:- image: harbor250.zhubl.xyz/zhubl-db/mysql:8.0.36-oraclename: dbports:- containerPort: 3306env:- name: MYSQL_DATABASEvalue: wordpress- name: MYSQL_ALLOW_EMPTY_PASSWORDvalue: "yes"- name: MYSQL_USERvalue: wordpress- name: MYSQL_PASSWORDvalue: wordpressargs:- --character-set-server=utf8 - --collation-server=utf8_bin- --default-authentication-plugin=mysql_native_password---apiVersion: v1
kind: Service
metadata:name: svc-mysql
spec:ports:- port: 3306selector:apps: mysql
创建资源
[root@master231 ingressroutes]# kubectl apply -f 05-deploy-mysql.yaml
deployment.apps/deploy-mysql created
service/svc-mysql created
[root@master231 ingressroutes]#
[root@master231 ingressroutes]# kubectl get pods -o wide -l apps=mysql
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
deploy-mysql-748bc68759-7pnt7 1/1 Running 0 15s 10.100.1.7 worker232 <none> <none>
创建路由规则
[root@master231 ingressroutes]# cat 06-IngressRouteTCP-mysql.yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRouteTCP
metadata:name: ingressroutetcp-mysqlnamespace: default
spec:# 使用自己定义的entryPoint。entryPoints:- mysqlroutes:- match: HostSNI(`*`)services:- name: svc-mysqlport: 3306[root@master231 ingressroutes]# kubectl apply -f 06-IngressRouteTCP-mysql.yaml
ingressroutetcp.traefik.io/ingressroutetcp-mysql created
[root@master231 ingressroutes]#
[root@master231 ingressroutes]# kubectl get ingressroutetcps ingressroutetcp-mysql
NAME AGE
ingressroutetcp-mysql 18s
[root@master231 ingressroutes]# kubectl edit svc traefik-server -n traefik
service/traefik-server edited
[root@master231 ingressroutes]# kubectl get svc traefik-server -n traefik
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik-server LoadBalancer 10.200.88.226 10.0.0.151 3306:37673/TCP,80:3859/TCP,443:13312/TCP 7m29s
查看Traefik的Dashboard验证
http://traefik.zhubaolin.com/dashboard/#/tcp/routers
修改Traefik的svc暴露端口
[root@master231 ingressroutes]# kubectl edit svc traefik-server -n traefik
service/traefik-server edited
...
spec:...ports:- name: mysqlport: 3306
...[root@master231 ingressroutes]# kubectl get svc traefik-server -n traefik
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik-server LoadBalancer 10.200.88.226 10.0.0.151 3306:37673/TCP,80:3859/TCP,443:13312/TCP 7m29s
客户端访问测试
[root@harbor250 ~]# mysql -h 10.0.0.151
mysql> USE wordpress
Database changed
mysql> CREATE TABLE student(id INT PRIMARY KEY AUTO_INCREMENT, name VARCHAR(255) NOT NULL, hobby VARCHAR(255) NOT NULL);
Query OK, 0 rows affected (0.01 sec)mysql> INSERT INTO student(name,hobby) VALUES ('XiXi','Sleep');
Query OK, 1 row affected (0.01 sec)mysql>
服务端测试验证
[root@master231 ingressroutes]# kubectl exec -it deploy-mysql-748bc68759-7pnt7 -- mysql
🌟配置tcp路由规则之Redis
部署Redis
[root@master231 ingressroutes]# cat 07-deploy-redis.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: deploy-redis
spec:replicas: 1selector:matchLabels:apps: redistemplate:metadata:labels:apps: redisspec:containers:- image: harbor250.zhubl.xyz/redis/redis:6.0.5name: dbports:- containerPort: 6379---apiVersion: v1
kind: Service
metadata:name: svc-redis
spec:ports:- port: 6379selector:apps: redis
[root@master231 ingressroutes]# kubectl apply -f 07-deploy-redis.yaml
deployment.apps/deploy-redis created
service/svc-redis created
重新部署Traefik
修改values.yaml配置文件
目的是为了添加’entryPoints’
[root@master231 ingressroutes]# vim /zhu/manifests/add-ons/traefik/values.yaml +635
...
ports:redis:port: 6379
卸载Traefik服务
[root@master231 ingressroutes]# helm -n traefik uninstall traefik-server
release "traefik-server" uninstalled
安装Traefik服务
[root@master231 ingressroutes]# helm -n traefik install traefik-server /zhu/manifests/add-ons/traefik/
NAME: traefik-server
LAST DEPLOYED: Thu Oct 2 20:18:57 2025
NAMESPACE: traefik
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
traefik-server with docker.io/traefik:v3.4.3 has been deployed successfully on traefik namespace !
创建路由规则
[root@master231 ingressroutes]# cat 08-IngressRouteTCP-redis.yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRouteTCP
metadata:name: ingressroutetcp-redisnamespace: default
spec:# 使用自己定义的entryPoint。entryPoints:- redisroutes:- match: HostSNI(`*`)services:- name: svc-redisport: 6379
[root@master231 ingressroutes]#
[root@master231 ingressroutes]# kubectl apply -f 08-IngressRouteTCP-redis.yaml
ingressroutetcp.traefik.io/ingressroutetcp-redis created
[root@master231 ingressroutes]#
[root@master231 ingressroutes]# kubectl get ingressroutetcps ingressroutetcp-redis
NAME AGE
ingressroutetcp-redis 25s
[root@master231 ingressroutes]#
访问traefki的WebUI
http://traefik.zhubaolin.com/dashboard/#/tcp/routers
k8s修改Traefik的svc解析记录
[root@master231 ingressroutes]# kubectl edit svc traefik-server -n traefik
...
spec:...ports:- name: redisport: 6379 ...
客户端测试
安装redis客户端工具
[root@harbor250 ~]# apt -y install redis-server
写入数据
[root@harbor250 ~]# echo 10.0.0.151 redis.zhubaolin.com >> /etc/hosts
[root@harbor250 ~]# tail -1 /etc/hosts
10.0.0.151 redis.zhubaolin.com
[root@harbor250 ~]# redis-cli --raw -n 5 -h redis.zhubaolin.com
redis.zhubaolin.com:6379[5]> KEYS *redis.zhubaolin.com:6379[5]> SET zhu xixi
OK
redis.zhubaolin.com:6379[5]> set zhubl haha
OK
redis.zhubaolin.com:6379[5]> KEYS *
zhu
zhubl
redis.zhubaolin.com:6379[5]>
服务端查看数据并验证
[root@master231 ingressroutes]# kubectl exec -it deploy-redis-5db97d7958-vxcj7 -- redis-cli -n 5
127.0.0.1:6379[5]> KEYS *
1) "zhu"
2) "zhubl"
127.0.0.1:6379[5]> get zhu
"xixi"
127.0.0.1:6379[5]> get zhubl
"haha"
127.0.0.1:6379[5]>
🌟配置UDP路由规则之whoamiudp
重新部署Traefik
修改values.yaml配置文件
[root@master231 ingressroutes]# vim /zhu/manifests/add-ons/traefik/values.yaml +635
...
ports:udpcase:port: 8081protocol: UDP
...
卸载Traefik服务
[root@master231 ingressroutes]# helm -n traefik uninstall traefik-server
release "traefik-server" uninstalled
安装Traefik服务
[root@master231 ingressroutes]# helm -n traefik install traefik-server /zhu/manifests/add-ons/traefik/
NAME: traefik-server
LAST DEPLOYED: Thu Oct 2 20:38:28 2025
NAMESPACE: traefik
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
traefik-server with docker.io/traefik:v3.4.3 has been deployed successfully on traefik namespace !
创建路由规则
[root@master231 ingressroutes]# cat 10-IngressRouteUDP-whoamiudp.yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRouteUDP
metadata:name: ingressrouteudp-whoamiudpnamespace: default
spec:entryPoints:- udpcaseroutes:- services:- name: svc-whoamiudpport: 8080
[root@master231 ingressroutes]#
[root@master231 ingressroutes]# kubectl apply -f 10-IngressRouteUDP-whoamiudp.yaml
ingressrouteudp.traefik.io/ingressrouteudp-whoamiudp created
[root@master231 ingressroutes]#
[root@master231 ingressroutes]# kubectl get ingressrouteudps ingressrouteudp-whoamiudp
NAME AGE
ingressrouteudp-whoamiudp 16s
[root@master231 ingressroutes]#
测试验证
查看whoamiudp的svc的ClusterIP
[root@master231 ingressroutes]# kubectl get svc svc-whoamiudp -o wide
[root@master231 ingressroutes]# kubectl get pods -o wide -l apps=whoamiudp
安装socat测试工具
[root@worker233 ~]# apt -y install socat
访问测试
[root@master231 ingressroutes]# echo "WHO" | socat - udp4-datagram:10.200.81.243:8080
Hostname: deploy-whoamiudp-f7657cc98-dmzfg
IP: 127.0.0.1
IP: 10.100.2.208
[root@master231 ingressroutes]# echo "WHO" | socat - udp4-datagram:10.200.81.243:8080
Hostname: deploy-whoamiudp-f7657cc98-cc8sg
IP: 127.0.0.1
IP: 10.100.1.3
[root@master231 ingressroutes]# echo "https://www.zhubl.xyz" | socat - udp4-datagram:10.200.81.243:8080
Received: https://www.zhubl.xyz
[root@master231 ingressroutes]#
修改svc的端口映射
[root@master231 ingressroutes]# kubectl get svc traefik-server -n traefik -o yaml > /tmp/xixi.yaml
[root@master231 ingressroutes]# vim /tmp/xixi.yaml
apiVersion: v1
kind: Service
metadata:name: traefik-servernamespace: traefik
spec:ports:- name: udpcaseport: 8081protocol: UDPtargetPort: udpcase- name: webnodePort: 30401port: 80protocol: TCPtargetPort: web- name: websecurenodePort: 47626port: 443protocol: TCPtargetPort: websecureselector:app.kubernetes.io/instance: traefik-server-traefikapp.kubernetes.io/name: traefik# type: LoadBalancertype: NodePort
[root@master231 ingressroutes]# kubectl apply -f /tmp/xixi.yaml
Warning: resource services/traefik-server is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
service/traefik-server configured# 将类型改为LoadBalancer即可
[root@master231 ingressroutes]# vim /tmp/xixi.yaml
[root@master231 ingressroutes]# kubectl apply -f /tmp/xixi.yaml
service/traefik-server configured
温馨提示:
此处一定要将svc的类型由LoadBalancer切换为NodePort,因为LoadBalancer不支持多端口暴露不同的协议。
但是改成NodePort再改回LoadBalancer又是可以的,在k8s 1.23.17测试有效。
集群外部测试
[root@harbor250 ~]# echo "WHO" | socat - udp4-datagram:10.0.0.151:8081
Hostname: deploy-whoamiudp-f7657cc98-cc8sg
IP: 127.0.0.1
IP: 10.100.1.3
[root@harbor250 ~]# echo "WHO" | socat - udp4-datagram:10.0.0.151:8081
Hostname: deploy-whoamiudp-f7657cc98-dmzfg
IP: 127.0.0.1
IP: 10.100.2.208
[root@harbor250 ~]# echo "https://www.zhubl.xyz" | socat - udp4-datagram:10.0.0.151:8081
Received: https://www.zhubl.xyz
[root@harbor250 ~]#
🌟Traefik中间件
中间件在Traefik的位置
连接到路由器的中间件是在将请求发送到您的服务之前(或在将服务的答案发送到客户端之前)调整请求的一种手段。
Traefik中有几个可用的中间件,有些可以修改请求、标头,有些负责重定向,有些添加身份验证等等。
使用相同协议的中间件可以组合成链,以适应每种情况。官方支持HTTP和TCP两种中间件。
参考链接: https://doc.traefik.io/traefik/middlewares/overview/
http中间件之ipallowlist中间件实战案例
部署测试服务
[root@master231 ingressroutes]# cat 11-deploy-xiuxian.yaml
apiVersion: apps/v1
kind: Deployment
metadata:name: deploy-xiuxian
spec:replicas: 1selector:matchLabels:apps: xiuxiantemplate:metadata:labels:apps: xiuxianspec:containers:- image: registry.cn-hangzhou.aliyuncs.com/yinzhengjie-k8s/apps:v1name: c1ports:- containerPort: 80---apiVersion: v1
kind: Service
metadata:name: svc-xiuxian
spec:ports:- port: 80selector:apps: xiuxian
[root@master231 ingressroutes]#
测试验证
[root@master231 ingressroutes]# kubectl apply -f 11-deploy-xiuxian.yaml
deployment.apps/deploy-xiuxian created
service/svc-xiuxian created
[root@master231 ingressroutes]# kubectl get pods -o wide -l apps=xiuxian
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
deploy-xiuxian-565ff98ccb-2qf87 1/1 Running 0 9s 10.100.2.210 worker233 <none> <none>
[root@master231 ingressroutes]#
[root@master231 ingressroutes]# kubectl get svc svc-xiuxian
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
svc-xiuxian ClusterIP 10.200.6.136 <none> 80/TCP 17s
[root@master231 ingressroutes]# curl 10.200.6.136
ipWhiteList|ipallowlist应用案例
在实际工作中,有一些URL并不希望对外暴露,比如prometheus、grafana等,我们就可以通过白名单IP来过到要求,可以使用Traefix中的ipWhiteList中间件来完成。
ipWhiteList组件可以限制客户端的IP地址是否可以进行访问,如果在白名单则允许访问,目前官方支持http和tcp两种配置方式。
值得注意的是,ipwhitelist官方已经弃用,推荐使用ipallowlist来替代。
[root@master231 ingressroutes]# cat 12-ipAllowList-IngressRoute.yaml
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:name: xiuxian-ipallowlistnamespace: default
spec:ipAllowList:sourceRange:- 127.0.0.1- 10.0.0.0/24---apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:name: ingressroute-xiuxian
spec:entryPoints:- webroutes:- match: Host(`middleware.zhubaolin.com`) && PathPrefix(`/`)kind: Ruleservices:- name: svc-xiuxianport: 80middlewares:- name: xiuxian-ipallowlistnamespace: default[root@master231 ingressroutes]# kubectl apply -f 12-ipAllowList-IngressRoute.yaml
middleware.traefik.io/xiuxian-ipallowlist created
ingressroute.traefik.io/ingressroute-xiuxian created
[root@master231 ingressroutes]# kubectl get middleware
NAME AGE
xiuxian-ipallowlist 26s
[root@master231 ingressroutes]#
测试访问
echo 10.0.0.151 middleware.zhubaolin.com >> /etc/hosts
tail -1 /etc/hosts
curl middleware.zhubaolin.com
将10.0.0.0/24网段移除
[root@master231 ingressroutes]# cat 12-ipAllowList-IngressRoute.yaml
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:name: xiuxian-ipallowlistnamespace: default
spec:ipAllowList:sourceRange:- 127.0.0.1#- 10.0.0.0/24---apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:name: ingressroute-xiuxian
spec:entryPoints:- webroutes:- match: Host(`middleware.zhubaolin.com`) && PathPrefix(`/`)kind: Ruleservices:- name: svc-xiuxianport: 80middlewares:- name: xiuxian-ipallowlistnamespace: default
测试访问
[root@harbor250 ~]# echo 10.0.0.151 middleware.zhubaolin.com >> /etc/hosts
[root@harbor250 ~]# tail -1 /etc/hosts
[root@harbor250 ~]# curl middleware.zhubaolin.com;echo
Forbidden
[root@harbor250 ~]#
访问不成功,尝试添加这一行参数 externalTrafficPolicy: Local
🌟IngressRoute绑定多中间件
编写资源清单
[root@master231 ingressroutes]# cat 13-basicAuth-secrets-IngressRoute.yaml
apiVersion: v1
kind: Secret
metadata:name: login-infonamespace: default
type: kubernetes.io/basic-auth
stringData:username: adminpassword: admin---apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:name: login-auth
spec:basicAuth:secret: login-info---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:name: xiuxian-ipallowlistnamespace: default
spec:ipAllowList:sourceRange:- 127.0.0.1- 10.0.0.0/24---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:name: ingressroute-xiuxian
spec:entryPoints:- webroutes:- match: Host(`auth.zhubaolin.com`) && PathPrefix(`/`)kind: Ruleservices:- name: svc-xiuxianport: 80middlewares:- name: login-authnamespace: default- name: xiuxian-ipallowlistnamespace: default
[root@master231 ingressroutes]#
创建资源
[root@master231 ingressroutes]# kubectl apply -f 13-basicAuth-secrets-IngressRoute.yaml
secret/login-info created
middleware.traefik.io/login-auth created
middleware.traefik.io/xiuxian-ipallowlist configured
ingressroute.traefik.io/ingressroute-xiuxian configured
[root@master231 ingressroutes]#
直接测试访问
[root@harbor250 ~]# curl auth.zhubaolin.com
401 Unauthorized
[root@harbor250 ~]# curl -u admin:admin auth.zhubaolin.com
<!DOCTYPE html>
<html><head><meta charset="utf-8"/><title>yinzhengjie apps v1</title><style>div img {width: 900px;height: 600px;margin: 0;}</style></head><body><h1 style="color: green">凡人修仙传 v1 </h1><div><img src="1.jpg"><div></body></html>
[root@harbor250 ~]#
温馨提示:
注意哈,一定要将Flannel网络的工作模式设置为: "host-gw"模式,否则上面测试IP黑名单可能存在问题
🌟TraefikService实战案例
测试环境准备
[root@master231 ~]# cat 14-deploy-svc-cm-lb-web.yaml
apiVersion: v1
kind: ConfigMap
metadata:name: cm-lb-web
data:web01: |server {listen 81;listen [::]:81;server_name localhost;location / {root /usr/share/nginx/html;index index.html index.htm;}error_page 500 502 503 504 /50x.html;location = /50x.html {root /usr/share/nginx/html;}}web02: |server {listen 82;listen [::]:82;server_name localhost;location / {root /usr/share/nginx/html;index index.html index.htm;}error_page 500 502 503 504 /50x.html;location = /50x.html {root /usr/share/nginx/html;}}---apiVersion: apps/v1
kind: Deployment
metadata:name: deploy-web01
spec:replicas: 1selector:matchLabels:apps: web01template:metadata:labels:apps: web01spec:volumes:- emptyDir: {}name: data- name: webconfconfigMap:name: cm-lb-webitems:- key: web01path: default.confinitContainers:- name: i1image: registry.cn-hangzhou.aliyuncs.com/yinzhengjie-k8s/apps:v1env:- name: PODNAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: PODNSvalueFrom:fieldRef:fieldPath: metadata.namespace- name: PODIPvalueFrom:fieldRef:fieldPath: status.podIPvolumeMounts:- name: datamountPath: /zhucommand: - /bin/sh- -c- 'echo "<h1> web01 NameSpace: ${PODNS}, PodName: ${PODNAME}, PodIP:${PODIP}</h1>" > /zhu/index.html'containers:- name: c1image: registry.cn-hangzhou.aliyuncs.com/yinzhengjie-k8s/apps:v1volumeMounts:- name: datamountPath: /usr/share/nginx/html- name: webconfmountPath: /etc/nginx/conf.d/default.confsubPath: default.conf
---apiVersion: apps/v1
kind: Deployment
metadata:name: deploy-web02
spec:replicas: 1selector:matchLabels:apps: web02template:metadata:labels:apps: web02spec:volumes:- emptyDir: {}name: data- name: webconfconfigMap:name: cm-lb-webitems:- key: web02path: default.confinitContainers:- name: i1image: registry.cn-hangzhou.aliyuncs.com/yinzhengjie-k8s/apps:v1env:- name: PODNAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: PODNSvalueFrom:fieldRef:fieldPath: metadata.namespace- name: PODIPvalueFrom:fieldRef:fieldPath: status.podIPvolumeMounts:- name: datamountPath: /zhucommand: - /bin/sh- -c- 'echo "<h1> web02 NameSpace: ${PODNS}, PodName: ${PODNAME}, PodIP:${PODIP}</h1>" > /zhu/index.html'containers:- name: c1image: registry.cn-hangzhou.aliyuncs.com/yinzhengjie-k8s/apps:v1volumeMounts:- name: datamountPath: /usr/share/nginx/html- name: webconfmountPath: /etc/nginx/conf.d/default.confsubPath: default.conf---apiVersion: v1
kind: Service
metadata:name: svc-web01
spec:ports:- port: 80targetPort: 81selector:apps: web01type: ClusterIP---apiVersion: v1
kind: Service
metadata:name: svc-web02
spec:ports:- port: 80targetPort: 82selector:apps: web02type: ClusterIP
[root@master231 ~]#
创建资源
[root@master231 ~]# kubectl apply -f 14-deploy-svc-cm-lb-web.yaml
configmap/cm-lb-web created
deployment.apps/deploy-web01 created
deployment.apps/deploy-web02 created
service/svc-web01 created
service/svc-web02 created
[root@master231 ~]#
Traefik高级应用实战案例之负载均衡案例
创建资源
[root@master231 ingressroutes]# cat 15-IngressRoute-lb-xiuxian.yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:name: ingressroute-lbnamespace: default
spec:entryPoints:- webroutes:- match: Host(`lb.zhubaolin.com`) && PathPrefix(`/`)kind: Ruleservices:- name: svc-web01port: 80namespace: default- name: svc-web02port: 80namespace: default
[root@master231 ingressroutes]# kubectl apply -f 15-IngressRoute-lb-xiuxian.yaml
ingressroute.traefik.io/ingressroute-lb created
[root@master231 ingressroutes]#
测试验证
[root@harbor250 ~]# echo 10.0.0.151 lb.zhubaolin.com >> /etc/hosts
[root@harbor250 ~]# for i in `seq 10`; do curl -s lb.zhubaolin.com;done
删除环境
[root@master231 ingressroutes]# kubectl delete -f 15-IngressRoute-lb-xiuxian.yaml
ingressroute.traefik.io "ingressroute-lb" deleted
[root@master231 ingressroutes]#
🌟Traefik高级应用实战案例之灰度发布案例
创建资源
[root@master231 ingressroutes]# cat 16-TraefikService-weighted.yaml
apiVersion: traefik.io/v1alpha1
kind: TraefikService
metadata:name: traefikservices-wrrnamespace: default
spec:# 基于权重调度weighted:services:- name: svc-web01port: 80# 定义调度到该svc的权重weight: 4# 指定类型有效值为: Service(default), TraefikServicekind: Service- name: svc-web02port: 80 weight: 1
[root@master231 ingressroutes]# kubectl apply -f 16-TraefikService-weighted.yaml
traefikservice.traefik.io/traefikservices-wrr created
[root@master231 ingressroutes]#
[root@master231 ingressroutes]# cat 17-IngressRoute-TraefikService.yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:name: ingressroute-lb-wrrnamespace: default
spec:entryPoints:- webroutes:- match: Host(`lb.zhubaolin.com`) && PathPrefix(`/`)kind: Ruleservices:# 指定TraefikService的名称- name: traefikservices-wrrnamespace: default# 注意,类型不再是k8s的Service,而是Traefik自实现的TraefikServicekind: TraefikService
[root@master231 ingressroutes]#
[root@master231 ingressroutes]# kubectl apply -f 17-IngressRoute-TraefikService.yaml
ingressroute.traefik.io/ingressroute-lb-wrr created
[root@master231 ingressroutes]#
测试验证
[root@harbor250 ~]# for i in `seq 10`; do curl -s lb.zhubaolin.com;done
[root@harbor250 ~]# for i in `seq 10`; do curl -s lb.zhubaolin.com; done | sort | uniq -c
删除测试案例
[root@master231 ingressroutes]# kubectl delete -f 16-TraefikService-weighted.yaml -f 17-IngressRoute-TraefikService.yaml
traefikservice.traefik.io "traefikservices-wrr" deleted
ingressroute.traefik.io "ingressroute-lb-wrr" deleted
Traefik高级应用实战案例之流量镜像|影子流量
编写资源清单
[root@master231 ingressroutes]# cat 18-TraefikService-mirroring.yaml
apiVersion: traefik.io/v1alpha1
kind: TraefikService
metadata:name: traefikservices-mirroringnamespace: default
spec:# 发送 100% 的请求到K8S名为"svc-web01"的Service。mirroring:kind: Servicename: svc-web01port: 80# 将其中20%的请求调度到k8s名为"svc-web02"的Service。mirrors:- name: svc-web02port: 80# 是指将20%请求的流量复制一份发送给其它'svc-web02'服务,并且会忽略这部分请求的响应,这个功能在做一些压测或者问题复现的时候很有用。percent: 20---apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:name: ingressroute-mirrornamespace: default
spec:entryPoints:- webroutes:- match: Host(`lb.zhubaolin.com`) && PathPrefix(`/`)kind: Ruleservices:- name: traefikservices-mirroringnamespace: defaultkind: TraefikService
[root@master231 ingressroutes]#
[root@master231 ingressroutes]# kubectl apply -f 18-TraefikService-mirroring.yaml
traefikservice.traefik.io/traefikservices-mirroring created
ingressroute.traefik.io/ingressroute-mirror created
[root@master231 ingressroutes]#
测试验证
[root@harbor250 ~]# for i in `seq 10`; do curl -s lb.zhubaolin.com; done
查看后端日志
[root@master231 ingressroutes]# kubectl get svc traefik-server -n traefik
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik-server LoadBalancer 10.200.91.121 10.0.0.151 8081:28893/UDP,80:30401/TCP,443:47626/TCP 39m
[root@master231 ingressroutes]#