当前位置： 首页 > news >正文

Service详解

news 2025/8/24 14:53:14

1、Service介绍

在kubernetes中，pod是应用程序的载体，我们可以通过pod的ip来访问应用程序，但是pod的ip地址不是固定的，这也就意味着不方便直接采用pod的ip对服务进行访问。为了解决这个问题，kubernetes提供了Service资源，Service会对提供同一个服务的多个pod进行聚合，并且提供一个统一的入口地址。通过访问Service的入口地址就能访问到后面的pod服务。

Service在很多情况下只是一个概念，真正起作用的其实是kube-proxy服务进程，每个Node节点上都运行着一个kube-proxy服务进程。当创建Service的时候会通过api-server向etcd写入创建的service的信息，而kube-proxy会基于监听的机制发现这种Service的变动，然后它会将最新的Service信息转换成对应的访问规则。

# 172.17.0.1:30380 是service提供的访问入口
# 当访问这个入口的时候，可以发现后面有三个pod的服务在等待调用，
# kube-proxy会基于rr（轮询）的策略，将请求分发到其中一个pod上去
# 这个规则会同时在集群内的所有节点上都生成，所以在任何一个节点上访问都可以。
[root@openEuler-1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.17.0.1:30380 rr-> 100.105.212.137:80           Masq    1      0          0-> 100.105.212.138:80           Masq    1      0          0-> 100.115.147.127:80           Masq    1      0          0

kube-proxy目前支持三种工作模式：

 userspace 模式

userspace模式下，kube-proxy会为每一个Service创建一个监听端口，发向Cluster IP的请求被 Iptables规则重定向到kube-proxy监听的端口上，kube-proxy根据LB算法选择一个提供服务的Pod并和其建立链接，以将请求转发到Pod上。该模式下，kube-proxy充当了一个四层负责均衡器的角色。由于kube-proxy运行在userspace中，在进行转发处理时会增加内核和用户空间之间的数据拷贝，虽然比较稳定，但是效率比较低。

iptables 模式

iptables模式下，kube-proxy为service后端的每个Pod创建对应的iptables规则，直接将发向Cluster IP的请求重定向到一个Pod IP。该模式下kube-proxy不承担四层负责均衡器的角色，只负责创建iptables规则。该模式的优点是较userspace模式效率更高，但不能提供灵活的LB策略，当后端Pod不可用时也无法进行重试。

ipvs 模式

ipvs模式和iptables类似，kube-proxy监控Pod的变化并创建相应的ipvs规则。ipvs相对iptables转发效率更高。除此以外，ipvs支持更多的LB算法。 

2、Service类型

Service的资源清单文件： 

kind: Service # 资源类型
apiVersion: v1 # 资源版本
metadata: # 元数据name: service # 资源名称namespace: dev # 命名空间
spec: # 描述selector: # 标签选择器，用于确定当前service代理哪些podapp: nginxtype: # Service类型，指定service的访问方式clusterIP: # 虚拟服务的ip地址sessionAffinity: # session亲和性，支持ClientIP、None两个选项ports: # 端口信息- protocol: TCPport: 3017 # service端口targetPort: 5003 # pod端口nodePort: 31122 # 主机端口
  • ClusterIP：默认值，它是Kubernetes系统自动分配的虚拟IP，只能在集群内部访问
  • NodePort：将Service通过指定的Node上的端口暴露给外部，通过此方法，就可以在集群外部访问服务
  • LoadBalancer：使用外接负载均衡器完成到服务的负载分发，注意此模式需要外部云环境支持
  • ExternalName： 把集群外部的服务引入集群内部，直接使用

3、Service使用

3.1、实验环境准备

在使用service之前，首先利用Deployment创建出3个pod，注意要为pod设置 app=nginx 的标签创建depoly-nginx.yaml，内容如下：

apiVersion: apps/v1
kind: Deployment
metadata:name: nginx-deploynamespace: dev
spec:replicas: 3selector:matchLabels:app: nginxtemplate:metadata:labels:app: nginxspec:containers:- name: nginximage: nginx:1.17.1ports:- containerPort: 80

[root@openEuler-1 ~]# kubectl create -f depoly-nginx.yaml
deployment.apps/nginx-deploy created
[root@openEuler-1 ~]# kubectl get deploy,pod -n dev
NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-deploy   3/3     3            3           2sNAME                                READY   STATUS    RESTARTS   AGE
pod/nginx-deploy-5b979d57d5-6wc74   1/1     Running   0          2s
pod/nginx-deploy-5b979d57d5-8v4bw   1/1     Running   0          2s
pod/nginx-deploy-5b979d57d5-rrgb7   1/1     Running   0          2s# 为了方便后面的测试，修改下三台nginx的index.html页面（三台修改的IP地址不一致）
# kubectl exec -it pc-deployment-66cb59b984-8p84h -n dev --/bin/sh
# echo "hostname -I, web2" > /usr/share/nginx/html/index.html[root@openEuler-1 ~]# kubectl exec -it nginx-deploy-5b979d57d5-6wc74 -n dev -- /bin/bash
root@nginx-deploy-5b979d57d5-6wc74:/# echo "hostname -I, web1" > /usr/share/nginx/html/index.html
root@nginx-deploy-5b979d57d5-6wc74:/# exit
[root@openEuler-1 ~]# kubectl exec -it nginx-deploy-5b979d57d5-8v4bw -n dev -- /bin/bash
root@nginx-deploy-5b979d57d5-8v4bw:/# echo "hostname -I, web2" > /usr/share/nginx/html/index.html
root@nginx-deploy-5b979d57d5-8v4bw:/# exit
[root@openEuler-1 ~]# kubectl exec -it nginx-deploy-5b979d57d5-rrgb7 -n dev -- /bin/bash
root@nginx-deploy-5b979d57d5-rrgb7:/# echo "hostname -I, web3" > /usr/share/nginx/html/index.html
root@nginx-deploy-5b979d57d5-rrgb7:/# exit
[root@openEuler-1 ~]# kubectl get pod -n dev -o wide
NAME                            READY   STATUS    RESTARTS   AGE     IP                NODE          NOMINATED NODE   READINESS GATES
nginx-deploy-5b979d57d5-6wc74   1/1     Running   0          7m34s   100.115.147.71    openeuler-2   <none>           <none>
nginx-deploy-5b979d57d5-8v4bw   1/1     Running   0          7m34s   100.115.147.74    openeuler-2   <none>           <none>
nginx-deploy-5b979d57d5-rrgb7   1/1     Running   0          7m34s   100.105.212.146   openeuler-3   <none>           <none>
[root@openEuler-1 ~]# curl 100.115.147.71
hostname -I, web1
[root@openEuler-1 ~]# curl 100.115.147.74
hostname -I, web2
[root@openEuler-1 ~]# curl 100.105.212.146
hostname -I, web3

3.2、ClusterIP类型的Service

创建一个nginx-svc1.yaml文件，如果初学觉得编辑困难可以使用如下命令：

kubectl expose deployment nginx-deploy --port=80 --target-port=80 --type=ClusterIP --name=nginx-svc1 -n dev -o yaml --dry-run=client > nginx-svc1.yaml

编辑后的文件如下：

apiVersion: v1
kind: Service
metadata:name: nginx-svc1namespace: dev
spec:ports:- port: 80protocol: TCPtargetPort: 80selector:app: nginxtype: ClusterIP

发现每次集群ip都不一样

[root@openEuler-1 ~]# kubectl apply -f nginx-svc1.yaml
service/nginx-svc1 created
[root@openEuler-1 ~]# kubectl get svc -n dev
NAME         TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
nginx-svc1   ClusterIP   10.96.0.121   <none>        80/TCP         27s[root@openEuler-1 ~]# vim nginx-svc1.yaml
[root@openEuler-1 ~]# kubectl delete -f nginx-svc1.yaml
service "nginx-svc1" deleted
[root@openEuler-1 ~]# kubectl apply -f nginx-svc1.yaml
service/nginx-svc1 created
[root@openEuler-1 ~]# kubectl get svc -n dev
NAME         TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
nginx-svc1   ClusterIP   10.96.0.253   <none>        80/TCP         5s

编辑配置文件：

apiVersion: v1
kind: Service
metadata:name: nginx-svc1namespace: dev
spec:ports:- port: 80protocol: TCPtargetPort: 80selector:app: nginxtype: ClusterIPclusterIP: 10.96.1.10

观察效果：

[root@openEuler-1 ~]# curl 10.96.1.10
hostname -I, web2
[root@openEuler-1 ~]# curl 10.96.1.10
hostname -I, web1
[root@openEuler-1 ~]# curl 10.96.1.10
hostname -I, web3

3.3、Endpoint类型的Service

Endpoint是kubernetes中的一个资源对象，存储在etcd中，用来记录一个service对应的所有pod的访问地址，它是根据service配置文件中selector描述产生的。一个Service由一组Pod组成，这些Pod通过Endpoints暴露出来，Endpoints是实现实际服务的端点集合。换句话说，service和pod之间的联系是通过endpoints实现的。

# 查看命令
[root@openEuler-1 ~]# kubectl get endpoints -n dev -o wide
NAME         ENDPOINTS                                                AGE
nginx-svc1   100.105.212.146:80,100.115.147.71:80,100.115.147.74:80   5m48s

负载分发策略

对Service的访问被分发到了后端的Pod上去，目前kubernetes提供了两种负载分发策略：

  • 如果不定义，默认使用kube-proxy的策略，比如随机、轮询
  • 基于客户端地址的会话保持模式，即来自同一个客户端发起的所有请求都会转发到固定的一个Pod上此模式可以使在spec中添加 sessionAffinity:ClientIP 选项

3.4、HeadLiness类型的Service

在某些场景中，开发人员可能不想使用Service提供的负载均衡功能，而希望自己来控制负载均衡策略，针对这种情况，kubernetes提供了HeadLiness Service，这类Service不会分配Cluster IP，如果想要访问service，只能通过service的域名进行查询。

配置nginx-svc2.yaml

apiVersion: v1
kind: Service
metadata:name: nginx-svc2namespace: dev
spec:ports:- port: 80protocol: TCPtargetPort: 80selector:app: nginxtype: ClusterIPclusterIP: None

# 创建
[root@openEuler-1 ~]# kubectl create -f nginx-svc2.yaml
service/nginx-svc2 created# 获取service， 发现CLUSTER-IP未分配
[root@openEuler-1 ~]# kubectl get svc -n dev -o wide
NAME         TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE     SELECTOR
nginx-svc1   ClusterIP   10.96.1.10    <none>        80/TCP         15m     app=nginx
nginx-svc2   ClusterIP   None          <none>        80/TCP         3m14s   app=nginx# 查看service详情
[root@openEuler-1 ~]# kubectl describe svc nginx-svc2 -n dev
Name:              nginx-svc2
Namespace:         dev
Labels:            <none>
Annotations:       <none>
Selector:          app=nginx
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                None
IPs:               None
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         100.105.212.146:80,100.115.147.71:80,100.115.147.74:80
Session Affinity:  None
Events:            <none># 查看域名的解析情况
[root@openEuler-1 ~]# kubectl get pod -n dev
NAME                            READY   STATUS    RESTARTS   AGE
nginx-deploy-5b979d57d5-6wc74   1/1     Running   0          52m
nginx-deploy-5b979d57d5-8v4bw   1/1     Running   0          52m
nginx-deploy-5b979d57d5-rrgb7   1/1     Running   0          52m
[root@openEuler-1 ~]# kubectl exec -it nginx-deploy-5b979d57d5-6wc74 -n dev -- /bin/bash
root@nginx-deploy-5b979d57d5-6wc74:/# cat /etc/resolv.conf
nameserver 10.96.0.10
search dev.svc.cluster.local svc.cluster.local cluster.local
options ndots:5
root@nginx-deploy-5b979d57d5-6wc74:/# exit
[root@openEuler-1 ~]# nslookup nginx-svc2.dev.svc.cluster.local 10.96.0.10
Server:         10.96.0.10
Address:        10.96.0.10#53Name:   nginx-svc2.dev.svc.cluster.local
Address: 100.105.212.146
Name:   nginx-svc2.dev.svc.cluster.local
Address: 100.115.147.71
Name:   nginx-svc2.dev.svc.cluster.local
Address: 100.115.147.74

3.5、NodePort类型的Service

在之前的样例中，创建的Service的ip地址只有集群内部才可以访问，如果希望将Service暴露给集群外部使用，那么就要使用到另外一种类型的Service，称为NodePort类型。NodePort的工作原理其实就是将service的端口映射到Node的一个端口上，然后就可以通过 NodeIp:NodePort 来访问service了。

创建service-nodeport.yaml：

kubectl expose deployment nginx-deploy --name=nginx-svc3 --type=NodePort --port=80 --target-port=80 -n dev -o yaml --dry-run=client > nginx-svc3.yaml
[root@openEuler-1 ~]# kubectl create -f nginx-svc3.yaml
service/nginx-svc3 created# 编辑后
apiVersion: v1
kind: Service
metadata:name: nginx-svc3namespace: dev
spec:ports:- port: 80protocol: TCPtargetPort: 80selector:app: nginxtype: NodePort

# 查看service
[root@openEuler-1 ~]# kubectl get svc -n dev -o wide
NAME         TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE    SELECTOR
nginx-svc1   ClusterIP   10.96.1.10    <none>        80/TCP         33m    app=nginx
nginx-svc2   ClusterIP   None          <none>        80/TCP         21m    app=nginx
nginx-svc3   NodePort    10.96.1.103   <none>        80:30363/TCP   98s    app=nginx

测试：

若是想用自定义的端口，修改配置文件

[root@openEuler-1 ~]# cat nginx-svc3.yaml
apiVersion: v1
kind: Service
metadata:name: nginx-svc3namespace: dev
spec:ports:- port: 80protocol: TCPtargetPort: 80nodePort: 30888selector:app: nginxtype: NodePort

[root@openEuler-1 ~]# kubectl delete -f nginx-svc3.yaml
service "nginx-svc3" deleted
[root@openEuler-1 ~]# kubectl create -f nginx-svc3.yaml
service/nginx-svc3 created

测试：

3.6、LoadBalancer类型的Service

LoadBalancer和NodePort很相似，目的都是向外部暴露一个端口，区别在于LoadBalancer会在集群的外部再来做一个负载均衡设备，而这个设备需要外部环境支持的，外部服务发送到这个设备上的请求，会被设备负载之后转发到集群中。

三款开源 Kubernetes 负载均衡器: MetalLB vs PureLB vs OpenELB

OpenELB的安装与配置

前提：

首先需要为 kube-proxy 启用 strictARP ，以便 Kubernetes 集群中的所有网卡停止响应其他网卡的ARP 请求，而由 OpenELB 处理 ARP 请求。

# 注意：是修改，默认strictARP 是 false
# kubectl edit configmap kube-proxy -n kube-system
......
ipvs:
strictARP: true
......

安装（参考官方文档）：

https://openelb.io/docs/getting-started/installation/install-openelb-on-kubernetes/

下载openelb.yaml文件

apiVersion: v1
kind: Namespace
metadata:name: openelb-system
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:annotations:controller-gen.kubebuilder.io/version: v0.12.1name: bgpconfs.network.kubesphere.io
spec:group: network.kubesphere.ionames:kind: BgpConflistKind: BgpConfListplural: bgpconfssingular: bgpconfscope: Clusterversions:- name: v1alpha1schema:openAPIV3Schema:description: BgpConf is the Schema for the bgpconfs APIproperties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: struct for container bgp:config. Configuration parametersrelating to the global BGP router.properties:as:description: original -> bgp:as bgp:as's original type is inet:as-number.Local autonomous system number of the router.  Uses the 32-bit as-numbertype from the model in RFC 6991.format: int32type: integerport:description: original -> gobgp:portformat: int32maximum: 65535minimum: 1type: integerrouterID:description: original -> bgp:router-id bgp:router-id's original typeis inet:ipv4-address. Router id of the router, expressed as an 32-bitvalue, IPv4 address.pattern: ^([0-9]{1,3}\.){3}[0-9]{1,3}$type: stringrequired:- as- port- routerIDtype: objectstatus:description: BgpConfStatus defines the observed state of BgpConftype: objecttype: objectserved: truestorage: false- name: v1alpha2schema:openAPIV3Schema:description: BgpConf is the Schema for the bgpconfs APIproperties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: Configuration parameters relating to the global BGP router.properties:as:format: int32type: integerasPerRack:additionalProperties:format: int32type: integertype: objectfamilies:items:format: int32type: integertype: arraygracefulRestart:properties:deferralTime:format: int32type: integerenabled:type: booleanhelperOnly:type: booleanlocalRestarting:type: booleanlonglivedEnabled:type: booleanmode:type: stringnotificationEnabled:type: booleanpeerRestartTime:format: int32type: integerpeerRestarting:type: booleanrestartTime:format: int32type: integerstaleRoutesTime:format: int32type: integertype: objectlistenAddresses:items:type: stringtype: arraylistenPort:format: int32type: integerpolicy:type: stringrouterId:type: stringuseMultiplePaths:type: booleantype: objectstatus:description: BgpConfStatus defines the observed state of BgpConfproperties:nodesConfStatus:additionalProperties:properties:as:format: int32type: integerrouterId:type: stringtype: objecttype: objecttype: objecttype: objectserved: truestorage: truesubresources:status: {}
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:annotations:controller-gen.kubebuilder.io/version: v0.12.1name: bgppeers.network.kubesphere.io
spec:group: network.kubesphere.ionames:kind: BgpPeerlistKind: BgpPeerListplural: bgppeerssingular: bgppeerscope: Clusterversions:- name: v1alpha1schema:openAPIV3Schema:description: BgpPeer is the Schema for the bgppeers APIproperties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:properties:addPaths:description: original -> bgp:add-paths Parameters relating to theadvertisement and receipt of multiple paths for a single NLRI (add-paths).properties:sendMax:description: original -> bgp:send-max The maximum number of pathsto advertise to neighbors for a single NLRI.type: integertype: objectconfig:description: original -> bgp:neighbor-address original -> bgp:neighbor-configConfiguration parameters relating to the BGP neighbor or group.properties:neighborAddress:description: original -> bgp:neighbor-address bgp:neighbor-address'soriginal type is inet:ip-address. Address of the BGP peer, eitherin IPv4 or IPv6.pattern: ^([0-9]{1,3}\.){3}[0-9]{1,3}$type: stringpeerAs:description: original -> bgp:peer-as bgp:peer-as's original typeis inet:as-number. AS number of the peer.format: int32type: integerrequired:- neighborAddress- peerAstype: objecttransport:description: original -> bgp:transport Transport session parametersfor the BGP neighbor or group.properties:passiveMode:description: original -> bgp:passive-mode bgp:passive-mode's originaltype is boolean. Wait for peers to issue requests to open aBGP session, rather than initiating sessions from the localrouter.type: booleanremotePort:description: original -> gobgp:remote-port gobgp:remote-port'soriginal type is inet:port-number.maximum: 65535minimum: 1type: integertype: objectusingPortForward:type: booleantype: objectstatus:description: BgpPeerStatus defines the observed state of BgpPeertype: objecttype: objectserved: truestorage: false- name: v1alpha2schema:openAPIV3Schema:description: BgpPeer is the Schema for the bgppeers APIproperties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:properties:afiSafis:items:properties:addPaths:properties:config:properties:receive:type: booleansendMax:format: int32type: integertype: objecttype: objectconfig:properties:enabled:type: booleanfamily:properties:afi:type: stringsafi:type: stringtype: objecttype: objectmpGracefulRestart:properties:config:properties:enabled:type: booleantype: objecttype: objecttype: objecttype: arrayconf:properties:adminDown:type: booleanallowOwnAs:format: int32type: integerauthPassword:type: stringdescription:type: stringlocalAs:format: int32type: integerneighborAddress:type: stringneighborInterface:type: stringpeerAs:format: int32type: integerpeerGroup:type: stringpeerType:format: int32type: integerremovePrivateAs:type: stringreplacePeerAs:type: booleanrouteFlapDamping:type: booleansendCommunity:format: int32type: integervrf:type: stringtype: objectebgpMultihop:properties:enabled:type: booleanmultihopTtl:format: int32type: integertype: objectgracefulRestart:properties:deferralTime:format: int32type: integerenabled:type: booleanhelperOnly:type: booleanlocalRestarting:type: booleanlonglivedEnabled:type: booleanmode:type: stringnotificationEnabled:type: booleanpeerRestartTime:format: int32type: integerpeerRestarting:type: booleanrestartTime:format: int32type: integerstaleRoutesTime:format: int32type: integertype: objectnodeSelector:description: A label selector is a label query over a set of resources.The result of matchLabels and matchExpressions are ANDed. An emptylabel selector matches all objects. A null label selector matchesno objects.properties:matchExpressions:description: matchExpressions is a list of label selector requirements.The requirements are ANDed.items:description: A label selector requirement is a selector thatcontains values, a key, and an operator that relates the keyand values.properties:key:description: key is the label key that the selector appliesto.type: stringoperator:description: operator represents a key's relationship toa set of values. Valid operators are In, NotIn, Existsand DoesNotExist.type: stringvalues:description: values is an array of string values. If theoperator is In or NotIn, the values array must be non-empty.If the operator is Exists or DoesNotExist, the valuesarray must be empty. This array is replaced during a strategicmerge patch.items:type: stringtype: arrayrequired:- key- operatortype: objecttype: arraymatchLabels:additionalProperties:type: stringdescription: matchLabels is a map of {key,value} pairs. A single{key,value} in the matchLabels map is equivalent to an elementof matchExpressions, whose key field is "key", the operatoris "In", and the values array contains only "value". The requirementsare ANDed.type: objecttype: objectx-kubernetes-map-type: atomictimers:properties:config:description: https://stackoverflow.com/questions/21151765/cannot-unmarshal-string-into-go-value-of-type-int64properties:connectRetry:type: stringholdTime:type: stringkeepaliveInterval:type: stringminimumAdvertisementInterval:type: stringtype: objecttype: objecttransport:properties:mtuDiscovery:type: booleanpassiveMode:type: booleanremoteAddress:type: stringremotePort:format: int32type: integertcpMss:format: int32type: integertype: objecttype: objectstatus:description: BgpPeerStatus defines the observed state of BgpPeerproperties:nodesPeerStatus:additionalProperties:properties:peerState:properties:adminState:type: stringauthPassword:type: stringdescription:type: stringflops:format: int32type: integerlocalAs:format: int32type: integermessages:properties:received:properties:discarded:type: stringkeepalive:type: stringnotification:type: stringopen:type: stringrefresh:type: stringtotal:type: stringupdate:type: stringwithdrawPrefix:type: stringwithdrawUpdate:type: stringtype: objectsent:properties:discarded:type: stringkeepalive:type: stringnotification:type: stringopen:type: stringrefresh:type: stringtotal:type: stringupdate:type: stringwithdrawPrefix:type: stringwithdrawUpdate:type: stringtype: objecttype: objectneighborAddress:type: stringoutQ:format: int32type: integerpeerAs:format: int32type: integerpeerGroup:type: stringpeerType:format: int32type: integerqueues:properties:input:format: int32type: integeroutput:format: int32type: integertype: objectremovePrivateAs:format: int32type: integerrouteFlapDamping:type: booleanrouterId:type: stringsendCommunity:format: int32type: integersessionState:type: stringtype: objecttimersState:properties:connectRetry:type: stringdowntime:type: stringholdTime:type: stringkeepaliveInterval:type: stringminimumAdvertisementInterval:type: stringnegotiatedHoldTime:type: stringuptime:type: stringtype: objecttype: objectdescription: 'INSERT ADDITIONAL STATUS FIELD - define observed stateof cluster Important: Run "make" to regenerate code after modifyingthis file'type: objecttype: objecttype: objectserved: truestorage: truesubresources:status: {}
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:annotations:controller-gen.kubebuilder.io/version: v0.12.1name: eips.network.kubesphere.io
spec:group: network.kubesphere.ionames:categories:- networkingkind: EiplistKind: EipListplural: eipssingular: eipscope: Clusterversions:- additionalPrinterColumns:- jsonPath: .spec.addressname: cidrtype: string- jsonPath: .status.usagename: usagetype: integer- jsonPath: .status.poolSizename: totaltype: integername: v1alpha1schema:openAPIV3Schema:description: Eip is the Schema for the eips APIproperties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: EipSpec defines the desired state of EIPproperties:address:pattern: ^([0-9]{1,3}\.){3}[0-9]{1,3}((\/([0-9]|[1-2][0-9]|3[0-2]))|(\-([0-9]{1,3}\.){3}[0-9]{1,3}))?$type: stringdisable:type: booleanprotocol:enum:- bgp- layer2type: stringusingKnownIPs:type: booleanrequired:- addresstype: objectstatus:description: EipStatus defines the observed state of EIPproperties:occupied:type: booleanpoolSize:type: integerusage:type: integertype: objecttype: objectserved: truestorage: falsesubresources: {}- additionalPrinterColumns:- jsonPath: .spec.addressname: cidrtype: string- jsonPath: .status.usagename: usagetype: integer- jsonPath: .status.poolSizename: totaltype: integername: v1alpha2schema:openAPIV3Schema:description: Eip is the Schema for the eips APIproperties:apiVersion:description: 'APIVersion defines the versioned schema of this representationof an object. Servers should convert recognized schemas to the latestinternal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'type: stringkind:description: 'Kind is a string value representing the REST resource thisobject represents. Servers may infer this from the endpoint the clientsubmits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'type: stringmetadata:type: objectspec:description: EipSpec defines the desired state of EIPproperties:address:type: stringdisable:type: booleaninterface:type: stringnamespaceSelector:additionalProperties:type: stringdescription: specify the namespace for allocation by selectortype: objectnamespaces:description: specify the namespace for the allocation by nameitems:type: stringtype: arraypriority:description: priority for automatically assigning addressestype: integerprotocol:enum:- bgp- layer2- viptype: stringusingKnownIPs:type: booleanrequired:- addresstype: objectstatus:description: EipStatus defines the observed state of EIPproperties:firstIP:type: stringlastIP:type: stringoccupied:type: booleanpoolSize:type: integerready:type: booleanusage:type: integerused:additionalProperties:type: stringtype: objectv4:type: booleantype: objecttype: objectserved: truestorage: truesubresources:status: {}
---
apiVersion: v1
kind: ServiceAccount
metadata:name: openelb-admissionnamespace: openelb-system
---
apiVersion: v1
kind: ServiceAccount
metadata:name: openelb-controllernamespace: openelb-system
---
apiVersion: v1
kind: ServiceAccount
metadata:name: openelb-speakernamespace: openelb-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:name: openelb-admissionnamespace: openelb-system
rules:
- apiGroups:- ""resources:- secretsverbs:- get- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:name: openelb-admission
rules:
- apiGroups:- admissionregistration.k8s.ioresources:- validatingwebhookconfigurationsverbs:- get- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:name: openelb-controller
rules:
- apiGroups:- coordination.k8s.ioresources:- leasesverbs:- get- list- watch- create- update- patch- delete
- apiGroups:- coordination.k8s.ioresources:- leases/statusverbs:- get- update- patch
- apiGroups:- ""resources:- configmapsverbs:- get- list- watch
- apiGroups:- appsresources:- daemonsets- deploymentsverbs:- create- delete- get- list- patch- update- watch
- apiGroups:- ""resources:- eventsverbs:- create- patch- update
- apiGroups:- ""resources:- namespaces- nodes- pods- pods/statusverbs:- get- list- watch
- apiGroups:- ""resources:- servicesverbs:- create- delete- get- list- patch- update- watch
- apiGroups:- ""resources:- services/finalizersverbs:- update
- apiGroups:- ""resources:- services/status- nodes/statusverbs:- get- patch- update
- apiGroups:- network.kubesphere.ioresources:- eipsverbs:- create- delete- get- list- patch- update- watch
- apiGroups:- network.kubesphere.ioresources:- eips/statusverbs:- get- patch- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:name: openelb-speaker
rules:
- apiGroups:- ""resources:- configmapsverbs:- create- delete- get- list- patch- update- watch
- apiGroups:- appsresources:- daemonsetsverbs:- create- delete- get- list- patch- update- watch
- apiGroups:- ""resources:- endpointsverbs:- get- list- watch
- apiGroups:- ""resources:- eventsverbs:- create- patch- update
- apiGroups:- ""resources:- nodes- podsverbs:- get- list- watch
- apiGroups:- ""resources:- servicesverbs:- get- list- patch- update- watch
- apiGroups:- network.kubesphere.ioresources:- bgpconfs- bgppeers- eipsverbs:- create- delete- get- list- patch- update- watch
- apiGroups:- network.kubesphere.ioresources:- bgpconfs/finalizersverbs:- update
- apiGroups:- network.kubesphere.ioresources:- bgppeers/status- eips/status- bgpconfs/statusverbs:- get- patch- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:name: openelb-admissionnamespace: openelb-system
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: openelb-admission
subjects:
- kind: ServiceAccountname: openelb-admissionnamespace: openelb-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: openelb-admission
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: openelb-admission
subjects:
- kind: ServiceAccountname: openelb-admissionnamespace: openelb-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: openelb-controller
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: openelb-controller
subjects:
- kind: ServiceAccountname: openelb-controllernamespace: openelb-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:name: openelb-speaker
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: openelb-speaker
subjects:
- kind: ServiceAccountname: openelb-speakernamespace: openelb-system
---
apiVersion: v1
data:key: b3BlbmVsYi1zcGVha2Vycw==
kind: Secret
metadata:name: memberlistnamespace: openelb-system
---
apiVersion: v1
kind: Service
metadata:name: openelb-controllernamespace: openelb-system
spec:ports:- name: https-webhookport: 443targetPort: webhookselector:app: openelbcomponent: controllertype: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:labels:app: openelbcomponent: controllername: openelb-controllernamespace: openelb-system
spec:selector:matchLabels:app: openelbcomponent: controllerstrategy:rollingUpdate:maxUnavailable: 1type: RollingUpdatetemplate:metadata:labels:app: openelbcomponent: controllerspec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: appoperator: Invalues:- openelb-controllertopologyKey: kubernetes.io/hostnamecontainers:- args:- --metrics-addr=:50052- --webhook-port=443- --leader-electcommand:- openelb-controllerenv:- name: OPENELB_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: NODE_NAMEvalueFrom:fieldRef:fieldPath: spec.nodeNameimage: kubesphere/openelb-controller:masterimagePullPolicy: IfNotPresentname: openelb-controllerports:- containerPort: 443name: webhookprotocol: TCPresources:limits:cpu: 100mmemory: 300Mirequests:cpu: 100mmemory: 100MivolumeMounts:- mountPath: /tmp/k8s-webhook-server/serving-certs/name: webhook-certreadOnly: truenodeSelector:kubernetes.io/os: linuxserviceAccountName: openelb-controllerterminationGracePeriodSeconds: 10tolerations:- key: CriticalAddonsOnlyoperator: Exists- effect: NoSchedulekey: node-role.kubernetes.io/master- effect: NoSchedulekey: node-role.kubernetes.io/control-planevolumes:- name: webhook-certsecret:items:- key: keypath: tls.key- key: certpath: tls.crtsecretName: openelb-admission
---
apiVersion: apps/v1
kind: DaemonSet
metadata:labels:app: openelbcomponent: speakername: openelb-speakernamespace: openelb-system
spec:selector:matchLabels:app: openelbcomponent: speakertemplate:metadata:labels:app: openelbcomponent: speakerspec:containers:- args:- --api-hosts=:50051- --enable-keepalived-vip=false- --enable-layer2=falsecommand:- openelb-speakerenv:- name: OPENELB_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: NODE_NAMEvalueFrom:fieldRef:fieldPath: spec.nodeName- name: MEMBER_LIST_SECRETvalueFrom:secretKeyRef:key: keyname: memberlist- name: OPENELB_DSNAMEvalueFrom:fieldRef:fieldPath: metadata.nameimage: kubesphere/openelb-speaker:masterimagePullPolicy: IfNotPresentname: openelb-speakerreadinessProbe:exec:command:- sh- -c- |gobgp -p 50051 globalfailureThreshold: 3periodSeconds: 10successThreshold: 1timeoutSeconds: 1resources:limits:cpu: 100mmemory: 300Mirequests:cpu: 100mmemory: 100MisecurityContext:capabilities:add:- NET_ADMIN- SYS_TIME- NET_RAW- NET_BIND_SERVICEhostNetwork: truenodeSelector:kubernetes.io/os: linuxserviceAccountName: openelb-speakerterminationGracePeriodSeconds: 10tolerations:- effect: NoSchedulekey: node-role.kubernetes.io/masteroperator: Exists- effect: NoSchedulekey: node-role.kubernetes.io/control-planeoperator: Exists
---
apiVersion: batch/v1
kind: Job
metadata:name: openelb-admission-createnamespace: openelb-system
spec:template:metadata:name: openelb-admission-createspec:containers:- args:- create- --host=openelb-controller,openelb-controller.$(POD_NAMESPACE).svc- --namespace=$(POD_NAMESPACE)- --secret-name=openelb-admissionenv:- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace# If you cannot access "registry.k8s.io/ingress-nginx/kube-webhook-certgen", you can replace it with "kubesphere/kube-webhook-certgen"image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1imagePullPolicy: IfNotPresentname: createrestartPolicy: OnFailuresecurityContext:runAsNonRoot: truerunAsUser: 2000serviceAccountName: openelb-admission
---
apiVersion: batch/v1
kind: Job
metadata:name: openelb-admission-patchnamespace: openelb-system
spec:template:metadata:name: openelb-admission-patchspec:containers:- args:- patch- --webhook-name=openelb-admission- --namespace=$(POD_NAMESPACE)- --patch-mutating=false- --secret-name=openelb-admission- --patch-failure-policy=Failenv:- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace# If you cannot access "registry.k8s.io/ingress-nginx/kube-webhook-certgen", you can replace it with "kubesphere/kube-webhook-certgen"image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1imagePullPolicy: IfNotPresentname: patchrestartPolicy: OnFailuresecurityContext:runAsNonRoot: truerunAsUser: 2000serviceAccountName: openelb-admission
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:name: openelb-admission
webhooks:
- admissionReviewVersions:- v1beta1- v1clientConfig:service:name: openelb-controllernamespace: openelb-systempath: /validate-network-kubesphere-io-v1alpha2-eipfailurePolicy: FailmatchPolicy: Equivalentname: validate.eip.network.kubesphere.iorules:- apiGroups:- network.kubesphere.ioapiVersions:- v1alpha2operations:- CREATE- UPDATEresources:- eipssideEffects: None

贴心的主播已经给大家粘贴在这里了

安装
# kubectl apply -f openelb.yaml[root@openEuler-1 ~]# kubectl get pod -n openelb-system
NAME                                  READY   STATUS      RESTARTS   AGE
openelb-admission-create-g8tsz        0/1     Completed   0          70s
openelb-admission-patch-57qjf         0/1     Completed   2          70s
openelb-controller-7f8788f446-gbddm   1/1     Running     0          71s
openelb-speaker-2g25x                 1/1     Running     0          70s
openelb-speaker-fdb9s                 1/1     Running     0          71s
openelb-speaker-glv7r                 1/1     Running     0          70s
添加 EIP 池

EIP 地址要与集群主机节点在同一网段内，且不可绑定任何网卡；

apiVersion: network.kubesphere.io/v1alpha2
kind: Eip
metadata:name: eip-pool
spec:address: 192.168.93.231-192.168.93.238protocol: layer2disable: falseinterface: eth33[root@openEuler-1 ~]# kubectl create -f ip_pool.yaml
eip.network.kubesphere.io/eip-pool created[root@openEuler-1 ~]# kubectl get eip
NAME       CIDR                            USAGE   TOTAL
eip-pool   192.168.93.231-192.168.93.238           8
配置 Service 为 LoadBalancer

把 Service 类型修改为 LoadBalancer，同时 annotations 中添加如下三行：

apiVersion: v1
kind: Service
metadata:name: svc-lbnamespace: devannotations:lb.kubesphere.io/v1alpha1: openelbprotocol.openelb.kubesphere.io/v1alpha1: layer2eip.openelb.kubesphere.io/v1alpha2: eip-pool
spec:selector:app: nginxtype: LoadBalancerports:- port: 80targetPort: 80[root@openEuler-1 ~]# kubectl apply -f svc-lb.yaml
[root@openEuler-1 ~]# kubectl get svc svc-lb -n dev
NAME     TYPE           CLUSTER-IP    EXTERNAL-IP      PORT(S)        AGE
svc-lb   LoadBalancer   10.96.1.124   192.168.93.231   80:32484/TCP   5m23s

测试负载均衡：

[root@openEuler-1 ~]# for i in {1..9}
> do
> curl 192.168.93.231
> done
hostname -I, web2
hostname -I, web1
hostname -I, web3
hostname -I, web2
hostname -I, web1
hostname -I, web3
hostname -I, web2
hostname -I, web1
hostname -I, web3

4、Ingress

4.1、Ingress介绍

在前面已经提到，Service对集群之外暴露服务的主要方式有两种：NotdePort和LoadBalancer，但是这两种方式，都有一定的缺点：

  • NodePort方式的缺点是会占用很多集群机器的端口，那么当集群服务变多的时候，这个缺点就愈发明显
  • LB方式的缺点是每个service需要一个LB，浪费、麻烦，并且需要kubernetes之外设备的支持

基于这种现状，kubernetes提供了Ingress资源对象，Ingress只需要一个NodePort或者一个LB就可以满足暴露多个Service的需求。工作机制大致如下图表示：

实际上，Ingress相当于一个7层的负载均衡器，是kubernetes对反向代理的一个抽象，它的工作原理类似于Nginx，可以理解成在Ingress里建立诸多映射规则，Ingress Controller通过监听这些配置规则并转化成Nginx的反向代理配置 , 然后对外部提供服务。在这里有两个核心概念：

  • ingress：kubernetes中的一个对象，作用是定义请求如何转发到service的规则
  • ingress controller：具体实现反向代理及负载均衡的程序，对ingress定义的规则进行解析，根据配置的规则来实现请求转发，实现方式有很多，比如Nginx, Contour, Haproxy等等

Ingress（以Nginx为例）的工作原理如下：

  • 用户编写Ingress规则，说明哪个域名对应kubernetes集群中的哪个Service
  • Ingress控制器动态感知Ingress服务规则的变化，然后生成一段对应的Nginx反向代理配置
  • Ingress控制器会将生成的Nginx配置写入到一个运行着的Nginx服务中，并动态更新
  • 到此为止，其实真正在工作的就是一个Nginx了，内部配置了用户定义的请求转发规则

4.2、Ingress使用

环境准备（https://github.com/kubernetes/ingress-nginx）

4.2.1、下载ingress部署的yaml文件
wget -c https://gitee.com/kong-xiangyuxcz/svn/releases/download/ingress-nginx-controller-v1.10.1/deploy.yaml
4.2.2、修改镜像地址
sed -i 's#registry.k8s.io/ingress-nginx/controller:v1.10.1.*#registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.10.1#' deploy.yamlsed -i 's#registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.1.*#registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.1#g' deploy.yaml
4.2.3、配置负载均衡需要添加注解
apiVersion: v1
kind: Service
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.4.0name: ingress-nginx-controllernamespace: ingress-nginx#： 添加注解annotations:lb.kubesphere.io/v1alpha1: openelbprotocol.openelb.kubesphere.io/v1alpha1: layer2eip.openelb.kubesphere.io/v1alpha2: eip-pool
---
apiVersion: v1
kind: Service
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-controllernamespace: ingress-nginxannotations:lb.kubesphere.io/v1alpha1: openelbprotocol.openelb.kubesphere.io/v1alpha1: layer2eip.openelb.kubesphere.io/v1alpha2: eip-pool
spec:externalTrafficPolicy: LocalipFamilies:- IPv4ipFamilyPolicy: SingleStackports:- appProtocol: httpname: httpport: 80protocol: TCPtargetPort: http- appProtocol: httpsname: httpsport: 443protocol: TCPtargetPort: httpsselector:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxtype: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-controller-admissionnamespace: ingress-nginx
spec:ports:- appProtocol: httpname: httpport: 80protocol: TCPtargetPort: http- appProtocol: httpsname: https-webhookport: 443targetPort: webhookselector:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxtype: LoadBalancer
---
4.2.4、部署ingress-ngin
[root@openEuler-1 ~]# kubectl apply -f deploy.yaml
namespace/ingress-nginx created[root@openEuler-1 ~]# kubectl get svc,pod -n ingress-nginx
NAME                                         TYPE           CLUSTER-IP    EXTERNAL-IP      PORT(S)                      AGE
service/ingress-nginx-controller             LoadBalancer   10.96.1.116   192.168.93.232   80:32072/TCP,443:32688/TCP   20s
service/ingress-nginx-controller-admission   LoadBalancer   10.96.3.92    <pending>        80:32432/TCP,443:31285/TCP   20sNAME                                            READY   STATUS      RESTARTS   AGE
pod/ingress-nginx-admission-create-5ksrd        0/1     Completed   0          20s
pod/ingress-nginx-admission-patch-gjkkm         0/1     Completed   0          20s
pod/ingress-nginx-controller-67667dc9fc-cpdw7   0/1     Running     0

4.3、Ingress暴露服务的方式

①：Deployment+LoadBalancer模式的Servic

如果要把ingress部署在公有云，那用这种方式比较合适。用Deployment部署ingress-

controller，创建一个type为 LoadBalancer的 service关联这组pod。大部分公有云，都会为

LoadBalancer的 service自动创建一个负载均衡器，通常还绑定了公网地址。只要把域名解析指向该地址，就实现了集群服务的对外暴露

# 创建ingress-1.yaml文件
[root@openEuler-1 ~]# cat ingress-1.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: nginx-ingressnamespace: dev
spec:ingressClassName: nginxrules:- host: www.itopenlab.comhttp:paths:- path: /pathType: Prefixbackend:service:name: svc-lbport:number: 80[root@openEuler-1 ~]# kubectl apply -f ingress-1.yaml
ingress.networking.k8s.io/nginx-ingress created[root@openEuler-1 ~]# kubectl get svc,ing -n dev
NAME                 TYPE           CLUSTER-IP    EXTERNAL-IP      PORT(S)        AGE
service/nginx-svc1   ClusterIP      10.96.1.10    <none>           80/TCP         3h31m
service/nginx-svc2   ClusterIP      None          <none>           80/TCP         3h19m
service/nginx-svc3   NodePort       10.96.3.79    <none>           80:30888/TCP   170m
service/svc-lb       LoadBalancer   10.96.1.124   192.168.93.231   80:32484/TCP   83m
service/svc-nginx    NodePort       10.96.2.29    <none>           80:30380/TCP   2d5h
service/svc-nginx2   NodePort       10.96.1.128   <none>           80:30449/TCP   2d5hNAME                                      CLASS   HOSTS               ADDRESS          PORTS   AGE
ingress.networking.k8s.io/nginx-ingress   nginx   www.itopenlab.com   192.168.93.232   80      2m45s# 在本机上配置hosts解析192.168.93.232 www.itopenlab.com

②：DaemonSet+HostNetwork+nodeselector

用DaemonSet结合nodeselector来部署ingress-controller到特定的node 上，然后使用

HostNetwork直接把该pod与宿主机node的网络打通，直接使用宿主机的80/433端口就能访问服务。这时，ingress-controller所在的node机器就很类似传统架构的边缘节点，比如机房入口的nginx服务器。该方式整个请求链路最简单，性能相对NodePort模式更好。缺点是由于直接利用宿主机节点的网络和端口，一个node只能部署一个ingress-controller pod。比较适合大并发的生产环境使用

配置文件：

apiVersion: v1
kind: Namespace
metadata:labels:app.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxname: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginxnamespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-admissionnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginxnamespace: ingress-nginx
rules:
- apiGroups:- ""resources:- namespacesverbs:- get
- apiGroups:- ""resources:- configmaps- pods- secrets- endpointsverbs:- get- list- watch
- apiGroups:- ""resources:- servicesverbs:- get- list- watch
- apiGroups:- networking.k8s.ioresources:- ingressesverbs:- get- list- watch
- apiGroups:- networking.k8s.ioresources:- ingresses/statusverbs:- update
- apiGroups:- networking.k8s.ioresources:- ingressclassesverbs:- get- list- watch
- apiGroups:- coordination.k8s.ioresourceNames:- ingress-nginx-leaderresources:- leasesverbs:- get- update
- apiGroups:- coordination.k8s.ioresources:- leasesverbs:- create
- apiGroups:- ""resources:- eventsverbs:- create- patch
- apiGroups:- discovery.k8s.ioresources:- endpointslicesverbs:- list- watch- get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-admissionnamespace: ingress-nginx
rules:
- apiGroups:- ""resources:- secretsverbs:- get- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:app.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx
rules:
- apiGroups:- ""resources:- configmaps- endpoints- nodes- pods- secrets- namespacesverbs:- list- watch
- apiGroups:- coordination.k8s.ioresources:- leasesverbs:- list- watch
- apiGroups:- ""resources:- nodesverbs:- get
- apiGroups:- ""resources:- servicesverbs:- get- list- watch
- apiGroups:- networking.k8s.ioresources:- ingressesverbs:- get- list- watch
- apiGroups:- ""resources:- eventsverbs:- create- patch
- apiGroups:- networking.k8s.ioresources:- ingresses/statusverbs:- update
- apiGroups:- networking.k8s.ioresources:- ingressclassesverbs:- get- list- watch
- apiGroups:- discovery.k8s.ioresources:- endpointslicesverbs:- list- watch- get
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-admission
rules:
- apiGroups:- admissionregistration.k8s.ioresources:- validatingwebhookconfigurationsverbs:- get- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginxnamespace: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: ingress-nginx
subjects:
- kind: ServiceAccountname: ingress-nginxnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-admissionnamespace: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: ingress-nginx-admission
subjects:
- kind: ServiceAccountname: ingress-nginx-admissionnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:app.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: ingress-nginx
subjects:
- kind: ServiceAccountname: ingress-nginxnamespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-admission
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: ingress-nginx-admission
subjects:
- kind: ServiceAccountname: ingress-nginx-admissionnamespace: ingress-nginx
---
apiVersion: v1
data:allow-snippet-annotations: "false"
kind: ConfigMap
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-controllernamespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-controllernamespace: ingress-nginxannotations:lb.kubesphere.io/v1alpha1: openelbprotocol.openelb.kubesphere.io/v1alpha1: layer2eip.openelb.kubesphere.io/v1alpha2: eip-pool
spec:externalTrafficPolicy: LocalipFamilies:- IPv4ipFamilyPolicy: SingleStackports:- appProtocol: httpname: httpport: 80protocol: TCPtargetPort: http- appProtocol: httpsname: httpsport: 443protocol: TCPtargetPort: httpsselector:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxtype: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-controller-admissionnamespace: ingress-nginx
spec:ports:- appProtocol: httpname: httpport: 80protocol: TCPtargetPort: http- appProtocol: httpsname: https-webhookport: 443targetPort: webhookselector:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxtype: LoadBalancer
---
apiVersion: apps/v1
kind: DaemonSet
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-controllernamespace: ingress-nginx
spec:minReadySeconds: 0revisionHistoryLimit: 10selector:matchLabels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginx#strategy:#  rollingUpdate:#    maxUnavailable: 1#  type: RollingUpdatetemplate:metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1spec:containers:- args:- /nginx-ingress-controller- --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller- --election-id=ingress-nginx-leader- --controller-class=k8s.io/ingress-nginx- --ingress-class=nginx- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller- --validating-webhook=:8443- --validating-webhook-certificate=/usr/local/certificates/cert- --validating-webhook-key=/usr/local/certificates/key- --enable-metrics=falseenv:- name: POD_NAMEvalueFrom:fieldRef:fieldPath: metadata.name- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespace- name: LD_PRELOADvalue: /usr/local/lib/libmimalloc.soimage: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.10.1imagePullPolicy: IfNotPresentlifecycle:preStop:exec:command:- /wait-shutdownlivenessProbe:failureThreshold: 5httpGet:path: /healthzport: 10254scheme: HTTPinitialDelaySeconds: 10periodSeconds: 10successThreshold: 1timeoutSeconds: 1name: controllerports:- containerPort: 80name: httpprotocol: TCP- containerPort: 443name: httpsprotocol: TCP- containerPort: 8443name: webhookprotocol: TCPreadinessProbe:failureThreshold: 3httpGet:path: /healthzport: 10254scheme: HTTPinitialDelaySeconds: 10periodSeconds: 10successThreshold: 1timeoutSeconds: 1resources:requests:cpu: 100mmemory: 90MisecurityContext:allowPrivilegeEscalation: falsecapabilities:add:- NET_BIND_SERVICEdrop:- ALLreadOnlyRootFilesystem: falserunAsNonRoot: truerunAsUser: 101seccompProfile:type: RuntimeDefaultvolumeMounts:- mountPath: /usr/local/certificates/name: webhook-certreadOnly: truednsPolicy: ClusterFirstnodeSelector:#kubernetes.io/os: linuxingress: "true"hostNetwork: trueserviceAccountName: ingress-nginxterminationGracePeriodSeconds: 300volumes:- name: webhook-certsecret:secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-admission-createnamespace: ingress-nginx
spec:template:metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-admission-createspec:containers:- args:- create- --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc- --namespace=$(POD_NAMESPACE)- --secret-name=ingress-nginx-admissionenv:- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespaceimage: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.1imagePullPolicy: IfNotPresentname: createsecurityContext:allowPrivilegeEscalation: falsecapabilities:drop:- ALLreadOnlyRootFilesystem: truerunAsNonRoot: truerunAsUser: 65532seccompProfile:type: RuntimeDefaultnodeSelector:kubernetes.io/os: linuxrestartPolicy: OnFailureserviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-admission-patchnamespace: ingress-nginx
spec:template:metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-admission-patchspec:containers:- args:- patch- --webhook-name=ingress-nginx-admission- --namespace=$(POD_NAMESPACE)- --patch-mutating=false- --secret-name=ingress-nginx-admission- --patch-failure-policy=Failenv:- name: POD_NAMESPACEvalueFrom:fieldRef:fieldPath: metadata.namespaceimage: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.4.1imagePullPolicy: IfNotPresentname: patchsecurityContext:allowPrivilegeEscalation: falsecapabilities:drop:- ALLreadOnlyRootFilesystem: truerunAsNonRoot: truerunAsUser: 65532seccompProfile:type: RuntimeDefaultnodeSelector:kubernetes.io/os: linuxrestartPolicy: OnFailureserviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:labels:app.kubernetes.io/component: controllerapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: nginx
spec:controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:labels:app.kubernetes.io/component: admission-webhookapp.kubernetes.io/instance: ingress-nginxapp.kubernetes.io/name: ingress-nginxapp.kubernetes.io/part-of: ingress-nginxapp.kubernetes.io/version: 1.10.1name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:- v1clientConfig:service:name: ingress-nginx-controller-admissionnamespace: ingress-nginxpath: /networking/v1/ingressesfailurePolicy: FailmatchPolicy: Equivalentname: validate.nginx.ingress.kubernetes.iorules:- apiGroups:- networking.k8s.ioapiVersions:- v1operations:- CREATE- UPDATEresources:- ingressessideEffects: None

部署：

③：Deployment+NodePort模式的Service

同样用deployment模式部署ingres-controller，并创建对应的服务，但是type为NodePort。这

样，ingress就会暴露在集群节点ip的特定端口上。由于nodeport暴露的端口是随机端口，一般会在前面再搭建一套负载均衡器来转发请求。该方式一般用于宿主机是相对固定的环境ip地址不变的场景。NodePort方式暴露ingress虽然简单方便，但是NodePort多了一层NAT，在请求量级很大时可能对性能会有一定影响

[root@openEuler-1 ~]# cat ingress-2.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: nginx-ingressnamespace: dev
spec:ingressClassName: nginxrules:- host: www.itopenlab.cnhttp:paths:- path: /pathType: Prefixbackend:service:name: nginx-svc2port:number: 80[root@openEuler-1 ~]# kubectl get svc,ing -n dev
NAME                 TYPE           CLUSTER-IP    EXTERNAL-IP      PORT(S)        AGE
service/nginx-svc1   ClusterIP      10.96.1.10    <none>           80/TCP         4h28m
service/nginx-svc2   ClusterIP      None          <none>           80/TCP         4h16m
service/nginx-svc3   NodePort       10.96.3.79    <none>           80:30888/TCP   3h47m
service/svc-lb       LoadBalancer   10.96.1.124   192.168.93.231   80:32484/TCP   140m
service/svc-nginx    NodePort       10.96.2.29    <none>           80:30380/TCP   2d6h
service/svc-nginx2   NodePort       10.96.1.128   <none>           80:30449/TCP   2d6hNAME                                      CLASS   HOSTS              ADDRESS          PORTS   AGE
ingress.networking.k8s.io/nginx-ingress   nginx   www.itopenlab.cn   192.168.93.232   80      5m3s

http://www.dtcms.com/a/347838.html

相关文章：

  • 鸿蒙异步处理从入门到实战：Promise、async/await、并发池、超时重试全套攻略
  • 【C++】全局变量/静态变量的初始化时机
  • 基于电力电子变压器的高压脉冲电源方案复现
  • 最小覆盖子串+滑动窗口
  • 【JVM内存结构系列】二、线程私有区域详解：程序计数器、虚拟机栈、本地方法栈——搞懂栈溢出与线程隔离
  • mysql为什么使用b+树不使用红黑树
  • tcpdump命令打印抓包信息
  • 用vscode使用git工具
  • 深度优先搜索（DFS)和广度优先搜索（BFS）
  • 【内网渗透】Relay2LDAP之NTLMKerberos两种利用
  • windows中bat脚本的一些操作（三）
  • 如和在不同目录之间引用模块-python
  • 微调系列：LoRA原理
  • MVC模式在个人博客系统中的应用
  • 【通俗易懂】TypeScript 增加了 JavaScript 的可选链 (?.) 和空值合并运算符 (??)理解
  • 【集合和映射】USACO Bronze 2019 December - 我在哪Where Am I?
  • 机器学习案例——预测矿物类型（模型训练）
  • DS18B20温度传感器详解
  • 电阻的功率
  • 多光谱相机检测石油石化行业的跑冒滴漏的可行性分析
  • 电蚊拍的原理及电压电容参数深度解析：从高频振荡到倍压整流的完整技术剖析
  • 决策树基础学习教育第二课：量化最优分裂——信息熵与基尼系数
  • 01_Python的in运算符判断列表等是否包含特定元素
  • [Vid-LLM] 创建和训练Vid-LLMs的各种方法体系
  • crypto.randomUUID is not a function
  • 一个备份、去除、新增k8s的node标签脚本
  • Redis(八股二弹)
  • 玳瑁的嵌入式日记D24-0823（数据结构）
  • 每日一题8.23
  • Day26 树的层序遍历 哈希表 排序算法 内核链表