Kubernetes 证书监控--x509-certificate-exporter
监控手段
该 Exporter 是通过监控集群所有 node 的指定目录或 path 下的证书文件以及 kubeconfig 文件来获取证书信息.
如果是使用 kubeadm 搭建的 Kubernetes 集群, 则可以监控如下包含证书的文件和 kubeconfig:
watchFiles:
- /var/lib/kubelet/pki/kubelet-client-current.pem
- /etc/kubernetes/pki/apiserver.crt
- /etc/kubernetes/pki/apiserver-etcd-client.crt
- /etc/kubernetes/pki/apiserver-kubelet-client.crt
- /etc/kubernetes/pki/ca.crt
- /etc/kubernetes/pki/front-proxy-ca.crt
- /etc/kubernetes/pki/front-proxy-client.crt
- /etc/kubernetes/pki/etcd/ca.crt
- /etc/kubernetes/pki/etcd/healthcheck-client.crt
- /etc/kubernetes/pki/etcd/peer.crt
- /etc/kubernetes/pki/etcd/server.crt
watchKubeconfFiles:
- /etc/kubernetes/admin.conf
- /etc/kubernetes/controller-manager.conf
- /etc/kubernetes/scheduler.conf
YAML下载并解压
https://github.com/enix/x509-certificate-exporter/tree/main
根据实际情况修改values.yaml， Chart.yaml 其他配置可不做修改
修改 Chart.yaml 里面版本号信息
[root@k8s-uat-m01 x509-certificate-exporter]# cat Chart.yaml
version: '3.19.1'
appVersion: '3.19.1'
........................................修改values.yaml地址
# 修改为镜像名称，默认是Chart中的appVersion
# 修改仓库名称image:# -- x509-certificate-exporter image registryregistry: swr.cn-north-4.myhuaweicloud.com# -- x509-certificate-exporter image repositoryrepository: ddn-k8s/docker.io/enix/x509-certificate-exporter
--------------------------------------------------------------------------# 节点标签，根据实际情况调整，基本也不用修改
# 容忍，根据实际情况调整，基本也不用修改
# 证书所在目录，根据实际情况调整# -- Additional environment variables for containerenv: []# - name: GOMAXPROCS# value: "1"# -- [SEE README] Map to define one or many DaemonSets running hostPath exporters. Key is used as a name ; value is a map to override all default settings set by `hostPathsExporter.*`.daemonSets: master:nodeSelector:node-role.kubernetes.io/control-plane: ""tolerations:- effect: NoSchedulekey: node-role.kubernetes.io/control-planeoperator: ExistswatchFiles:- /etc/kubernetes/pki/apiserver.crt- /etc/kubernetes/pki/apiserver-etcd-client.crt- /etc/kubernetes/pki/apiserver-kubelet-client.crt- /etc/kubernetes/pki/ca.crt- /etc/kubernetes/pki/front-proxy-ca.crt- /etc/kubernetes/pki/front-proxy-client.crt# 配置文件所在目录，根据实际情况调整，也可不做配置watchKubeconfFiles:- /etc/kubernetes/admin.conf- /etc/kubernetes/controller-manager.conf- /etc/kubernetes/kubelet.conf- /etc/kubernetes/scheduler.conf[root@k8s-uat-m01 x509-certificate-exporter]# helm install x509-certificate-exporter --values values.yaml .[root@k8s-uat-m01 x509-certificate-exporter]# kubectl get pod
NAME READY STATUS RESTARTS AGE
x509-certificate-exporter-56567b56b9-xdmrf 1/1 Running 0 7h50m
x509-certificate-exporter-master-5tspp 1/1 Running 0 7h50m
x509-certificate-exporter-master-6ffhr 1/1 Running 0 7h50m
x509-certificate-exporter-master-6q94j 1/1 Running 0 7h50m[root@k8s-uat-m01 x509-certificate-exporter]# kubectl get ds
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
x509-certificate-exporter-master 3 3 3 3 3 node-role.kubernetes.io/control-plane= 7h52m
[root@k8s-uat-m01 x509-certificate-exporter]# kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
x509-certificate-exporter 1/1 1 1 7h52m[root@k8s-uat-m01 x509-certificate-exporter]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
x509-certificate-exporter ClusterIP 10.202.156.255 <none> 9793/TCP 8h
[root@k8s-uat-m01 x509-certificate-exporter]# curl 10.202.156.255:9793/metrics
...............................................................................异常处理
# 检查pod是否都运行kubectl get po -n ssl-monitor -owide
# 若运行异常，查看日志，一般都为某证书没读取权限或证书不存在kubectl logs -n ssl-monitor 【pod name】
# 若证书无读取权限，则前往该节点赋权chmod +r 【证书路径】