SeaweedFS深度解析(九):k8s环境使用helm部署Seaweedfs集群
上一篇:《SeaweedFS深度解析(八):k8s环境使用Operator部署Seaweedfs集群》
链接: link
#作者:闫乾苓
文章目录
- k8s环境使用helm部署Seaweedfs集群
- 准备镜像
- seaweed-master-localpv-storageclass.yaml
- seaweed-volume-localpv-storageclass.yaml
- seaweed-filer-localpv-storageclass.yaml
- 创建3个组件的storageclass
- master PV的yaml文件master-pv.yaml
- volume PV的yaml文件 volume-pv.yaml
- filer PV的yaml文件 filer-pv.yaml
- 查看创建的pv
- 如果提示有以下错误:
- 如果需要删除,可以使用下面的命令:
k8s环境使用helm部署Seaweedfs集群
~# helm repo add seaweedfs https://seaweedfs.github.io/seaweedfs/helm~# helm search repo seaweedfs --versions
NAME CHART VERSION APP VERSION DESCRIPTION
seaweedfs/seaweedfs 4.0.392 3.92 SeaweedFS
seaweedfs/seaweedfs 4.0.391 3.91 SeaweedFS
seaweedfs/seaweedfs 4.0.390 3.90 SeaweedFS
准备镜像
~# docker pull chrislusf/seaweedfs:3.92
~# docker tag chrislusf/seaweedfs:3.92 192.168.61.3/seaweedfs/seaweedfs:3.92
~# docker push 192.168.61.3/seaweedfs/seaweedfs:3.92
SeaweedFS Helm Chart 的默认参数值完整导出到本地
~# helm show values seaweedfs/seaweedfs --version 4.0.392 > seaweedfs-values-4.0.392.yaml
为了master、volume、filer组件的数据持久化,简单起见使用local-pv方案,为避免各组件间pv绑定错乱,这里使用为不同组件使用不同storageclass名称。
seaweed-master-localpv-storageclass.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:name: seaweed-master-localpv-storageclass
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
seaweed-volume-localpv-storageclass.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:name: seaweed-volume-localpv-storageclass
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
seaweed-filer-localpv-storageclass.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:name: seaweed-filer-localpv-storageclass
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
创建3个组件的storageclass
[root@master storageclass]# kubectl apply -f seaweed-master-localpv-storageclass.yaml -f seaweed-volume-localpv-storageclass.yaml -f seaweed-filer-localpv-storageclass.yaml
[root@master storageclass]# kubectl get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
seaweed-filer-localpv-storageclass kubernetes.io/no-provisioner Delete WaitForFirstConsumer false 10d
seaweed-master-localpv-storageclass kubernetes.io/no-provisioner Delete WaitForFirstConsumer false 10d
seaweed-volume-localpv-storageclass kubernetes.io/no-provisioner Delete WaitForFirstConsumer false 10d
master PV的yaml文件master-pv.yaml
---
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-master-data-0labels:app: seaweedfs-masterrole: data
spec:capacity:storage: 2GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-master-localpv-storageclasslocal:path: /data/seaweedfs/master/data-0nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node1
---
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-master-logs-0labels:app: seaweedfs-masterrole: logs
spec:capacity:storage: 1GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-master-localpv-storageclasslocal:path: /data/seaweedfs/master/logs-0nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node1
---
# master 副本1
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-master-data-1labels:app: seaweedfs-masterrole: data
spec:capacity:storage: 2GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-master-localpv-storageclasslocal:path: /data/seaweedfs/master/data-1nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node2
---
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-master-logs-1labels:app: seaweedfs-masterrole: logs
spec:capacity:storage: 1GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-master-localpv-storageclasslocal:path: /data/seaweedfs/master/logs-1nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node2
---
# master 副本2
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-master-data-2labels:app: seaweedfs-masterrole: data
spec:capacity:storage: 2GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-master-localpv-storageclasslocal:path: /data/seaweedfs/master/data-2nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node3
---
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-master-logs-2labels:app: seaweedfs-masterrole: logs
spec:capacity:storage: 1GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-master-localpv-storageclasslocal:path: /data/seaweedfs/master/logs-2nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node3
volume PV的yaml文件 volume-pv.yaml
---
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-volume-data-0labels:app: seaweedfs-volumerole: data
spec:capacity:storage: 20GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-volume-localpv-storageclasslocal:path: /data/seaweedfs/volume/data-0nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node1
---
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-volume-logs-0labels:app: seaweedfs-volumerole: logs
spec:capacity:storage: 1GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-volume-localpv-storageclasslocal:path: /data/seaweedfs/volume/logs-0nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node1
---
# volume 副本1
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-volume-data-1labels:app: seaweedfs-volumerole: data
spec:capacity:storage: 20GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-volume-localpv-storageclasslocal:path: /data/seaweedfs/volume/data-1nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node2
---
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-volume-logs-1labels:app: seaweedfs-volumerole: logs
spec:capacity:storage: 1GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-volume-localpv-storageclasslocal:path: /data/seaweedfs/volume/logs-1nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node2
---
# volume 副本2
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-volume-data-2labels:app: seaweedfs-volumerole: data
spec:capacity:storage: 20GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-volume-localpv-storageclasslocal:path: /data/seaweedfs/volume/data-2nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node3
---
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-volume-logs-2labels:app: seaweedfs-volumerole: logs
spec:capacity:storage: 1GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-volume-localpv-storageclasslocal:path: /data/seaweedfs/volume/logs-2nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node3
filer PV的yaml文件 filer-pv.yaml
---
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-filer-data-0labels:app: seaweedfs-filerrole: data
spec:capacity:storage: 2GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-filer-localpv-storageclasslocal:path: /data/seaweedfs/filer/data-0nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node1
---
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-filer-logs-0labels:app: seaweedfs-filerrole: logs
spec:capacity:storage: 1GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-filer-localpv-storageclasslocal:path: /data/seaweedfs/filer/logs-0nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node1
---
# filer 副本1
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-filer-data-1labels:app: seaweedfs-filerrole: data
spec:capacity:storage: 2GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-filer-localpv-storageclasslocal:path: /data/seaweedfs/filer/data-1nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node2
---
apiVersion: v1
kind: PersistentVolume
metadata:name: seaweedfs-filer-logs-1labels:app: seaweedfs-filerrole: logs
spec:capacity:storage: 1GivolumeMode: FilesystemaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: seaweed-filer-localpv-storageclasslocal:path: /data/seaweedfs/filer/logs-1nodeAffinity:required:nodeSelectorTerms:- matchExpressions:- key: kubernetes.io/hostnameoperator: Invalues:- node2
因为local-pv的storageclass不支持根据pvc动态创建pv,因此手动创建3个组件的pv
[root@master pv]# kubectl apply -f master-pv.yaml -f volume-pv.yaml -f filer-pv.yaml
查看创建的pv
~# helm install seaweedfs seaweedfs/seaweedfs --version 4.0.392 -f seaweedfs-values.yaml
如果提示有以下错误:
Error: INSTALLATION FAILED: Get "https://seaweedfs.github.io/seaweedfs/helm/seaweedfs-4.0.392.tgz": read tcp 192.168.61.20:60156->185.199.108.153:443: read: connection reset by peer
可以现手动下载Helm Chart包
[root@master ~]# wget https://seaweedfs.github.io/seaweedfs/helm/seaweedfs-4.0.392.tgz
在使用已经下载的包进行安装
[root@master helm_deploy_cluster]# helm install seaweedfs ./seaweedfs-4.0.392.tgz -f seaweedfs-values.yaml
NAME: seaweedfs
LAST DEPLOYED: Tue Jul 15 11:24:18 2025
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
如果修改了values.yaml,可以更新部署
helm upgrade seaweedfs ./seaweedfs-4.0.392.tgz -f seaweedfs-values.yaml
如果需要删除,可以使用下面的命令:
[root@master helm_deploy_cluster]# helm ls --all
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
seaweedfs default 4 2025-07-15 14:24:45.450034677 +0800 CST deployed seaweedfs-4.0.392 3.92
[root@master helm_deploy_cluster]# helm uninstall seaweedfs
release "seaweedfs" uninstalled[root@master helm_deploy_cluster]# kubectl get pod
NAME READY STATUS RESTARTS AGE
seaweedfs-filer-0 0/1 CrashLoopBackOff 3 (9s ago) 6m41s
seaweedfs-filer-1 0/1 CrashLoopBackOff 3 (10s ago) 6m41s
seaweedfs-master-0 0/1 CrashLoopBackOff 3 (13s ago) 6m41s
seaweedfs-master-1 0/1 CrashLoopBackOff 3 (7s ago) 6m41s
seaweedfs-master-2 0/1 CrashLoopBackOff 3 (8s ago) 6m41s
seaweedfs-volume-0 0/1 Running 0 6m41s
seaweedfs-volume-1 0/1 Running 0 6m41s
seaweedfs-volume-2 0/1 Running 0 6m41s
[root@master helm_deploy_cluster]# kubectl logs pods/seaweedfs-master-0
I0704 08:13:39.030513 config.go:53 Reading : Config File "security" Not Found in "[/data /.seaweedfs /usr/local/etc/seaweedfs /etc/seaweedfs]"
flag provided but not defined: -raftElectionTimeout
Example: weed master -port=9333
Default Usage:-cpuprofile stringcpu profile output file-defaultReplication stringDefault replication type if not specified.-disableHttpdisable http requests, only gRPC operations are allowed.-electionTimeout durationelection timeout of master servers (default 10s)