当前位置：首页 > news >正文

kubernetes单机部署踩坑笔记

news 2025/7/12 14:08:29

安装 minikube

# curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
# sudo install minikube-linux-amd64 /usr/local/bin/minikube
# minikube
minikube provisions and manages local Kubernetes clusters optimized for development workflows.Basic Commands:start            Starts a local Kubernetes clusterstatus           Gets the status of a local Kubernetes clusterstop             Stops a running local Kubernetes clusterdelete           Deletes a local Kubernetes clusterdashboard        Access the Kubernetes dashboard running within the minikube clusterpause            pause Kubernetesunpause          unpause KubernetesImages Commands:docker-env       Provides instructions to point your terminal's docker-cli to the Docker Engine inside minikube.
(Useful for building docker images directly inside minikube)podman-env       Configure environment to use minikube's Podman servicecache            Manage cache for imagesimage            Manage imagesConfiguration and Management Commands:addons           Enable or disable a minikube addonconfig           Modify persistent configuration valuesprofile          Get or list the current profiles (clusters)update-context   Update kubeconfig in case of an IP or port changeNetworking and Connectivity Commands:service          Returns a URL to connect to a servicetunnel           Connect to LoadBalancer servicesAdvanced Commands:mount            Mounts the specified directory into minikubessh              Log into the minikube environment (for debugging)kubectl          Run a kubectl binary matching the cluster versionnode             Add, remove, or list additional nodescp               Copy the specified file into minikubeTroubleshooting Commands:ssh-key          Retrieve the ssh identity key path of the specified nodessh-host         Retrieve the ssh host key of the specified nodeip               Retrieves the IP address of the specified nodelogs             Returns logs to debug a local Kubernetes clusterupdate-check     Print current and latest version numberversion          Print the version of minikubeoptions          Show a list of global command-line options (applies to all commands).Other Commands:completion       Generate command completion for a shelllicense          Outputs the licenses of dependencies to a directoryUse "minikube <command> --help" for more information about a given command.

安装 kubernetes on minikube

坑1：本来是打算尝试kubeflow，但kubernetes 版本还要考虑 kubeflow 版本匹配，不能随便装，比如k3s 1.23.官网没推荐：

https://www.kubeflow.org/docs/releases/kubeflow-1.10/: https://www.kubeflow.org/docs/releases/kubeflow-1.10/

# docker pull kicbase/stable:v0.0.47
# docker images
REPOSITORY                                 TAG                  IMAGE ID       CREATED         SIZE
kicbase/stable                             v0.0.47              795ea6a69ce6   5 weeks ago     1.31GB# minikube start --base-image=kicbase/stable:v0.0.47 --kubernetes-version=v1.23.0 --image-mirror-country='cn' --driver=docker --force
😄  minikube v1.36.0 on Openeuler 20.03
❗  minikube skips various validations when --force is supplied; this may lead to unexpected behavior
✨  Using the docker driver based on user configuration
🛑  The "docker" driver should not be used with root privileges. If you wish to continue as root, use --force.
💡  If you are running minikube within a VM, consider using --driver=none:
📘    https://minikube.sigs.k8s.io/docs/reference/drivers/none/
✅  Using image repository registry.cn-hangzhou.aliyuncs.com/google_containers
📌  Using Docker driver with root privileges
👍  Starting "minikube" primary control-plane node in "minikube" cluster
🚜  Pulling base image v0.0.47 ...
🔥  Creating docker container (CPUs=2, Memory=7900MB) ...
🐳  Preparing Kubernetes v1.23.0 on Docker 28.1.1 ...▪ Generating certificates and keys ...▪ Booting up control plane ...▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/storage-provisioner:v5
🌟  Enabled addons: default-storageclass, storage-provisioner
💡  kubectl not found. If you need it, try: 'minikube kubectl -- get pods -A'
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default# docker ps
CONTAINER ID   IMAGE                                 COMMAND                  CREATED         STATUS                  PORTS                                                                                                                                                           NAMES
1a06ba1fabad   kicbase/stable:v0.0.47                "/usr/local/bin/entr…"   7 minutes ago   Up 7 minutes            127.0.0.1:32782->22/tcp, 127.0.0.1:32781->2376/tcp, 127.0.0.1:32780->5000/tcp, 127.0.0.1:32779->8443/tcp, 127.0.0.1:32778->32443/tcp                            minikube# alias kubectl='minikube kubectl -- '
# kubectl get all
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   9m24s# kubectl get po -A
NAMESPACE     NAME                               READY   STATUS    RESTARTS        AGE
kube-system   coredns-65c54cc984-m5mqj           1/1     Running   0               10m
kube-system   etcd-minikube                      1/1     Running   0               10m
kube-system   kube-apiserver-minikube            1/1     Running   0               10m
kube-system   kube-controller-manager-minikube   1/1     Running   0               10m
kube-system   kube-proxy-kdm4h                   1/1     Running   0               10m
kube-system   kube-scheduler-minikube            1/1     Running   0               10m
kube-system   storage-provisioner                1/1     Running   1 (9m39s ago)   10m

minikube dashboard 打开失败

原因：拉镜像失败，把镜像源头改成：registry.cn-hangzhou.aliyuncs.com/google_containers/...`

# minikube dashboard
🔌  Enabling dashboard ...▪ Using image docker.io/kubernetesui/dashboard:v2.7.0▪ Using image docker.io/kubernetesui/metrics-scraper:v1.0.8
💡  Some dashboard features require the metrics-server addon. To enable all features please run:minikube addons enable metrics-server🤔  Verifying dashboard health ...
🚀  Launching proxy ...
🤔  Verifying proxy health ...❌  Exiting due to SVC_URL_TIMEOUT: http://127.0.0.1:41369/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ is not accessible: Temporary Error: unexpected response code: 503# minikube addons enable metrics-server
💡  metrics-server is an addon maintained by Kubernetes. For any concerns contact minikube on GitHub.
You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server:v0.7.2
🌟  The 'metrics-server' addon is enabled

查看pods状态，发现都是image下载的问题

# kubectl get pods -A
NAMESPACE              NAME                                         READY   STATUS             RESTARTS       AGE
kube-system            coredns-65c54cc984-m5mqj                     1/1     Running            0              138m
kube-system            etcd-minikube                                1/1     Running            0              139m
kube-system            kube-apiserver-minikube                      1/1     Running            0              139m
kube-system            kube-controller-manager-minikube             1/1     Running            0              139m
kube-system            kube-proxy-kdm4h                             1/1     Running            0              138m
kube-system            kube-scheduler-minikube                      1/1     Running            0              139m
kube-system            metrics-server-97757d947-687sv               0/1     ErrImagePull       0              6m24s
kube-system            storage-provisioner                          1/1     Running            1 (138m ago)   138m
kubernetes-dashboard   dashboard-metrics-scraper-7db978b848-jdz9r   0/1     ImagePullBackOff   0              24m
kubernetes-dashboard   kubernetes-dashboard-6f4c897964-cknwz        0/1     ImagePullBackOff   0              24m# kubectl get pods --all-namespaces | grep dashboard
kubernetes-dashboard   dashboard-metrics-scraper-7db978b848-jdz9r   0/1     ImagePullBackOff   0              21m
kubernetes-dashboard   kubernetes-dashboard-6f4c897964-cknwz        0/1     ImagePullBackOff   0              21m

describe 详细信息，从events发现错误详情

# kubectl describe pod dashboard-metrics-scraper-7db978b848-jdz9r -n kubernetes-dashboard
Name:         dashboard-metrics-scraper-7db978b848-jdz9r
Namespace:    kubernetes-dashboard
Priority:     0
Node:         minikube/192.168.49.2
Start Time:   Wed, 02 Jul 2025 13:18:26 +0800
Labels:       k8s-app=dashboard-metrics-scraperpod-template-hash=7db978b848
Annotations:  seccomp.security.alpha.kubernetes.io/pod: runtime/default
Status:       Pending
IP:           172.17.0.4
IPs:IP:           172.17.0.4
Controlled By:  ReplicaSet/dashboard-metrics-scraper-7db978b848
Containers:dashboard-metrics-scraper:Container ID:   Image:          docker.io/kubernetesui/metrics-scraper:v1.0.8@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765cImage ID:       Port:           8000/TCPHost Port:      0/TCPState:          WaitingReason:       ImagePullBackOffReady:          FalseRestart Count:  0Liveness:       http-get http://:8000/ delay=30s timeout=30s period=10s #success=1 #failure=3Environment:    <none>Mounts:/tmp from tmp-volume (rw)/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-n28kh (ro)
Conditions:Type              StatusInitialized       True Ready             False ContainersReady   False PodScheduled      True 
Volumes:tmp-volume:Type:       EmptyDir (a temporary directory that shares a pod''s lifetime)Medium:     SizeLimit:  <unset>kube-api-access-n28kh:Type:                    Projected (a volume that contains injected data from multiple sources)TokenExpirationSeconds:  3607ConfigMapName:           kube-root-ca.crtConfigMapOptional:       <nil>DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              kubernetes.io/os=linux
Tolerations:                 node-role.kubernetes.io/master:NoSchedulenode.kubernetes.io/not-ready:NoExecute op=Exists for 300snode.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:Type     Reason     Age                 From               Message----     ------     ----                ----               -------Normal   Scheduled  25m                 default-scheduler  Successfully assigned kubernetes-dashboard/dashboard-metrics-scraper-7db978b848-jdz9r to minikubeWarning  Failed     23m (x2 over 24m)   kubelet            Failed to pull image "docker.io/kubernetesui/metrics-scraper:v1.0.8@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c": rpc error: code = Unknown desc = Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)Normal   Pulling    23m (x4 over 25m)   kubelet            Pulling image "docker.io/kubernetesui/metrics-scraper:v1.0.8@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c"Warning  Failed     22m (x4 over 24m)   kubelet            Error: ErrImagePullWarning  Failed     22m (x2 over 24m)   kubelet            Failed to pull image "docker.io/kubernetesui/metrics-scraper:v1.0.8@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c": rpc error: code = Unknown desc = Error response from daemon: Get "https://registry-1.docker.io/v2/": context deadline exceededWarning  Failed     22m (x6 over 24m)   kubelet            Error: ImagePullBackOffNormal   BackOff    13s (x95 over 24m)  kubelet            Back-off pulling image "docker.io/kubernetesui/metrics-scraper:v1.0.8@sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c"

修改deployment中的镜像仓库，选择国内镜像源

# kubectl -n kubernetes-dashboard edit deployment dashboard-metrics-scraperspec:containers:- image: registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-scraper:v1.0.8imagePullPolicy: IfNotPresent# kubectl get pods --all-namespaces | grep dashboard
kubernetes-dashboard   dashboard-metrics-scraper-7fc7cd8b6-gd7kd   1/1     Running            0              114s
kubernetes-dashboard   kubernetes-dashboard-6f4c897964-cknwz       0/1     ImagePullBackOff   0              38m# kubectl -n kubernetes-dashboard edit deployment kubernetes-dashboardspec:containers:- args:- --namespace=kubernetes-dashboard- --enable-skip-login- --disable-settings-authorizerimage: registry.cn-hangzhou.aliyuncs.com/google_containers/dashboard:v2.7.0imagePullPolicy: IfNotPresent:# kubectl get pods --all-namespaces | grep dashboard
kubernetes-dashboard   dashboard-metrics-scraper-7fc7cd8b6-gd7kd   1/1     Running             0              5m10s
kubernetes-dashboard   kubernetes-dashboard-6cfc6d649d-c59gx       0/1     ContainerCreating   0              35s
kubernetes-dashboard   kubernetes-dashboard-6f4c897964-cknwz       0/1     ImagePullBackOff    0              42m# kubectl get pods --all-namespaces | grep dashboard
kubernetes-dashboard   dashboard-metrics-scraper-7fc7cd8b6-gd7kd   1/1     Running            0              5m12s
kubernetes-dashboard   kubernetes-dashboard-6cfc6d649d-c59gx       1/1     Running            0              37s
kubernetes-dashboard   kubernetes-dashboard-6f4c897964-cknwz       0/1     Terminating        0              42m# kubectl get pods --all-namespaces | grep dashboard
kubernetes-dashboard   dashboard-metrics-scraper-7fc7cd8b6-gd7kd   1/1     Running            0              5m14s
kubernetes-dashboard   kubernetes-dashboard-6cfc6d649d-c59gx       1/1     Running            0              39s

minikube dashboard 开放外部访问

minikube实际上是docker容器，所以即使k8s服务开启了nodeport，也只是开放了minikube容器的端口，宿主机外部是无法访问的。
为了让宿主机外可以访问minikube上的k8s服务，一种方式是在宿主机上使用kubectl proxy开辟临时隧道，比较简单粗暴；另一种在docker网络中，用nginx搭建一个宿主机端口->nginx端口->minikube nodeport的转发链路，如下图：

# kubectl -n kubernetes-dashboard get svc
NAME                        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
dashboard-metrics-scraper   ClusterIP   10.98.239.69   <none>        8000/TCP   49m
kubernetes-dashboard        ClusterIP   10.98.139.28   <none>        80/TCP     49m# curl localhost:80
curl: (7) Failed to connect to localhost port 8080: Connection refused# minikube ssh
docker@minikube:~$ curl http://10.98.139.28:80/
<!--
Copyright 2017 The Kubernetes Authors.Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
--><!DOCTYPE html><html lang="en" dir="ltr"><head><meta charset="utf-8"><title>Kubernetes Dashboard</title><link rel="icon" type="image/png" href="assets/images/kubernetes-logo.png"><meta name="viewport" content="width=device-width">
<style>html,body{height:100%;margin:0}*::-webkit-scrollbar{background:transparent;height:8px;width:8px}</style><link rel="stylesheet" href="styles.243e6d874431c8e8.css" media="print" onload="this.media='all'"><noscript><link rel="stylesheet" href="styles.243e6d874431c8e8.css"></noscript></head><body><kd-root></kd-root>
<script src="runtime.134ad7745384bed8.js" type="module"></script><script src="polyfills.5c84b93f78682d4f.js" type="module"></script><script src="scripts.2c4f58d7c579cacb.js" defer></script><script src="en.main.3550e3edca7d0ed8.js" type="module"></script></body></html>

# kubectl -n kubernetes-dashboard port-forward svc/kubernetes-dashboard 8080:80
Forwarding from 127.0.0.1:8080 -> 9090
Forwarding from [::1]:8080 -> 9090
^C中断# kubectl -n kubernetes-dashboard port-forward --address 0.0.0.0 svc/kubernetes-dashboard 8080:80
Forwarding from 0.0.0.0:8080 -> 9090
^C中断# curl localhost:8080/
<!--
Copyright 2017 The Kubernetes Authors.Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
--><!DOCTYPE html><html lang="en" dir="ltr"><head><meta charset="utf-8"><title>Kubernetes Dashboard</title><link rel="icon" type="image/png" href="assets/images/kubernetes-logo.png"><meta name="viewport" content="width=device-width">
<style>html,body{height:100%;margin:0}*::-webkit-scrollbar{background:transparent;height:8px;width:8px}</style><link rel="stylesheet" href="styles.243e6d874431c8e8.css" media="print" onload="this.media='all'"><noscript><link rel="stylesheet" href="styles.243e6d874431c8e8.css"></noscript></head><body><kd-root></kd-root>
<script src="runtime.134ad7745384bed8.js" type="module"></script><script src="polyfills.5c84b93f78682d4f.js" type="module"></script><script src="scripts.2c4f58d7c579cacb.js" defer></script><script src="en.main.3550e3edca7d0ed8.js" type="module"></script></body></html>

docker安装kuboard

鉴于dashboard不好用，直接上kuboard。
kuboard可以部署在k8s集群外，只要网络联通就行，用来管理minikube非常合适，因为只需要把kuboard和minikube容器放在同一个docker网络即可！

  kuboard:image: eipwork/kuboard:v3container_name: kuboardrestart: alwaysvolumes:- /data/docker_data/kuboard:/dataports:- "30080:80"- "10081:10081"environment:- KUBOARD_AGENT_SERVER_TCP_PORT=10081- KUBOARD_ROOT_USER=admin- KUBOARD_SERVER_HTTP_PORT=80- KUBOARD_SERVER_HTTPS_ENABLE=falsenetworks:- mynet- minikube

坑：开放端口10080被浏览器拦截(ERR_UNSAFE_PORT)，改用30080