记录日常工作关于系统运维,虚拟化云计算,数据库,网络安全等各方面问题。

windows 2016 Hyper-v虚拟机中安装docker,k8s 1.18.6,重启master节点系统,k8s无法正常启动。

查看docker 容器实例,一直在不停的重启,重建中。

使用vmstate -n 3,与top命令,查看硬盘IO占用很高,很卡顿。


Hyper-v中安装的kubernetes 1.18.6 单master节点集群,docker使用的cgroup-driver是systemd,并且安装了kuboard面板。

master节点系统不重启时,一直正常。但是系统重启后,k8s的master节点无法正常工作。


最后将 docker与kubelet 的cgroup-driver 修改成默认的cgroupfs后,

重启 docker服务与kubelet 服务后,才正常使用。

明白 k8s装到hyper-v虚拟机中,会有问题,建议还是装到物理机系统中比较好,

中间层太多,浪费性能,毛病也会变多。


查看 cat /var/log/messages 里面有大量此类信息。


cb0-42f2-a753-83ef4abc9208)
Aug 12 20:52:08 centosk8s kubelet[1060]: E0812 20:52:08.818796    1060 eviction_manager.go:575] eviction manager: cannot evict a critical pod kube-apiserver-centosk8s_kube-syste
m(20ba893addde533b5afb0c186476bdeb)
Aug 12 20:52:08 centosk8s kubelet[1060]: E0812 20:52:08.818843    1060 eviction_manager.go:575] eviction manager: cannot evict a critical pod etcd-centosk8s_kube-system(825ce2fd
629567db608a423e92c98c86)
Aug 12 20:52:08 centosk8s kubelet[1060]: E0812 20:52:08.818915    1060 eviction_manager.go:575] eviction manager: cannot evict a critical pod kube-controller-manager-centosk8s_k
ube-system(66f148146c60c051d0c133636fade921)
Aug 12 20:52:08 centosk8s kubelet[1060]: E0812 20:52:08.818958    1060 eviction_manager.go:575] eviction manager: cannot evict a critical pod kube-scheduler-centosk8s_kube-syste
m(d6a9a541043821658cd8668c99f9f657)
Aug 12 20:52:08 centosk8s kubelet[1060]: I0812 20:52:08.818985    1060 eviction_manager.go:396] eviction manager: unable to evict any pods from the node
Aug 12 20:52:09 centosk8s kubelet[1060]: W0812 20:52:09.053521    1060 cni.go:331] CNI failed to retrieve network namespace path: cannot find network namespace for the terminate
d container "fbd061ef3bc4d7277304f4a199cf2c794a7fad49307eebbc8ab7ee7b28348514"
Aug 12 20:52:17 centosk8s kubelet[1060]: E0812 20:52:17.844034    1060 secret.go:195] Couldn't get secret kube-system/coredns-token-hq7g4: failed to sync secret cache: timed out
 waiting for the condition
Aug 12 20:52:17 centosk8s kubelet[1060]: E0812 20:52:17.844304    1060 nestedpendingoperations.go:301] Operation for "{volumeName:kubernetes.io/secret/2934206c-4949-4bc9-bb11-e2
4fa9c1a2cb-coredns-token-hq7g4 podName:2934206c-4949-4bc9-bb11-e24fa9c1a2cb nodeName:}" failed. No retries permitted until 2020-08-12 20:54:19.844222271 +0800 CST m=+817.3848007
30 (durationBeforeRetry 2m2s). Error: "MountVolume.SetUp failed for volume \"coredns-token-hq7g4\" (UniqueName: \"kubernetes.io/secret/2934206c-4949-4bc9-bb11-e24fa9c1a2cb-cored
ns-token-hq7g4\") pod \"coredns-66db54ff7f-t92h5\" (UID: \"2934206c-4949-4bc9-bb11-e24fa9c1a2cb\") : failed to sync secret cache: timed out waiting for the condition"
Aug 12 20:52:17 centosk8s kubelet[1060]: E0812 20:52:17.844388    1060 configmap.go:200] Couldn't get configMap kube-system/coredns: failed to sync configmap cache: timed out wa
iting for the condition
Aug 12 20:52:17 centosk8s kubelet[1060]: E0812 20:52:17.844581    1060 nestedpendingoperations.go:301] Operation for "{volumeName:kubernetes.io/configmap/2934206c-4949-4bc9-bb11
-e24fa9c1a2cb-config-volume podName:2934206c-4949-4bc9-bb11-e24fa9c1a2cb nodeName:}" failed. No retries permitted until 2020-08-12 20:54:19.844466172 +0800 CST m=+817.385044631
(durationBeforeRetry 2m2s). Error: "MountVolume.SetUp failed for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2934206c-4949-4bc9-bb11-e24fa9c1a2cb-config-volu
me\") pod \"coredns-66db54ff7f-t92h5\" (UID: \"2934206c-4949-4bc9-bb11-e24fa9c1a2cb\") : failed to sync configmap cache: timed out waiting for the condition"
Aug 12 20:52:17 centosk8s kubelet[1060]: E0812 20:52:17.844706    1060 secret.go:195] Couldn't get secret kube-system/coredns-token-hq7g4: failed to sync secret cache: timed out
 waiting for the condition
Aug 12 20:52:17 centosk8s kubelet[1060]: E0812 20:52:17.844841    1060 nestedpendingoperations.go:301] Operation for "{volumeName:kubernetes.io/secret/dd1a6c17-1bd9-4ee0-94b0-e8
97777ee08b-coredns-token-hq7g4 podName:dd1a6c17-1bd9-4ee0-94b0-e897777ee08b nodeName:}" failed. No retries permitted until 2020-08-12 20:54:19.844780173 +0800 CST m=+817.3853586
32 (durationBeforeRetry 2m2s). Error: "MountVolume.SetUp failed for volume \"coredns-token-hq7g4\" (UniqueName: \"kubernetes.io/secret/dd1a6c17-1bd9-4ee0-94b0-e897777ee08b-cored
ns-token-hq7g4\") pod \"coredns-66db54ff7f-bmx5z\" (UID: \"dd1a6c17-1bd9-4ee0-94b0-e897777ee08b\") : failed to sync secret cache: timed out waiting for the condition"
Aug 12 20:52:17 centosk8s kubelet[1060]: E0812 20:52:17.844902    1060 configmap.go:200] Couldn't get configMap kube-system/coredns: failed to sync configmap cache: timed out wa
iting for the condition
Aug 12 20:52:17 centosk8s kubelet[1060]: E0812 20:52:17.845028    1060 nestedpendingoperations.go:301] Operation for "{volumeName:kubernetes.io/configmap/dd1a6c17-1bd9-4ee0-94b0
-e897777ee08b-config-volume podName:dd1a6c17-1bd9-4ee0-94b0-e897777ee08b nodeName:}" failed. No retries permitted until 2020-08-12 20:54:19.844966773 +0800 CST m=+817.385545332
(durationBeforeRetry 2m2s). Error: "MountVolume.SetUp failed for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/dd1a6c17-1bd9-4ee0-94b0-e897777ee08b-config-volu
me\") pod \"coredns-66db54ff7f-bmx5z\" (UID: \"dd1a6c17-1bd9-4ee0-94b0-e897777ee08b\") : failed to sync configmap cache: timed out waiting for the condition"
Aug 12 20:52:18 centosk8s kubelet[1060]: E0812 20:52:18.051878    1060 secret.go:195] Couldn't get secret kube-system/kube-proxy-token-mjj7b: failed to sync secret cache: timed
out waiting for the condition
Aug 12 20:52:18 centosk8s kubelet[1060]: E0812 20:52:18.052075    1060 nestedpendingoperations.go:301] Operation for "{volumeName:kubernetes.io/secret/61ed4ced-ecb0-42f2-a753-83
ef4abc9208-kube-proxy-token-mjj7b podName:61ed4ced-ecb0-42f2-a753-83ef4abc9208 nodeName:}" failed. No retries permitted until 2020-08-12 20:54:20.051996755 +0800 CST m=+817.5925
75214 (durationBeforeRetry 2m2s). Error: "MountVolume.SetUp failed for volume \"kube-proxy-token-mjj7b\" (UniqueName: \"kubernetes.io/secret/61ed4ced-ecb0-42f2-a753-83ef4abc9208
-kube-proxy-token-mjj7b\") pod \"kube-proxy-wspl4\" (UID: \"61ed4ced-ecb0-42f2-a753-83ef4abc9208\") : failed to sync secret cache: timed out waiting for the condition"
Aug 12 20:52:18 centosk8s kubelet[1060]: E0812 20:52:18.052132    1060 configmap.go:200] Couldn't get configMap kube-system/kube-proxy: failed to sync configmap cache: timed out
 waiting for the condition
Aug 12 20:52:18 centosk8s kubelet[1060]: E0812 20:52:18.052274    1060 nestedpendingoperations.go:301] Operation for "{volumeName:kubernetes.io/configmap/61ed4ced-ecb0-42f2-a753
-83ef4abc9208-kube-proxy podName:61ed4ced-ecb0-42f2-a753-83ef4abc9208 nodeName:}" failed. No retries permitted until 2020-08-12 20:54:20.052221456 +0800 CST m=+817.592799915 (du
rationBeforeRetry 2m2s). Error: "MountVolume.SetUp failed for volume \"kube-proxy\" (UniqueName: \"kubernetes.io/configmap/61ed4ced-ecb0-42f2-a753-83ef4abc9208-kube-proxy\") pod
 \"kube-proxy-wspl4\" (UID: \"61ed4ced-ecb0-42f2-a753-83ef4abc9208\") : failed to sync configmap cache: timed out waiting for the condition"
Aug 12 20:52:18 centosk8s kubelet[1060]: E0812 20:52:18.257924    1060 secret.go:195] Couldn't get secret kube-system/calico-node-token-6827j: failed to sync secret cache: timed
 out waiting for the condition
Aug 12 20:52:18 centosk8s kubelet[1060]: E0812 20:52:18.258226    1060 nestedpendingoperations.go:301] Operation for "{volumeName:kubernetes.io/secret/22306350-4212-4baf-9d7f-56
a4b30b2833-calico-node-token-6827j podName:22306350-4212-4baf-9d7f-56a4b30b2833 nodeName:}" failed. No retries permitted until 2020-08-12 20:54:20.258080835 +0800 CST m=+817.798
659294 (durationBeforeRetry 2m2s). Error: "MountVolume.SetUp failed for volume \"calico-node-token-6827j\" (UniqueName: \"kubernetes.io/secret/22306350-4212-4baf-9d7f-56a4b30b28
33-calico-node-token-6827j\") pod \"calico-node-9t9kv\" (UID: \"22306350-4212-4baf-9d7f-56a4b30b2833\") : failed to sync secret cache: timed out waiting for the condition"
Aug 12 20:52:18 centosk8s kubelet[1060]: E0812 20:52:18.667554    1060 secret.go:195] Couldn't get secret kube-system/calico-kube-controllers-token-bzc4w: failed to sync secret
cache: timed out waiting for the condition
Aug 12 20:52:18 centosk8s kubelet[1060]: E0812 20:52:18.667839    1060 nestedpendingoperations.go:301] Operation for "{volumeName:kubernetes.io/secret/01dc668c-7086-4a5d-9ed6-5a
e11c37e416-calico-kube-controllers-token-bzc4w podName:01dc668c-7086-4a5d-9ed6-5ae11c37e416 nodeName:}" failed. No retries permitted until 2020-08-12 20:54:20.667746386 +0800 CS
T m=+818.208324945 (durationBeforeRetry 2m2s). Error: "MountVolume.SetUp failed for volume \"calico-kube-controllers-token-bzc4w\" (UniqueName: \"kubernetes.io/secret/01dc668c-7
086-4a5d-9ed6-5ae11c37e416-calico-kube-controllers-token-bzc4w\") pod \"calico-kube-controllers-5b8b769fcd-pwtwh\" (UID: \"01dc668c-7086-4a5d-9ed6-5ae11c37e416\") : failed to sy
nc secret cache: timed out waiting for the condition"
Aug 12 20:52:18 centosk8s kubelet[1060]: E0812 20:52:18.874264    1060 secret.go:195] Couldn't get secret kube-system/default-token-jfzkv: failed to sync secret cache: timed out
 waiting for the condition
Aug 12 20:52:18 centosk8s kubelet[1060]: E0812 20:52:18.874522    1060 nestedpendingoperations.go:301] Operation for "{volumeName:kubernetes.io/secret/3e9070f1-17fd-4e7c-be77-85
29162dc0af-default-token-jfzkv podName:3e9070f1-17fd-4e7c-be77-8529162dc0af nodeName:}" failed. No retries permitted until 2020-08-12 20:54:20.874444368 +0800 CST m=+818.4150228
27 (durationBeforeRetry 2m2s). Error: "MountVolume.SetUp failed for volume \"default-token-jfzkv\" (UniqueName: \"kubernetes.io/secret/3e9070f1-17fd-4e7c-be77-8529162dc0af-defau
lt-token-jfzkv\") pod \"kuboard-6bf654bb78-8j46c\" (UID: \"3e9070f1-17fd-4e7c-be77-8529162dc0af\") : failed to sync secret cache: timed out waiting for the condition"
Aug 12 20:52:18 centosk8s kubelet[1060]: W0812 20:52:18.879744    1060 eviction_manager.go:344] eviction manager: attempting to reclaim memory
Aug 12 20:52:18 centosk8s kubelet[1060]: I0812 20:52:18.879798    1060 eviction_manager.go:355] eviction manager: must evict pod(s) to reclaim memory



转载请标明出处【windows 2016 Hyper-v虚拟机中安装k8s 1.18.6,非常 卡】。

《www.micoder.cc》 虚拟化云计算,系统运维,安全技术服务.

网站已经关闭评论