Kubernetes 卷“的MountVolume.SetUp失败;“策略适配器机密”:不能';t传播对象缓存:等待条件时超时
使用kubeadm创建了一个两节点集群 已安装的istio 1.1.11 kubectl版本Kubernetes 卷“的MountVolume.SetUp失败;“策略适配器机密”:不能';t传播对象缓存:等待条件时超时,kubernetes,kubectl,istio,Kubernetes,Kubectl,Istio,使用kubeadm创建了一个两节点集群 已安装的istio 1.1.11 kubectl版本 Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:40:16Z", GoVersion:"go1.12.5
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:40:16Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:32:14Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
执行istio文档中给出的命令
$用于安装/kubernetes/helm/istio init/files/crd*yaml中的i;kubectl是否适用-f$i;完成
$kubectl apply-f install/kubernetes/istio demo.yaml
创建了服务
$kubectl获取吊舱-istio系统
遥测和策略pod状态变为CrashLoopBackOff状态
istio-policy-648b5f5bb5-dv5np 1/2 **CrashLoopBackOff** 5 2m52s
istio-telemetry-57946b8569-9m7gd 1/2 **CrashLoopBackOff** 5 2m52s
描述pod时,出现以下错误
Warning FailedMount 2m16s (x2 over 2m18s) kubelet, ip-xxx-xxx-xxx-xxx MountVolume.SetUp failed for volume "policy-adapter-secret" : couldn't propagate object cache: timed out waiting for the condition
已尝试重新启动VM,已重新启动docker服务。这没有帮助
由于上述错误,pod反复尝试重新启动,然后崩溃
需要您帮助解决此问题如果您的节点没有足够的内存运行Istio,这些混音器服务可能会崩溃。越来越多的人使用诸如安装Istio(和其他服务网格)之类的工具,因为它会突出显示内存之类的争用点。在部署
istio-demo
或istio-demo-auth
配置文件时,您需要确保每个节点都有一个配置文件(特别是,如果istio控制平面仅部署到一个节点).在网络中,您可以发现许多与无法传播对象缓存相关的问题:等待条件时超时。
Github上已经有公开发行的版本-
作为解决此问题的众多步骤之一,请尝试:
- 重新启动群集
- 重新启动docker deamon
- 更新你的操作系统
在与ISTIO有关的案例中,我在Kubeadm、Minikube和GKE上试用过。
在所有情况下,istio policy XXX XXX XXX
和istio遥测XXX XXX
由于活动性问题失败而重新启动
telemetry example
---
Warning Unhealthy 8m49s (x9 over 9m29s) kubelet, gke-istio-default-pool-c41459f8-zbhn Liveness probe failed: Get http://10.56.0.6:15014/version: dial tcp 10.56.0.6:15014: connect: connection refused
Normal Killing 8m49s (x3 over 9m19s) kubelet, gke-istio-default-pool-c41459f8-zbhn Killing container with id docker://mixer:Container failed liveness probe.. Container will be killed and recreated.
policy example
---
Warning Unhealthy 7m28s (x9 over 8m8s) kubelet, gke-istio-default-pool-c41459f8-3c6d Liveness probe failed: Get http://10.56.2.6:15014/version: dial tcp 10.56.2.6:15014: connect: connection refused
Normal Killing 7m28s (x3 over 7m58s) kubelet, gke-istio-default-pool-c41459f8-3c6d Killing container with id docker://mixer:Container failed liveness probe.. Container will be killed and recreated.
即使在文档示例中,您也可以看到遥测和策略被重新启动了2次
在验证了两个yaml(istio-demo.yaml和istio-demo auth.yaml)后,我发现遥测和策略部署
已将liveness probe设置为5秒
livenessProbe:
httpGet:
path: /version
port: 15014
initialDelaySeconds: 5
periodSeconds: 5
如果您将从istio telemetry
pod在混合器容器上使用kubectl日志,您可能会看到一些错误,如
2019-07-18T15:16:01.887334Z info pickfirstBalancer: HandleSubConnStateChange: 0xc420751a80, CONNECTING
...
2019-07-18T15:16:21.887741Z info pickfirstBalancer: HandleSubConnStateChange: 0xc420751a80, TRANSIENT_FAILURE
2019-07-18T15:16:21.887862Z error mcp Failed to create a new MCP sink stream: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: Error while dialing dial tcp 10.0.25.27:9901: i/o timeout"
...
2019-07-18T15:16:44.282027Z info pickfirstBalancer: HandleSubConnStateChange: 0xc420751a80, CONNECTING
2019-07-18T15:16:44.287281Z info pickfirstBalancer: HandleSubConnStateChange: 0xc420751a80, READY
2019-07-18T15:16:44.888794Z info mcp (re)trying to establish new MCP sink stream
2019-07-18T15:16:44.888922Z info mcp New MCP sink stream created
因此,简而言之,两种(遥测和策略)部署中的混合器容器都需要大约44秒来建立所有连接
如果在两种部署中都将initialDelaySeconds:
更改为60秒,则不应重新启动POD,原因是liveness probe
希望这有帮助类似的地方:检查版本。这些版本是1.15版本,将CPU增加到8个vCore和32 GB RAM内存。重新部署istio演示。但问题依然存在。你能帮我做些调试吗?这解决了我的问题,我试着用不到4Gi的内存运行cassandra helm。谢谢!“针对卷“pvc-edd053ea-6eed-4763-85d5”的MountVolume.MountDevice失败:rpc错误:code=Internal desc=安装失败:退出状态255 cmd:'mount-t ext4/dev/disk/by id/scsi-0DO\u volume\u pvc-edd053ea-6eed-4763-85d5/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-edd053ea-6eed-4763-85d5-046d7bb6238c/globalmount”输出:”挂载:在/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-edd053ea-6eed-4763-85d5上挂载/dev/disk/by-id/scsi-0DO\u Volume\u pvc-edd053ea-6eed-4763-85d5/globalmount失败:资源繁忙\n“\uuuuuuu删除状态集po时出现此错误,重新启动节点解决了此问题。我使用了digitalocean csi