Kubernetes 库贝歇尔在过程中间停下来，https://127.0.0.1:6443/healthz，请求失败：<；urlopen错误隧道连接失败：403禁止>&引用；_Kubernetes_Kubespray

Kubernetes 库贝歇尔在过程中间停下来，https://127.0.0.1:6443/healthz，请求失败：<；urlopen错误隧道连接失败：403禁止>&引用；

kubernetes

Kubernetes 库贝歇尔在过程中间停下来，https://127.0.0.1:6443/healthz，请求失败：<；urlopen错误隧道连接失败：403禁止>&引用；,kubernetes,kubespray,Kubernetes,Kubespray,我想通过Kubespray在3个主机、3个ETCD和2个节点上安装Kubernetes。但是Kub祈祷PooLoad停在中间。有一次，它会打印此消息，但过程仍在继续： TASK [kubernetes/kubeadm : Join to cluster with ignores] * fatal: [lsrv-k8s-node1]: FAILED! => {"changed": true, "cmd": ["timeout", "-k", "120s", "120s", "/usr/lo

我想通过Kubespray在3个主机、3个ETCD和2个节点上安装Kubernetes。但是Kub祈祷PooLoad停在中间。有一次，它会打印此消息，但过程仍在继续：

TASK [kubernetes/kubeadm : Join to cluster with ignores] *
fatal: [lsrv-k8s-node1]: FAILED! => {"changed": true, "cmd": ["timeout", "-k", "120s", "120s", "/usr/local/bin/kubeadm", "join", "config", "/etc/kubernetes/kubeadm-client.conf", "ignore-preflight-errors=all"], "delta": "0:01:03.639553", "end": "2020-04-25 23:08:51.163709", "msg": "non-zero return code", "rc": 1, "start": "2020-04-25 23:07:47.524156", "stderr": "W0425 23:07:47.569297   49639 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.\nW0425 23:07:47.570267   49639 common.go:77] your configuration file uses a deprecated API spec: \"kubeadm.k8s.io/v1beta1\". Please use 'kubeadm config migrate old-config old.yaml new-config new.yaml', which will write the new, similar spec using a newer API version.\n\t[WARNING DirAvailableetc-kubernetes-manifests]: /etc/kubernetes/manifests is not empty\n\t[WARNING IsDockerSystemdCheck]: detected \"cgroupfs\" as the Docker cgroup driver. The recommended driver is \"systemd\". Please follow the guide at https://kubernetes.io/docs/setup/cri/\n\t[WARNING HTTPProxy]: Connection to \"https://192.168.72.133\" uses proxy \"https://192.168.70.145:3128\". If that is not intended, adjust your proxy settings\nerror execution phase preflight: couldn't validate the identity of the API Server: Get https://192.168.72.133:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s: proxyconnect tcp: tls: first record does not look like a TLS handshake\nTo see the stack trace of this error execute with v=5 or higher", "stderr_lines": ["W0425 23:07:47.569297   49639 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.", "W0425 23:07:47.570267   49639 common.go:77] your configuration file uses a deprecated API spec: \"kubeadm.k8s.io/v1beta1\". Please use 'kubeadm config migrate old-config old.yaml new-config new.yaml', which will write the new, similar spec using a newer API version.", "\t[WARNING DirAvailableetc-kubernetes-manifests]: /etc/kubernetes/manifests is not empty", "\t[WARNING IsDockerSystemdCheck]: detected \"cgroupfs\" as the Docker cgroup driver. The recommended driver is \"systemd\". Please follow the guide at https://kubernetes.io/docs/setup/cri/", "\t[WARNING HTTPProxy]: Connection to \"https://192.168.72.133\" uses proxy \"https://192.168.70.145:3128\". If that is not intended, adjust your proxy settings", "error execution phase preflight: couldn't validate the identity of the API Server: Get https://192.168.72.133:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s: proxyconnect tcp: tls: first record does not look like a TLS handshake", "To see the stack trace of this error execute with v=5 or higher"], "stdout": "[preflight] Running pre-flight checks", "stdout_lines": ["[preflight] Running pre-flight checks"]}
fatal: [lsrv-k8s-node2]: FAILED! => {"changed": true, "cmd": ["timeout", "-k", "120s", "120s", "/usr/local/bin/kubeadm", "join", "config", "/etc/kubernetes/kubeadm-client.conf", "ignore-preflight-errors=all"], "delta": "0:01:03.644100", "end": "2020-04-25 23:08:51.182100", "msg": "non-zero return code", "rc": 1, "start": "2020-04-25 23:07:47.538000", "stderr": "W0425 23:07:47.583487   30148 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.\nW0425 23:07:47.584414   30148 common.go:77] your configuration file uses a deprecated API spec: \"kubeadm.k8s.io/v1beta1\". Please use 'kubeadm config migrate old-config old.yaml new-config new.yaml', which will write the new, similar spec using a newer API version.\n\t[WARNING DirAvailableetc-kubernetes-manifests]: /etc/kubernetes/manifests is not empty\n\t[WARNING IsDockerSystemdCheck]: detected \"cgroupfs\" as the Docker cgroup driver. The recommended driver is \"systemd\". Please follow the guide at https://kubernetes.io/docs/setup/cri/\n\t[WARNING HTTPProxy]: Connection to \"https://192.168.72.133\" uses proxy \"https://192.168.70.145:3128\". If that is not intended, adjust your proxy settings\nerror execution phase preflight: couldn't validate the identity of the API Server: Get https://192.168.72.133:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s: proxyconnect tcp: tls: first record does not look like a TLS handshake\nTo see the stack trace of this error execute with v=5 or higher", "stderr_lines": ["W0425 23:07:47.583487   30148 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.", "W0425 23:07:47.584414   30148 common.go:77] your configuration file uses a deprecated API spec: \"kubeadm.k8s.io/v1beta1\". Please use 'kubeadm config migrate old-config old.yaml new-config new.yaml', which will write the new, similar spec using a newer API version.", "\t[WARNING DirAvailableetc-kubernetes-manifests]: /etc/kubernetes/manifests is not empty", "\t[WARNING IsDockerSystemdCheck]: detected \"cgroupfs\" as the Docker cgroup driver. The recommended driver is \"systemd\". Please follow the guide at https://kubernetes.io/docs/setup/cri/", "\t[WARNING HTTPProxy]: Connection to \"https://192.168.72.133\" uses proxy \"https://192.168.70.145:3128\". If that is not intended, adjust your proxy settings", "error execution phase preflight: couldn't validate the identity of the API Server: Get https://192.168.72.133:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s: proxyconnect tcp: tls: first record does not look like a TLS handshake", "To see the stack trace of this error execute with v=5 or higher"], "stdout": "[preflight] Running pre-flight checks", "stdout_lines": ["[preflight] Running pre-flight checks"]}
Saturday 25 April 2020  23:08:51 +0430 (0:01:03.866)       0:06:53.654  

TASK [kubernetes/kubeadm : Display kubeadm join stderr if any] *
ok: [lsrv-k8s-node1] => {
    "msg": "Joined with warnings\n['W0425 23:07:47.569297   49639 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.', 'W0425 23:07:47.570267   49639 common.go:77] your configuration file uses a deprecated API spec: \"kubeadm.k8s.io/v1beta1\". Please use \\'kubeadm config migrate old-config old.yaml new-config new.yaml\\', which will write the new, similar spec using a newer API version.', '\\t[WARNING DirAvailableetc-kubernetes-manifests]: /etc/kubernetes/manifests is not empty', '\\t[WARNING IsDockerSystemdCheck]: detected \"cgroupfs\" as the Docker cgroup driver. The recommended driver is \"systemd\". Please follow the guide at https://kubernetes.io/docs/setup/cri/', '\\t[WARNING HTTPProxy]: Connection to \"https://192.168.72.133\" uses proxy \"https://192.168.70.145:3128\". If that is not intended, adjust your proxy settings', \"error execution phase preflight: couldn't validate the identity of the API Server: Get https://192.168.72.133:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s: proxyconnect tcp: tls: first record does not look like a TLS handshake\", 'To see the stack trace of this error execute with v=5 or higher']\n"
}
ok: [lsrv-k8s-node2] => {
    "msg": "Joined with warnings\n['W0425 23:07:47.583487   30148 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.', 'W0425 23:07:47.584414   30148 common.go:77] your configuration file uses a deprecated API spec: \"kubeadm.k8s.io/v1beta1\". Please use \\'kubeadm config migrate old-config old.yaml new-config new.yaml\\', which will write the new, similar spec using a newer API version.', '\\t[WARNING DirAvailableetc-kubernetes-manifests]: /etc/kubernetes/manifests is not empty', '\\t[WARNING IsDockerSystemdCheck]: detected \"cgroupfs\" as the Docker cgroup driver. The recommended driver is \"systemd\". Please follow the guide at https://kubernetes.io/docs/setup/cri/', '\\t[WARNING HTTPProxy]: Connection to \"https://192.168.72.133\" uses proxy \"https://192.168.70.145:3128\". If that is not intended, adjust your proxy settings', \"error execution phase preflight: couldn't validate the identity of the API Server: Get https://192.168.72.133:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s: proxyconnect tcp: tls: first record does not look like a TLS handshake\", 'To see the stack trace of this error execute with v=5 or higher']\n"
}
Saturday 25 April 2020  23:08:51 +0430 (0:00:00.082)       0:06:53.737  
Saturday 25 April 2020  23:08:51 +0430 (0:00:00.050)       0:06:53.787

但最终它在这一点上停止了：

PLAY [kube-master] *


TASK [kubespray-defaults : Configure defaults] *
ok: [lsrv-k8s-mstr1] => {
    "msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
ok: [lsrv-k8s-mstr2] => {
    "msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
ok: [lsrv-k8s-mstr3] => {
    "msg": "Check roles/kubespray-defaults/defaults/main.yml"
}
Saturday 25 April 2020  23:09:41 +0430 (0:00:00.044)       0:07:44.209  
Saturday 25 April 2020  23:09:41 +0430 (0:00:00.043)       0:07:44.253  
Saturday 25 April 2020  23:09:41 +0430 (0:00:00.044)       0:07:44.297  
FAILED - RETRYING: Kubernetes Apps | Wait for kube-apiserver (20 retries left).
FAILED - RETRYING: Kubernetes Apps | Wait for kube-apiserver (19 retries left).
...
FAILED - RETRYING: Kubernetes Apps | Wait for kube-apiserver (2 retries left).
FAILED - RETRYING: Kubernetes Apps | Wait for kube-apiserver (1 retries left).

TASK [kubernetes-apps/ansible : Kubernetes Apps | Wait for kube-apiserver] *
fatal: [lsrv-k8s-mstr1]: FAILED! => {"attempts": 20, "changed": false, "content": "", "elapsed": 0, "msg": "Status code was -1 and not [200]: Request failed: <urlopen error Tunnel connection failed: 403 Forbidden>", "redirected": false, "status": -1, "url": "https://127.0.0.1:6443/healthz"}

NO MORE HOSTS LEFT *

PLAY RECAP *
localhost                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
lsrv-k8s-etcd1             : ok=152  changed=8    unreachable=0    failed=0    skipped=213  rescued=0    ignored=0   
lsrv-k8s-etcd2             : ok=142  changed=8    unreachable=0    failed=0    skipped=206  rescued=0    ignored=0   
lsrv-k8s-etcd3             : ok=142  changed=8    unreachable=0    failed=0    skipped=206  rescued=0    ignored=0   
lsrv-k8s-mstr1             : ok=626  changed=48   unreachable=0    failed=1    skipped=747  rescued=0    ignored=0   
lsrv-k8s-mstr2             : ok=464  changed=40   unreachable=0    failed=0    skipped=605  rescued=0    ignored=0   
lsrv-k8s-mstr3             : ok=466  changed=40   unreachable=0    failed=0    skipped=603  rescued=0    ignored=0   
lsrv-k8s-node1             : ok=385  changed=22   unreachable=0    failed=1    skipped=334  rescued=1    ignored=0   
lsrv-k8s-node2             : ok=385  changed=22   unreachable=0    failed=1    skipped=334  rescued=1    ignored=0   

Saturday 25 April 2020  23:10:07 +0430 (0:00:25.764)       0:08:10.061  
=============================================================================== 
kubernetes/kubeadm : Join to cluster - 64.06s
kubernetes/kubeadm : Join to cluster with ignores  63.87s
kubernetes-apps/ansible : Kubernetes Apps | Wait for kube-apiserver  25.76s
kubernetes/preinstall : Update package management cache (APT)  17.29s
etcd : Gen_certs | Write etcd master certs - 11.07s
kubernetes/master : Master | wait for kube-scheduler  7.76s
Gather necessary facts  6.35s
kubernetes-apps/ingress_controller/cert_manager : Cert Manager | Remove legacy namespace  5.64s
container-engine/docker : ensure docker packages are installed  5.14s
kubernetes-apps/ingress_controller/ingress_nginx : NGINX Ingress Controller | Create manifests  4.48s
kubernetes/master : kubeadm | write out kubeadm certs - 4.41s
kubernetes-apps/ingress_controller/cert_manager : Cert Manager | Create manifests - 3.99s
etcd : Gen_certs | Gather etcd master certs - 3.70s
bootstrap-os : Fetch /etc/os-release  3.63s
bootstrap-os : Install dbus for the hostname module - 3.29s
kubernetes-apps/external_provisioner/local_path_provisioner : Local Path Provisioner | Create manifests - 3.11s
kubernetes-apps/ingress_controller/ingress_nginx : NGINX Ingress Controller | Apply manifests - 3.05s
kubernetes/client : Generate admin kubeconfig with external api endpoint  2.70s
kubernetes/master : kubeadm | Check if apiserver.crt contains all needed SANs - 2.68s
download : download | Download files / images - 2.67s

PLAY[kube master]*
任务[kubespray默认值：配置默认值]*
确定：[lsrv-k8s-mstr1]=>{
“msg”：“检查角色/kubespray defaults/defaults/main.yml”
}
确定：[lsrv-k8s-mstr2]=>{
“msg”：“检查角色/kubespray defaults/defaults/main.yml”
}
确定：[lsrv-k8s-mstr3]=>{
“msg”：“检查角色/kubespray defaults/defaults/main.yml”
}
2020年4月25日星期六23:09:41+0430（0:00:00.044）0:07:44.209
2020年4月25日星期六23:09:41+0430（0:00:00.043）0:07:44.253
2020年4月25日星期六23:09:41+0430（0:00:00.044）0:07:44.297
失败-重试：Kubernetes应用程序|等待kube apiserver（还剩20次重试）。
失败-重试：Kubernetes应用程序|等待kube apiserver（还剩19次重试）。
...
失败-重试：Kubernetes应用程序|等待kube apiserver（还剩2次重试）。
失败-重试：Kubernetes应用程序|等待kube apiserver（还剩1次重试）。
任务[kubernetes应用程序/ansible:kubernetes应用程序|等待kube apiserver]*
致命：[lsrv-k8s-mstr1]：失败！=>{“尝试”：20，“更改”：false，“内容”：““已用”：0，“消息”：“状态代码为-1，而不是[200]：请求失败：”，“重定向”：false，“状态”：-1，“url”：”https://127.0.0.1:6443/healthz"}
没有更多的主人了*
重演*
localhost:ok=1已更改=0无法访问=0失败=0跳过=0已获救=0已忽略=0
lsrv-k8s-etcd1:正常=152更改=8无法访问=0失败=0跳过=213获救=0忽略=0
lsrv-k8s-etcd2:正常=142更改=8无法访问=0失败=0跳过=206获救=0忽略=0
lsrv-k8s-etcd3:正常=142更改=8无法访问=0失败=0跳过=206获救=0忽略=0
lsrv-k8s-mstr1:正常=626更改=48无法访问=0失败=1跳过=747获救=0忽略=0
lsrv-k8s-mstr2:正常=464更改=40无法访问=0失败=0跳过=605获救=0忽略=0
lsrv-k8s-mstr3:正常=466更改=40无法访问=0失败=0跳过=603获救=0忽略=0
lsrv-k8s-node1:正常=385更改=22无法访问=0失败=1跳过=334获救=1忽略=0
lsrv-k8s-node2:ok=385 changed=22 unreachable=0 failed=1 skipped=334 Saved=1 ignored=0
2020年4月25日星期六23:10:07+0430（0:00:25.764）0:08:10.061
=============================================================================== 
kubernetes/kubeadm：加入集群-64.06s
kubernetes/kubeadm：使用63.87s连接到集群
kubernetes应用程序/ansible:kubernetes应用程序|等待kube apiserver 25.76s
kubernetes/预安装：更新包管理缓存（APT）17.29s
etcd:Gen|U证书|写etcd主证书-11.07s
kubernetes/master:master |等待kube调度程序7.76s
收集必要的事实6.35s
kubernetes应用程序/入口控制器/证书管理器：证书管理器|删除旧命名空间5.64s
容器引擎/docker：确保docker软件包的安装时间为5.14秒
kubernetes应用程序/ingress_控制器/ingress_nginx:nginx ingress控制器|创建清单4.48s
kubernetes/硕士：kubeadm |写出kubeadm证书-4.41s
kubernetes应用程序/入口控制器/证书管理器：证书管理器|创建清单-3.99s
etcd:Gen|U证书|收集etcd主证书-3.70s
引导操作系统：Fetch/etc/os版本3.63s
引导操作系统：为主机名模块-3.29s安装dbus
kubernetes apps/external_provisioner/local_path_provisioner:local path provisioner|创建清单-3.11s
kubernetes应用程序/ingress_控制器/ingress_nginx:nginx ingress控制器|应用清单-3.05s
kubernetes/客户端：使用外部api端点2.70s生成管理员kubeconfig
kubernetes/master:kubeadm |检查apiserver.crt是否包含所有需要的san-2.68s
下载：下载|下载文件/图像-2.67s

健康检查不起作用。返回403：

致命：[lsrv-k8s-mstr1]：失败！=>{“尝试”：20，“更改”：false，“内容”：““已用”：0，“消息”：“状态代码为-1而不是[200]：请求失败：”，“重定向”：false，“状态”：-1，“url”：“}

请指导我。

您的错误消息表明您可能存在根问题身份验证问题。确保没有遗漏或错误配置任何预安装步骤

这些命令将提供有关群集状态的一些信息：

kubectl获取组件状态

kubectl获取节点

kubectl get pods——所有名称空间

该问题是由工作节点中/etc/environment文件中的https\u代理集引起的

删除https_代理和http_代理行后，问题得到解决。

/kubectl.sh获取组件状态名称状态消息错误控制器管理器健康正常计划程序健康正常etcd-1健康{“健康”：“true”}etcd-0健康{“健康”：“真”}etcd-2健康{“健康”：“真”}./kubectl.sh获取节点名称状态角色年龄版本lsrv-k8s-mstr1就绪主机6m52s v1.18.1 lsrv-k8s-mstr2就绪主机6m13s v1.18.1 lsrv-k8s-mstr3就绪主机6m16s v1.18.1