Kubernetes kubeadm join命令无法加入HA k8s主群集

我正在AWS上设置kubernetes 1.14公顷

我使用3个主节点和5个工作节点的堆叠etcd拓扑。我能够在第一个主节点上运行kubeadm init命令，并在第二个主节点上运行kubeadm join命令。我看到这两个都是成功的，并且能够使用kubectl get nodes命令列出

但是，相同的kubeadm join命令在第三个主节点上失败，该命令失败

[mark-control-plane] Marking the node ip-10-169-50-168 as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node ip-10-169-50-168 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[kubelet-check] Initial timeout of 40s passed.
error execution phase control-plane-join/mark-control-plane: error applying control-plane label and taints: timed out waiting for the condition

当我检查docker日志时，我看到第三个节点上的etcd能够加入集群，后来连接被拒绝。下面是日志

2019-04-10 17:44:59.409307 I | etcdserver/membership: added member 8ee1c831d170ef7f [https://XX.XX.XX.XX:2380] to cluster bedd10c18e149ae2
2019-04-10 17:44:59.409447 N | etcdserver/membership: set the initial cluster version to 3.3
2019-04-10 17:44:59.409506 I | etcdserver/api: enabled capabilities for version 3.3
2019-04-10 17:44:59.414195 I | rafthttp: established a TCP streaming connection with peer aa2e639fdfb57216 (stream Message reader)
2019-04-10 17:44:59.426797 I | etcdserver/membership: added member aa2e639fdfb57216 [https://XX.XX.XX.XX:2380] to cluster bedd10c18e149ae2
2019-04-10 17:44:59.428027 I | etcdserver/membership: added member 4d402309132b25d3 [https://XX.XX.XX.XX:2380] to cluster bedd10c18e149ae2
2019-04-10 17:44:59.436291 I | etcdserver: 4d402309132b25d3 initialzed peer connection; fast-forwarding 8 ticks (election ticks 10) with 2 active peer(s)
2019-04-10 17:44:59.448880 I | etcdserver: published {Name:ip-10-169-50-178 ClientURLs:[https://XX.XX.XX.XX:2379]} to cluster bedd10c18e149ae2
2019-04-10 17:44:59.448959 I | embed: ready to serve client requests
2019-04-10 17:44:59.449247 I | embed: ready to serve client requests
2019-04-10 17:44:59.450469 I | embed: serving client requests on XX.XX.XX.XX:2379
2019-04-10 17:44:59.450817 I | embed: serving client requests on 127.0.0.1:2379
2019-04-10 17:45:01.533145 I | embed: rejected connection from "127.0.0.1:46992" (error "EOF", ServerName "")
2019-04-10 17:45:03.146800 I | embed: rejected connection from "XX.XX.XX.XX:48888" (error "remote error: tls: bad certificate", ServerName "")
2019-04-10 17:45:03.788293 I | raft: 4d402309132b25d3 [logterm: 8, index: 892, vote: 0] ignored MsgVote from 8ee1c831d170ef7f [logterm: 8, index: 892] at term 8: lease is not expired (remaining ticks: 10)
2019-04-10 17:45:04.312725 W | wal: sync duration of 1.985619012s, expected less than 1s
2019-04-10 17:45:05.588410 I | raft: 4d402309132b25d3 [logterm: 8, index: 892, vote: 0] ignored MsgVote from 8ee1c831d170ef7f [logterm: 8, index: 892] at term 8: lease is not expired (remaining ticks: 3)
2019-04-10 17:45:05.589745 I | raft: 4d402309132b25d3 [term: 8] received a MsgApp message with higher term from 8ee1c831d170ef7f [term: 10]
2019-04-10 17:45:05.589762 I | raft: 4d402309132b25d3 became follower at term 10
2019-04-10 17:45:05.589781 I | raft: raft.node: 4d402309132b25d3 changed leader from aa2e639fdfb57216 to 8ee1c831d170ef7f at term 10
proto: no coders for int
proto: no encoder for ValueSize int [GetProperties]
2019-04-10 17:50:43.108887 I | mvcc: store.index: compact 978
2019-04-10 17:50:43.110211 I | mvcc: finished scheduled compaction at 978 (took 960.176µs)

你能分享一些解决问题的建议吗

拒绝来自XX.XX.XX.XX的连接：48888错误远程错误： tls：坏证书

我会开始调查第三个硕士学位的证书。 将其与第二个主机（成功引导到集群中的主机）的证书进行比较 最终，您应该拥有以下文件列表：

---

希望你没有错过这一步，我能够解决这个问题。我必须以节点RBAC的身份通过apiServer.authorization-mode。

请添加您要加入的节点的kubelet日志journalctl-xeu kubeletI所有这些证书都存在。

2019-04-10 17:44:59.409307 I | etcdserver/membership: added member 8ee1c831d170ef7f [https://XX.XX.XX.XX:2380] to cluster bedd10c18e149ae2
2019-04-10 17:44:59.409447 N | etcdserver/membership: set the initial cluster version to 3.3
2019-04-10 17:44:59.409506 I | etcdserver/api: enabled capabilities for version 3.3
2019-04-10 17:44:59.414195 I | rafthttp: established a TCP streaming connection with peer aa2e639fdfb57216 (stream Message reader)
2019-04-10 17:44:59.426797 I | etcdserver/membership: added member aa2e639fdfb57216 [https://XX.XX.XX.XX:2380] to cluster bedd10c18e149ae2
2019-04-10 17:44:59.428027 I | etcdserver/membership: added member 4d402309132b25d3 [https://XX.XX.XX.XX:2380] to cluster bedd10c18e149ae2
2019-04-10 17:44:59.436291 I | etcdserver: 4d402309132b25d3 initialzed peer connection; fast-forwarding 8 ticks (election ticks 10) with 2 active peer(s)
2019-04-10 17:44:59.448880 I | etcdserver: published {Name:ip-10-169-50-178 ClientURLs:[https://XX.XX.XX.XX:2379]} to cluster bedd10c18e149ae2
2019-04-10 17:44:59.448959 I | embed: ready to serve client requests
2019-04-10 17:44:59.449247 I | embed: ready to serve client requests
2019-04-10 17:44:59.450469 I | embed: serving client requests on XX.XX.XX.XX:2379
2019-04-10 17:44:59.450817 I | embed: serving client requests on 127.0.0.1:2379
2019-04-10 17:45:01.533145 I | embed: rejected connection from "127.0.0.1:46992" (error "EOF", ServerName "")
2019-04-10 17:45:03.146800 I | embed: rejected connection from "XX.XX.XX.XX:48888" (error "remote error: tls: bad certificate", ServerName "")
2019-04-10 17:45:03.788293 I | raft: 4d402309132b25d3 [logterm: 8, index: 892, vote: 0] ignored MsgVote from 8ee1c831d170ef7f [logterm: 8, index: 892] at term 8: lease is not expired (remaining ticks: 10)
2019-04-10 17:45:04.312725 W | wal: sync duration of 1.985619012s, expected less than 1s
2019-04-10 17:45:05.588410 I | raft: 4d402309132b25d3 [logterm: 8, index: 892, vote: 0] ignored MsgVote from 8ee1c831d170ef7f [logterm: 8, index: 892] at term 8: lease is not expired (remaining ticks: 3)
2019-04-10 17:45:05.589745 I | raft: 4d402309132b25d3 [term: 8] received a MsgApp message with higher term from 8ee1c831d170ef7f [term: 10]
2019-04-10 17:45:05.589762 I | raft: 4d402309132b25d3 became follower at term 10
2019-04-10 17:45:05.589781 I | raft: raft.node: 4d402309132b25d3 changed leader from aa2e639fdfb57216 to 8ee1c831d170ef7f at term 10
proto: no coders for int
proto: no encoder for ValueSize int [GetProperties]
2019-04-10 17:50:43.108887 I | mvcc: store.index: compact 978
2019-04-10 17:50:43.110211 I | mvcc: finished scheduled compaction at 978 (took 960.176µs)