使用kubeconfig在coreos上安装kubernetes(而不是弃用的--api服务器)
我有两台coreos机器,带有coreos测试版(1185.2.0) 我使用修改后的脚本安装kuberentes和rkt容器,原始脚本位于。修改后的版本位于 我为脚本设置的环境变量有:使用kubeconfig在coreos上安装kubernetes(而不是弃用的--api服务器),kubernetes,coreos,rkt,kubelet,Kubernetes,Coreos,Rkt,Kubelet,我有两台coreos机器,带有coreos测试版(1185.2.0) 我使用修改后的脚本安装kuberentes和rkt容器,原始脚本位于。修改后的版本位于 我为脚本设置的环境变量有: ADVERTISE_IP=10.79.218.2 ETCD_ENDPOINTS="https://coreos-2.tux-in.com:2379,https://coreos-3.tux-in.com:2379" K8S_VER=v1.4.3_coreos.0 HYPERKUBE_IMAGE_REPO=quay
ADVERTISE_IP=10.79.218.2
ETCD_ENDPOINTS="https://coreos-2.tux-in.com:2379,https://coreos-3.tux-in.com:2379"
K8S_VER=v1.4.3_coreos.0
HYPERKUBE_IMAGE_REPO=quay.io/coreos/hyperkube
POD_NETWORK=10.2.0.0/16
SERVICE_IP_RANGE=10.3.0.0/24
K8S_SERVICE_IP=10.3.0.1
DNS_SERVICE_IP=10.3.0.10
USE_CALICO=true
CONTAINER_RUNTIME=rkt
ETCD_CERT_FILE="/etc/ssl/etcd/etcd1.pem"
ETCD_KEY_FILE="/etc/ssl/etcd/etcd1-key.pem"
ETCD_TRUSTED_CA_FILE="/etc/ssl/etcd/ca.pem"
ETCD_CLIENT_CERT_AUTH=true
OVERWRITE_ALL_FILES=true
CONTROLLER_HOSTNAME="coreos-2.tux-in.com"
ETCD_CERT_ROOT_DIR="/etc/ssl/etcd"
ETCD_SCHEME="https"
ETCD_AUTHORITY="coreos-2.tux-in.com:2379"
IS_MASK_UPDATE_ENGINE=false
最值得注意的变化是增加了对etcd2 tls证书和kubeconfig yaml使用的支持,而不是被弃用的--api server
目前,我正在尝试使用coreos-2.tux-in.com的控制器脚本进行安装
控制器节点的kubeconfig yaml包含:
current-context: tuxin-coreos-context
apiVersion: v1
clusters:
- cluster:
certificate-authority: /etc/kubernetes/ssl/ca.pem
server: https://coreos-2.tux-in.com:443
name: tuxin-coreos-cluster
contexts:
- context:
cluster: tuxin-coreos-cluster
name: tuxin-coreos-context
kind: Config
preferences:
colors: true
users:
- name: kubelet
user:
client-certificate: /etc/kubernetes/ssl/apiserver.pem
client-key: /etc/kubernetes/ssl/apiserver-key.pem
生成的kubelet.service
文件包含
[Service]
Environment=KUBELET_VERSION=v1.4.3_coreos.0
Environment=KUBELET_ACI=quay.io/coreos/hyperkube
Environment="RKT_OPTS=--volume dns,kind=host,source=/etc/resolv.conf --mount volume=dns,target=/etc/resolv.conf --volume rkt,kind=host,source=/opt/bin/host-rkt --mount volume=rkt,target=/usr/bin/rkt --volume var-lib-rkt,kind=host,source=/var/lib/rkt --mount volume=var-lib-rkt,target=/var/lib/rkt --volume stage,kind=host,source=/tmp --mount volume=stage,target=/tmp --volume var-log,kind=host,source=/var/log --mount volume=var-log,target=/var/log"
ExecStartPre=/usr/bin/mkdir -p /etc/kubernetes/manifests
ExecStartPre=/usr/bin/mkdir -p /var/log/containers
ExecStart=/usr/lib/coreos/kubelet-wrapper --kubeconfig=/etc/kubernetes/controller-kubeconfig.yaml --register-schedulable=false --cni-conf-dir=/etc/kubernetes/cni/net.d --network-plugin=cni --container-runtime=rkt --rkt-path=/usr/bin/rkt --rkt-stage1-image=coreos.com/rkt/stage1-coreos --allow-privileged=true --pod-manifest-path=/etc/kubernetes/manifests --hostname-override=10.79.218.2 --cluster_dns=10.3.0.10 --cluster_domain=cluster.local
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
现在。。我很确定这与使用--kubeconfig
而不是--api server
有关,因为只有在进行了此更改之后,我才开始出现此错误
kubelet日志输出为
kubelet现在没有正确安装,在我的桌面上运行kubectl get nodes
时,它返回一个空列表
有什么想法吗
更新
kubectl get节点的输出——v=8
更新
etcdctl ls/registry/minions
输出:
Error: 100: Key not found (/registry/minions) [42662]
控制器上的ps-aef | grep kubelet
root 2054 1 3 12:49 ? 00:18:06 /kubelet --kubeconfig=/etc/kubernetes/controller-kubeconfig.yaml --register-schedulable=false --cni-conf-dir=/etc/kubernetes/cni/net.d --network-plugin=cni --container-runtime=rkt --rkt-path=/usr/bin/rkt --allow-privileged=true --pod-manifest-path=/etc/kubernetes/manifests --hostname-override=10.79.218.2 --cluster_dns=10.3.0.10 --cluster_domain=cluster.local
root 2605 1 0 12:51 ? 00:00:00 stage1/rootfs/usr/lib/ld-linux-x86-64.so.2 stage1/rootfs/usr/bin/systemd-nspawn --boot --register=true --link-journal=try-guest --keep-unit --quiet --uuid=b7008337-7b90-4fd7-8f1f-7bc45f056685 --machine=rkt-b7008337-7b90-4fd7-8f1f-7bc45f056685 --directory=stage1/rootfs --bind=/var/lib/kubelet/pods/52646008312b398ac0d3031ad8b9e280/containers/kube-scheduler/ce639294-9f68-11e6-a3bd-1c6f653e6f72:/opt/stage2/kube-scheduler/rootfs/dev/termination-log --bind=/var/lib/kubelet/pods/52646008312b398ac0d3031ad8b9e280/containers/kube-scheduler/etc-hosts:/opt/stage2/kube-scheduler/rootfs/etc/hosts --bind=/var/lib/kubelet/pods/52646008312b398ac0d3031ad8b9e280/containers/kube-scheduler/etc-resolv-conf:/opt/stage2/kube-scheduler/rootfs/etc/resolv.conf --capability=CAP_AUDIT_WRITE,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FSETID,CAP_FOWNER,CAP_KILL,CAP_MKNOD,CAP_NET_RAW,CAP_NET_BIND_SERVICE,CAP_SETUID,CAP_SETGID,CAP_SETPCAP,CAP_SETFCAP,CAP_SYS_CHROOT -- --default-standard-output=tty --log-target=null --show-status=0
root 2734 1 0 12:51 ? 00:00:00 stage1/rootfs/usr/lib/ld-linux-x86-64.so.2 stage1/rootfs/usr/bin/systemd-nspawn --boot --register=true --link-journal=try-guest --keep-unit --quiet --uuid=ee6be263-c4ed-4a70-879c-57e2dde4ab7a --machine=rkt-ee6be263-c4ed-4a70-879c-57e2dde4ab7a --directory=stage1/rootfs --bind=/var/lib/kubelet/pods/0c997ab29f8d032a29a952f578d9014c/containers/kube-apiserver/ceb3598e-9f68-11e6-a3bd-1c6f653e6f72:/opt/stage2/kube-apiserver/rootfs/dev/termination-log --bind=/var/lib/kubelet/pods/0c997ab29f8d032a29a952f578d9014c/containers/kube-apiserver/etc-hosts:/opt/stage2/kube-apiserver/rootfs/etc/hosts --bind=/var/lib/kubelet/pods/0c997ab29f8d032a29a952f578d9014c/containers/kube-apiserver/etc-resolv-conf:/opt/stage2/kube-apiserver/rootfs/etc/resolv.conf --bind-ro=/etc/ssl/etcd:/opt/stage2/kube-apiserver/rootfs/etc/ssl/etcd --bind-ro=/etc/kubernetes/ssl:/opt/stage2/kube-apiserver/rootfs/etc/kubernetes/ssl --bind-ro=/usr/share/ca-certificates:/opt/stage2/kube-apiserver/rootfs/etc/ssl/certs --capability=CAP_AUDIT_WRITE,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FSETID,CAP_FOWNER,CAP_KILL,CAP_MKNOD,CAP_NET_RAW,CAP_NET_BIND_SERVICE,CAP_SETUID,CAP_SETGID,CAP_SETPCAP,CAP_SETFCAP,CAP_SYS_CHROOT -- --default-standard-output=tty --log-target=null --show-status=0
root 2760 1 0 12:51 ? 00:00:00 stage1/rootfs/usr/lib/ld-linux-x86-64.so.2 stage1/rootfs/usr/bin/systemd-nspawn --boot --register=true --link-journal=try-guest --keep-unit --quiet --uuid=6a9e6598-3c1d-4563-bbdf-4ca1774f8f83 --machine=rkt-6a9e6598-3c1d-4563-bbdf-4ca1774f8f83 --directory=stage1/rootfs --bind-ro=/etc/kubernetes/ssl:/opt/stage2/kube-controller-manager/rootfs/etc/kubernetes/ssl --bind-ro=/usr/share/ca-certificates:/opt/stage2/kube-controller-manager/rootfs/etc/ssl/certs --bind=/var/lib/kubelet/pods/11d558df35524947fb7ed66cf7bed0eb/containers/kube-controller-manager/cebd2d3d-9f68-11e6-a3bd-1c6f653e6f72:/opt/stage2/kube-controller-manager/rootfs/dev/termination-log --bind=/var/lib/kubelet/pods/11d558df35524947fb7ed66cf7bed0eb/containers/kube-controller-manager/etc-hosts:/opt/stage2/kube-controller-manager/rootfs/etc/hosts --bind=/var/lib/kubelet/pods/11d558df35524947fb7ed66cf7bed0eb/containers/kube-controller-manager/etc-resolv-conf:/opt/stage2/kube-controller-manager/rootfs/etc/resolv.conf --capability=CAP_AUDIT_WRITE,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FSETID,CAP_FOWNER,CAP_KILL,CAP_MKNOD,CAP_NET_RAW,CAP_NET_BIND_SERVICE,CAP_SETUID,CAP_SETGID,CAP_SETPCAP,CAP_SETFCAP,CAP_SYS_CHROOT -- --default-standard-output=tty --log-target=null --show-status=0
root 3861 1 0 12:53 ? 00:00:00 stage1/rootfs/usr/lib/ld-linux-x86-64.so.2 stage1/rootfs/usr/bin/systemd-nspawn --boot --register=true --link-journal=try-guest --keep-unit --quiet --uuid=3dad014c-b31f-4e11-afb7-59214a7a4de9 --machine=rkt-3dad014c-b31f-4e11-afb7-59214a7a4de9 --directory=stage1/rootfs --bind=/var/lib/kubelet/pods/7889fbb0a1c86d9bfdb12908938dee20/containers/kube-policy-controller/etc-hosts:/opt/stage2/kube-policy-controller/rootfs/etc/hosts --bind=/var/lib/kubelet/pods/7889fbb0a1c86d9bfdb12908938dee20/containers/kube-policy-controller/etc-resolv-conf:/opt/stage2/kube-policy-controller/rootfs/etc/resolv.conf --bind-ro=/etc/ssl/etcd:/opt/stage2/kube-policy-controller/rootfs/etc/ssl/etcd --bind=/var/lib/kubelet/pods/7889fbb0a1c86d9bfdb12908938dee20/containers/kube-policy-controller/dfd7a7dc-9f68-11e6-a3bd-1c6f653e6f72:/opt/stage2/kube-policy-controller/rootfs/dev/termination-log --capability=CAP_AUDIT_WRITE,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FSETID,CAP_FOWNER,CAP_KILL,CAP_MKNOD,CAP_NET_RAW,CAP_NET_BIND_SERVICE,CAP_SETUID,CAP_SETGID,CAP_SETPCAP,CAP_SETFCAP,CAP_SYS_CHROOT --bind=/var/lib/kubelet/pods/7889fbb0a1c86d9bfdb12908938dee20/containers/leader-elector/etc-hosts:/opt/stage2/leader-elector/rootfs/etc/hosts --bind=/var/lib/kubelet/pods/7889fbb0a1c86d9bfdb12908938dee20/containers/leader-elector/etc-resolv-conf:/opt/stage2/leader-elector/rootfs/etc/resolv.conf --bind=/var/lib/kubelet/pods/7889fbb0a1c86d9bfdb12908938dee20/containers/leader-elector/f9e65e21-9f68-11e6-a3bd-1c6f653e6f72:/opt/stage2/leader-elector/rootfs/dev/termination-log --capability=CAP_AUDIT_WRITE,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FSETID,CAP_FOWNER,CAP_KILL,CAP_MKNOD,CAP_NET_RAW,CAP_NET_BIND_SERVICE,CAP_SETUID,CAP_SETGID,CAP_SETPCAP,CAP_SETFCAP,CAP_SYS_CHROOT -- --default-standard-output=tty --log-target=null --show-status=0
ps-aef | grep kubelet
on worker
root 2092 1 0 12:56 ? 00:03:56 /kubelet --cni-conf-dir=/etc/kubernetes/cni/net.d --network-plugin=cni --container-runtime=rkt --rkt-path=/usr/bin/rkt --register-node=true --allow-privileged=true --pod-manifest-path=/etc/kubernetes/manifests --hostname-override=10.79.218.3 --cluster_dns=10.3.0.10 --cluster_domain=cluster.local --kubeconfig=/etc/kubernetes/worker-kubeconfig.yaml --tls-cert-file=/etc/kubernetes/ssl/worker.pem --tls-private-key-file=/etc/kubernetes/ssl/worker-key.pem
更新
当我运行journalctl-f-u kubelet
时,我注意到每10秒我就会收到以下消息:
Nov 02 13:01:54 coreos-2.tux-in.com kubelet-wrapper[1751]: I1102 13:01:54.360929 1751 kubelet_node_status.go:203] Setting node annotation to enable volume controller attach/detach
此消息与哪个服务相关?可能由于某种故障,每10秒就有东西重新启动一次。选项--require kubeconfigfor应该会有帮助。你能粘贴“kubectl get nodes--v=8”@HrishikeshKumar-更新的主帖子的输出吗。感谢kubectl能够联系apiserver,apiserver返回一个空响应。@HrishikeshKumar-谢谢。你知道为什么吗?或者如何进一步调查此问题?您是否可以在主机上发出命令
etcdctl ls/registry/minions
,它必须列出所有工作节点,在您的情况下,它可能返回空。我猜工人没有正确地向船长登记。启动时,辅助设备上的kubelet应向主设备注册。在您进行更改后,可能无法找到apiserver。搜索名为kubelet.log-/var/log/kubernetes/kubelet.log的文件,查看是否有任何错误。此外,apiserver应该位于参数中。您可以执行ps-aef来检查是否有任何kubelet进程正在运行。