Ubuntu libvirt-coreos配置（kube-up.sh）后Kubernetes群集主机未启动_Ubuntu_Kubernetes_Coreos_Libvirt

Ubuntu libvirt-coreos配置（kube-up.sh）后Kubernetes群集主机未启动

ubuntu kubernetes

Ubuntu libvirt-coreos配置（kube-up.sh）后Kubernetes群集主机未启动,ubuntu,kubernetes,coreos,libvirt,Ubuntu,Kubernetes,Coreos,Libvirt,我是Kubernetes的新手，但我是一个长期的Linux系统管理员在完成本教程之后，我发现Ubuntu Xenial系统存在一个问题：主机背景： [core@kvmhost Automation]$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.1 LTS Release: 16.04 Codename: xenia

我是Kubernetes的新手，但我是一个长期的Linux系统管理员

在完成本教程之后，我发现Ubuntu Xenial系统存在一个问题：

主机背景：

[core@kvmhost Automation]$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.1 LTS
Release:        16.04
Codename:       xenial

[core@kvmhost Automation]$ export KUBERNETES_PROVIDER=libvirt-coreos; wget -q -O - https://get.k8s.io | bash
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                               Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0    232      0 --:--:-- --:--:-- --:--:--   232
100     7  100     7    0     0      7      0  0:00:01 --:--:--  0:00:01     0
Downloading kubernetes release v1.5.2
  from https://dl.k8s.io/v1.5.2/kubernetes.tar.gz
  to /mnt/largevol/Automation/kubernetes.tar.gz
Is this ok? [Y]/n
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0    240      0 --:--:-- --:--:-- --:--:--   240
100 4682k  100 4682k    0     0  3925k      0  0:00:01  0:00:01 --:--:-- 17.5M
Unpacking kubernetes release v1.5.2
Kubernetes release: v1.5.2
Server: linux/amd64  (to override, set KUBERNETES_SERVER_ARCH)
Client: linux/amd64  (autodetected)

Will download kubernetes-server-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release/release/v1.5.2
Will download and extract kubernetes-client-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release/release/v1.5.2
Is this ok? [Y]/n
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  299M  100  299M    0     0  16.9M      0  0:00:17  0:00:17 --:--:-- 18.5M

md5sum(kubernetes-server-linux-amd64.tar.gz)=7947bd430c4ffc358a6784e51c1d2b0f
sha1sum(kubernetes-server-linux-amd64.tar.gz)=4dbdcfa623412dac6be8fd5a4209a1f1423e8d30

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 22.2M  100 22.2M    0     0  12.4M      0  0:00:01  0:00:01 --:--:-- 12.4M

md5sum(kubernetes-client-linux-amd64.tar.gz)=bca61eea6c0ddc5a5c771ac4d01cf784
sha1sum(kubernetes-client-linux-amd64.tar.gz)=081c096636f3f262b0e494dc496d97e67bc6c3db

Extracting /mnt/largevol/Automation/kubernetes/client/kubernetes-client-linux-amd64.tar.gz into /mnt/largevol/Automation/kubernetes/platforms/linux/amd64
Add '/mnt/largevol/Automation/kubernetes/client/bin' to your PATH to use newly-installed binaries.

[core@kvmhost Automation]$ cd kubernetes/
[core@kvmhost kubernetes]$ echo "POOL_PATH=\"/mnt/largevol/Automation/images/\"" >> cluster/libvirt-coreos/config-default.sh
[core@kvmhost kubernetes]$ cluster/kube-up.sh
... Starting cluster using provider: libvirt-coreos
... calling verify-prereqs
... calling verify-kube-binaries
... calling kube-up
KUBE_MASTER_IP: 192.168.10.1
KUBE_MASTER: kubernetes-master
Pool kubernetes created

--2017-02-13 01:03:49--  https://alpha.release.core-os.net/amd64-usr/current/coreos_production_qemu_image.img.bz2
Resolving alpha.release.core-os.net (alpha.release.core-os.net)... 104.16.20.26, 104.16.21.26, 2400:cb00:2048:1::6810:141a, ...
Connecting to alpha.release.core-os.net (alpha.release.core-os.net)|104.16.20.26|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 278995186 (266M) [application/octet-stream]
Saving to: ‘cluster/../cluster/../cluster/libvirt-coreos/coreos_production_qemu_image.img.bz2’

coreos_production_qemu_image. 100%[=================================================>] 266.07M  19.0MB/s    in 14s

2017-02-13 01:04:03 (19.0 MB/s) - ‘cluster/../cluster/../cluster/libvirt-coreos/coreos_production_qemu_image.img.bz2’ saved [278995186/278995186]


Vol coreos_base.img created


Pool kubernetes refreshed

TLS assets generated...
Network kubernetes_global created from cluster/../cluster/../cluster/libvirt-coreos/network_kubernetes_global.xml

Network kubernetes_pods created from cluster/../cluster/../cluster/libvirt-coreos/network_kubernetes_pods.xml

Vol kubernetes-node-1.img created

Pool kubernetes refreshed

Domain kubernetes-node-1 created from /tmp/tmp.Uak23Z1JIQ

Vol kubernetes-node-2.img created

Pool kubernetes refreshed

Domain kubernetes-node-2 created from /tmp/tmp.8TC3eDjEwK

Vol kubernetes-node-3.img created

Pool kubernetes refreshed

Domain kubernetes-node-3 created from /tmp/tmp.MYRaUwdpBw

Vol kubernetes-master.img created

Pool kubernetes refreshed

Domain kubernetes-master created from /tmp/tmp.2C2JUbH993

Cluster "libvirt-coreos" set.
User "libvirt-coreos" set.
Context "libvirt-coreos" set.
Switched to context "libvirt-coreos".
Wrote config for libvirt-coreos to /home/core/.kube/config
Wait for cluster readiness
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3

--8<-- REPEATED MANY TIMES BUT CLUSTER IS NEVER READY--8<--

Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3

[core@kvmhost kubernetes]$ ssh 192.168.10.1

The authenticity of host '192.168.10.1 (192.168.10.1)' can't be established.
ECDSA key fingerprint is SHA256:yZ1rw8WV0ECdep3TDEydnznx8S5TagdvZEVoWtNaHK0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.10.1' (ECDSA) to the list of known hosts.

Container Linux by CoreOS alpha (1313.0.0)
core@kubernetes-master ~ $ cd /opt/kubernetes/bin/
core@kubernetes-master /opt/kubernetes/bin $ ls -l

ls: cannot open directory '.': Permission denied

core@kubernetes-master /opt/kubernetes/bin $ cd ..
core@kubernetes-master /opt/kubernetes $ ls -l

total 16
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 addons
drwxr-x---. 2 core core 4096 Jan 12 05:08 bin
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 certs
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 manifests

core@kubernetes-master /opt/kubernetes $ id

uid=500(core) gid=500(core) groups=500(core),10(wheel),233(docker),248(systemd-journal),250(portage),251(rkt),253(fleet) context=system_u:system_r:kernel_t:s0

core@kubernetes-master /opt/kubernetes $

[core@kvmhost bin]$ cd /mnt/largevol/Automation/images/kubernetes/
[core@kvmhost kubernetes]$ chmod -R a+rx bin
[core@kvmhost kubernetes]$ virsh reboot kubernetes-master
Domain kubernetes-master is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-1
Domain kubernetes-node-1 is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-2
Domain kubernetes-node-2 is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-3
Domain kubernetes-node-3 is being rebooted

[core@kvmhost kubernetes]$ kubectl cluster-info
Kubernetes master is running at http://192.168.10.1:8080
KubeDNS is running at http://192.168.10.1:8080/api/v1/proxy/namespaces/kube-system/services/kube-dns

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

获取Kubernetes并部署集群（成功）：

[core@kvmhost Automation]$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.1 LTS
Release:        16.04
Codename:       xenial

[core@kvmhost Automation]$ export KUBERNETES_PROVIDER=libvirt-coreos; wget -q -O - https://get.k8s.io | bash
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                               Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0    232      0 --:--:-- --:--:-- --:--:--   232
100     7  100     7    0     0      7      0  0:00:01 --:--:--  0:00:01     0
Downloading kubernetes release v1.5.2
  from https://dl.k8s.io/v1.5.2/kubernetes.tar.gz
  to /mnt/largevol/Automation/kubernetes.tar.gz
Is this ok? [Y]/n
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0    240      0 --:--:-- --:--:-- --:--:--   240
100 4682k  100 4682k    0     0  3925k      0  0:00:01  0:00:01 --:--:-- 17.5M
Unpacking kubernetes release v1.5.2
Kubernetes release: v1.5.2
Server: linux/amd64  (to override, set KUBERNETES_SERVER_ARCH)
Client: linux/amd64  (autodetected)

Will download kubernetes-server-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release/release/v1.5.2
Will download and extract kubernetes-client-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release/release/v1.5.2
Is this ok? [Y]/n
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  299M  100  299M    0     0  16.9M      0  0:00:17  0:00:17 --:--:-- 18.5M

md5sum(kubernetes-server-linux-amd64.tar.gz)=7947bd430c4ffc358a6784e51c1d2b0f
sha1sum(kubernetes-server-linux-amd64.tar.gz)=4dbdcfa623412dac6be8fd5a4209a1f1423e8d30

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 22.2M  100 22.2M    0     0  12.4M      0  0:00:01  0:00:01 --:--:-- 12.4M

md5sum(kubernetes-client-linux-amd64.tar.gz)=bca61eea6c0ddc5a5c771ac4d01cf784
sha1sum(kubernetes-client-linux-amd64.tar.gz)=081c096636f3f262b0e494dc496d97e67bc6c3db

Extracting /mnt/largevol/Automation/kubernetes/client/kubernetes-client-linux-amd64.tar.gz into /mnt/largevol/Automation/kubernetes/platforms/linux/amd64
Add '/mnt/largevol/Automation/kubernetes/client/bin' to your PATH to use newly-installed binaries.

[core@kvmhost Automation]$ cd kubernetes/
[core@kvmhost kubernetes]$ echo "POOL_PATH=\"/mnt/largevol/Automation/images/\"" >> cluster/libvirt-coreos/config-default.sh
[core@kvmhost kubernetes]$ cluster/kube-up.sh
... Starting cluster using provider: libvirt-coreos
... calling verify-prereqs
... calling verify-kube-binaries
... calling kube-up
KUBE_MASTER_IP: 192.168.10.1
KUBE_MASTER: kubernetes-master
Pool kubernetes created

--2017-02-13 01:03:49--  https://alpha.release.core-os.net/amd64-usr/current/coreos_production_qemu_image.img.bz2
Resolving alpha.release.core-os.net (alpha.release.core-os.net)... 104.16.20.26, 104.16.21.26, 2400:cb00:2048:1::6810:141a, ...
Connecting to alpha.release.core-os.net (alpha.release.core-os.net)|104.16.20.26|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 278995186 (266M) [application/octet-stream]
Saving to: ‘cluster/../cluster/../cluster/libvirt-coreos/coreos_production_qemu_image.img.bz2’

coreos_production_qemu_image. 100%[=================================================>] 266.07M  19.0MB/s    in 14s

2017-02-13 01:04:03 (19.0 MB/s) - ‘cluster/../cluster/../cluster/libvirt-coreos/coreos_production_qemu_image.img.bz2’ saved [278995186/278995186]


Vol coreos_base.img created


Pool kubernetes refreshed

TLS assets generated...
Network kubernetes_global created from cluster/../cluster/../cluster/libvirt-coreos/network_kubernetes_global.xml

Network kubernetes_pods created from cluster/../cluster/../cluster/libvirt-coreos/network_kubernetes_pods.xml

Vol kubernetes-node-1.img created

Pool kubernetes refreshed

Domain kubernetes-node-1 created from /tmp/tmp.Uak23Z1JIQ

Vol kubernetes-node-2.img created

Pool kubernetes refreshed

Domain kubernetes-node-2 created from /tmp/tmp.8TC3eDjEwK

Vol kubernetes-node-3.img created

Pool kubernetes refreshed

Domain kubernetes-node-3 created from /tmp/tmp.MYRaUwdpBw

Vol kubernetes-master.img created

Pool kubernetes refreshed

Domain kubernetes-master created from /tmp/tmp.2C2JUbH993

Cluster "libvirt-coreos" set.
User "libvirt-coreos" set.
Context "libvirt-coreos" set.
Switched to context "libvirt-coreos".
Wrote config for libvirt-coreos to /home/core/.kube/config
Wait for cluster readiness
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3

--8<-- REPEATED MANY TIMES BUT CLUSTER IS NEVER READY--8<--

Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3

[core@kvmhost kubernetes]$ ssh 192.168.10.1

The authenticity of host '192.168.10.1 (192.168.10.1)' can't be established.
ECDSA key fingerprint is SHA256:yZ1rw8WV0ECdep3TDEydnznx8S5TagdvZEVoWtNaHK0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.10.1' (ECDSA) to the list of known hosts.

Container Linux by CoreOS alpha (1313.0.0)
core@kubernetes-master ~ $ cd /opt/kubernetes/bin/
core@kubernetes-master /opt/kubernetes/bin $ ls -l

ls: cannot open directory '.': Permission denied

core@kubernetes-master /opt/kubernetes/bin $ cd ..
core@kubernetes-master /opt/kubernetes $ ls -l

total 16
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 addons
drwxr-x---. 2 core core 4096 Jan 12 05:08 bin
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 certs
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 manifests

core@kubernetes-master /opt/kubernetes $ id

uid=500(core) gid=500(core) groups=500(core),10(wheel),233(docker),248(systemd-journal),250(portage),251(rkt),253(fleet) context=system_u:system_r:kernel_t:s0

core@kubernetes-master /opt/kubernetes $

[core@kvmhost bin]$ cd /mnt/largevol/Automation/images/kubernetes/
[core@kvmhost kubernetes]$ chmod -R a+rx bin
[core@kvmhost kubernetes]$ virsh reboot kubernetes-master
Domain kubernetes-master is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-1
Domain kubernetes-node-1 is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-2
Domain kubernetes-node-2 is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-3
Domain kubernetes-node-3 is being rebooted

[core@kvmhost kubernetes]$ kubectl cluster-info
Kubernetes master is running at http://192.168.10.1:8080
KubeDNS is running at http://192.168.10.1:8080/api/v1/proxy/namespaces/kube-system/services/kube-dns

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

启动群集（失败）：

[core@kvmhost Automation]$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.1 LTS
Release:        16.04
Codename:       xenial

[core@kvmhost Automation]$ export KUBERNETES_PROVIDER=libvirt-coreos; wget -q -O - https://get.k8s.io | bash
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                               Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0    232      0 --:--:-- --:--:-- --:--:--   232
100     7  100     7    0     0      7      0  0:00:01 --:--:--  0:00:01     0
Downloading kubernetes release v1.5.2
  from https://dl.k8s.io/v1.5.2/kubernetes.tar.gz
  to /mnt/largevol/Automation/kubernetes.tar.gz
Is this ok? [Y]/n
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0    240      0 --:--:-- --:--:-- --:--:--   240
100 4682k  100 4682k    0     0  3925k      0  0:00:01  0:00:01 --:--:-- 17.5M
Unpacking kubernetes release v1.5.2
Kubernetes release: v1.5.2
Server: linux/amd64  (to override, set KUBERNETES_SERVER_ARCH)
Client: linux/amd64  (autodetected)

Will download kubernetes-server-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release/release/v1.5.2
Will download and extract kubernetes-client-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release/release/v1.5.2
Is this ok? [Y]/n
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  299M  100  299M    0     0  16.9M      0  0:00:17  0:00:17 --:--:-- 18.5M

md5sum(kubernetes-server-linux-amd64.tar.gz)=7947bd430c4ffc358a6784e51c1d2b0f
sha1sum(kubernetes-server-linux-amd64.tar.gz)=4dbdcfa623412dac6be8fd5a4209a1f1423e8d30

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 22.2M  100 22.2M    0     0  12.4M      0  0:00:01  0:00:01 --:--:-- 12.4M

md5sum(kubernetes-client-linux-amd64.tar.gz)=bca61eea6c0ddc5a5c771ac4d01cf784
sha1sum(kubernetes-client-linux-amd64.tar.gz)=081c096636f3f262b0e494dc496d97e67bc6c3db

Extracting /mnt/largevol/Automation/kubernetes/client/kubernetes-client-linux-amd64.tar.gz into /mnt/largevol/Automation/kubernetes/platforms/linux/amd64
Add '/mnt/largevol/Automation/kubernetes/client/bin' to your PATH to use newly-installed binaries.

[core@kvmhost Automation]$ cd kubernetes/
[core@kvmhost kubernetes]$ echo "POOL_PATH=\"/mnt/largevol/Automation/images/\"" >> cluster/libvirt-coreos/config-default.sh
[core@kvmhost kubernetes]$ cluster/kube-up.sh
... Starting cluster using provider: libvirt-coreos
... calling verify-prereqs
... calling verify-kube-binaries
... calling kube-up
KUBE_MASTER_IP: 192.168.10.1
KUBE_MASTER: kubernetes-master
Pool kubernetes created

--2017-02-13 01:03:49--  https://alpha.release.core-os.net/amd64-usr/current/coreos_production_qemu_image.img.bz2
Resolving alpha.release.core-os.net (alpha.release.core-os.net)... 104.16.20.26, 104.16.21.26, 2400:cb00:2048:1::6810:141a, ...
Connecting to alpha.release.core-os.net (alpha.release.core-os.net)|104.16.20.26|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 278995186 (266M) [application/octet-stream]
Saving to: ‘cluster/../cluster/../cluster/libvirt-coreos/coreos_production_qemu_image.img.bz2’

coreos_production_qemu_image. 100%[=================================================>] 266.07M  19.0MB/s    in 14s

2017-02-13 01:04:03 (19.0 MB/s) - ‘cluster/../cluster/../cluster/libvirt-coreos/coreos_production_qemu_image.img.bz2’ saved [278995186/278995186]


Vol coreos_base.img created


Pool kubernetes refreshed

TLS assets generated...
Network kubernetes_global created from cluster/../cluster/../cluster/libvirt-coreos/network_kubernetes_global.xml

Network kubernetes_pods created from cluster/../cluster/../cluster/libvirt-coreos/network_kubernetes_pods.xml

Vol kubernetes-node-1.img created

Pool kubernetes refreshed

Domain kubernetes-node-1 created from /tmp/tmp.Uak23Z1JIQ

Vol kubernetes-node-2.img created

Pool kubernetes refreshed

Domain kubernetes-node-2 created from /tmp/tmp.8TC3eDjEwK

Vol kubernetes-node-3.img created

Pool kubernetes refreshed

Domain kubernetes-node-3 created from /tmp/tmp.MYRaUwdpBw

Vol kubernetes-master.img created

Pool kubernetes refreshed

Domain kubernetes-master created from /tmp/tmp.2C2JUbH993

Cluster "libvirt-coreos" set.
User "libvirt-coreos" set.
Context "libvirt-coreos" set.
Switched to context "libvirt-coreos".
Wrote config for libvirt-coreos to /home/core/.kube/config
Wait for cluster readiness
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3

--8<-- REPEATED MANY TIMES BUT CLUSTER IS NEVER READY--8<--

Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3

[core@kvmhost kubernetes]$ ssh 192.168.10.1

The authenticity of host '192.168.10.1 (192.168.10.1)' can't be established.
ECDSA key fingerprint is SHA256:yZ1rw8WV0ECdep3TDEydnznx8S5TagdvZEVoWtNaHK0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.10.1' (ECDSA) to the list of known hosts.

Container Linux by CoreOS alpha (1313.0.0)
core@kubernetes-master ~ $ cd /opt/kubernetes/bin/
core@kubernetes-master /opt/kubernetes/bin $ ls -l

ls: cannot open directory '.': Permission denied

core@kubernetes-master /opt/kubernetes/bin $ cd ..
core@kubernetes-master /opt/kubernetes $ ls -l

total 16
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 addons
drwxr-x---. 2 core core 4096 Jan 12 05:08 bin
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 certs
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 manifests

core@kubernetes-master /opt/kubernetes $ id

uid=500(core) gid=500(core) groups=500(core),10(wheel),233(docker),248(systemd-journal),250(portage),251(rkt),253(fleet) context=system_u:system_r:kernel_t:s0

core@kubernetes-master /opt/kubernetes $

[core@kvmhost bin]$ cd /mnt/largevol/Automation/images/kubernetes/
[core@kvmhost kubernetes]$ chmod -R a+rx bin
[core@kvmhost kubernetes]$ virsh reboot kubernetes-master
Domain kubernetes-master is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-1
Domain kubernetes-node-1 is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-2
Domain kubernetes-node-2 is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-3
Domain kubernetes-node-3 is being rebooted

[core@kvmhost kubernetes]$ kubectl cluster-info
Kubernetes master is running at http://192.168.10.1:8080
KubeDNS is running at http://192.168.10.1:8080/api/v1/proxy/namespaces/kube-system/services/kube-dns

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

摘要：

[core@kvmhost Automation]$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.1 LTS
Release:        16.04
Codename:       xenial

[core@kvmhost Automation]$ export KUBERNETES_PROVIDER=libvirt-coreos; wget -q -O - https://get.k8s.io | bash
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                               Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0    232      0 --:--:-- --:--:-- --:--:--   232
100     7  100     7    0     0      7      0  0:00:01 --:--:--  0:00:01     0
Downloading kubernetes release v1.5.2
  from https://dl.k8s.io/v1.5.2/kubernetes.tar.gz
  to /mnt/largevol/Automation/kubernetes.tar.gz
Is this ok? [Y]/n
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0    240      0 --:--:-- --:--:-- --:--:--   240
100 4682k  100 4682k    0     0  3925k      0  0:00:01  0:00:01 --:--:-- 17.5M
Unpacking kubernetes release v1.5.2
Kubernetes release: v1.5.2
Server: linux/amd64  (to override, set KUBERNETES_SERVER_ARCH)
Client: linux/amd64  (autodetected)

Will download kubernetes-server-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release/release/v1.5.2
Will download and extract kubernetes-client-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release/release/v1.5.2
Is this ok? [Y]/n
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  299M  100  299M    0     0  16.9M      0  0:00:17  0:00:17 --:--:-- 18.5M

md5sum(kubernetes-server-linux-amd64.tar.gz)=7947bd430c4ffc358a6784e51c1d2b0f
sha1sum(kubernetes-server-linux-amd64.tar.gz)=4dbdcfa623412dac6be8fd5a4209a1f1423e8d30

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 22.2M  100 22.2M    0     0  12.4M      0  0:00:01  0:00:01 --:--:-- 12.4M

md5sum(kubernetes-client-linux-amd64.tar.gz)=bca61eea6c0ddc5a5c771ac4d01cf784
sha1sum(kubernetes-client-linux-amd64.tar.gz)=081c096636f3f262b0e494dc496d97e67bc6c3db

Extracting /mnt/largevol/Automation/kubernetes/client/kubernetes-client-linux-amd64.tar.gz into /mnt/largevol/Automation/kubernetes/platforms/linux/amd64
Add '/mnt/largevol/Automation/kubernetes/client/bin' to your PATH to use newly-installed binaries.

[core@kvmhost Automation]$ cd kubernetes/
[core@kvmhost kubernetes]$ echo "POOL_PATH=\"/mnt/largevol/Automation/images/\"" >> cluster/libvirt-coreos/config-default.sh
[core@kvmhost kubernetes]$ cluster/kube-up.sh
... Starting cluster using provider: libvirt-coreos
... calling verify-prereqs
... calling verify-kube-binaries
... calling kube-up
KUBE_MASTER_IP: 192.168.10.1
KUBE_MASTER: kubernetes-master
Pool kubernetes created

--2017-02-13 01:03:49--  https://alpha.release.core-os.net/amd64-usr/current/coreos_production_qemu_image.img.bz2
Resolving alpha.release.core-os.net (alpha.release.core-os.net)... 104.16.20.26, 104.16.21.26, 2400:cb00:2048:1::6810:141a, ...
Connecting to alpha.release.core-os.net (alpha.release.core-os.net)|104.16.20.26|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 278995186 (266M) [application/octet-stream]
Saving to: ‘cluster/../cluster/../cluster/libvirt-coreos/coreos_production_qemu_image.img.bz2’

coreos_production_qemu_image. 100%[=================================================>] 266.07M  19.0MB/s    in 14s

2017-02-13 01:04:03 (19.0 MB/s) - ‘cluster/../cluster/../cluster/libvirt-coreos/coreos_production_qemu_image.img.bz2’ saved [278995186/278995186]


Vol coreos_base.img created


Pool kubernetes refreshed

TLS assets generated...
Network kubernetes_global created from cluster/../cluster/../cluster/libvirt-coreos/network_kubernetes_global.xml

Network kubernetes_pods created from cluster/../cluster/../cluster/libvirt-coreos/network_kubernetes_pods.xml

Vol kubernetes-node-1.img created

Pool kubernetes refreshed

Domain kubernetes-node-1 created from /tmp/tmp.Uak23Z1JIQ

Vol kubernetes-node-2.img created

Pool kubernetes refreshed

Domain kubernetes-node-2 created from /tmp/tmp.8TC3eDjEwK

Vol kubernetes-node-3.img created

Pool kubernetes refreshed

Domain kubernetes-node-3 created from /tmp/tmp.MYRaUwdpBw

Vol kubernetes-master.img created

Pool kubernetes refreshed

Domain kubernetes-master created from /tmp/tmp.2C2JUbH993

Cluster "libvirt-coreos" set.
User "libvirt-coreos" set.
Context "libvirt-coreos" set.
Switched to context "libvirt-coreos".
Wrote config for libvirt-coreos to /home/core/.kube/config
Wait for cluster readiness
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3

--8<-- REPEATED MANY TIMES BUT CLUSTER IS NEVER READY--8<--

Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3

[core@kvmhost kubernetes]$ ssh 192.168.10.1

The authenticity of host '192.168.10.1 (192.168.10.1)' can't be established.
ECDSA key fingerprint is SHA256:yZ1rw8WV0ECdep3TDEydnznx8S5TagdvZEVoWtNaHK0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.10.1' (ECDSA) to the list of known hosts.

Container Linux by CoreOS alpha (1313.0.0)
core@kubernetes-master ~ $ cd /opt/kubernetes/bin/
core@kubernetes-master /opt/kubernetes/bin $ ls -l

ls: cannot open directory '.': Permission denied

core@kubernetes-master /opt/kubernetes/bin $ cd ..
core@kubernetes-master /opt/kubernetes $ ls -l

total 16
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 addons
drwxr-x---. 2 core core 4096 Jan 12 05:08 bin
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 certs
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 manifests

core@kubernetes-master /opt/kubernetes $ id

uid=500(core) gid=500(core) groups=500(core),10(wheel),233(docker),248(systemd-journal),250(portage),251(rkt),253(fleet) context=system_u:system_r:kernel_t:s0

core@kubernetes-master /opt/kubernetes $

[core@kvmhost bin]$ cd /mnt/largevol/Automation/images/kubernetes/
[core@kvmhost kubernetes]$ chmod -R a+rx bin
[core@kvmhost kubernetes]$ virsh reboot kubernetes-master
Domain kubernetes-master is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-1
Domain kubernetes-node-1 is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-2
Domain kubernetes-node-2 is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-3
Domain kubernetes-node-3 is being rebooted

[core@kvmhost kubernetes]$ kubectl cluster-info
Kubernetes master is running at http://192.168.10.1:8080
KubeDNS is running at http://192.168.10.1:8080/api/v1/proxy/namespaces/kube-system/services/kube-dns

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

我们的群集已成功部署（主节点加3个节点），但由于文件访问权限，kube apiserver进程无法启动

解决方法（更改9p文件系统的权限以提供+rx）：

[core@kvmhost Automation]$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.1 LTS
Release:        16.04
Codename:       xenial

[core@kvmhost Automation]$ export KUBERNETES_PROVIDER=libvirt-coreos; wget -q -O - https://get.k8s.io | bash
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                               Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0    232      0 --:--:-- --:--:-- --:--:--   232
100     7  100     7    0     0      7      0  0:00:01 --:--:--  0:00:01     0
Downloading kubernetes release v1.5.2
  from https://dl.k8s.io/v1.5.2/kubernetes.tar.gz
  to /mnt/largevol/Automation/kubernetes.tar.gz
Is this ok? [Y]/n
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0    240      0 --:--:-- --:--:-- --:--:--   240
100 4682k  100 4682k    0     0  3925k      0  0:00:01  0:00:01 --:--:-- 17.5M
Unpacking kubernetes release v1.5.2
Kubernetes release: v1.5.2
Server: linux/amd64  (to override, set KUBERNETES_SERVER_ARCH)
Client: linux/amd64  (autodetected)

Will download kubernetes-server-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release/release/v1.5.2
Will download and extract kubernetes-client-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release/release/v1.5.2
Is this ok? [Y]/n
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  299M  100  299M    0     0  16.9M      0  0:00:17  0:00:17 --:--:-- 18.5M

md5sum(kubernetes-server-linux-amd64.tar.gz)=7947bd430c4ffc358a6784e51c1d2b0f
sha1sum(kubernetes-server-linux-amd64.tar.gz)=4dbdcfa623412dac6be8fd5a4209a1f1423e8d30

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 22.2M  100 22.2M    0     0  12.4M      0  0:00:01  0:00:01 --:--:-- 12.4M

md5sum(kubernetes-client-linux-amd64.tar.gz)=bca61eea6c0ddc5a5c771ac4d01cf784
sha1sum(kubernetes-client-linux-amd64.tar.gz)=081c096636f3f262b0e494dc496d97e67bc6c3db

Extracting /mnt/largevol/Automation/kubernetes/client/kubernetes-client-linux-amd64.tar.gz into /mnt/largevol/Automation/kubernetes/platforms/linux/amd64
Add '/mnt/largevol/Automation/kubernetes/client/bin' to your PATH to use newly-installed binaries.

[core@kvmhost Automation]$ cd kubernetes/
[core@kvmhost kubernetes]$ echo "POOL_PATH=\"/mnt/largevol/Automation/images/\"" >> cluster/libvirt-coreos/config-default.sh
[core@kvmhost kubernetes]$ cluster/kube-up.sh
... Starting cluster using provider: libvirt-coreos
... calling verify-prereqs
... calling verify-kube-binaries
... calling kube-up
KUBE_MASTER_IP: 192.168.10.1
KUBE_MASTER: kubernetes-master
Pool kubernetes created

--2017-02-13 01:03:49--  https://alpha.release.core-os.net/amd64-usr/current/coreos_production_qemu_image.img.bz2
Resolving alpha.release.core-os.net (alpha.release.core-os.net)... 104.16.20.26, 104.16.21.26, 2400:cb00:2048:1::6810:141a, ...
Connecting to alpha.release.core-os.net (alpha.release.core-os.net)|104.16.20.26|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 278995186 (266M) [application/octet-stream]
Saving to: ‘cluster/../cluster/../cluster/libvirt-coreos/coreos_production_qemu_image.img.bz2’

coreos_production_qemu_image. 100%[=================================================>] 266.07M  19.0MB/s    in 14s

2017-02-13 01:04:03 (19.0 MB/s) - ‘cluster/../cluster/../cluster/libvirt-coreos/coreos_production_qemu_image.img.bz2’ saved [278995186/278995186]


Vol coreos_base.img created


Pool kubernetes refreshed

TLS assets generated...
Network kubernetes_global created from cluster/../cluster/../cluster/libvirt-coreos/network_kubernetes_global.xml

Network kubernetes_pods created from cluster/../cluster/../cluster/libvirt-coreos/network_kubernetes_pods.xml

Vol kubernetes-node-1.img created

Pool kubernetes refreshed

Domain kubernetes-node-1 created from /tmp/tmp.Uak23Z1JIQ

Vol kubernetes-node-2.img created

Pool kubernetes refreshed

Domain kubernetes-node-2 created from /tmp/tmp.8TC3eDjEwK

Vol kubernetes-node-3.img created

Pool kubernetes refreshed

Domain kubernetes-node-3 created from /tmp/tmp.MYRaUwdpBw

Vol kubernetes-master.img created

Pool kubernetes refreshed

Domain kubernetes-master created from /tmp/tmp.2C2JUbH993

Cluster "libvirt-coreos" set.
User "libvirt-coreos" set.
Context "libvirt-coreos" set.
Switched to context "libvirt-coreos".
Wrote config for libvirt-coreos to /home/core/.kube/config
Wait for cluster readiness
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3

--8<-- REPEATED MANY TIMES BUT CLUSTER IS NEVER READY--8<--

Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3

[core@kvmhost kubernetes]$ ssh 192.168.10.1

The authenticity of host '192.168.10.1 (192.168.10.1)' can't be established.
ECDSA key fingerprint is SHA256:yZ1rw8WV0ECdep3TDEydnznx8S5TagdvZEVoWtNaHK0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.10.1' (ECDSA) to the list of known hosts.

Container Linux by CoreOS alpha (1313.0.0)
core@kubernetes-master ~ $ cd /opt/kubernetes/bin/
core@kubernetes-master /opt/kubernetes/bin $ ls -l

ls: cannot open directory '.': Permission denied

core@kubernetes-master /opt/kubernetes/bin $ cd ..
core@kubernetes-master /opt/kubernetes $ ls -l

total 16
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 addons
drwxr-x---. 2 core core 4096 Jan 12 05:08 bin
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 certs
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 manifests

core@kubernetes-master /opt/kubernetes $ id

uid=500(core) gid=500(core) groups=500(core),10(wheel),233(docker),248(systemd-journal),250(portage),251(rkt),253(fleet) context=system_u:system_r:kernel_t:s0

core@kubernetes-master /opt/kubernetes $

[core@kvmhost bin]$ cd /mnt/largevol/Automation/images/kubernetes/
[core@kvmhost kubernetes]$ chmod -R a+rx bin
[core@kvmhost kubernetes]$ virsh reboot kubernetes-master
Domain kubernetes-master is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-1
Domain kubernetes-node-1 is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-2
Domain kubernetes-node-2 is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-3
Domain kubernetes-node-3 is being rebooted

[core@kvmhost kubernetes]$ kubectl cluster-info
Kubernetes master is running at http://192.168.10.1:8080
KubeDNS is running at http://192.168.10.1:8080/api/v1/proxy/namespaces/kube-system/services/kube-dns

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

问题：

[core@kvmhost Automation]$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.1 LTS
Release:        16.04
Codename:       xenial

[core@kvmhost Automation]$ export KUBERNETES_PROVIDER=libvirt-coreos; wget -q -O - https://get.k8s.io | bash
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                               Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0    232      0 --:--:-- --:--:-- --:--:--   232
100     7  100     7    0     0      7      0  0:00:01 --:--:--  0:00:01     0
Downloading kubernetes release v1.5.2
  from https://dl.k8s.io/v1.5.2/kubernetes.tar.gz
  to /mnt/largevol/Automation/kubernetes.tar.gz
Is this ok? [Y]/n
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   161  100   161    0     0    240      0 --:--:-- --:--:-- --:--:--   240
100 4682k  100 4682k    0     0  3925k      0  0:00:01  0:00:01 --:--:-- 17.5M
Unpacking kubernetes release v1.5.2
Kubernetes release: v1.5.2
Server: linux/amd64  (to override, set KUBERNETES_SERVER_ARCH)
Client: linux/amd64  (autodetected)

Will download kubernetes-server-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release/release/v1.5.2
Will download and extract kubernetes-client-linux-amd64.tar.gz from https://storage.googleapis.com/kubernetes-release/release/v1.5.2
Is this ok? [Y]/n
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  299M  100  299M    0     0  16.9M      0  0:00:17  0:00:17 --:--:-- 18.5M

md5sum(kubernetes-server-linux-amd64.tar.gz)=7947bd430c4ffc358a6784e51c1d2b0f
sha1sum(kubernetes-server-linux-amd64.tar.gz)=4dbdcfa623412dac6be8fd5a4209a1f1423e8d30

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 22.2M  100 22.2M    0     0  12.4M      0  0:00:01  0:00:01 --:--:-- 12.4M

md5sum(kubernetes-client-linux-amd64.tar.gz)=bca61eea6c0ddc5a5c771ac4d01cf784
sha1sum(kubernetes-client-linux-amd64.tar.gz)=081c096636f3f262b0e494dc496d97e67bc6c3db

Extracting /mnt/largevol/Automation/kubernetes/client/kubernetes-client-linux-amd64.tar.gz into /mnt/largevol/Automation/kubernetes/platforms/linux/amd64
Add '/mnt/largevol/Automation/kubernetes/client/bin' to your PATH to use newly-installed binaries.

[core@kvmhost Automation]$ cd kubernetes/
[core@kvmhost kubernetes]$ echo "POOL_PATH=\"/mnt/largevol/Automation/images/\"" >> cluster/libvirt-coreos/config-default.sh
[core@kvmhost kubernetes]$ cluster/kube-up.sh
... Starting cluster using provider: libvirt-coreos
... calling verify-prereqs
... calling verify-kube-binaries
... calling kube-up
KUBE_MASTER_IP: 192.168.10.1
KUBE_MASTER: kubernetes-master
Pool kubernetes created

--2017-02-13 01:03:49--  https://alpha.release.core-os.net/amd64-usr/current/coreos_production_qemu_image.img.bz2
Resolving alpha.release.core-os.net (alpha.release.core-os.net)... 104.16.20.26, 104.16.21.26, 2400:cb00:2048:1::6810:141a, ...
Connecting to alpha.release.core-os.net (alpha.release.core-os.net)|104.16.20.26|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 278995186 (266M) [application/octet-stream]
Saving to: ‘cluster/../cluster/../cluster/libvirt-coreos/coreos_production_qemu_image.img.bz2’

coreos_production_qemu_image. 100%[=================================================>] 266.07M  19.0MB/s    in 14s

2017-02-13 01:04:03 (19.0 MB/s) - ‘cluster/../cluster/../cluster/libvirt-coreos/coreos_production_qemu_image.img.bz2’ saved [278995186/278995186]


Vol coreos_base.img created


Pool kubernetes refreshed

TLS assets generated...
Network kubernetes_global created from cluster/../cluster/../cluster/libvirt-coreos/network_kubernetes_global.xml

Network kubernetes_pods created from cluster/../cluster/../cluster/libvirt-coreos/network_kubernetes_pods.xml

Vol kubernetes-node-1.img created

Pool kubernetes refreshed

Domain kubernetes-node-1 created from /tmp/tmp.Uak23Z1JIQ

Vol kubernetes-node-2.img created

Pool kubernetes refreshed

Domain kubernetes-node-2 created from /tmp/tmp.8TC3eDjEwK

Vol kubernetes-node-3.img created

Pool kubernetes refreshed

Domain kubernetes-node-3 created from /tmp/tmp.MYRaUwdpBw

Vol kubernetes-master.img created

Pool kubernetes refreshed

Domain kubernetes-master created from /tmp/tmp.2C2JUbH993

Cluster "libvirt-coreos" set.
User "libvirt-coreos" set.
Context "libvirt-coreos" set.
Switched to context "libvirt-coreos".
Wrote config for libvirt-coreos to /home/core/.kube/config
Wait for cluster readiness
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3

--8<-- REPEATED MANY TIMES BUT CLUSTER IS NEVER READY--8<--

Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3
Nb ready nodes: 0 / 3

[core@kvmhost kubernetes]$ ssh 192.168.10.1

The authenticity of host '192.168.10.1 (192.168.10.1)' can't be established.
ECDSA key fingerprint is SHA256:yZ1rw8WV0ECdep3TDEydnznx8S5TagdvZEVoWtNaHK0.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.10.1' (ECDSA) to the list of known hosts.

Container Linux by CoreOS alpha (1313.0.0)
core@kubernetes-master ~ $ cd /opt/kubernetes/bin/
core@kubernetes-master /opt/kubernetes/bin $ ls -l

ls: cannot open directory '.': Permission denied

core@kubernetes-master /opt/kubernetes/bin $ cd ..
core@kubernetes-master /opt/kubernetes $ ls -l

total 16
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 addons
drwxr-x---. 2 core core 4096 Jan 12 05:08 bin
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 certs
drwxrwxr-x. 2 core core 4096 Feb 13 01:04 manifests

core@kubernetes-master /opt/kubernetes $ id

uid=500(core) gid=500(core) groups=500(core),10(wheel),233(docker),248(systemd-journal),250(portage),251(rkt),253(fleet) context=system_u:system_r:kernel_t:s0

core@kubernetes-master /opt/kubernetes $

[core@kvmhost bin]$ cd /mnt/largevol/Automation/images/kubernetes/
[core@kvmhost kubernetes]$ chmod -R a+rx bin
[core@kvmhost kubernetes]$ virsh reboot kubernetes-master
Domain kubernetes-master is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-1
Domain kubernetes-node-1 is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-2
Domain kubernetes-node-2 is being rebooted

[core@kvmhost kubernetes]$ virsh reboot kubernetes-node-3
Domain kubernetes-node-3 is being rebooted

[core@kvmhost kubernetes]$ kubectl cluster-info
Kubernetes master is running at http://192.168.10.1:8080
KubeDNS is running at http://192.168.10.1:8080/api/v1/proxy/namespaces/kube-system/services/kube-dns

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

为什么libvirt来宾看不到文件？这些文件由ID 500拥有，该ID与CoreOS用户的UID匹配
我做错了什么

我是由k8sport指导的，但我发现这个问题已经6个月了；这仍然是一个问题吗？1.5.2在“库伯内特斯时代”实际上是史前的；你用现代版本试过了吗？虽然我没有libvirt设置来测试，但当我看到uid权限不匹配时，我认为是“卷装载”；你有没有检查过那个目录是否是挂载（卷挂载、nfs挂载、其他奇怪的非本地目录类型的东西）？我是由k8sport指导的，但我看到这个问题已经6个月了；这仍然是一个问题吗？1.5.2在“库伯内特斯时代”实际上是史前的；你用现代版本试过了吗？虽然我没有libvirt设置来测试，但当我看到uid权限不匹配时，我认为是“卷装载”；您是否检查过该目录是否为装载（卷装载、nfs装载、其他奇怪的非本地目录类型）？