Kubernetes 运行kubelet时出错
我试图在fedora 24/lxc容器上启动kubelet,但出现了一个错误,似乎与libvirt/iptables有关 Docker(使用dnf/yum安装): Kubernetes(下载v1.3.3并提取tar): 启动、参数和错误:Kubernetes 运行kubelet时出错,kubernetes,lxc,Kubernetes,Lxc,我试图在fedora 24/lxc容器上启动kubelet,但出现了一个错误,似乎与libvirt/iptables有关 Docker(使用dnf/yum安装): Kubernetes(下载v1.3.3并提取tar): 启动、参数和错误: [root@node2 bin]# ./kubelet --address=0.0.0.0 --api-servers=http://master1:8080 --container-runtime=docker --hostname-override=nod
[root@node2 bin]# ./kubelet --address=0.0.0.0 --api-servers=http://master1:8080 --container-runtime=docker --hostname-override=node1 --port=10250
I0802 17:43:04.264454 2348 docker.go:327] Start docker client with request timeout=2m0s
W0802 17:43:04.271850 2348 server.go:487] Could not load kubeconfig file /var/lib/kubelet/kubeconfig: stat /var/lib/kubelet/kubeconfig: no such file or directory. Trying auth path instead.
W0802 17:43:04.271906 2348 server.go:448] Could not load kubernetes auth path /var/lib/kubelet/kubernetes_auth: stat /var/lib/kubelet/kubernetes_auth: no such file or directory. Continuing with defaults.
I0802 17:43:04.272241 2348 manager.go:138] cAdvisor running in container: "/"
W0802 17:43:04.275956 2348 manager.go:146] unable to connect to Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp 127.0.0.1:15441: getsockopt: connection refused
I0802 17:43:04.280283 2348 fs.go:139] Filesystem partitions: map[/dev/mapper/fedora_kg--fedora-root:{mountpoint:/ major:253 minor:0 fsType:ext4 blockSize:0}]
I0802 17:43:04.284868 2348 manager.go:192] Machine: {NumCores:4 CpuFrequency:3192789
MemoryCapacity:4125679616 MachineID:1e80444278b7442385a762b9545cec7b
SystemUUID:5EC24D56-9CA6-B237-EE21-E0899C3C16AB BootID:44212209-ff1d-4340-8433-11a93274d927
Filesystems:[{Device:/dev/mapper/fedora_kg--fedora-root
Capacity:52710469632 Type:vfs Inodes:3276800}]
DiskMap:map[8:0:{Name:sda Major:8 Minor:0 Size:85899345920 Scheduler:cfq}
253:0:{Name:dm-0 Major:253 Minor:0 Size:53687091200 Scheduler:none}
253:1:{Name:dm-1 Major:253 Minor:1 Size:4160749568 Scheduler:none}
253:2:{Name:dm-2 Major:253 Minor:2 Size:27518828544 Scheduler:none}
253:3:{Name:dm-3 Major:253 Minor:3 Size:107374182400 Scheduler:none}]
NetworkDevices:[
{Name:eth0 MacAddress:00:16:3e:b9:ce:f3 Speed:10000 Mtu:1500}
{Name:flannel.1 MacAddress:fa:ed:34:75:d6:1d Speed:0 Mtu:1450}]
Topology:[
{Id:0 Memory:4125679616
Cores:[{Id:0 Threads:[0]
Caches:[]} {Id:1 Threads:[1] Caches:[]}]
Caches:[{Size:8388608 Type:Unified Level:3}]}
{Id:1 Memory:0 Cores:[{Id:0 Threads:[2]
Caches:[]} {Id:1 Threads:[3] Caches:[]}]
Caches:[{Size:8388608 Type:Unified Level:3}]}]
CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
I0802 17:43:04.285649 2348 manager.go:198]
Version: {KernelVersion:4.6.4-301.fc24.x86_64 ContainerOsVersion:Fedora 24 (Twenty Four)
DockerVersion:1.12.0 CadvisorVersion: CadvisorRevision:}
I0802 17:43:04.286366 2348 server.go:768] Watching apiserver
W0802 17:43:04.286477 2348 kubelet.go:561] Hairpin mode set to "promiscuous-bridge" but configureCBR0 is false, falling back to "hairpin-veth"
I0802 17:43:04.286575 2348 kubelet.go:384] Hairpin mode set to "hairpin-veth"
W0802 17:43:04.303188 2348 plugins.go:170] can't set sysctl net/bridge/bridge-nf-call-iptables: open /proc/sys/net/bridge/bridge-nf-call-iptables: no such file or directory
I0802 17:43:04.307700 2348 docker_manager.go:235] Setting dockerRoot to /var/lib/docker
I0802 17:43:04.310175 2348 server.go:730] Started kubelet v1.3.3
E0802 17:43:04.311636 2348 kubelet.go:933] Image garbage collection failed: unable to find data for container /
E0802 17:43:04.312800 2348 kubelet.go:994] Failed to start ContainerManager [open /proc/sys/kernel/panic: read-only file system, open /proc/sys/kernel/panic_on_oops: read-only file system, open /proc/sys/vm/overcommit_memory: read-only file system]
I0802 17:43:04.312962 2348 status_manager.go:123] Starting to sync pod status with apiserver
I0802 17:43:04.313080 2348 kubelet.go:2468] Starting kubelet main sync loop.
I0802 17:43:04.313187 2348 kubelet.go:2477] skipping pod synchronization - [Failed to start ContainerManager [open /proc/sys/kernel/panic: read-only file system, open /proc/sys/kernel/panic_on_oops: read-only file system, open /proc/sys/vm/overcommit_memory: read-only file system] network state unknown container runtime is down]
I0802 17:43:04.313525 2348 server.go:117] Starting to listen on 0.0.0.0:10250
I0802 17:43:04.315021 2348 volume_manager.go:216] Starting Kubelet Volume Manager
I0802 17:43:04.325998 2348 factory.go:228] Registering Docker factory
E0802 17:43:04.326049 2348 manager.go:240] Registration of the rkt container factory failed: unable to communicate with Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp 127.0.0.1:15441: getsockopt: connection refused
I0802 17:43:04.326073 2348 factory.go:54] Registering systemd factory
I0802 17:43:04.326545 2348 factory.go:86] Registering Raw factory
I0802 17:43:04.326993 2348 manager.go:1072] Started watching for new ooms in manager
I0802 17:43:04.331164 2348 oomparser.go:185] oomparser using systemd
I0802 17:43:04.331904 2348 manager.go:281] Starting recovery of all containers
I0802 17:43:04.368958 2348 manager.go:286] Recovery completed
I0802 17:43:04.419959 2348 kubelet.go:1185] Node node1 was previously registered
I0802 17:43:09.313871 2348 kubelet.go:2477] skipping pod synchronization - [Failed to start ContainerManager [open /proc/sys/kernel/panic: read-only file system, open /proc/sys/kernel/panic_on_oops: read-only file system, open /proc/sys/vm/overcommit_memory: read-only file system]]
法兰绒(使用dnf/yum安装):
容器的LXC设置:
[root@kg-fedora node2]# cat config
# Template used to create this container: /usr/share/lxc/templates/lxc-fedora
# Parameters passed to the template:
# For additional config options, please look at lxc.container.conf(5)
# Uncomment the following line to support nesting containers:
#lxc.include = /usr/share/lxc/config/nesting.conf
# (Be aware this has security implications)
lxc.network.type = veth
lxc.network.link = virbr0
lxc.network.hwaddr = 00:16:3e:b9:ce:f3
lxc.network.flags = up
lxc.network.ipv4 = 192.168.122.23/24
lxc.network.ipv4.gateway = 192.168.80.2
# Include common configuration
lxc.include = /usr/share/lxc/config/fedora.common.conf
lxc.arch = x86_64
# When using LXC with apparmor, uncomment the next line to run unconfined:
#lxc.aa_profile = unconfined
# example simple networking setup, uncomment to enable
#lxc.network.type = veth
#lxc.network.flags = up
#lxc.network.link = lxcbr0
#lxc.network.name = eth0
# Additional example for veth network type
# static MAC address,
#lxc.network.hwaddr = 00:16:3e:77:52:20
# persistent veth device name on host side
# Note: This may potentially collide with other containers of same name!
#lxc.network.veth.pair = v-fedora-template-e0
lxc.cgroup.devices.allow = a
lxc.cap.drop =
lxc.rootfs = /var/lib/lxc/node2/rootfs
lxc.rootfs.backend = dir
lxc.utsname = node2
libvirt-1.3.3.2-1.fc24.x86_64:
[root@kg-fedora node2]# systemctl status libvirtd
● libvirtd.service - Virtualization daemon
Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2016-07-29 16:33:09 EDT; 3 days ago
Docs: man:libvirtd(8)
http://libvirt.org
Main PID: 1191 (libvirtd)
Tasks: 18 (limit: 512)
Memory: 7.3M
CPU: 9.108s
CGroup: /system.slice/libvirtd.service
├─1191 /usr/sbin/libvirtd
├─1597 /sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
└─1599 /sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
法兰绒/码头工人配置:
[root@node2 ~]# systemctl stop docker
[root@node2 ~]# ip link delete docker0
[root@node2 ~]# systemctl start docker
[root@node2 ~]# ip -4 a|grep inet
inet 127.0.0.1/8 scope host lo
inet 10.100.72.0/16 scope global flannel.1
inet 172.17.0.1/16 scope global docker0
inet 192.168.122.23/24 brd 192.168.122.255 scope global dynamic eth0
请注意,docker0接口与flannel.1接口使用的ip范围不同
任何指点都将不胜感激 对于可能寻求此问题解决方案的任何人: 因为您使用的是LXC,所以需要确保所讨论的文件系统是作为rw安装的。这需要在LXC的配置文件中指定以下选项: raw.lxc:“lxc.apparmor.profile=unconfined\nlxc.cap.drop=\nlxc.cgroup.devices.allow=a\nlxc.mount.auto=proc:rw sys:rw” 或者只是 lxc.mount.auto:proc:rw sys:rw 以下是参考资料:
SElinux是否可能会干扰?
getenforce
->已禁用
<代码>sestatus->SELinux状态:已禁用
[root@kg-fedora node2]# cat config
# Template used to create this container: /usr/share/lxc/templates/lxc-fedora
# Parameters passed to the template:
# For additional config options, please look at lxc.container.conf(5)
# Uncomment the following line to support nesting containers:
#lxc.include = /usr/share/lxc/config/nesting.conf
# (Be aware this has security implications)
lxc.network.type = veth
lxc.network.link = virbr0
lxc.network.hwaddr = 00:16:3e:b9:ce:f3
lxc.network.flags = up
lxc.network.ipv4 = 192.168.122.23/24
lxc.network.ipv4.gateway = 192.168.80.2
# Include common configuration
lxc.include = /usr/share/lxc/config/fedora.common.conf
lxc.arch = x86_64
# When using LXC with apparmor, uncomment the next line to run unconfined:
#lxc.aa_profile = unconfined
# example simple networking setup, uncomment to enable
#lxc.network.type = veth
#lxc.network.flags = up
#lxc.network.link = lxcbr0
#lxc.network.name = eth0
# Additional example for veth network type
# static MAC address,
#lxc.network.hwaddr = 00:16:3e:77:52:20
# persistent veth device name on host side
# Note: This may potentially collide with other containers of same name!
#lxc.network.veth.pair = v-fedora-template-e0
lxc.cgroup.devices.allow = a
lxc.cap.drop =
lxc.rootfs = /var/lib/lxc/node2/rootfs
lxc.rootfs.backend = dir
lxc.utsname = node2
[root@kg-fedora node2]# systemctl status libvirtd
● libvirtd.service - Virtualization daemon
Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2016-07-29 16:33:09 EDT; 3 days ago
Docs: man:libvirtd(8)
http://libvirt.org
Main PID: 1191 (libvirtd)
Tasks: 18 (limit: 512)
Memory: 7.3M
CPU: 9.108s
CGroup: /system.slice/libvirtd.service
├─1191 /usr/sbin/libvirtd
├─1597 /sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
└─1599 /sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
[root@node2 ~]# systemctl stop docker
[root@node2 ~]# ip link delete docker0
[root@node2 ~]# systemctl start docker
[root@node2 ~]# ip -4 a|grep inet
inet 127.0.0.1/8 scope host lo
inet 10.100.72.0/16 scope global flannel.1
inet 172.17.0.1/16 scope global docker0
inet 192.168.122.23/24 brd 192.168.122.255 scope global dynamic eth0