Kubernetes 无法从pod内部ping ClusterIP，DNS不适用于外部域，如google.com_Kubernetes_Skydns

Kubernetes 无法从pod内部ping ClusterIP，DNS不适用于外部域，如google.com

kubernetes

Kubernetes 无法从pod内部ping ClusterIP，DNS不适用于外部域，如google.com,kubernetes,skydns,Kubernetes,Skydns,我已经在裸机/Ubuntu上安装了Kubernetes。我在git中提交。我使用了cd-kubernetes/cluster；KUBERNETES_PROVIDER=ubuntu./kube up.sh后跟cd KUBERNETES/cluster/ubuntu/deployAddons.sh以启动集群。一切顺利，集群启动了我的/ubuntu/config default.sh如下： # Define all your cluster nodes, MASTER node comes firs

我已经在裸机/Ubuntu上安装了Kubernetes。我在git中提交。我使用了

cd-kubernetes/cluster；KUBERNETES_PROVIDER=ubuntu./kube up.sh

后跟

cd KUBERNETES/cluster/ubuntu/deployAddons.sh

以启动集群。一切顺利，集群启动了

我的

/ubuntu/config default.sh

如下：

# Define all your cluster nodes, MASTER node comes first"
# And separated with blank space like <user_1@ip_1> <user_2@ip_2> <user_3@ip_3> 
export nodes=${nodes:-"root@192.168.48.170 root@192.168.48.180"}

# Define all your nodes role: a(master) or i(minion) or ai(both master and minion), must be the order same 
role=${role:-"ai i"}
# If it practically impossible to set an array as an environment variable
# from a script, so assume variable is a string then convert it to an array
export roles=($role)

# Define minion numbers
export NUM_NODES=${NUM_NODES:-2}
# define the IP range used for service cluster IPs.
# according to rfc 1918 ref: https://tools.ietf.org/html/rfc1918 choose a private ip range here.
export SERVICE_CLUSTER_IP_RANGE=${SERVICE_CLUSTER_IP_RANGE:-192.168.3.0/24}  # formerly PORTAL_NET
# define the IP range used for flannel overlay network, should not conflict with above SERVICE_CLUSTER_IP_RANGE
export FLANNEL_NET=${FLANNEL_NET:-172.16.0.0/16}

# Optionally add other contents to the Flannel configuration JSON
# object normally stored in etcd as /coreos.com/network/config.  Use
# JSON syntax suitable for insertion into a JSON object constructor
# after other field name:value pairs.  For example:
# FLANNEL_OTHER_NET_CONFIG=', "SubnetMin": "172.16.10.0", "SubnetMax": "172.16.90.0"'

export FLANNEL_OTHER_NET_CONFIG
FLANNEL_OTHER_NET_CONFIG=''

# Admission Controllers to invoke prior to persisting objects in cluster
export ADMISSION_CONTROL=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,SecurityContextDeny

# Path to the config file or directory of files of kubelet
export KUBELET_CONFIG=${KUBELET_CONFIG:-""}

# A port range to reserve for services with NodePort visibility
SERVICE_NODE_PORT_RANGE=${SERVICE_NODE_PORT_RANGE:-"30000-32767"}

# Optional: Enable node logging.
ENABLE_NODE_LOGGING=false
LOGGING_DESTINATION=${LOGGING_DESTINATION:-elasticsearch}

# Optional: When set to true, Elasticsearch and Kibana will be setup as part of the cluster bring up.
ENABLE_CLUSTER_LOGGING=false
ELASTICSEARCH_LOGGING_REPLICAS=${ELASTICSEARCH_LOGGING_REPLICAS:-1}

# Optional: When set to true, heapster, Influxdb and Grafana will be setup as part of the cluster bring up.
ENABLE_CLUSTER_MONITORING="${KUBE_ENABLE_CLUSTER_MONITORING:-true}"

# Extra options to set on the Docker command line.  This is useful for setting
# --insecure-registry for local registries.
DOCKER_OPTS=${DOCKER_OPTS:-""}

# Extra options to set on the kube-proxy command line.  This is useful
# for selecting the iptables proxy-mode, for example.
KUBE_PROXY_EXTRA_OPTS=${KUBE_PROXY_EXTRA_OPTS:-""}

# Optional: Install cluster DNS.
ENABLE_CLUSTER_DNS="${KUBE_ENABLE_CLUSTER_DNS:-true}"
# DNS_SERVER_IP must be a IP in SERVICE_CLUSTER_IP_RANGE
DNS_SERVER_IP=${DNS_SERVER_IP:-"192.168.3.10"}
DNS_DOMAIN=${DNS_DOMAIN:-"cluster.local"}
DNS_REPLICAS=${DNS_REPLICAS:-1}

# Optional: Install Kubernetes UI
ENABLE_CLUSTER_UI="${KUBE_ENABLE_CLUSTER_UI:-true}"

# Optional: Enable setting flags for kube-apiserver to turn on behavior in active-dev
RUNTIME_CONFIG="--basic-auth-file=password.csv"

# Optional: Add http or https proxy when download easy-rsa.
# Add envitonment variable separated with blank space like "http_proxy=http://10.x.x.x:8080 https_proxy=https://10.x.x.x:8443"
PROXY_SETTING=${PROXY_SETTING:-""}

DEBUG=${DEBUG:-"false"}

nameserver 8.8.8.8
nameserver 127.0.1.1

search default.svc.cluster.local svc.cluster.local cluster.local
nameserver 192.168.3.10
nameserver 8.8.8.8
nameserver 127.0.1.1
options ndots:5

以及使用以下yml的服务：

apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  ports:
  - port: 8000
    targetPort: 80
    protocol: TCP
  selector:
    app: nginx
  type: NodePort

然后，我使用

docker exec-it[container\u ID]bash

进入启动的集装箱码头。主要有两个问题:

我不能ping像google.com这样的外部域名，但我可以ping像8.8.8.8这样的外部IP。因此，容器可以访问internet

内部服务解析为更正内部集群，但我无法从容器内部ping该IP

主机的

/etc/resolve.conf

文件如下：

# Define all your cluster nodes, MASTER node comes first"
# And separated with blank space like <user_1@ip_1> <user_2@ip_2> <user_3@ip_3> 
export nodes=${nodes:-"root@192.168.48.170 root@192.168.48.180"}

# Define all your nodes role: a(master) or i(minion) or ai(both master and minion), must be the order same 
role=${role:-"ai i"}
# If it practically impossible to set an array as an environment variable
# from a script, so assume variable is a string then convert it to an array
export roles=($role)

# Define minion numbers
export NUM_NODES=${NUM_NODES:-2}
# define the IP range used for service cluster IPs.
# according to rfc 1918 ref: https://tools.ietf.org/html/rfc1918 choose a private ip range here.
export SERVICE_CLUSTER_IP_RANGE=${SERVICE_CLUSTER_IP_RANGE:-192.168.3.0/24}  # formerly PORTAL_NET
# define the IP range used for flannel overlay network, should not conflict with above SERVICE_CLUSTER_IP_RANGE
export FLANNEL_NET=${FLANNEL_NET:-172.16.0.0/16}

# Optionally add other contents to the Flannel configuration JSON
# object normally stored in etcd as /coreos.com/network/config.  Use
# JSON syntax suitable for insertion into a JSON object constructor
# after other field name:value pairs.  For example:
# FLANNEL_OTHER_NET_CONFIG=', "SubnetMin": "172.16.10.0", "SubnetMax": "172.16.90.0"'

export FLANNEL_OTHER_NET_CONFIG
FLANNEL_OTHER_NET_CONFIG=''

# Admission Controllers to invoke prior to persisting objects in cluster
export ADMISSION_CONTROL=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,SecurityContextDeny

# Path to the config file or directory of files of kubelet
export KUBELET_CONFIG=${KUBELET_CONFIG:-""}

# A port range to reserve for services with NodePort visibility
SERVICE_NODE_PORT_RANGE=${SERVICE_NODE_PORT_RANGE:-"30000-32767"}

# Optional: Enable node logging.
ENABLE_NODE_LOGGING=false
LOGGING_DESTINATION=${LOGGING_DESTINATION:-elasticsearch}

# Optional: When set to true, Elasticsearch and Kibana will be setup as part of the cluster bring up.
ENABLE_CLUSTER_LOGGING=false
ELASTICSEARCH_LOGGING_REPLICAS=${ELASTICSEARCH_LOGGING_REPLICAS:-1}

# Optional: When set to true, heapster, Influxdb and Grafana will be setup as part of the cluster bring up.
ENABLE_CLUSTER_MONITORING="${KUBE_ENABLE_CLUSTER_MONITORING:-true}"

# Extra options to set on the Docker command line.  This is useful for setting
# --insecure-registry for local registries.
DOCKER_OPTS=${DOCKER_OPTS:-""}

# Extra options to set on the kube-proxy command line.  This is useful
# for selecting the iptables proxy-mode, for example.
KUBE_PROXY_EXTRA_OPTS=${KUBE_PROXY_EXTRA_OPTS:-""}

# Optional: Install cluster DNS.
ENABLE_CLUSTER_DNS="${KUBE_ENABLE_CLUSTER_DNS:-true}"
# DNS_SERVER_IP must be a IP in SERVICE_CLUSTER_IP_RANGE
DNS_SERVER_IP=${DNS_SERVER_IP:-"192.168.3.10"}
DNS_DOMAIN=${DNS_DOMAIN:-"cluster.local"}
DNS_REPLICAS=${DNS_REPLICAS:-1}

# Optional: Install Kubernetes UI
ENABLE_CLUSTER_UI="${KUBE_ENABLE_CLUSTER_UI:-true}"

# Optional: Enable setting flags for kube-apiserver to turn on behavior in active-dev
RUNTIME_CONFIG="--basic-auth-file=password.csv"

# Optional: Add http or https proxy when download easy-rsa.
# Add envitonment variable separated with blank space like "http_proxy=http://10.x.x.x:8080 https_proxy=https://10.x.x.x:8443"
PROXY_SETTING=${PROXY_SETTING:-""}

DEBUG=${DEBUG:-"false"}

nameserver 8.8.8.8
nameserver 127.0.1.1

search default.svc.cluster.local svc.cluster.local cluster.local
nameserver 192.168.3.10
nameserver 8.8.8.8
nameserver 127.0.1.1
options ndots:5

容器的

/etc/resolve.conf

文件如下：

# Define all your cluster nodes, MASTER node comes first"
# And separated with blank space like <user_1@ip_1> <user_2@ip_2> <user_3@ip_3> 
export nodes=${nodes:-"root@192.168.48.170 root@192.168.48.180"}

# Define all your nodes role: a(master) or i(minion) or ai(both master and minion), must be the order same 
role=${role:-"ai i"}
# If it practically impossible to set an array as an environment variable
# from a script, so assume variable is a string then convert it to an array
export roles=($role)

# Define minion numbers
export NUM_NODES=${NUM_NODES:-2}
# define the IP range used for service cluster IPs.
# according to rfc 1918 ref: https://tools.ietf.org/html/rfc1918 choose a private ip range here.
export SERVICE_CLUSTER_IP_RANGE=${SERVICE_CLUSTER_IP_RANGE:-192.168.3.0/24}  # formerly PORTAL_NET
# define the IP range used for flannel overlay network, should not conflict with above SERVICE_CLUSTER_IP_RANGE
export FLANNEL_NET=${FLANNEL_NET:-172.16.0.0/16}

# Optionally add other contents to the Flannel configuration JSON
# object normally stored in etcd as /coreos.com/network/config.  Use
# JSON syntax suitable for insertion into a JSON object constructor
# after other field name:value pairs.  For example:
# FLANNEL_OTHER_NET_CONFIG=', "SubnetMin": "172.16.10.0", "SubnetMax": "172.16.90.0"'

export FLANNEL_OTHER_NET_CONFIG
FLANNEL_OTHER_NET_CONFIG=''

# Admission Controllers to invoke prior to persisting objects in cluster
export ADMISSION_CONTROL=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,SecurityContextDeny

# Path to the config file or directory of files of kubelet
export KUBELET_CONFIG=${KUBELET_CONFIG:-""}

# A port range to reserve for services with NodePort visibility
SERVICE_NODE_PORT_RANGE=${SERVICE_NODE_PORT_RANGE:-"30000-32767"}

# Optional: Enable node logging.
ENABLE_NODE_LOGGING=false
LOGGING_DESTINATION=${LOGGING_DESTINATION:-elasticsearch}

# Optional: When set to true, Elasticsearch and Kibana will be setup as part of the cluster bring up.
ENABLE_CLUSTER_LOGGING=false
ELASTICSEARCH_LOGGING_REPLICAS=${ELASTICSEARCH_LOGGING_REPLICAS:-1}

# Optional: When set to true, heapster, Influxdb and Grafana will be setup as part of the cluster bring up.
ENABLE_CLUSTER_MONITORING="${KUBE_ENABLE_CLUSTER_MONITORING:-true}"

# Extra options to set on the Docker command line.  This is useful for setting
# --insecure-registry for local registries.
DOCKER_OPTS=${DOCKER_OPTS:-""}

# Extra options to set on the kube-proxy command line.  This is useful
# for selecting the iptables proxy-mode, for example.
KUBE_PROXY_EXTRA_OPTS=${KUBE_PROXY_EXTRA_OPTS:-""}

# Optional: Install cluster DNS.
ENABLE_CLUSTER_DNS="${KUBE_ENABLE_CLUSTER_DNS:-true}"
# DNS_SERVER_IP must be a IP in SERVICE_CLUSTER_IP_RANGE
DNS_SERVER_IP=${DNS_SERVER_IP:-"192.168.3.10"}
DNS_DOMAIN=${DNS_DOMAIN:-"cluster.local"}
DNS_REPLICAS=${DNS_REPLICAS:-1}

# Optional: Install Kubernetes UI
ENABLE_CLUSTER_UI="${KUBE_ENABLE_CLUSTER_UI:-true}"

# Optional: Enable setting flags for kube-apiserver to turn on behavior in active-dev
RUNTIME_CONFIG="--basic-auth-file=password.csv"

# Optional: Add http or https proxy when download easy-rsa.
# Add envitonment variable separated with blank space like "http_proxy=http://10.x.x.x:8080 https_proxy=https://10.x.x.x:8443"
PROXY_SETTING=${PROXY_SETTING:-""}

DEBUG=${DEBUG:-"false"}

nameserver 8.8.8.8
nameserver 127.0.1.1

search default.svc.cluster.local svc.cluster.local cluster.local
nameserver 192.168.3.10
nameserver 8.8.8.8
nameserver 127.0.1.1
options ndots:5

关于第一个问题，我认为可能与SkyDNS名称服务器的错误配置有关，也可能与我必须进行的自定义配置有关，但我不知道

然而，我不知道为什么容器不能ping集群

有解决办法吗？

我找到了一个解决办法。命令行参数部分中的SkyDNS文档，特别是“名称服务器”参数意味着：

名称服务器：将DNS请求转发到这些（递归）名称服务器（IP:端口组合的数组），如果对域没有权威性。这默认为/etc/resolv.conf中列出的服务器

但事实并非如此！要解决此问题，应将dns加载项复制控制器配置文件（cluster/addons/dns/skydns rc.yaml.in）更改为包含名称服务器配置。我改变了skydns容器的部分如下，它就像一个魅力

  - name: skydns
    image: gcr.io/google_containers/skydns:2015-10-13-8c72f8c
    resources:
      # keep request = limit to keep this container in guaranteed class
      limits:
        cpu: 100m
        memory: 50Mi
      requests:
        cpu: 100m
        memory: 50Mi
    args:
    # command = "/skydns"
    - -machines=http://127.0.0.1:4001
    - -addr=0.0.0.0:53
    - -nameservers=8.8.8.8:53
    - -ns-rotate=false
    - -domain={{ pillar['dns_domain'] }}.
    ports:
    - containerPort: 53
      name: dns
      protocol: UDP
    - containerPort: 53
      name: dns-tcp
      protocol: TCP
    livenessProbe:
      httpGet:
        path: /healthz
        port: 8080
        scheme: HTTP
      initialDelaySeconds: 30
      timeoutSeconds: 5
    readinessProbe:
      httpGet:
        path: /healthz
        port: 8080
        scheme: HTTP
      initialDelaySeconds: 1
      timeoutSeconds: 5

我可以回答你的

ping clusterIP

问题。我遇到了同样的问题，想从Pod ping服务的集群IP

解决方案似乎是无法ping集群IP，但可以使用curl with port访问端点

我只是想找到有关ping虚拟IP的详细信息。

另一种处理DNS相同问题的方法是在configMap中设置上游服务器：

apiVersion: v1
    kind: ConfigMap
    metadata:
      name: kube-dns
      namespace: kube-system
    data:
        upstreamNameservers: |
        ["8.8.8.8", "8.8.4.4"]

如果服务使用iptables来实现，那么集群不能ping，因为iptables只允许tcp数据包。但是，当您卷曲clusterIP+端口时，iptables规则会将此tcp数据包发送到pod

#ping 10.96.229.40
PING 10.96.229.40 (10.96.229.40) 56(84) bytes of data.
^C
--- 10.96.229.40 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms


#iptables-save |grep 10.96.229.40
-A KUBE-SERVICES -d 10.96.229.40/32 -p tcp -m comment --comment "***-service:https has no endpoints" -m tcp --dport 8443 -j REJECT --reject-with icmp-port-unreachable

如果服务使用IPV，则可以ping clusterIP。但是本地环回设备发送的响应，因为kube代理向lo添加了路由规则

# ip route get 10.68.155.139
local 10.68.155.139 dev lo src 10.68.155.139 
    cache <local> 
# ping -c 1 10.68.155.139
PING 10.68.155.139 (10.68.155.139) 56(84) bytes of data.
64 bytes from 10.68.155.139: icmp_seq=1 ttl=64 time=0.045 ms

--- 10.68.155.139 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.045/0.045/0.045/0.000 ms

#ip路由获取10.68.155.139
本地10.68.155.139 dev lo src 10.68.155.139
隐藏物
#平-C110.68.155.139
PING 10.68.155.139（10.68.155.139）56（84）字节的数据。
10.68.155.139中的64字节：icmp_seq=1 ttl=64时间=0.045毫秒
---10.68.155.139平统计---
发送1个数据包，接收1个，0%数据包丢失，时间0ms
rtt最小值/平均值/最大值/mdev=0.045/0.045/0.045/0.000毫秒

事实证明，流量路由是使用基于

（ip，端口）

对的

iptables来完成的。因此，虚拟IP不可ping，但可以通过该特定端口访问。DNS问题仍然存在。较新版本（现在称为kube DNS）显然不再支持名称服务器选项。仍在试图找出如何让kube dns转发到上游名称服务器我对当前的kubernetes和kube dns也有同样的问题。