elasticsearch elasticsearch StatefolSet pod无法装入卷
我使用持久卷在EKS(
elasticsearch elasticsearch StatefolSet pod无法装入卷,
elasticsearch,kubernetes,amazon-eks,
elasticsearch,Kubernetes,Amazon Eks,我使用持久卷在EKS(服务器版本:v1.13.7-EKS-c57ff8)上运行了由3个节点组成的Elasticsearch集群(StatefulSet) 我将EKS集群从1.12升级到1.13,成功。但其中一个elasticsearch群集节点未能启动并卡在init状态: NAME READY STATUS RESTARTS AGE es-master-0 0/1
服务器版本:v1.13.7-EKS-c57ff8
)上运行了由3个节点组成的Elasticsearch集群(StatefulSet
)
我将EKS集群从1.12
升级到1.13
,成功。但其中一个elasticsearch群集节点未能启动并卡在init
状态:
NAME READY STATUS RESTARTS AGE
es-master-0 0/1 Init:0/3 0 15h
es-master-1 1/1 Running 0 44h
es-master-2 1/1 Running 0 44h
我试图杀死吊舱,但是新的吊舱又卡在了同样的状态
当我检查pod部署时(kubectl description pod es-master-0
),我注意到pod无法装载卷:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 2m13s default-scheduler Successfully assigned kube-logging/es-master-0 to ip-10-2-18-16.us-west-2.compute.internal
Normal SuccessfulAttachVolume 2m10s attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-f2e27430-af11-11e9-b10d-02a8eba067e2"
Warning FailedMount 10s kubelet, ip-10-2-18-16.us-west-2.compute.internal Unable to mount volumes for pod "es-master-0_kube-logging(bc27e29c-b539-11e9-9958-06eeabb0603e)": timeout expired waiting for volumes to attach or mount for pod "kube-logging"/"es-master-0". list of unmounted volumes=[data]. list of unattached volumes=[data default-token-bz6w9]
kubectl get pv的输出
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-06cd5cfe-af12-11e9-b10d-02a8eba067e2 100Gi RWO Retain Bound kube-logging/data-es-master-1 aws-gp2 7d19h
pvc-178b5aba-af12-11e9-b10d-02a8eba067e2 100Gi RWO Retain Bound kube-logging/data-es-master-2 aws-gp2 7d19h
pvc-f2e27430-af11-11e9-b10d-02a8eba067e2 100Gi RWO Retain Bound kube-logging/data-es-master-0 aws-gp2 7d19h
kubectl get pvc的输出
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
data-es-master-0 Bound pvc-f2e27430-af11-11e9-b10d-02a8eba067e2 100Gi RWO aws-gp2 7d19h
data-es-master-1 Bound pvc-06cd5cfe-af12-11e9-b10d-02a8eba067e2 100Gi RWO aws-gp2 7d19h
data-es-master-2 Bound pvc-178b5aba-af12-11e9-b10d-02a8eba067e2 100Gi RWO aws-gp2 7d19h
我还尝试重新启动该pod所在的节点
这是我的清单文件:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: es-master
namespace: kube-logging
spec:
serviceName: elasticsearch
replicas: 3
selector:
matchLabels:
app: elasticsearch
template:
metadata:
labels:
app: elasticsearch
spec:
containers:
- name: elasticsearch
image: docker.elastic.co/elasticsearch/elasticsearch:7.2.0
resources:
limits:
cpu: 1000m
memory: 2.5G
requests:
cpu: 100m
ports:
- containerPort: 9200
name: rest
protocol: TCP
- containerPort: 9300
name: inter-node
protocol: TCP
volumeMounts:
- name: data
mountPath: /usr/share/elasticsearch/data
env:
- name: cluster.name
value: prod-eks-logs
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: node.name
value: "$(NODE_NAME).elasticsearch"
- name: discovery.zen.ping.unicast.hosts
value: "es-master-0.elasticsearch,es-master-1.elasticsearch,es-master-2.elasticsearch"
- name: cluster.initial_master_nodes
value: "es-master-0.elasticsearch,es-master-1.elasticsearch,es-master-2.elasticsearch"
- name: discovery.zen.minimum_master_nodes
value: "2"
- name: ES_JAVA_OPTS
value: "-Xmx1g -Xmx1g"
initContainers:
- name: fix-permissions
image: busybox
command: ["sh", "-c", "chown -R 1000:1000 /usr/share/elasticsearch/data"]
securityContext:
privileged: true
volumeMounts:
- name: data
mountPath: /usr/share/elasticsearch/data
- name: increase-vm-max-map
image: busybox
command: ["sysctl", "-w", "vm.max_map_count=262144"]
securityContext:
privileged: true
- name: increase-fd-ulimit
image: busybox
command: ["sh", "-c", "ulimit -n 65536"]
securityContext:
privileged: true
volumeClaimTemplates:
- metadata:
name: data
labels:
app: elasticsearch
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: aws-gp2
resources:
requests:
storage: 100Gi
关于如何通过elasticsearch状态有什么帮助吗