elasticsearch filebeat无法连接到elasticsearch
我已经在Kubernetes(EKS)上运行了elasticsearch filebeat无法连接到elasticsearch,elasticsearch,kubernetes,filebeat,elasticsearch,Kubernetes,Filebeat,我已经在Kubernetes(EKS)上运行了Elasticsearch,在Kubernetes上运行了filebeat 现在我正试图从其他EC2机器(EKS之外)获取日志,因此在EC2上安装了确切版本的filebeat,并将其配置为将日志发送到Kubernetes上运行的Elasticsearch 但无法在Elasticsearch(Kibana)中查看任何日志。这是filebeat的日志 2019-08-26T18:18:16.005Z INFO instance/beat.go
Elasticsearch
,在Kubernetes上运行了filebeat
现在我正试图从其他EC2
机器(EKS之外)获取日志,因此在EC2
上安装了确切版本的filebeat
,并将其配置为将日志发送到Kubernetes上运行的Elasticsearch
但无法在Elasticsearch(Kibana)中查看任何日志。这是filebeat的日志
2019-08-26T18:18:16.005Z INFO instance/beat.go:292 Setup Beat: filebeat; Version: 7.2.1
2019-08-26T18:18:16.005Z INFO [index-management] idxmgmt/std.go:178 Set output.elasticsearch.index to 'filebeat-7.2.1' as ILM is enabled.
2019-08-26T18:18:16.005Z INFO elasticsearch/client.go:166 Elasticsearch url: http://elasticsearch.dev.domain.net:9200
2019-08-26T18:18:16.005Z INFO add_cloud_metadata/add_cloud_metadata.go:351 add_cloud_metadata: hosting provider type detected as aws, metadata={"availability_zone":"us-west-2a","instance":{"id":"i-0185e1d68306f95b4"},"machine":{"type":"t2.medium"},"provider":"aws","region":"us-west-2"}
2019-08-26T18:18:16.005Z INFO [publisher] pipeline/module.go:97 Beat name: dev-web1
2019-08-26T18:18:16.006Z INFO elasticsearch/client.go:166 Elasticsearch url: http://elasticsearch.dev.domain.net:9200
日志中没有多少信息
然后我注意到:
root@dev-web1:~# sudo systemctl status filebeat
● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
Loaded: loaded (/lib/systemd/system/filebeat.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2019-08-26 18:18:47 UTC; 18min ago
Docs: https://www.elastic.co/products/beats/filebeat
Main PID: 7768 (filebeat)
CGroup: /system.slice/filebeat.service
└─7768 /usr/share/filebeat/bin/filebeat -e -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs
Aug 26 18:35:38 dev-web1 filebeat[7768]: 2019-08-26T18:35:38.156Z ERROR pipeline/output.go:100 Failed to connect to backoff(elasticsearch(http://elasticsear
Aug 26 18:35:38 dev-web1 filebeat[7768]: 2019-08-26T18:35:38.156Z INFO pipeline/output.go:93 Attempting to reconnect to backoff(elasticsearch(http://elastic
Aug 26 18:35:38 dev-web1 filebeat[7768]: 2019-08-26T18:35:38.156Z INFO [publisher] pipeline/retry.go:189 retryer: send unwait-signal to consumer
Aug 26 18:35:38 dev-web1 filebeat[7768]: 2019-08-26T18:35:38.157Z INFO [publisher] pipeline/retry.go:191 done
Aug 26 18:35:38 dev-web1 filebeat[7768]: 2019-08-26T18:35:38.157Z INFO [publisher] pipeline/retry.go:166 retryer: send wait signal to consumer
Aug 26 18:35:38 dev-web1 filebeat[7768]: 2019-08-26T18:35:38.157Z INFO [publisher] pipeline/retry.go:168 done
Aug 26 18:35:47 dev-web1 filebeat[7768]: 2019-08-26T18:35:47.028Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitori
Aug 26 18:36:17 dev-web1 filebeat[7768]: 2019-08-26T18:36:17.028Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitori
root@dev-web1:~#
但我无法阅读上述状态消息中的完整行
所以我试着:
root@dev-web1:~# curl elasticsearch.dev.domain.net/_cat/health
1566844775 18:39:35 dev-eks-logs green 3 3 48 24 0 0 0 0 - 100.0%
root@dev-web1:~#
这对port有效,但对port无效
root@dev-web1:~# curl elasticsearch.dev.domain.net:9200/_cat/health
filebeat具有以下配置
output.elasticsearch:
hosts: ["elasticsearch.dev.domain.net"]
username: "elastic"
password: "changeme"
如何在filebeat端修复此问题
远程登录测试:
root@dev-web1:~# telnet <ip> 5044
Trying <ip>...
telnet: Unable to connect to remote host: Connection refused
root@dev-web1:~# telnet localhost 5044
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
root@dev-web1:~#
root@dev-web1:~#telnet 5044
尝试
telnet:无法连接到远程主机:连接被拒绝
root@dev-web1:~#telnet本地主机5044
正在尝试127.0.0.1。。。
telnet:无法连接到远程主机:连接被拒绝
root@dev-web1:~#
我认为是网络问题,检查telnet到本地主机/IP 5044我认为是网络问题,检查telnet到本地主机/IP 5044说:
主机
…如果未指定端口,则使用9200
在filbeat配置中添加主机:[“elasticsearch.dev.domain.net:80”]可以解决此问题。说明:
主机
…如果未指定端口,则使用9200
在filbeat配置中添加
主机:[“elasticsearch.dev.domain.net:80”]
应该可以解决问题。我认为问题在于端口未打开……如果您使用linux,只需将iptables-F(如果是测试环境)root@dev-web2:~#sudo ufw状态:不活动,不活动。请稍候,ufw只是一个接口…你需要开放端口如何将端口5044
与filebeat
相关?你应该阅读这篇文章…对我来说非常有用,直接点5我认为问题是端口没有开放…如果你使用linux,只需放置iptables-F(如果是测试环境)root@dev-web2:~#sudo ufw status status:inactive它不是active。等等,ufw只是一个接口…你需要打开端口5044
与filebeat
相关吗?你应该读一下这篇文章…对我来说,第5点非常有用