elasticsearch filebeat无法连接到elasticsearch,elasticsearch,kubernetes,filebeat,elasticsearch,Kubernetes,Filebeat" /> elasticsearch filebeat无法连接到elasticsearch,elasticsearch,kubernetes,filebeat,elasticsearch,Kubernetes,Filebeat" />

elasticsearch filebeat无法连接到elasticsearch

elasticsearch filebeat无法连接到elasticsearch,elasticsearch,kubernetes,filebeat,elasticsearch,Kubernetes,Filebeat,我已经在Kubernetes(EKS)上运行了Elasticsearch,在Kubernetes上运行了filebeat 现在我正试图从其他EC2机器(EKS之外)获取日志,因此在EC2上安装了确切版本的filebeat,并将其配置为将日志发送到Kubernetes上运行的Elasticsearch 但无法在Elasticsearch(Kibana)中查看任何日志。这是filebeat的日志 2019-08-26T18:18:16.005Z INFO instance/beat.go

我已经在Kubernetes(EKS)上运行了
Elasticsearch
,在Kubernetes上运行了
filebeat

现在我正试图从其他
EC2
机器(EKS之外)获取日志,因此在
EC2
上安装了确切版本的
filebeat
,并将其配置为将日志发送到Kubernetes上运行的
Elasticsearch

但无法在Elasticsearch(Kibana)中查看任何日志。这是filebeat的日志

2019-08-26T18:18:16.005Z    INFO    instance/beat.go:292    Setup Beat: filebeat; Version: 7.2.1
2019-08-26T18:18:16.005Z    INFO    [index-management]  idxmgmt/std.go:178  Set output.elasticsearch.index to 'filebeat-7.2.1' as ILM is enabled.
2019-08-26T18:18:16.005Z    INFO    elasticsearch/client.go:166 Elasticsearch url: http://elasticsearch.dev.domain.net:9200
2019-08-26T18:18:16.005Z    INFO    add_cloud_metadata/add_cloud_metadata.go:351    add_cloud_metadata: hosting provider type detected as aws, metadata={"availability_zone":"us-west-2a","instance":{"id":"i-0185e1d68306f95b4"},"machine":{"type":"t2.medium"},"provider":"aws","region":"us-west-2"}
2019-08-26T18:18:16.005Z    INFO    [publisher] pipeline/module.go:97   Beat name: dev-web1
2019-08-26T18:18:16.006Z    INFO    elasticsearch/client.go:166 Elasticsearch url: http://elasticsearch.dev.domain.net:9200
日志中没有多少信息

然后我注意到:

root@dev-web1:~# sudo systemctl status filebeat
● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
   Loaded: loaded (/lib/systemd/system/filebeat.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2019-08-26 18:18:47 UTC; 18min ago
     Docs: https://www.elastic.co/products/beats/filebeat
 Main PID: 7768 (filebeat)
   CGroup: /system.slice/filebeat.service
           └─7768 /usr/share/filebeat/bin/filebeat -e -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs

Aug 26 18:35:38 dev-web1 filebeat[7768]: 2019-08-26T18:35:38.156Z        ERROR        pipeline/output.go:100        Failed to connect to backoff(elasticsearch(http://elasticsear
Aug 26 18:35:38 dev-web1 filebeat[7768]: 2019-08-26T18:35:38.156Z        INFO        pipeline/output.go:93        Attempting to reconnect to backoff(elasticsearch(http://elastic
Aug 26 18:35:38 dev-web1 filebeat[7768]: 2019-08-26T18:35:38.156Z        INFO        [publisher]        pipeline/retry.go:189        retryer: send unwait-signal to consumer
Aug 26 18:35:38 dev-web1 filebeat[7768]: 2019-08-26T18:35:38.157Z        INFO        [publisher]        pipeline/retry.go:191          done
Aug 26 18:35:38 dev-web1 filebeat[7768]: 2019-08-26T18:35:38.157Z        INFO        [publisher]        pipeline/retry.go:166        retryer: send wait signal to consumer
Aug 26 18:35:38 dev-web1 filebeat[7768]: 2019-08-26T18:35:38.157Z        INFO        [publisher]        pipeline/retry.go:168          done
Aug 26 18:35:47 dev-web1 filebeat[7768]: 2019-08-26T18:35:47.028Z        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitori
Aug 26 18:36:17 dev-web1 filebeat[7768]: 2019-08-26T18:36:17.028Z        INFO        [monitoring]        log/log.go:145        Non-zero metrics in the last 30s        {"monitori
root@dev-web1:~#
但我无法阅读上述状态消息中的完整行

所以我试着:

root@dev-web1:~# curl elasticsearch.dev.domain.net/_cat/health
1566844775 18:39:35 dev-eks-logs green 3 3 48 24 0 0 0 0 - 100.0%
root@dev-web1:~#
这对port有效,但对port无效

root@dev-web1:~# curl elasticsearch.dev.domain.net:9200/_cat/health
filebeat具有以下配置

output.elasticsearch:
  hosts: ["elasticsearch.dev.domain.net"]
  username: "elastic"
  password: "changeme"
如何在filebeat端修复此问题

远程登录测试:

root@dev-web1:~# telnet <ip> 5044
Trying <ip>...
telnet: Unable to connect to remote host: Connection refused
root@dev-web1:~# telnet localhost 5044
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
root@dev-web1:~#
root@dev-web1:~#telnet 5044
尝试
telnet:无法连接到远程主机:连接被拒绝
root@dev-web1:~#telnet本地主机5044
正在尝试127.0.0.1。。。
telnet:无法连接到远程主机:连接被拒绝
root@dev-web1:~#

我认为是网络问题,检查telnet到本地主机/IP 5044

我认为是网络问题,检查telnet到本地主机/IP 5044

说:

主机
…如果未指定端口,则使用9200

在filbeat配置中添加主机:[“elasticsearch.dev.domain.net:80”]可以解决此问题。

说明:

主机
…如果未指定端口,则使用9200


在filbeat配置中添加
主机:[“elasticsearch.dev.domain.net:80”]
应该可以解决问题。

我认为问题在于端口未打开……如果您使用linux,只需将iptables-F(如果是测试环境)root@dev-web2:~#sudo ufw状态:不活动,不活动。请稍候,ufw只是一个接口…你需要开放端口如何将端口
5044
filebeat
相关?你应该阅读这篇文章…对我来说非常有用,直接点5我认为问题是端口没有开放…如果你使用linux,只需放置iptables-F(如果是测试环境)root@dev-web2:~#sudo ufw status status:inactive它不是active。等等,ufw只是一个接口…你需要打开端口
5044
filebeat
相关吗?你应该读一下这篇文章…对我来说,第5点非常有用