如何使用fluent位转发输出插件将数据发送到tcp端口

如何使用fluent位转发输出插件将数据发送到tcp端口,tcp,docker-compose,telegraf,fluent-bit,telegraf-plugins,Tcp,Docker Compose,Telegraf,Fluent Bit,Telegraf Plugins,我正在尝试使用fluent bit将日志从日志文件发送到telegraf,telegraf正在监听端口8094。我可以通过这样的终端向这个端口发送数据 回显“一些日志数据”| nc本地主机8094 但是当我使用fluent bit formward输出插件将数据发送到同一端口时,它在fluent bit日志中给出了这个错误 fluent-bit_1|[2019/11/21 11:14:44][error][io]TCP连接失败:本地主机:8094(连接被拒绝) fluent-bit|[2019/

我正在尝试使用fluent bit将日志从日志文件发送到telegraf,telegraf正在监听端口8094。我可以通过这样的终端向这个端口发送数据

回显“一些日志数据”| nc本地主机8094

但是当我使用fluent bit formward输出插件将数据发送到同一端口时,它在fluent bit日志中给出了这个错误

fluent-bit_1|[2019/11/21 11:14:44][error][io]TCP连接失败:本地主机:8094(连接被拒绝)
fluent-bit|[2019/11/21 11:14:44][error][out|fw]没有可用的上游连接

这是我的docker撰写文件:

version: '3'

services:
  # Define a Telegraf service
  telegraf:
    image: telegraf
    volumes:
      - ./telegraf/telegraf.conf:/etc/telegraf/telegraf.conf:ro
    ports:
      - "8092:8092/udp"
      - "8094:8094"
      - "8125:8125/udp"
      - "9126:9126"
    networks:
      - mynet

  fluent-bit:
    image: fluent/fluent-bit:1.3.2-debug
    volumes:
      - ./fluent-bit:/fluent-bit/etc
      - ./access_logs/localhost_access_log:/logs
    depends_on:
      - telegraf
    networks:
      - mynet

networks:
  mynet:
fluent-bit.conf:

[SERVICE]
    Flush           2
    Parsers_File    parsers.conf

[INPUT]
    Name                tail
    Tag                 cuic.logs
    Path                /logs/*.log
    Path_Key            File_Path
    Multiline           On
    Parser_Firstline    start

[OUTPUT]
    Name          forward
    Match         *
    Host          localhost
    Port          8094
    Tag           cuic.logs
telegraf.conf:

[[outputs.file]]
  files = ["/tmp/metrics.out"]
  data_format = "json"
  json_timestamp_units = "1s"

[[inputs.socket_listener]]
  service_address = "tcp://:8094"
  socket_mode = "777"
  data_format = "grok"
  grok_patterns = ["%{CUSTOM_LOG}"]
  grok_custom_patterns = '''
    SOME_GROK_PATTERN
    '''

[[aggregators.histogram]]
 period = "10s"
 drop_original = false
 [[aggregators.histogram.config]]
   buckets = [0.0, 10.0, 20.0, 30.0, 40.0, 50.0, 60.0, 70.0, 80.0, 90.0, 100.0]
   measurement_name = "access_log"
   fields = ["resp_time"]

有人能帮我找出我做错了什么吗?

我想问题在于使用主机名“localhost”时。由于网络级别的容器localhost将是其自己的网络作用域,因此它将无法根据需要访问其他容器TCP端口

您可以在此处阅读有关同一问题的更多信息:


而且。。。请注意,Fluent位中的前向输出协议使用的是二进制协议,而不是我怀疑您想要使用的普通JSON。请改用tcp输出插件。

这肯定是您正在使用的输出/输入插件。Telegraf有一个FluentD插件,它看起来像这样:

    # Read metrics exposed by fluentd in_monitor plugin
[[inputs.fluentd]]
  ## This plugin reads information exposed by fluentd (using /api/plugins.json endpoint).
  ##
  ## Endpoint:
  ## - only one URI is allowed
  ## - https is not supported
  endpoint = "http://localhost:24220/api/plugins.json"

  ## Define which plugins have to be excluded (based on "type" field - e.g. monitor_agent)
  exclude = [
      "monitor_agent",
      "dummy",
  ]
[INPUT]
    Name  cpu
    Tag   cpu

[OUTPUT]
    Name  http
    Match *
    Host  192.168.2.3
    Port  80
    URI   /something
您的应用程序将如下所示:

    # Read metrics exposed by fluentd in_monitor plugin
[[inputs.fluentd]]
  ## This plugin reads information exposed by fluentd (using /api/plugins.json endpoint).
  ##
  ## Endpoint:
  ## - only one URI is allowed
  ## - https is not supported
  endpoint = "http://localhost:24220/api/plugins.json"

  ## Define which plugins have to be excluded (based on "type" field - e.g. monitor_agent)
  exclude = [
      "monitor_agent",
      "dummy",
  ]
[INPUT]
    Name  cpu
    Tag   cpu

[OUTPUT]
    Name  http
    Match *
    Host  192.168.2.3
    Port  80
    URI   /something

但是Fluent Bit也有一个。

Fluent Bit有一个名为forward的out插件,它可以根据fluentd协议转发输出。您可以根据此单据进行设置:

然后,您可以发现telegraf有一个名为fluentd的输入插件,将其设置为输入,然后从fluentd客户端端点收集可以满足您需求的指标