Docker Logstash管道不从pipelines.yml运行

Docker Logstash管道不从pipelines.yml运行,docker,logstash,configuration-files,elastic-stack,Docker,Logstash,Configuration Files,Elastic Stack,我试图运行logstash:latest容器,但运行时管道抛出错误 当我直接使用-f“/path/to/pipeline”运行它时,一切都正常。 一旦我从pipelines.yml运行它,它会说管道的第一行第一列的格式不好 14:39:28.728[LogStash::Runner]错误LogStash.agent-无法创建管道{:reason=>“}后第1行第1列(字节1)处的#、输入、筛选、输出应为} 注意:在我对第一行有评论之前,它说错误发生在评论之后,所以文件访问和路径肯定是可以的 如果

我试图运行logstash:latest容器,但运行时管道抛出错误

当我直接使用-f“/path/to/pipeline”运行它时,一切都正常。 一旦我从pipelines.yml运行它,它会说管道的第一行第一列的格式不好

14:39:28.728[LogStash::Runner]错误LogStash.agent-无法创建管道{:reason=>“}后第1行第1列(字节1)处的#、输入、筛选、输出应为}

注意:在我对第一行有评论之前,它说错误发生在评论之后,所以文件访问和路径肯定是可以的

如果有人知道我是如何或为什么经历这种行为的,我愿意接受建议。 在底部,我包括我的配置

管道。yml

- pipeline.id: log-pipeline
  path.config: "/etc/logstash/pipelines/log_elastic_write.conf"
  pipeline.workers: 1
log\u elastic\u write.conf

input { 
        kafka {
                bootstrap_servers => "kafka:9092"
                topics => ["logs"]
                group_id => "logs_write"
                auto_offset_reset => "earliest"
        }
}
filter {
        date {
                match => [ "timestamp" , "yyyy-MM-dd HH:mm:ss.SSSSSS" ]
                timezone => "UTC"
        }
}
filter {
        json {
                source => "message"
        }
}
filter {
        json {
                source => "message"
                target => "raw_message"
        }

        mutate {
                rename => {"@timestamp" => "timestamp_message_received"}
        }

        mutate {
                remove_field => [ "message" ]
                rename => {"[raw_message][timestamp]" => "timestamp_message_sent"}
        }
}
filter {
        date {
                match => ["timestamp_message_sent","yyyy-MM-dd HH:mm:ss.SSS"]
                target => "timestamp_message_sent"
                locale => "en"
        }
}
filter {
        mutate { remove_field => [ "raw_message" ] }
        mutate { remove_field => [ "timestamp" ] }
}
output {
        elasticsearch {
                hosts => "ipelastic:82"
                index => "logging"
        }
}
entrypoint.sh

#!/usr/bin/env bash

echo 'Inspecting file structure'
find /etc/logstash/ -maxdepth 4 | grep -v git | grep -v idea

echo 'Setting up runtime and pipeline configuration files(entrypoint.sh):'
echo "----------        'logstash.yml'        ----------"
envsubst < "${LOGSTASH_DIR_TARGET}/${LOGSTASH_CONFIG_SOURCE}" > "${LOGSTASH_DIR_TARGET}/${LOGSTASH_CONFIG_TARGET}"
rm "${LOGSTASH_DIR_TARGET}/${LOGSTASH_CONFIG_SOURCE}"

echo "----------        'pipelines.yml'        ----------"
envsubst < "${LOGSTASH_DIR_TARGET}/${PIPELINES_CONFIG_SOURCE}" > "${LOGSTASH_DIR_TARGET}/${PIPELINES_CONFIG_TARGET}"
rm "${LOGSTASH_DIR_TARGET}/${PIPELINES_CONFIG_SOURCE}"
less "${LOGSTASH_DIR_TARGET}/${PIPELINES_CONFIG_TARGET}"

echo "---------- pipeline: elasticsearch logs ----------"
envsubst < "${PIPELINE_DIR_TARGET}/${LOG_WRITE_PIPELINE_SOURCE}" > "${PIPELINE_DIR_TARGET}/${LOG_WRITE_PIPELINE_TARGET}"
rm "${PIPELINE_DIR_TARGET}/${LOG_WRITE_PIPELINE_SOURCE}"
less "${PIPELINE_DIR_TARGET}/${LOG_WRITE_PIPELINE_TARGET}"

chmod a+rx "${LOGSTASH_DIR_TARGET}/${LOGSTASH_CONFIG_TARGET}"
chmod a+rx "${LOGSTASH_DIR_TARGET}/${PIPELINES_CONFIG_TARGET}"
chmod a+rx "${PIPELINE_DIR_TARGET}/${LOG_WRITE_PIPELINE_TARGET}"

logstash
#/usr/bin/env bash
echo“检查文件结构”
查找/etc/logstash/-maxdepth 4 | grep-vgit | grep-videa
echo“设置运行时和管道配置文件(entrypoint.sh):”
echo“------------'logstash.yml'------------”
envsubst<“${LOGSTASH_DIR_TARGET}/${LOGSTASH_CONFIG_SOURCE}”>“${LOGSTASH_DIR_TARGET}/${LOGSTASH_CONFIG_TARGET}”
rm“${LOGSTASH\u DIR\u TARGET}/${LOGSTASH\u CONFIG\u SOURCE}”
echo“------------”pipelines.yml'------------”
envsubst<“${LOGSTASH_DIR_TARGET}/${PIPELINES_CONFIG_SOURCE}”>“${LOGSTASH_DIR_TARGET}/${PIPELINES_CONFIG_TARGET}”
rm“${LOGSTASH\u DIR\u TARGET}/${PIPELINES\u CONFIG\u SOURCE}”
减去“${LOGSTASH\u DIR\u TARGET}/${PIPELINES\u CONFIG\u TARGET}”
echo“------------管道:elasticsearch日志----------------”
envsubst<“${PIPELINE\u DIR\u TARGET}/${LOG\u WRITE\u PIPELINE\u SOURCE}”>“${PIPELINE\u DIR\u TARGET}/${LOG\u WRITE\u PIPELINE\u TARGET}”
rm“${PIPELINE\u DIR\u TARGET}/${LOG\u WRITE\u PIPELINE\u SOURCE}”
减去“${PIPELINE\u DIR\u TARGET}/${LOG\u WRITE\u PIPELINE\u TARGET}”
chmod a+rx“${LOGSTASH\u DIR\u TARGET}/${LOGSTASH\u CONFIG\u TARGET}”
chmod a+rx“${LOGSTASH\u DIR\u TARGET}/${PIPELINES\u CONFIG\u TARGET}”
chmod a+rx“${PIPELINE\u DIR\u TARGET}/${LOG\u WRITE\u PIPELINE\u TARGET}”
伐木场

这些文件是如何进入容器的?@DavidMaze我在dockerfile中复制了一个虚拟文件,然后使用envsubt替换environmentals使其动态,但您会看到替换后的直接打印输出,因此,它应该是logstash在ENTRYPOINT上运行后可用的脚本。您可以将ENTRYPOINT脚本添加到问题中吗?你能
docker跑吗。。。sh
(使用图像的入口点脚本)来验证配置文件是否存在且是否为空?@DavidMaze我使用入口点中的logstash-f“/etc/logstash/pipelines/log_elastic_write.conf”运行了脚本,它可以正常工作。我只想在未来增加管道。然后,我通过访问shell检查了这些文件,它们在那里并且具有正确的值。我完全没有主意了。我最后一次机会是在-f选项中指定多个管道,但这似乎不干净,应该可以工作。