elasticsearch Logstash和ElasticSearch之间缺少数据

我试图用Logstash解析IIS错误XML文件，然后将这些错误存储在Elasticsearch中，并显示在Kibana中

这是我的logstash.conf文件：

input 
{
    file{
        path => ["C:/inetpub/logs/FailedReqLogFiles/*/*.xml"]
        start_position => "beginning"
        # filter is not thread safe, so have to move the multiline into the input
        codec => multiline{
            #pattern => "^</failedRequest>"
            pattern => "^<\?xml version"
            negate => true
            what => "previous"
            max_lines => 12000
        }
        sincedb_path => "C:/Users/ss/Source/elk/logstash/bin/.sincedb"
    }
}

filter{
    xml{
        store_xml => "false"
        source => "message"
        target => "EVENT"
        xpath => [
          "/failedRequest/@url", "url",
          "/failedRequest/@appPoolId", "appPoolId",
          "/failedRequest/@verb", "verb",
          "/failedRequest/@statusCode", "statusCode"
        ]
    }    
}

output 
{
    elasticsearch{
        hosts => ["100.202.191.77:9200"]
        index => "testserver-logstash"   
        flush_size => 1     
    }
    stdout 
    {
        codec => rubydebug
    }
    file{
        path => "C:/Users/ss/Source/elk/logstash/bin/test.log"
    }
}

这些目录中包含的日志文件有：fr000001.xml、fr000002.xml、fr000003.xml、fr000004.xml等等

xml文件的结构如下所示：

<?xml version="1.0" encoding="UTF-8" ?>
<?xml-stylesheet type='text/xsl' href='freb.xsl'?>
<!-- saved from url=(0014)about:internet -->
<failedRequest url="https://aaa.bbb.com:443/ddd.aspx" 
               siteId="3"
               appPoolId="aaa.bbb.com" 
               processId="15168"
               verb="GET" 
               remoteUserName=""
               userName=""
               tokenUserName="NT AUTHORITY\IUSR"
               authenticationType="anonymous"
               activityId="{00000000-0000-0000-0200-0080030000FF}"
               failureReason="STATUS_CODE"
               statusCode="500" 
               triggerStatusCode="500"
               timeTaken="368584"
               xmlns:freb="http://schemas.microsoft.com/win/2006/06/iis/freb"
               >
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">blablabla
</Event>
</failedRequest> 

喋喋不休

我遇到的问题是：总共有13个xml日志文件。我可以看到logstash输出在stdout和test.log中正确打印所有这13个文件的解析xml，但是在Kibana中，我只能看到13个xml文件中的4个文件的数据，其余9个文件丢失

---

我是麋鹿的新手，真的被困在这里了。希望有人能帮我。提前谢谢

我对这句话发表了评论，看起来logstash很管用

 flush_size => 1    

---

我在网上发表了评论，看起来logstash很管用

 flush_size => 1    

---

你查过elasticsearch日志了吗？也许它在为您吐出一些错误消息。@EvaldasBuinauskas感谢您的帮助。我想我已经弄明白了。你查过elasticsearch日志了吗？也许它在为您吐出一些错误消息。@EvaldasBuinauskas感谢您的帮助。我想我已经明白了。