如何使用LogStash在tomcat日志文件上创建索引

我想从tomcat日志文件中进行一些日常分析，比如引发了多少错误和异常，以及它们的类别等。因此，我选择ELK来做这件事，并且我对日志索引是新手

这是我的conf文件：

input {
    file {
        path => "\localhost.2016-01-09.log"
        start_position => beginning
    }
}
filter {
    grok {
        match => { "message" => "%{COMBINEDAPACHELOG}"}
    }
    geoip {
        source => "clientip"
    }
}
output {
    stdout {codec => "rubydebug"}
    elasticsearch {
        hosts => ["localhost:9200"]

    }
}

下面是日志文件的一些行：

09-Jan-2016 18:30:38.722 INFO [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log No Spring WebApplicationInitializer types detected on classpath
09-Jan-2016 18:30:38.796 INFO [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log Initializing log4j from [C:\tomcat\apache-tomcat-8.0.26\temp\0-contact-statecollab-ws-15.12-SNAPSHOT-unknown-20151230-1152\WEB-INF\log4j.properties]
09-Jan-2016 18:30:38.901 INFO [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log Initializing Spring root WebApplicationContext
09-Jan-2016 18:30:54.271 INFO [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log No Spring WebApplicationInitializer types detected on classpath
09-Jan-2016 18:30:54.316 INFO [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log Initializing log4j from [C:\tomcat\apache-tomcat-8.0.26\temp\1-ohsms-ws-15.10.16-SNAPSHOT-unknown-20151119-1832\WEB-INF\log4j.properties]
09-Jan-2016 18:30:54.361 INFO [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log Initializing Spring root WebApplicationContext
09-Jan-2016 18:31:14.627 INFO [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log ContextListener: contextInitialized()
09-Jan-2016 18:31:14.628 INFO [localhost-startStop-1] org.apache.catalina.core.ApplicationContext.log SessionListener: contextInitialized()
09-Jan-2016 18:43:18.329 INFO [http-apr-8080-exec-9] org.apache.catalina.core.ApplicationContext.log No Spring WebApplicationInitializer types detected on classpath
09-Jan-2016 18:43:18.369 INFO [http-apr-8080-exec-9] org.apache.catalina.core.ApplicationContext.log Initializing log4j from [C:\tomcat\apache-tomcat-8.0.26\temp\5-exports-service-15.12-SNAPSHOT-unknown-20160108-1712\WEB-INF\log4j.properties]
09-Jan-2016 18:43:18.405 INFO [http-apr-8080-exec-9] org.apache.catalina.core.ApplicationContext.log Initializing Spring root WebApplicationContext
09-Jan-2016 18:43:29.405 SEVERE [http-apr-8080-exec-9] org.apache.catalina.core.StandardContext.listenerStart Exception sending context initialized event to listener instance of class org.springframework.web.context.ContextLoaderListener
 org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'exports-ws': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: protected gov.hhs.acf.ohs.hses.model.exports.service.ExportsService gov.hhs.acf.ohs.hses.services.exports.ExportsWebService.exportsService; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type [gov.hhs.acf.ohs.hses.model.exports.service.ExportsService] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true), @org.springframework.beans.factory.annotation.Qualifier(value=exportsService)}
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'gisDataSource': Invocation of init method failed; nested exception is javax.naming.NameNotFoundException: Name [hses_service] is not bound in this Context. Unable to find [hses_service].
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'contactService' defined in class path resource [contactContext.xml]: Cannot resolve reference to bean 'contactPgTransactionManager' while setting bean property 'transactionManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'contactPgTransactionManager' defined in class path resource [contactDaoPgContext.xml]: Cannot resolve reference to bean 'contactPgSessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'contactPgSessionFactory' defined in class path resource [contactDaoPgContext.xml]: Cannot resolve reference to bean 'gisDataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'gisDataSource': Invocation of init method failed; nested exception is javax.naming.NameNotFoundException: Name [hses_service] is not bound in this Context. Unable to find [hses_service].
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'drsExportsService' defined in class path resource [drsExportsContext.xml]: Cannot resolve reference to bean 'drsExportTransactionManager' while setting bean property 'transactionManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'drsExportTransactionManager' defined in class path resource [drsExportsContext.xml]: Cannot resolve reference to bean 'drsExportSessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'drsExportSessionFactory' defined in class path resource [drsExportsContext.xml]: Cannot resolve reference to bean 'gisDataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'gisDataSource': Invocation of init method failed; nested exception is javax.naming.NameNotFoundException: Name [hses_service] is not bound in this Context. Unable to find [hses_service].
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'exports-ws': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: protected gov.hhs.acf.ohs.hhhh.model.exports.service.ExportsService gov.hhs.acf.ohs.hhhh.services.exports.ExportsWebService.exportsService; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type [gov.hhs.acf.ohs.hhhh.model.exports.service.ExportsService] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true), @org.springframework.beans.factory.annotation.Qualifier(value=exportsService)}
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'gisDataSource': Invocation of init method failed; nested exception is javax.naming.NameNotFoundException: Name [hhhh_service] is not bound in this Context. Unable to find [hhhh_service].
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'contactService' defined in class path resource [contactContext.xml]: Cannot resolve reference to bean 'contactPgTransactionManager' while setting bean property 'transactionManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'contactPgTransactionManager' defined in class path resource [contactDaoPgContext.xml]: Cannot resolve reference to bean 'contactPgSessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'contactPgSessionFactory' defined in class path resource [contactDaoPgContext.xml]: Cannot resolve reference to bean 'gisDataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'gisDataSource': Invocation of init method failed; nested exception is javax.naming.NameNotFoundException: Name [hhhh_service] is not bound in this Context. Unable to find [hhhh_service].
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ccrRegistrantExportService' defined in class path resource [ccrRegistrantExportPgContext.xml]: Cannot resolve reference to bean 'ccrRegistrantExportTransactionManager' while setting bean property 'transactionManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ccrRegistrantExportTransactionManager' defined in class path resource [ccrRegistrantExportPgContext.xml]: Cannot resolve reference to bean 'ccrRegistrantExportSessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'ccrRegistrantExportSessionFactory' defined in class path resource [ccrRegistrantExportPgContext.xml]: Cannot resolve reference to bean 'gisDataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'gisDataSource': Invocation of init method failed; nested exception is javax.naming.NameNotFoundException: Name [hhhh_service] is not bound in this Context. Unable to find [hhhh_service].
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'drsExportsService' defined in class path resource [drsExportsContext.xml]: Cannot resolve reference to bean 'drsExportTransactionManager' while setting bean property 'transactionManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'drsExportTransactionManager' defined in class path resource [drsExportsContext.xml]: Cannot resolve reference to bean 'drsExportSessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'drsExportSessionFactory' defined in class path resource [drsExportsContext.xml]: Cannot resolve reference to bean 'gisDataSource' while setting bean property 'dataSource'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'gisDataSource': Invocation of init method failed; nested exception is javax.naming.NameNotFoundException: Name [hhhh_service] is not bound in this Context. Unable to find [hhhh_service].

当我运行一个脚本：“logstash-f first pipeline.conf”时，它说

日志存储启动已完成

但当我使用以下行检查索引时：

没有创建这样的索引。我错过什么了吗？我不想更改服务器上的任何tomcat配置，我想从tomcat日志文件中获取每日报告，而不是运行服务器

我切换到Ubuntu进行更多的游戏，并查看了elastic搜索的日志。它说：

[2016-02-10 12:15:32,496][INFO ][node                     ] [Archie Corrigan] version[2.2.0], pid[6365], build[8ff36d1/2016-01-27T13:32:39Z]
[2016-02-10 12:15:32,496][INFO ][node                     ] [Archie Corrigan] initializing ...
[2016-02-10 12:15:33,232][INFO ][plugins                  ] [Archie Corrigan] modules [lang-expression, lang-groovy], plugins [], sites []
[2016-02-10 12:15:33,269][INFO ][env                      ] [Archie Corrigan] using [1] data paths, mounts [[/ (/dev/loop0)]], net usable_space [8.7gb], net total_space [16.9gb], spins? [possibly], types [ext4]
[2016-02-10 12:15:33,269][INFO ][env                      ] [Archie Corrigan] heap size [990.7mb], compressed ordinary object pointers [true]
[2016-02-10 12:15:35,932][INFO ][node                     ] [Archie Corrigan] initialized
[2016-02-10 12:15:35,932][INFO ][node                     ] [Archie Corrigan] starting ...
[2016-02-10 12:15:36,044][INFO ][transport                ] [Archie Corrigan] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2016-02-10 12:15:36,052][INFO ][discovery                ] [Archie Corrigan] elasticsearch/CNUHuSKySm6osx2heQAIqg
[2016-02-10 12:15:39,132][INFO ][cluster.service          ] [Archie Corrigan] new_master {Archie Corrigan}{CNUHuSKySm6osx2heQAIqg}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2016-02-10 12:15:39,173][INFO ][http                     ] [Archie Corrigan] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2016-02-10 12:15:39,173][INFO ][node                     ] [Archie Corrigan] started
[2016-02-10 12:15:39,362][INFO ][gateway                  ] [Archie Corrigan] recovered [0] indices into cluster_state

---

我发现了问题。我必须在输入->文件属性中提供sincedb\u路径属性（sincedb\u路径=>“\null”）。因为弹性搜索认为该索引已经创建，无需再次创建。否则，ES将等待新的行添加到日志文件。

根据您的配置，您的标准输出日志应包含发送到elasticsearch的所有事件的副本；有什么？此外，请查看日志存储错误日志和elasticsearch群集日志，以获取任何信息性消息。嗨，Alain，感谢您的回复。我应该如何检查你提到的那些日志？顺便说一句，当我在他们的网站上使用教程中提供的apache类型日志文件运行它时，一切正常。因此，这是否意味着我必须提供不同的模式？通常，日志存储日志位于日志存储主机上的/var/log/logstash中，而elasticsearch日志位于elasticsearch主机上的/var/log/elasticsearch中。您需要先确定问题，然后才能确定解决方案。阿兰，我通过添加弹性搜索日志编辑了我的问题。“var\log\logstash”中没有文件