elasticsearch 当在Logstash中定义Elasticsearch索引模板时,Logstash抛出[message=>;";undefined method`update';for nil:NilClass";]
我正在尝试使用logstash将数据从sql server加载到弹性搜索。对于索引创建,我使用logstash的配置文件中指定的索引模板 logstash.config的输出字段:elasticsearch 当在Logstash中定义Elasticsearch索引模板时,Logstash抛出[message=>;";undefined method`update';for nil:NilClass";],elasticsearch,logstash,logstash-configuration,elk,elasticsearch,Logstash,Logstash Configuration,Elk,我正在尝试使用logstash将数据从sql server加载到弹性搜索。对于索引创建,我使用logstash的配置文件中指定的索引模板 logstash.config的输出字段: output{ stdout { codec => json_lines } elasticsearch { hosts => ["localhost:9200"]
output{
stdout { codec => json_lines }
elasticsearch {
hosts => ["localhost:9200"]
document_id => "%{[id]}"
manage_template => true
template => "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/logstash_test.json"
template_name => "logstash_test"
template_overwrite => true
}
}
[2021-01-25T19:48:28,232][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2021-01-25T19:48:28,298][INFO ][logstash.outputs.elasticsearch][main] Using mapping template from {:path=>"C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/logstash_test.json"}
[2021-01-25T19:48:28,370][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/sampleconf.conf"], :thread=>"#<Thread:0x797ce1a8 run>"}
[2021-01-25T19:48:28,509][ERROR][logstash.outputs.elasticsearch][main] Failed to install template. {:message=>"undefined method `update' for nil:NilClass", :class=>"NoMethodError", :backtrace=>["C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:42:in `add_ilm_settings_to_template'", "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:15:in `install_template'", "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/common.rb:218:in `install_template'", "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/common.rb:49:in `block in setup_after_successful_connection'"]}
[2021-01-25T19:48:28,546][INFO ][logstash.outputs.elasticsearch][main] Creating rollover alias <logstash-{now/d}-000001>
[2021-01-25T19:48:29,463][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>1.08}
[2021-01-25T19:48:29,802][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
[2021-01-25T19:48:29,888][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
{
"index_patterns":"*-logstash_test-*",
"priority":1,
"template":{
"settings":{
"analysis":{
"analyzer":{
"test_autocomplete":{
"tokenizer":"test_tokenizer"
}
},
"tokenizer":{
"test_tokenizer":{
"custom_token_chars":"'-",
"max_gram":"3",
"min_gram":"3",
"token_chars":[
],
"type":"ngram"
}
}
},
"mappings":{
"properties":{
"test_id":{
"type":"text",
"analyzer":"test_autocomplete"
}
}
}
}
}
}
运行日志存储时出错:
output{
stdout { codec => json_lines }
elasticsearch {
hosts => ["localhost:9200"]
document_id => "%{[id]}"
manage_template => true
template => "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/logstash_test.json"
template_name => "logstash_test"
template_overwrite => true
}
}
[2021-01-25T19:48:28,232][INFO ][logstash.outputs.elasticsearch][main] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2021-01-25T19:48:28,298][INFO ][logstash.outputs.elasticsearch][main] Using mapping template from {:path=>"C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/logstash_test.json"}
[2021-01-25T19:48:28,370][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>1000, "pipeline.sources"=>["C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/sampleconf.conf"], :thread=>"#<Thread:0x797ce1a8 run>"}
[2021-01-25T19:48:28,509][ERROR][logstash.outputs.elasticsearch][main] Failed to install template. {:message=>"undefined method `update' for nil:NilClass", :class=>"NoMethodError", :backtrace=>["C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:42:in `add_ilm_settings_to_template'", "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb:15:in `install_template'", "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/common.rb:218:in `install_template'", "C:/Users/Lich/Documents/logstash-7.10.1-windows-x86_64/logstash-7.10.1/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.7.0-java/lib/logstash/outputs/elasticsearch/common.rb:49:in `block in setup_after_successful_connection'"]}
[2021-01-25T19:48:28,546][INFO ][logstash.outputs.elasticsearch][main] Creating rollover alias <logstash-{now/d}-000001>
[2021-01-25T19:48:29,463][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>1.08}
[2021-01-25T19:48:29,802][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
[2021-01-25T19:48:29,888][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
{
"index_patterns":"*-logstash_test-*",
"priority":1,
"template":{
"settings":{
"analysis":{
"analyzer":{
"test_autocomplete":{
"tokenizer":"test_tokenizer"
}
},
"tokenizer":{
"test_tokenizer":{
"custom_token_chars":"'-",
"max_gram":"3",
"min_gram":"3",
"token_chars":[
],
"type":"ngram"
}
}
},
"mappings":{
"properties":{
"test_id":{
"type":"text",
"analyzer":"test_autocomplete"
}
}
}
}
}
}
虽然失败,但是使用数据创建索引,但是设置{..}
中指定的分析器不反映。我是新来的麋鹿堆栈,不知道为什么会发生这种情况
Elasticsearch版本:7.10.1
日志存储版本:7.10.1
确保您的
logstash_test.json
文件具有以下格式:
{
"index_patterns": "*-logstash_test-*",
"order": 1,
"settings": {
"analysis": {
"analyzer": {
"test_autocomplete": {
"tokenizer": "test_tokenizer"
}
},
"tokenizer": {
"test_tokenizer": {
"custom_token_chars": "'-",
"max_gram": "3",
"min_gram": "3",
"token_chars": [],
"type": "ngram"
}
}
}
},
"mappings": {
"properties": {
"test_id": {
"type": "text",
"analyzer": "test_autocomplete"
}
}
}
}
您在
模板
部分中包含了映射
和设置
,但这仅适用于elasticsearch
日志存储输出尚不支持的新映射。您需要使用。您好,非常感谢,它很有效。但是我可以看到索引并不是用我在索引模板中提到的新索引模式名称创建的。相反,它是按如下方式创建的:[logstash.outputs.elasticsearch][main]试图安装模板{:manage\u template=>{“index\u patterns”=>“logstash-*”,“order”=>1,“settings”=>{“shard的数量”=>2,“analysis”=>{“analysis”=>…
您需要在elasticsearch
输出中指定与您的索引模式匹配的所需索引名称,否则Logstash将默认命名为Logstash xyz
,谢谢!您是一个救生员!!太棒了,很高兴它成功了;-)