elasticsearch 如何使用Elasticsearch映射API更改现有索引的字段类型
我使用的是elasticsearch 如何使用Elasticsearch映射API更改现有索引的字段类型,elasticsearch,kibana,elastic-stack,elasticsearch,Kibana,Elastic Stack,我使用的是ELK,文档结构如下 { "_index": "prod1-db.log-*", "_type": "db.log", "_id": "AVadEaq7", "_score": null, "_source": { "message": "2016-07-08T12:52:42.026+0000 I NETWORK [conn4928242] end connection 192.168.170.62:47530 (31 connections now
ELK
,文档结构如下
{
"_index": "prod1-db.log-*",
"_type": "db.log",
"_id": "AVadEaq7",
"_score": null,
"_source": {
"message": "2016-07-08T12:52:42.026+0000 I NETWORK [conn4928242] end connection 192.168.170.62:47530 (31 connections now open)",
"@version": "1",
"@timestamp": "2016-08-18T09:50:54.247Z",
"type": "log",
"input_type": "log",
"count": 1,
"beat": {
"hostname": "prod1",
"name": "prod1"
},
"offset": 1421607236,
"source": "/var/log/db/db.log",
"fields": null,
"host": "prod1",
"tags": [
"beats_input_codec_plain_applied"
]
},
"fields": {
"@timestamp": [
1471513854247
]
},
"sort": [
1471513854247
]
}
我想将消息
字段更改为未分析
。我想知道如何使用Elasticsedarch映射API来实现这一点?例如,如何使用PUT-Mapping-API
向现有索引添加新类型
我使用的是kibana4.5
和Elasticsearch 2.3
更新
在logstash
中尝试了以下template.json
1 {
2 "template": "logstash-*",
3 "mappings": {
4 "_default_": {
5 "properties": {
6 "message" : {
7 "type" : "string",
8 "index" : "not_analyzed"
9 }
10 }
11 }
12 }
13 }
logstash_1 | {:timestamp=>"2016-08-24T11:00:26.097000+0000", :message=>"Invalid setting for elasticsearch output plugin:\n\n output {\n elasticsearch {\n # This setting must be a path\n # File does not exist or cannot be opened /home/dw/docker-elk/logstash/core_mapping_template.json\n template => \"/home/dw/docker-elk/logstash/core_mapping_template.json\"\n ...\n }\n }", :level=>:error}
logstash_1 | {:timestamp=>"2016-08-24T11:00:26.153000+0000", :message=>"Pipeline aborted due to error", :exception=>#<LogStash::ConfigurationError: Something is wrong with your configuration.>, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/config/mixin.rb:134:in `config_init'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/outputs/base.rb:63:in `initialize'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/output_delegator.rb:74:in `register'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:181:in `start_workers'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:181:in `start_workers'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:136:in `run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/agent.rb:473:in `start_pipeline'"], :level=>:error}
logstash_1 | {:timestamp=>"2016-08-24T11:00:29.168000+0000", :message=>"stopping pipeline", :id=>"main"}
启动logstash
时出现以下错误
1 {
2 "template": "logstash-*",
3 "mappings": {
4 "_default_": {
5 "properties": {
6 "message" : {
7 "type" : "string",
8 "index" : "not_analyzed"
9 }
10 }
11 }
12 }
13 }
logstash_1 | {:timestamp=>"2016-08-24T11:00:26.097000+0000", :message=>"Invalid setting for elasticsearch output plugin:\n\n output {\n elasticsearch {\n # This setting must be a path\n # File does not exist or cannot be opened /home/dw/docker-elk/logstash/core_mapping_template.json\n template => \"/home/dw/docker-elk/logstash/core_mapping_template.json\"\n ...\n }\n }", :level=>:error}
logstash_1 | {:timestamp=>"2016-08-24T11:00:26.153000+0000", :message=>"Pipeline aborted due to error", :exception=>#<LogStash::ConfigurationError: Something is wrong with your configuration.>, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/config/mixin.rb:134:in `config_init'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/outputs/base.rb:63:in `initialize'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/output_delegator.rb:74:in `register'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:181:in `start_workers'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:181:in `start_workers'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:136:in `run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/agent.rb:473:in `start_pipeline'"], :level=>:error}
logstash_1 | {:timestamp=>"2016-08-24T11:00:29.168000+0000", :message=>"stopping pipeline", :id=>"main"}
logstash\u 1{:timestamp=>“2016-08-24T11:00:26.097000+0000”,message=>“elasticsearch输出插件的设置无效:\n\n output{\n elasticsearch{\n#此设置必须是路径\n#文件不存在或无法打开/home/dw/docker elk/logstash/core\u mapping\u template.json\n template=>”/home/dw/docker elk/logstash/core\u mapping\u template.json\“\n…\n}\n}”,level=>:error}
logstash|{:timestamp=>“2016-08-24T11:00:26.153000+0000”,:message=>“由于错误导致管道中止”,:exception=>,:backtrace=>[”/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/config/mixin.rb:134:in'config|init',“/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/outputs/base.rb:63:in‘initialize’”/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/output_delegator.rb:74:in‘register’”/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:181:in‘start_workers’,“org/jruby/RubyArray.java:1613:in‘each’,/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:181:in‘start_workers’。”/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/pipeline.rb:136:in‘run’,/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.4-java/lib/logstash/agent.rb:473:in‘start_pipeline’,:level=>:error}
logstash|{:时间戳=>“2016-08-24T11:00:29.168000+0000”,“消息=>“正在停止管道”,“id=>“主”}
索引已经存在时,不能更改索引的映射,除非创建新字段到对象或多字段
如果您想使用映射API实现此目的,您的请求将如下所示:
PUT /prod1-db.log-*/_mapping/log
{
"properties": {
"message": {
"type": "string",
"index": "not_analyzed"
}
}
}
elasticsearch {
template => "/etc/logstash/template/template.json"
template_overwrite => true
}
不过,我建议您使用映射创建一个JSON文件,并将其添加到日志存储配置中
模板文件可能如下所示(您需要对此进行自定义):
日志存储配置中的elasticsearch
条目如下所示:
PUT /prod1-db.log-*/_mapping/log
{
"properties": {
"message": {
"type": "string",
"index": "not_analyzed"
}
}
}
elasticsearch {
template => "/etc/logstash/template/template.json"
template_overwrite => true
}
如果您在创建索引时没有为字段指定任何映射,则在您第一次将文档索引到索引时,elastic search会根据提供的数据自动为每个字段选择最佳映射。查看您在问题中提供的文档,elasticsearch可能已经为字段
消息的Analyser
。一旦分配了它,您就无法更改它。唯一的方法是创建一个新的索引。尝试PUT/prod1 db.log-*/\u映射/log{“属性”:{“消息”:{“类型”:“字符串”,“索引”:“未分析”}
但是从elasticsearch
中得到一个错误,`java.lang.IllegalArgumentException:无效的版本格式:{“属性”:{“消息”:{“类型”:“字符串”,“索引”:“未分析”}HTTP/1.1`@daiyue是否已重新创建索引?重新创建索引是什么意思?如何结合添加映射来实现此操作?@daiyue无法重新映射现有索引(某些预期除外).Mapping仅适用于正在创建的索引。我强烈建议您使用模板文件,因为这样您就不必处理curl,并且可以非常轻松地编辑更改。是的。它定义了应将映射应用于哪个索引。