elasticsearch ElasticSearch单滚动索引配置
我想设置一个json配置文件,我可以通过加载和使用该文件,如下所示:
elasticsearch ElasticSearch单滚动索引配置,
elasticsearch,configuration,python-3.6,
elasticsearch,Configuration,Python 3.6,我想设置一个json配置文件,我可以通过加载和使用该文件,如下所示: import elasticsearch es = elasticsearch.Elasticsearch(hosts=[{'host': host, 'port': port}]) es.indices.create(index="my_special_index", body=es_config) 该指数必须是每月一次的 这用于创建静态索引: es_config = { "settings": { "inde
import elasticsearch
es = elasticsearch.Elasticsearch(hosts=[{'host': host, 'port': port}])
es.indices.create(index="my_special_index", body=es_config)
该指数必须是每月一次的
这用于创建静态索引:
es_config = {
"settings": {
"index": {
"number_of_shards": 3,
"number_of_replicas": 1
},
"analysis": {
"analyzer": {
"default": {
"tokenizer": "standard",
"filter": ["lowercase", "stop", "asciifolding"]
}
}
}
},
"mappings": {
"alert": {
"properties": {
"id": {"type": "long"},
"title": {"type": "text"},
"user": {
"type": "nested",
"properties": {
"id": {"type": "long"},
"name": {"type": "text"}
}
}
}
}
}
}
我还没有弄清楚如何设置滚动群集,但是否可以在一个config+调用中完成所有操作?让我困惑的是,我认为滚动是一个被动过程。但是当调用
elasticsearch.index.rollover()
时,它会对别名/活动索引进行活动检查,以查看是否满足滚动条件,如果满足,则会增加活动索引。执行活动的滚动显示如下所示:
import elasticsearch
es = elasticsearch.Elasticsearch(hosts=[{'host': host, 'port': port}])
es.indices.create(index="my_special_index", body=es_config)
给定问题中的索引配置为index\u config
es = elasticsearch.Elasticsearch(hosts=[{'host': host, 'port': port}])
index_name = "test-index-00001" # initial index name ends with number
alias_name = "test-index" # we will refer to the alias name from now on
doctype = "test"
# first create index and alias
es.indices.create(index=index_name, body=index_config)
# create alias
es.indices.put_alias(index=index_name, name=alias_name)
for i in range(5):
import datetime
doc_body = {"id": i, "title": str(datetime.datetime.now())}
es.index(index=alias_name, doc_type=doctype, body=doc_body)
time.sleep(0.1)
time.sleep(1) # es needs about a second
rollover_config = {"conditions": {"max_age": "10d", "max_docs": 3}}
rollover_ret = es.indices.rollover(alias=alias_name, body=rollover_config)
print(rollover_ret)
for i in range(5):
import datetime
doc_body = {"id": i, "title": str(datetime.datetime.now())}
es.index(index=alias_name, doc_type=doctype, body=doc_body)
time.sleep(0.1)
当您检查索引时,您会发现test-index-00001
包含五项,而不是三项(如rollover\u config声明),因为只有在这五项已经被索引之后才会发生滚动
为了获得被动滚动,我们可以利用ElasticSearch的自动索引创建和模板系统。假设我们希望为每个日历月创建一个新索引(第二个用于复制粘贴测试): 首先,使用别名和模板字段创建:
{
"aliases": {"alerts-index": {}},
"template": "alerts-index-*",
"settings": {
"index": {
"number_of_shards": 3,
"number_of_replicas": 1
},
"mappings": {
"alert": {
"properties": {
"id": {"type": "long"},
"title": {"type": "text"}
}
}
}
}
然后使用YYYYMM后缀后面的初始索引名创建模板:
es = elasticsearch.Elasticsearch(hosts=[{'host': host, 'port': port}])
alias_name = "test-index"
doctype = "test"
initial_index_name = "{}-{}".format(alias_name, datetime.datetime.now().strftime("%Y%m%S"))
es.indices.create(index=initial_index_name, body=index_config)
time.sleep(1)
for i in range(5):
doc_body = {"id": i, "title": str(datetime.datetime.now())}
indexname = "{}-{}".format(aliasname, datetime.datetime.now().strftime("%Y%m%S"))
rets = es.index(index=alias_name, doc_type=doctype, body=doc_body)
time.sleep(0.3)
在这里,您应该可以看到您的文档根据时间以多个索引结束。使用索引模板+别名,在低使用时间对旧索引进行单独的删除/收缩调用