elasticsearch Elasticsearch最新N条记录的聚合,elasticsearch,filter,aggregation,elasticsearch,Filter,Aggregation" /> elasticsearch Elasticsearch最新N条记录的聚合,elasticsearch,filter,aggregation,elasticsearch,Filter,Aggregation" />
Fatal编程技术网
  • elasticsearch/
  • elasticsearch Elasticsearch最新N条记录的聚合

elasticsearch Elasticsearch最新N条记录的聚合

filter

elasticsearch Elasticsearch最新N条记录的聚合,elasticsearch,filter,aggregation,elasticsearch,Filter,Aggregation,有没有办法对最新的N条记录进行聚合 这个解决方案不起作用 { "query": {...}, "size": N, "order": ..., "aggs": { .... } } 有关更多详细信息：我想从“服务名称”字段为“x”的记录中获取最后10条记录，然后对这10条记录进行汇总，以确定其中有多少条记录在“响应代码”字段中“成功” 我的数据是这样的： [ {

有没有办法对最新的N条记录进行聚合

这个解决方案不起作用

{
   "query": {...},
   "size": N,
   "order": ...,
   "aggs": {
       ....
   }
}
有关更多详细信息：我想从“服务名称”字段为“x”的记录中获取最后10条记录，然后对这10条记录进行汇总，以确定其中有多少条记录在“响应代码”字段中“成功”

我的数据是这样的：

[
  {
    "_index": "logs",
    "_type": "_doc",
    "_id": "1232525",
    "_score": 1,
    "_source": {
      "resp_body": "",
      "client_ip": "127.0.0.1",
      "resp_time": "2021-04-15T10:24:51+01:00",
      "@timestamp": "2021-04-15T05:55:00.452Z",
      "resp_code": "412",
      "service_name": "service1",
      "log_id": "1232525"
    }
  },
  {
    "_index": "logs",
    "_type": "_doc",
    "_id": "1232524",
    "_score": 1,
    "_source": {
      "resp_body": "",
      "client_ip": "127.0.0.1",
      "resp_time": "2021-04-15T10:23:51+01:00",
      "@timestamp": "2021-04-15T05:53:00.452Z",
      "resp_code": "0",
      "service_name": "service2",
      "log_id": "1232524"
    }
  },
  {
    "_index": "logs",
    "_type": "_doc",
    "_id": "1232523",
    "_score": 1,
    "_source": {
      "resp_body": "",
      "client_ip": "127.0.0.1",
      "resp_time": "2021-04-15T10:22:51+01:00",
      "@timestamp": "2021-04-15T05:52:00.452Z",
      "resp_code": "0",
      "service_name": "service1",
      "log_id": "1232523"
    }
  },
  {
    "_index": "logs",
    "_type": "_doc",
    "_id": "1232522",
    "_score": 1,
    "_source": {
      "resp_body": "",
      "client_ip": "127.0.0.1",
      "resp_time": "2021-04-15T10:21:51+01:00",
      "@timestamp": "2021-04-15T05:51:00.452Z",
      "resp_code": "0",
      "service_name": "service1",
      "log_id": "1232522"
    }
  },
  {
    "_index": "logs",
    "_type": "_doc",
    "_id": "1232521",
    "_score": 1,
    "_source": {
      "resp_body": "",
      "client_ip": "127.0.0.1",
      "resp_time": "2021-04-15T10:20:51+01:00",
      "@timestamp": "2021-04-15T05:50:00.452Z",
      "resp_code": "0",
      "service_name": "service2",
      "log_id": "1232521"
    }
  }
]
例如：我想获取最后两条记录的“service_name=service1”，并找出其中有多少条记录的“resp_code=0”

您需要使用和的组合才能获得所需的结果

  • 使用过滤器聚合（
    first\u filter
    ），首先，这些文档被过滤为
    “service\u name=service1”
  • 然后使用术语聚合（
    top\u terms\u aggregation
    ）根据
    log\u id
    字段创建过滤文档的存储桶。根据
    @timestamp
    字段（使用最大聚合），按
    desc
    顺序对这些存储桶进行排序
  • 再次使用过滤器聚合（
    second\u filter
    ），这些文档被过滤掉，并具有
    “resp\u code=0”
    • 添加带有索引映射、数据（与问题相同）、搜索查询和搜索结果的工作示例

    索引映射：

    {
  "mappings": {
    "properties": {
      "@timestamp": {
        "type": "date",
        "format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"
      }
    }
  }
}

    {
  "size": 0,
  "aggs": {
    "first_filter": {
      "filter": {
        "bool": {
          "must": [
            {
              "term": {
                "service_name.keyword": "service1"
              }
            }
          ]
        }
      },
      "aggs": {
        "top_terms_aggregation": {
          "terms": {
            "field": "log_id.keyword",
            "size": 10,
            "order": {
              "second_filter>latestRecord": "desc"
            }
          },
          "aggs": {
            "second_filter": {
              "filter": {
                "bool": {
                  "must": [
                    {
                      "term": {
                        "resp_code": "0"
                      }
                    }
                  ]
                }
              },
              "aggs": {
                "latestRecord": {
                  "max": {
                    "field": "@timestamp"
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

    "aggregations": {
    "first_filter": {
      "doc_count": 3,
      "top_terms_aggregation": {
        "doc_count_error_upper_bound": 0,
        "sum_other_doc_count": 0,
        "buckets": [
          {
            "key": "1232523",
            "doc_count": 1,
            "second_filter": {
              "doc_count": 1,
              "latestOrder": {
                "value": 1.618465920452E12,
                "value_as_string": "2021-04-15T05:52:00.452Z"      // note this
              }
            }
          },
          {
            "key": "1232522",
            "doc_count": 1,
            "second_filter": {
              "doc_count": 1,
              "latestOrder": {
                "value": 1.618465860452E12,
                "value_as_string": "2021-04-15T05:51:00.452Z"          // note this
              }
            }
          },
          {
            "key": "1232525",
            "doc_count": 1,
            "second_filter": {
              "doc_count": 0,
              "latestOrder": {
                "value": null
              }
            }
          }
        ]
      }
    }
  }
    搜索查询：

    {
  "mappings": {
    "properties": {
      "@timestamp": {
        "type": "date",
        "format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"
      }
    }
  }
}

    {
  "size": 0,
  "aggs": {
    "first_filter": {
      "filter": {
        "bool": {
          "must": [
            {
              "term": {
                "service_name.keyword": "service1"
              }
            }
          ]
        }
      },
      "aggs": {
        "top_terms_aggregation": {
          "terms": {
            "field": "log_id.keyword",
            "size": 10,
            "order": {
              "second_filter>latestRecord": "desc"
            }
          },
          "aggs": {
            "second_filter": {
              "filter": {
                "bool": {
                  "must": [
                    {
                      "term": {
                        "resp_code": "0"
                      }
                    }
                  ]
                }
              },
              "aggs": {
                "latestRecord": {
                  "max": {
                    "field": "@timestamp"
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

    "aggregations": {
    "first_filter": {
      "doc_count": 3,
      "top_terms_aggregation": {
        "doc_count_error_upper_bound": 0,
        "sum_other_doc_count": 0,
        "buckets": [
          {
            "key": "1232523",
            "doc_count": 1,
            "second_filter": {
              "doc_count": 1,
              "latestOrder": {
                "value": 1.618465920452E12,
                "value_as_string": "2021-04-15T05:52:00.452Z"      // note this
              }
            }
          },
          {
            "key": "1232522",
            "doc_count": 1,
            "second_filter": {
              "doc_count": 1,
              "latestOrder": {
                "value": 1.618465860452E12,
                "value_as_string": "2021-04-15T05:51:00.452Z"          // note this
              }
            }
          },
          {
            "key": "1232525",
            "doc_count": 1,
            "second_filter": {
              "doc_count": 0,
              "latestOrder": {
                "value": null
              }
            }
          }
        ]
      }
    }
  }
    搜索结果：

    {
  "mappings": {
    "properties": {
      "@timestamp": {
        "type": "date",
        "format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"
      }
    }
  }
}

    {
  "size": 0,
  "aggs": {
    "first_filter": {
      "filter": {
        "bool": {
          "must": [
            {
              "term": {
                "service_name.keyword": "service1"
              }
            }
          ]
        }
      },
      "aggs": {
        "top_terms_aggregation": {
          "terms": {
            "field": "log_id.keyword",
            "size": 10,
            "order": {
              "second_filter>latestRecord": "desc"
            }
          },
          "aggs": {
            "second_filter": {
              "filter": {
                "bool": {
                  "must": [
                    {
                      "term": {
                        "resp_code": "0"
                      }
                    }
                  ]
                }
              },
              "aggs": {
                "latestRecord": {
                  "max": {
                    "field": "@timestamp"
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

    "aggregations": {
    "first_filter": {
      "doc_count": 3,
      "top_terms_aggregation": {
        "doc_count_error_upper_bound": 0,
        "sum_other_doc_count": 0,
        "buckets": [
          {
            "key": "1232523",
            "doc_count": 1,
            "second_filter": {
              "doc_count": 1,
              "latestOrder": {
                "value": 1.618465920452E12,
                "value_as_string": "2021-04-15T05:52:00.452Z"      // note this
              }
            }
          },
          {
            "key": "1232522",
            "doc_count": 1,
            "second_filter": {
              "doc_count": 1,
              "latestOrder": {
                "value": 1.618465860452E12,
                "value_as_string": "2021-04-15T05:51:00.452Z"          // note this
              }
            }
          },
          {
            "key": "1232525",
            "doc_count": 1,
            "second_filter": {
              "doc_count": 0,
              "latestOrder": {
                "value": null
              }
            }
          }
        ]
      }
    }
  }
    你能分享一些样本索引数据和期望的搜索结果吗？你想得到基于时间戳的最后10条记录吗？您的数据中是否有任何字段可用于检索最新的N条记录？我希望最后10条记录基于ID或字段时间。很难假设字段值，您能否共享一些示例索引数据？@ESCoder我用示例编辑了问题data@Behzad爸爸，请把答案通读一遍，让我知道这是否解决了你的问题？