备案控制台
探索云世界
开发者社区 数据库 文章 正文

一起来学ElasticSearch(九)

2023-05-31 8048
版权
版权声明：
本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《 阿里云开发者社区用户服务协议》和 《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写 侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。
本文涉及的产品
检索分析服务 Elasticsearch 版，2核4GB开发者规格 1个月
推荐场景：
数据可视化分析航班信息
简介： 前言目前正在出一个Es专题系列教程, 篇幅会较多, 喜欢的话，给个关注❤️ ~本节给大家讲下es中如何做聚合操作, 内容有点多，需要耐心看完~本文偏实战一些，好了, 废话不多说直接开整吧~什么是聚合聚合的概念有点类似mysql中group by,sum(...)，这么说大家可能就有点印象了, 但是在es中聚合操作功能更强大。在了解es中聚合的概念之前，先来看下这两个概念, 聚合就是一个或多个桶和零个或多个指标的组合。

前言

目前正在出一个Es专题系列教程, 篇幅会较多, 喜欢的话，给个关注❤️ ~

本节给大家讲下es中如何做聚合操作, 内容有点多，需要耐心看完~

本文偏实战一些，好了, 废话不多说直接开整吧~

什么是聚合

聚合的概念有点类似mysql中group by,sum(...)，这么说大家可能就有点印象了, 但是在es中聚合操作功能更强大。

在了解es中聚合的概念之前，先来看下这两个概念, 聚合就是一个或多个桶和零个或多个指标的组合。

聚合有着非常多的场景应用，比如后台报表，通常要做非常多的复杂统计，而且数据量庞大，如果单纯依靠数据库统计，速度会非常慢甚至拖垮数据库，而使用es就会相对容易一些

桶（Buckets）

可以理解为es中存储的文档，通过满足特定条件的集合，这就叫做桶。

当聚合开始被执行，每个文档里面的值通过计算来决定符合哪个桶的条件。如果匹配到，文档将放入相应的桶并接着进行聚合操作。桶也可以被嵌套在其他桶里面，提供层次化的或者有条件的划分方案。

指标（Metrics）

对桶内的文档进行计算，比如通过文档的值计算平均值，计算最大值最小值， 下面就带大家看看如何去进行聚合操作

聚合指令

通常语法如下:

GET index_name/_search
{
"aggs": {
 "NAME": {
   "AGG_TYPE": {}
 }
}
}

 

  • index_name: 索引名称
  • aggs: 聚合修饰符
  • NAME：自定义变量名称，用于返回聚合结果时的变量名
  • AGG_TYPE：聚合类型

注意NAME,  AGG_TYPE是特定参数, 下面我们看看有哪些聚合类型：

  • terms：按照匹配条件进行聚合，可以按照条件将文档存入不同的桶中，进行后续操作
  • histogram：条形图（折方图），可以指定步长，按照步长递增进行聚合
  • date_histogram：时间条形图（折方图），可以指定时间频率，按照时间频率进行聚合
  • cardinality：去重计算，存在一定的误差值
  • percentiles：获取字段不同百分比数对应的值
  • percentile_ranks：获取值对应的百分比数
  • filter：对聚合结果进行过滤，对查询结果不过滤
  • post_filter：对聚合结果不过滤，对查询结果过滤
  • avg：计算平均值
  • sum：求和
  • min：最小值
  • max：最大值

下面看一个示例:

本节我们新建索引,下面是一个简单的请求日志索引, 定义了请求方法，路径，耗时，日志创建时间这几个字段

PUT req_log
{
"mappings": {
 "properties" : {
   "method" : {
     "type" : "keyword"
   },
   "path" : {
     "type" : "keyword"
   },
   "times" : {
     "type" : "long"
   },
   "created" : {
     "type" : "date"
   }
 }
}
}

紧接着，往里边塞点数据

POST /req_log/_bulk
{ "index": {}}
{ "times" : 80, "method" : "GET", "path" : "/api/post/1", "created" : "2023-02-09" }
{ "index": {}}
{ "times" : 30, "method" : "GET", "path" : "/api/post/2", "created" : "2023-02-07" }
{ "index": {}}
{ "times" : 20, "method" : "GET", "path" : "/api/post/3", "created" : "2023-02-08" }
{ "index": {}}
{ "times" : 120, "method" : "GET", "path" : "/api/post/20", "created" : "2023-02-06" }
{ "index": {}}
{ "times" : 150, "method" : "GET", "path" : "/api/post/1", "created" : "2023-02-05" }
{ "index": {}}
{ "times" : 80, "method" : "GET", "path" : "/api/post/3", "created" : "2023-02-04" }
{ "index": {}}
{ "times" : 960, "method" : "GET", "path" : "/api/post/6", "created" : "2023-02-03" }
{ "index": {}}
{ "times" : 9000, "method" : "GET", "path" : "/api/post/8", "created" : "2023-02-02" }
{ "index": {}}
{ "times" : 1300, "method" : "GET", "path" : "/api/post/6", "created" : "2023-02-01" }
{ "index": {}}
{ "times" : 400, "method" : "GET", "path" : "/api/post/4", "created" : "2023-02-10" }
{ "index": {}}
{ "times" : 89, "method" : "GET", "path" : "/api/post/3", "created" : "2023-02-11" }
{ "index": {}}
{ "times" : 380, "method" : "GET", "path" : "/api/post/2", "created" : "2023-02-12" }
{ "index": {}}
{ "times" : 270, "method" : "GET", "path" : "/api/post/10", "created" : "2023-02-13" }
{ "index": {}}
{ "times" : 630, "method" : "GET", "path" : "/api/post/12", "created" : "2023-02-14" }
{ "index": {}}
{ "times" : 210 , "method" : "GET", "path" : "/api/post/4", "created" : "2023-02-15" }
{ "index": {}}
{ "times" : 900, "method" : "GET", "path" : "/api/post/6", "created" : "2023-02-16" }
{ "index": {}}
{ "times" : 870, "method" : "GET", "path" : "/api/post/7", "created" : "2023-02-17" }

查询每个请求路径(path)下的请求数量

GET req_log/_search
{
"aggs": {
  "counts": {
    "terms": {
      "field": "path"
    }
  }
}
}

返回:

{
  "took" : 995,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 17,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-09"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 30,
          "method" : "GET",
          "path" : "/api/post/2",
          "created" : "2023-02-07"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "G0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 20,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-08"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 120,
          "method" : "GET",
          "path" : "/api/post/20",
          "created" : "2023-02-06"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 150,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-05"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-04"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "H0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 960,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-03"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 9000,
          "method" : "GET",
          "path" : "/api/post/8",
          "created" : "2023-02-02"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 1300,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-01"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 400,
          "method" : "GET",
          "path" : "/api/post/4",
          "created" : "2023-02-10"
        }
      }
    ]
  },
  "aggregations" : {
    "counts" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "/api/post/3",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/6",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/1",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/2",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/4",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/10",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/12",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/20",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/7",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/8",
          "doc_count" : 1
        }
      ]
    }
  }
}

可以看到自定义返回的字段counts，聚合的类型为terms,聚合字段为path,也就是按照path进行桶划分。

从结果来看也很明显,buckets分为了4个桶，key代表聚合的字段名称，doc_count代表文档的数量

terms还支持以下命令格式:

GET req_log/_search
{
"aggs": {
 "counts": {
   "terms": {
     "field": "path",
     "size": 10,
     "collect_mode": "depth_first",
     "order": {
       "_count": "desc"
     }
   }
 }
}
}

返回:

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  ......,
  "aggregations" : {
    "counts" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "/api/post/3",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/6",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/1",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/2",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/4",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/10",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/12",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/20",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/7",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/8",
          "doc_count" : 1
        }
      ]
    }
  }
}
  • size：返回桶中的多少个数据，通常可以结合排序模式进行使用，默认值=10
  • collect_mode: 集合模式，包括深度优先遍历（depth_first）和广度优先遍历（breadth_first）两种。对于数组类型的字段，在使用深度优先遍历的情况下，可能会导致占用内存过多的情况。因为深度优先遍历会将数据全部加载到内存中后再进行操作
  • order 排序，默认按照doc_count倒序排列，可以指定默认字段或子聚合字段进行排序

嵌套聚合

es中默认支持聚合的嵌套，可以在一个桶中再次进行桶的划分, 嵌套有分为同级和子级,下面看一个例子:

同级嵌套

GET req_log/_search
{
"aggs": {
 "path_count": {
   "terms": {
     "field": "path"
   }
 },
 "method_count": {
   "terms": {
     "field": "method"
   }
 }
}
}

返回:

{
  "took" : 7,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 17,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-09"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 30,
          "method" : "GET",
          "path" : "/api/post/2",
          "created" : "2023-02-07"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "G0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 20,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-08"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 120,
          "method" : "GET",
          "path" : "/api/post/20",
          "created" : "2023-02-06"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 150,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-05"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-04"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "H0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 960,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-03"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 9000,
          "method" : "GET",
          "path" : "/api/post/8",
          "created" : "2023-02-02"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 1300,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-01"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 400,
          "method" : "GET",
          "path" : "/api/post/4",
          "created" : "2023-02-10"
        }
      }
    ]
  },
  "aggregations" : {
    "method_count" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "GET",
          "doc_count" : 17
        }
      ]
    },
    "path_count" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "/api/post/3",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/6",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/1",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/2",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/4",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/10",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/12",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/20",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/7",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/8",
          "doc_count" : 1
        }
      ]
    }
  }
}

从结果来看，很明显的看出，不同method和path下的请求数量

子级嵌套

假如，我现在有一个需求：

  • 同级各个请求方法下的请求数量
  • 各个请求method下各个path的请求数量
  • 各个path下请求耗时的平均值

查询示例:

GET req_log/_search
{
"aggs": {
 "method_count": {
   "terms": {
     "field": "method"
   },
   "aggs": {
     "path_count": {
       "terms": {
         "field": "path"
       },
       "aggs": {
         "avg_times": {
           "avg": {
             "field": "times"
           }
         }
       }
     }
   }
 }
}
}

返回:

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 17,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-09"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 30,
          "method" : "GET",
          "path" : "/api/post/2",
          "created" : "2023-02-07"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "G0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 20,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-08"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 120,
          "method" : "GET",
          "path" : "/api/post/20",
          "created" : "2023-02-06"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 150,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-05"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-04"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "H0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 960,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-03"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 9000,
          "method" : "GET",
          "path" : "/api/post/8",
          "created" : "2023-02-02"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 1300,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-01"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 400,
          "method" : "GET",
          "path" : "/api/post/4",
          "created" : "2023-02-10"
        }
      }
    ]
  },
  "aggregations" : {
    "method_count" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "GET",
          "doc_count" : 17,
          "path_count" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "/api/post/3",
                "doc_count" : 3,
                "avg_times" : {
                  "value" : 63.0
                }
              },
              {
                "key" : "/api/post/6",
                "doc_count" : 3,
                "avg_times" : {
                  "value" : 1053.3333333333333
                }
              },
              {
                "key" : "/api/post/1",
                "doc_count" : 2,
                "avg_times" : {
                  "value" : 115.0
                }
              },
              {
                "key" : "/api/post/2",
                "doc_count" : 2,
                "avg_times" : {
                  "value" : 205.0
                }
              },
              {
                "key" : "/api/post/4",
                "doc_count" : 2,
                "avg_times" : {
                  "value" : 305.0
                }
              },
              {
                "key" : "/api/post/10",
                "doc_count" : 1,
                "avg_times" : {
                  "value" : 270.0
                }
              },
              {
                "key" : "/api/post/12",
                "doc_count" : 1,
                "avg_times" : {
                  "value" : 630.0
                }
              },
              {
                "key" : "/api/post/20",
                "doc_count" : 1,
                "avg_times" : {
                  "value" : 120.0
                }
              },
              {
                "key" : "/api/post/7",
                "doc_count" : 1,
                "avg_times" : {
                  "value" : 870.0
                }
              },
              {
                "key" : "/api/post/8",
                "doc_count" : 1,
                "avg_times" : {
                  "value" : 9000.0
                }
              }
            ]
          }
        }
      ]
    }
  }
}

从结果来看，桶按照层级嵌套关系

聚合过滤

查询和聚合过滤

这种是最常见的过滤方法，就是对查询结果和聚合结果都进行过滤，在aggs同级加上一个query即可，query前几节都给大家讲过, 下面看一个示例:

GET req_log/_search
{
"query": {
 "constant_score": {
   "filter": {
     "term": {
       "method": "POST"
     }
   }
 }
},
"aggs": {
 "path_count": {
   "terms": {
     "field": "path"
   }
 }
}
}

返回:

{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "path_count" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [ ]
    }
  }
}

可以看到结果是空的，原因是我们过滤了方法，因为整个文档都不存在POST所以为空

有时候我们的需求是这样的，想要拿某个数据和整个文档数据做比较，这个怎么做呢？global:{}可以很方便的聚合全部文档,下面看一个示例:

查询path=/api/post/3下的平均请求耗时和整个请求下的平均请求耗时

GET req_log/_search
{
"query": {
 "constant_score": {
   "filter": {
     "term": {
       "path": "/api/post/3"
     }
   }
 }
},
"aggs": {
 "path_avg": {
   "avg": {
     "field": "times"
   }
 },
 "all_order":{
   "global": {},
   "aggs": {
     "all_avg": {
       "avg": {
         "field": "times"
       }
     }
   }
 }
}
}

返回:

{
  "took" : 11,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 3,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "G0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 20,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-08"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-04"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "I0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 89,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-11"
        }
      }
    ]
  },
  "aggregations" : {
    "path_avg" : {
      "value" : 63.0
    },
    "all_order" : {
      "doc_count" : 17,
      "all_avg" : {
        "value" : 911.1176470588235
      }
    }
  }
}

从结果比较来看api/post/3下的接口请求速度还是很快的，平均是63, 整个接口平均耗时是911

聚合过滤

有时候，我们不需要过滤查询结果，只需要过滤聚合结果，这个怎么做呢？下面接着看一个示例:

查询method=GET下的请求并计算出path=/api/post/3下的请求平均耗时

  • filter
GET req_log/_search
{
 "query":{
     "constant_score":{
         "filter":{
             "term":{
                 "method":"GET"
             }
         }
     }
 },
 "aggs":{
     "req_count":{
         "aggs":{
             "req_path_order":{
                 "terms":{
                     "field":"path"
                 },
                 "aggs":{
                     "avg_times":{
                         "avg":{
                             "field":"times"
                         }
                     }
                 }
             }
         },
         "filter":{
             "term":{
                 "path":"/api/post/3"
             }
         }
     }
 }
}

来看返回结果:

{
  "took" : 6,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 17,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-09"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 30,
          "method" : "GET",
          "path" : "/api/post/2",
          "created" : "2023-02-07"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "G0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 20,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-08"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 120,
          "method" : "GET",
          "path" : "/api/post/20",
          "created" : "2023-02-06"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 150,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-05"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-04"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "H0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 960,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-03"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 9000,
          "method" : "GET",
          "path" : "/api/post/8",
          "created" : "2023-02-02"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 1300,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-01"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 400,
          "method" : "GET",
          "path" : "/api/post/4",
          "created" : "2023-02-10"
        }
      }
    ]
  },
  "aggregations" : {
    "req_count" : {
      "doc_count" : 3,
      "req_path_order" : {
        "doc_count_error_upper_bound" : 0,
        "sum_other_doc_count" : 0,
        "buckets" : [
          {
            "key" : "/api/post/3",
            "doc_count" : 3,
            "avg_times" : {
              "value" : 63.0
            }
          }
        ]
      }
    }
  }
}

从结果来看,请求查询的结果并没有被过滤，只有聚合的结果被过滤了

查询过滤 & 聚合不过滤

与上相反，值过滤查询，不过滤聚合结果, 下面看个示例:

查询path=/api/post/3并且method=GET下的请求,并聚合结果各个path下的请求数

  • post_filter
GET req_log/_search
{
"aggs": {
 "path_count": {
   "terms": {
     "field": "path"
   }
 }
},
"post_filter": {
 "bool": {
   "must": [
     {
       "term": {
         "path": {
           "value": "/api/post/3"
         }
       }
     },
     {
       "term": {
         "method": {
           "value": "GET"
         }
       }
     }
   ]
 }
}
}

返回:

{
  "took" : 4,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 3,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "G0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 20,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-08"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-04"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "I0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 89,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-11"
        }
      }
    ]
  },
  "aggregations" : {
    "path_count" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "/api/post/3",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/6",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/1",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/2",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/4",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/10",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/12",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/20",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/7",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/8",
          "doc_count" : 1
        }
      ]
    }
  }
}

从结果来看,查询结果只有api/post/3但是聚合结果并没有过滤这个条件

结束语

本节就到此结束了，还有一些聚合内容放到下节讲，先消化这么多 ~

本着把自己知道的都告诉大家，如果本文对您有所帮助，点赞+关注鼓励一下呗~

相关文章

项目源码(源码已更新 欢迎star⭐️)

往期并发编程内容推荐

博客(阅读体验较佳)

推荐 SpringBoot & SpringCloud (源码已更新 欢迎star⭐️)












































文章标签：
检索分析服务 Elasticsearch版
Java
容器
BI
关系型数据库
索引
Docker
MySQL
API
数据库
存储
相关实践学习
使用阿里云Elasticsearch体验信息检索加速
通过创建登录阿里云Elasticsearch集群，使用DataWorks将MySQL数据同步至Elasticsearch，体验多条件检索效果，简单展示数据同步和信息检索加速的过程和操作。
ElasticSearch 入门精讲
ElasticSearch是一个开源的、基于Lucene的、分布式、高扩展、高实时的搜索与数据分析引擎。根据DB-Engines的排名显示，Elasticsearch是最受欢迎的企业搜索引擎，其次是Apache Solr（也是基于Lucene）。 ElasticSearch的实现原理主要分为以下几个步骤： 用户将数据提交到Elastic Search 数据库中 通过分词控制器去将对应的语句分词，将其权重和分词结果一并存入数据 当用户搜索数据时候，再根据权重将结果排名、打分 将返回结果呈现给用户 Elasticsearch可以用于搜索各种文档。它提供可扩展的搜索，具有接近实时的搜索，并支持多租户。
程序员皮卡秋(公众号程序员皮卡秋)
目录
相关文章
hsfxuebao
|
2月前
|
存储 搜索推荐 关系型数据库
为什么需要 Elasticsearch
为什么需要 Elasticsearch
hsfxuebao
21 0
龙哥手记
|
JSON Java API
ElasticSearch - WrapperQuery
笔记
龙哥手记
480 0
陶然同学
|
3月前
|
搜索推荐 关系型数据库 MySQL
【Elasticsearch】初识elasticsearch
【Elasticsearch】初识elasticsearch
陶然同学
41 1
人不走空
|
4月前
|
存储 JSON 监控
Elasticsearch简单讲解
Elasticsearch简单讲解
人不走空
51 0
凌虚Wayne
|
8月前
|
存储 关系型数据库 MySQL
Elasticsearch（一）
Elasticsearch（一）
凌虚Wayne
41 0
凌虚Wayne
|
8月前
|
存储 关系型数据库 MySQL
Elasticsearch（二）
Elasticsearch（二）
凌虚Wayne
39 0
凌虚Wayne
|
8月前
|
存储 缓存 索引
Elasticsearch（四）
Elasticsearch（四）
凌虚Wayne
33 0
陶然同学
|
10月前
|
存储 监控 搜索推荐
【Elasticsearch】初识elasticsearch（上）
【Elasticsearch】初识elasticsearch
陶然同学
54 0
程序员皮卡秋(公众号程序员皮卡秋)
|
12月前
|
前端开发 Java API
一起来学ElasticSearch(十)
前言 目前正在出一个Es专题系列教程, 篇幅会较多, 喜欢的话，给个关注❤️ ~ 承接上文，上节给大家讲的es聚合还有一点内容，本节给大家更完~ 本文偏实战一些，为了方便演示,本节示例沿用上节索引,好了, 废话不多说直接开整吧~ 聚合排序 我们如何在聚合结果中进行自定义字段排序呢?
程序员皮卡秋(公众号程序员皮卡秋)
47 0
陶然同学
|
自然语言处理 索引
如果还不会Elasticsearch这七个问题 那么你的Elasticsearch白学
如果还不会Elasticsearch这七个问题 那么你的Elasticsearch白学
陶然同学
56 0

热门文章

最新文章

  • 1
    《CMOS集成电路后端设计与实战》——3.3　标准单元设计流程
  • 2
    无人驾驶背后的技术 - PostGIS点云(pointcloud)应用
  • 3
    反垃圾邮件技术已转移到云计算中
  • 4
    MYSQL数据库主从复制(一)
  • 5
    Keepalived+MySQL主从切换演示【书里讲解】
  • 6
    数据库总结20160728
  • 7
    c++11新特性(4) lambda捕捉块
  • 8
    0225Snapshot Too Old Error ora-01555.txt
  • 9
    libvirtVirsh
  • 10
    What are TCHAR, WCHAR, LPSTR, LPWSTR, LPCTSTR (etc.)?
  • 1
    【大模型】讨论 LLM 在更广泛的通用人工智能 （AGI） 领域中的作用
    30
  • 2
    【大模型】在实际应用中大规模部署 LLM 会带来哪些挑战？
    23
  • 3
    【大模型】小样本学习的概念及其在微调 LLM 中的应用
    24
  • 4
    【大模型】LLM 如何处理域外或无意义的提示？
    19
  • 5
    【大模型】在使用大语言模型的过程中，我们需要考虑到哪些道德问题？
    18
  • 6
    【大模型】大语言模型存在的一些限制
    14
  • 7
    【大模型】描述一些评估 LLM 性能的技术
    11
  • 8
    【大模型】如何使用提示工程来改善 LLM 输出？
    9
  • 9
    【大模型】大语言模型训练数据中的偏差概念及其可能的影响？
    8
  • 10
    【大模型】在大语言模型的架构中，Transformer有何作用？
    14

    • 相关课程

    更多
  • ElasticSearch 最新快速入门教程
  • ElasticSearch 入门精讲
  • Kibana Dashboard 入门
  • MongoDB快速入门

    • 相关电子书

    更多
  • 阿里云Elasticsearch 2.0发布
  • 阿里云数据智能-Elasticsearch分享
  • 开源与云：Elasticsearch应用剖析

    • 相关实验场景

    更多
  • 基于Elasticsearch向量检索的以文搜图
  • 利用Elasticsearch实现地理位置查询
  • 使用阿里云Elasticsearch快速搭建可观测系统
    • 下一篇
    将Stable Diffusion模型文件转存到FC环境的NAS