产品解决方案文档与社区免费试用定价云市场合作伙伴支持与服务了解阿里云
备案控制台登录/注册
开发者社区 数据库 文章 正文

一起来学ElasticSearch(九)

2023-05-31 8028
版权
版权声明：
本文内容由阿里云实名注册用户自发贡献，版权归原作者所有，阿里云开发者社区不拥有其著作权，亦不承担相应法律责任。具体规则请查看《 阿里云开发者社区用户服务协议》和 《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容，填写 侵权投诉表单进行举报，一经查实，本社区将立刻删除涉嫌侵权内容。
简介： 前言 目前正在出一个Es专题系列教程, 篇幅会较多, 喜欢的话，给个关注❤️ ~ 本节给大家讲下es中如何做聚合操作, 内容有点多，需要耐心看完~ 本文偏实战一些，好了, 废话不多说直接开整吧~ 什么是聚合 聚合的概念有点类似mysql中group by,sum(...)，这么说大家可能就有点印象了, 但是在es中聚合操作功能更强大。 在了解es中聚合的概念之前，先来看下这两个概念, 聚合就是一个或多个桶和零个或多个指标的组合。
+关注继续查看

前言

目前正在出一个Es专题系列教程, 篇幅会较多, 喜欢的话，给个关注❤️ ~

本节给大家讲下es中如何做聚合操作, 内容有点多，需要耐心看完~

本文偏实战一些，好了, 废话不多说直接开整吧~

什么是聚合

聚合的概念有点类似mysql中group by,sum(...)，这么说大家可能就有点印象了, 但是在es中聚合操作功能更强大。

在了解es中聚合的概念之前，先来看下这两个概念, 聚合就是一个或多个桶和零个或多个指标的组合。

聚合有着非常多的场景应用，比如后台报表，通常要做非常多的复杂统计，而且数据量庞大，如果单纯依靠数据库统计，速度会非常慢甚至拖垮数据库，而使用es就会相对容易一些

桶（Buckets）

可以理解为es中存储的文档，通过满足特定条件的集合，这就叫做桶。

当聚合开始被执行，每个文档里面的值通过计算来决定符合哪个桶的条件。如果匹配到，文档将放入相应的桶并接着进行聚合操作。桶也可以被嵌套在其他桶里面，提供层次化的或者有条件的划分方案。

指标（Metrics）

对桶内的文档进行计算，比如通过文档的值计算平均值，计算最大值最小值， 下面就带大家看看如何去进行聚合操作

聚合指令

通常语法如下:

GET index_name/_search
{
"aggs": {
 "NAME": {
   "AGG_TYPE": {}
 }
}
}

  • index_name: 索引名称
  • aggs: 聚合修饰符
  • NAME：自定义变量名称，用于返回聚合结果时的变量名
  • AGG_TYPE：聚合类型

注意NAME, AGG_TYPE是特定参数, 下面我们看看有哪些聚合类型：

  • terms：按照匹配条件进行聚合，可以按照条件将文档存入不同的桶中，进行后续操作
  • histogram：条形图（折方图），可以指定步长，按照步长递增进行聚合
  • date_histogram：时间条形图（折方图），可以指定时间频率，按照时间频率进行聚合
  • cardinality：去重计算，存在一定的误差值
  • percentiles：获取字段不同百分比数对应的值
  • percentile_ranks：获取值对应的百分比数
  • filter：对聚合结果进行过滤，对查询结果不过滤
  • post_filter：对聚合结果不过滤，对查询结果过滤
  • avg：计算平均值
  • sum：求和
  • min：最小值
  • max：最大值

下面看一个示例:

本节我们新建索引,下面是一个简单的请求日志索引, 定义了请求方法，路径，耗时，日志创建时间这几个字段

PUT req_log
{
"mappings": {
 "properties" : {
   "method" : {
     "type" : "keyword"
   },
   "path" : {
     "type" : "keyword"
   },
   "times" : {
     "type" : "long"
   },
   "created" : {
     "type" : "date"
   }
 }
}
}

紧接着，往里边塞点数据

POST /req_log/_bulk
{ "index": {}}
{ "times" : 80, "method" : "GET", "path" : "/api/post/1", "created" : "2023-02-09" }
{ "index": {}}
{ "times" : 30, "method" : "GET", "path" : "/api/post/2", "created" : "2023-02-07" }
{ "index": {}}
{ "times" : 20, "method" : "GET", "path" : "/api/post/3", "created" : "2023-02-08" }
{ "index": {}}
{ "times" : 120, "method" : "GET", "path" : "/api/post/20", "created" : "2023-02-06" }
{ "index": {}}
{ "times" : 150, "method" : "GET", "path" : "/api/post/1", "created" : "2023-02-05" }
{ "index": {}}
{ "times" : 80, "method" : "GET", "path" : "/api/post/3", "created" : "2023-02-04" }
{ "index": {}}
{ "times" : 960, "method" : "GET", "path" : "/api/post/6", "created" : "2023-02-03" }
{ "index": {}}
{ "times" : 9000, "method" : "GET", "path" : "/api/post/8", "created" : "2023-02-02" }
{ "index": {}}
{ "times" : 1300, "method" : "GET", "path" : "/api/post/6", "created" : "2023-02-01" }
{ "index": {}}
{ "times" : 400, "method" : "GET", "path" : "/api/post/4", "created" : "2023-02-10" }
{ "index": {}}
{ "times" : 89, "method" : "GET", "path" : "/api/post/3", "created" : "2023-02-11" }
{ "index": {}}
{ "times" : 380, "method" : "GET", "path" : "/api/post/2", "created" : "2023-02-12" }
{ "index": {}}
{ "times" : 270, "method" : "GET", "path" : "/api/post/10", "created" : "2023-02-13" }
{ "index": {}}
{ "times" : 630, "method" : "GET", "path" : "/api/post/12", "created" : "2023-02-14" }
{ "index": {}}
{ "times" : 210 , "method" : "GET", "path" : "/api/post/4", "created" : "2023-02-15" }
{ "index": {}}
{ "times" : 900, "method" : "GET", "path" : "/api/post/6", "created" : "2023-02-16" }
{ "index": {}}
{ "times" : 870, "method" : "GET", "path" : "/api/post/7", "created" : "2023-02-17" }

查询每个请求路径(path)下的请求数量

GET req_log/_search
{
"aggs": {
  "counts": {
    "terms": {
      "field": "path"
    }
  }
}
}

返回:

{
  "took" : 995,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 17,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-09"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 30,
          "method" : "GET",
          "path" : "/api/post/2",
          "created" : "2023-02-07"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "G0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 20,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-08"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 120,
          "method" : "GET",
          "path" : "/api/post/20",
          "created" : "2023-02-06"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 150,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-05"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-04"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "H0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 960,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-03"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 9000,
          "method" : "GET",
          "path" : "/api/post/8",
          "created" : "2023-02-02"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 1300,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-01"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 400,
          "method" : "GET",
          "path" : "/api/post/4",
          "created" : "2023-02-10"
        }
      }
    ]
  },
  "aggregations" : {
    "counts" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "/api/post/3",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/6",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/1",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/2",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/4",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/10",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/12",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/20",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/7",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/8",
          "doc_count" : 1
        }
      ]
    }
  }
}

可以看到自定义返回的字段counts，聚合的类型为terms,聚合字段为path,也就是按照path进行桶划分。

从结果来看也很明显,buckets分为了4个桶，key代表聚合的字段名称，doc_count代表文档的数量

terms还支持以下命令格式:

GET req_log/_search
{
"aggs": {
 "counts": {
   "terms": {
     "field": "path",
     "size": 10,
     "collect_mode": "depth_first",
     "order": {
       "_count": "desc"
     }
   }
 }
}
}

返回:


{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  ......,
  "aggregations" : {
    "counts" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "/api/post/3",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/6",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/1",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/2",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/4",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/10",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/12",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/20",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/7",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/8",
          "doc_count" : 1
        }
      ]
    }
  }
}
  • size：返回桶中的多少个数据，通常可以结合排序模式进行使用，默认值=10
  • collect_mode: 集合模式，包括深度优先遍历（depth_first）和广度优先遍历（breadth_first）两种。对于数组类型的字段，在使用深度优先遍历的情况下，可能会导致占用内存过多的情况。因为深度优先遍历会将数据全部加载到内存中后再进行操作
  • order 排序，默认按照doc_count倒序排列，可以指定默认字段或子聚合字段进行排序

嵌套聚合

es中默认支持聚合的嵌套，可以在一个桶中再次进行桶的划分, 嵌套有分为同级和子级,下面看一个例子:

同级嵌套

GET req_log/_search
{
"aggs": {
 "path_count": {
   "terms": {
     "field": "path"
   }
 },
 "method_count": {
   "terms": {
     "field": "method"
   }
 }
}
}

返回:

{
  "took" : 7,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 17,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-09"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 30,
          "method" : "GET",
          "path" : "/api/post/2",
          "created" : "2023-02-07"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "G0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 20,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-08"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 120,
          "method" : "GET",
          "path" : "/api/post/20",
          "created" : "2023-02-06"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 150,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-05"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-04"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "H0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 960,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-03"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 9000,
          "method" : "GET",
          "path" : "/api/post/8",
          "created" : "2023-02-02"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 1300,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-01"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 400,
          "method" : "GET",
          "path" : "/api/post/4",
          "created" : "2023-02-10"
        }
      }
    ]
  },
  "aggregations" : {
    "method_count" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "GET",
          "doc_count" : 17
        }
      ]
    },
    "path_count" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "/api/post/3",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/6",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/1",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/2",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/4",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/10",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/12",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/20",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/7",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/8",
          "doc_count" : 1
        }
      ]
    }
  }
}

从结果来看，很明显的看出，不同method和path下的请求数量

子级嵌套

假如，我现在有一个需求：

  • 同级各个请求方法下的请求数量
  • 各个请求method下各个path的请求数量
  • 各个path下请求耗时的平均值

查询示例:

GET req_log/_search
{
"aggs": {
 "method_count": {
   "terms": {
     "field": "method"
   },
   "aggs": {
     "path_count": {
       "terms": {
         "field": "path"
       },
       "aggs": {
         "avg_times": {
           "avg": {
             "field": "times"
           }
         }
       }
     }
   }
 }
}
}

返回:

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 17,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-09"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 30,
          "method" : "GET",
          "path" : "/api/post/2",
          "created" : "2023-02-07"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "G0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 20,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-08"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 120,
          "method" : "GET",
          "path" : "/api/post/20",
          "created" : "2023-02-06"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 150,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-05"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-04"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "H0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 960,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-03"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 9000,
          "method" : "GET",
          "path" : "/api/post/8",
          "created" : "2023-02-02"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 1300,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-01"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 400,
          "method" : "GET",
          "path" : "/api/post/4",
          "created" : "2023-02-10"
        }
      }
    ]
  },
  "aggregations" : {
    "method_count" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "GET",
          "doc_count" : 17,
          "path_count" : {
            "doc_count_error_upper_bound" : 0,
            "sum_other_doc_count" : 0,
            "buckets" : [
              {
                "key" : "/api/post/3",
                "doc_count" : 3,
                "avg_times" : {
                  "value" : 63.0
                }
              },
              {
                "key" : "/api/post/6",
                "doc_count" : 3,
                "avg_times" : {
                  "value" : 1053.3333333333333
                }
              },
              {
                "key" : "/api/post/1",
                "doc_count" : 2,
                "avg_times" : {
                  "value" : 115.0
                }
              },
              {
                "key" : "/api/post/2",
                "doc_count" : 2,
                "avg_times" : {
                  "value" : 205.0
                }
              },
              {
                "key" : "/api/post/4",
                "doc_count" : 2,
                "avg_times" : {
                  "value" : 305.0
                }
              },
              {
                "key" : "/api/post/10",
                "doc_count" : 1,
                "avg_times" : {
                  "value" : 270.0
                }
              },
              {
                "key" : "/api/post/12",
                "doc_count" : 1,
                "avg_times" : {
                  "value" : 630.0
                }
              },
              {
                "key" : "/api/post/20",
                "doc_count" : 1,
                "avg_times" : {
                  "value" : 120.0
                }
              },
              {
                "key" : "/api/post/7",
                "doc_count" : 1,
                "avg_times" : {
                  "value" : 870.0
                }
              },
              {
                "key" : "/api/post/8",
                "doc_count" : 1,
                "avg_times" : {
                  "value" : 9000.0
                }
              }
            ]
          }
        }
      ]
    }
  }
}

从结果来看，桶按照层级嵌套关系

聚合过滤

查询和聚合过滤

这种是最常见的过滤方法，就是对查询结果和聚合结果都进行过滤，在aggs同级加上一个query即可，query前几节都给大家讲过, 下面看一个示例:

GET req_log/_search
{
"query": {
 "constant_score": {
   "filter": {
     "term": {
       "method": "POST"
     }
   }
 }
},
"aggs": {
 "path_count": {
   "terms": {
     "field": "path"
   }
 }
}
}

返回:

{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "path_count" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [ ]
    }
  }
}

可以看到结果是空的，原因是我们过滤了方法，因为整个文档都不存在POST所以为空

有时候我们的需求是这样的，想要拿某个数据和整个文档数据做比较，这个怎么做呢？global:{}可以很方便的聚合全部文档,下面看一个示例:

查询path=/api/post/3下的平均请求耗时和整个请求下的平均请求耗时

GET req_log/_search
{
"query": {
 "constant_score": {
   "filter": {
     "term": {
       "path": "/api/post/3"
     }
   }
 }
},
"aggs": {
 "path_avg": {
   "avg": {
     "field": "times"
   }
 },
 "all_order":{
   "global": {},
   "aggs": {
     "all_avg": {
       "avg": {
         "field": "times"
       }
     }
   }
 }
}
}

返回:

{
  "took" : 11,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 3,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "G0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 20,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-08"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-04"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "I0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 89,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-11"
        }
      }
    ]
  },
  "aggregations" : {
    "path_avg" : {
      "value" : 63.0
    },
    "all_order" : {
      "doc_count" : 17,
      "all_avg" : {
        "value" : 911.1176470588235
      }
    }
  }
}

从结果比较来看api/post/3下的接口请求速度还是很快的，平均是63, 整个接口平均耗时是911

聚合过滤

有时候，我们不需要过滤查询结果，只需要过滤聚合结果，这个怎么做呢？下面接着看一个示例:

查询method=GET下的请求并计算出path=/api/post/3下的请求平均耗时

  • filter
GET req_log/_search
{
 "query":{
     "constant_score":{
         "filter":{
             "term":{
                 "method":"GET"
             }
         }
     }
 },
 "aggs":{
     "req_count":{
         "aggs":{
             "req_path_order":{
                 "terms":{
                     "field":"path"
                 },
                 "aggs":{
                     "avg_times":{
                         "avg":{
                             "field":"times"
                         }
                     }
                 }
             }
         },
         "filter":{
             "term":{
                 "path":"/api/post/3"
             }
         }
     }
 }
}

来看返回结果:


{
  "took" : 6,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 17,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-09"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "GkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 30,
          "method" : "GET",
          "path" : "/api/post/2",
          "created" : "2023-02-07"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "G0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 20,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-08"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 120,
          "method" : "GET",
          "path" : "/api/post/20",
          "created" : "2023-02-06"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 150,
          "method" : "GET",
          "path" : "/api/post/1",
          "created" : "2023-02-05"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-04"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "H0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 960,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-03"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IEK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 9000,
          "method" : "GET",
          "path" : "/api/post/8",
          "created" : "2023-02-02"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IUK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 1300,
          "method" : "GET",
          "path" : "/api/post/6",
          "created" : "2023-02-01"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "IkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 400,
          "method" : "GET",
          "path" : "/api/post/4",
          "created" : "2023-02-10"
        }
      }
    ]
  },
  "aggregations" : {
    "req_count" : {
      "doc_count" : 3,
      "req_path_order" : {
        "doc_count_error_upper_bound" : 0,
        "sum_other_doc_count" : 0,
        "buckets" : [
          {
            "key" : "/api/post/3",
            "doc_count" : 3,
            "avg_times" : {
              "value" : 63.0
            }
          }
        ]
      }
    }
  }
}

从结果来看,请求查询的结果并没有被过滤，只有聚合的结果被过滤了

查询过滤 & 聚合不过滤

与上相反，值过滤查询，不过滤聚合结果, 下面看个示例:

查询path=/api/post/3并且method=GET下的请求,并聚合结果各个path下的请求数

  • post_filter
GET req_log/_search
{
"aggs": {
 "path_count": {
   "terms": {
     "field": "path"
   }
 }
},
"post_filter": {
 "bool": {
   "must": [
     {
       "term": {
         "path": {
           "value": "/api/post/3"
         }
       }
     },
     {
       "term": {
         "method": {
           "value": "GET"
         }
       }
     }
   ]
 }
}
}

返回:

{
  "took" : 4,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 3,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "G0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 20,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-08"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "HkK3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 80,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-04"
        }
      },
      {
        "_index" : "req_log",
        "_type" : "_doc",
        "_id" : "I0K3NIYBdXrpvlCF01bz",
        "_score" : 1.0,
        "_source" : {
          "times" : 89,
          "method" : "GET",
          "path" : "/api/post/3",
          "created" : "2023-02-11"
        }
      }
    ]
  },
  "aggregations" : {
    "path_count" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "/api/post/3",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/6",
          "doc_count" : 3
        },
        {
          "key" : "/api/post/1",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/2",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/4",
          "doc_count" : 2
        },
        {
          "key" : "/api/post/10",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/12",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/20",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/7",
          "doc_count" : 1
        },
        {
          "key" : "/api/post/8",
          "doc_count" : 1
        }
      ]
    }
  }
}

从结果来看,查询结果只有api/post/3但是聚合结果并没有过滤这个条件

结束语

本节就到此结束了，还有一些聚合内容放到下节讲，先消化这么多 ~

本着把自己知道的都告诉大家，如果本文对您有所帮助，点赞+关注鼓励一下呗~

相关文章

项目源码(源码已更新 欢迎star⭐️)

往期并发编程内容推荐

博客(阅读体验较佳)

推荐 SpringBoot & SpringCloud (源码已更新 欢迎star⭐️)












































文章标签：
检索分析服务 Elasticsearch版
Java
容器
BI
关系型数据库
索引
Docker
MySQL
API
数据库
存储
相关实践学习
使用阿里云Elasticsearch体验信息检索加速
通过创建登录阿里云Elasticsearch集群，使用DataWorks将MySQL数据同步至Elasticsearch，体验多条件检索效果，简单展示数据同步和信息检索加速的过程和操作。
ElasticSearch 入门精讲
ElasticSearch是一个开源的、基于Lucene的、分布式、高扩展、高实时的搜索与数据分析引擎。根据DB-Engines的排名显示，Elasticsearch是最受欢迎的企业搜索引擎，其次是Apache Solr（也是基于Lucene）。 ElasticSearch的实现原理主要分为以下几个步骤： 用户将数据提交到Elastic Search 数据库中 通过分词控制器去将对应的语句分词，将其权重和分词结果一并存入数据 当用户搜索数据时候，再根据权重将结果排名、打分 将返回结果呈现给用户 Elasticsearch可以用于搜索各种文档。它提供可扩展的搜索，具有接近实时的搜索，并支持多租户。
程序员皮卡秋(公众号程序员皮卡秋)
目录
相关文章
凌虚Wayne
|
7天前
|
存储 关系型数据库 MySQL
Elasticsearch（一）
Elasticsearch（一）
凌虚Wayne
10 0
程序员皮卡秋(公众号程序员皮卡秋)
|
7月前
|
自然语言处理 Java 中间件
一起来学ElasticSearch(八)
一起来学ElasticSearch(八)
程序员皮卡秋(公众号程序员皮卡秋)
62 0
程序员皮卡秋(公众号程序员皮卡秋)
|
7月前
|
存储 JSON 自然语言处理
一起来学ElasticSearch(二)
一起来学ElasticSearch(二)
程序员皮卡秋(公众号程序员皮卡秋)
101 0
程序员皮卡秋(公众号程序员皮卡秋)
|
7月前
|
JSON Java API
一起来学ElasticSearch(一)
一起来学ElasticSearch(一)
程序员皮卡秋(公众号程序员皮卡秋)
131 0
游客4jrkw4st6g47g
|
11月前
|
SQL 搜索推荐 数据挖掘
大话ElasticSearch（下）
搜索引擎简述 什么是搜索？ 搜索：就是在任何场景下，找寻你想要的信息，这个时候，会输入一段你要搜索的关键字，然后就期望找到这个关键字相关的有些信息。 搜索分类： 普通的网页搜索、垂直搜索引擎等
游客4jrkw4st6g47g
58 0
互联网-小阿宇
|
11月前
|
缓存 自然语言处理 安全
elasticsearch
elasticsearch
互联网-小阿宇
65 0
elasticsearch
vohelon
|
数据采集 存储 数据可视化
Elasticsearch 是什么？
Elasticsearch 是什么？
vohelon
76 0
唐家四少官微
|
存储 JSON 关系型数据库
ElasticSearch介绍
es学习笔记
唐家四少官微
114 0
vohelon
|
存储 JSON 搜索推荐
Elasticsearch是什么
Elasticsearch是什么
vohelon
77 0
vohelon
|
搜索推荐 Java Apache
Elasticsearch
Elasticsearch
vohelon
83 0
热门文章
最新文章
1
全球最大成人网站YouPorn推出漏洞悬赏计划：最高可获25000美元
2
从零入门 Serverless | 一文详解 Serverless 架构模式
3
iphoneX都面世了，你的数据仓库还停留在诺基亚时代吗?
4
《企业大数据系统构建实战：技术、架构、实施与应用》一2.4　本章小结
5
浅谈Java中的对象和对象引用
6
MySQL抑制binlog日志中的BINLOG部分
7
酷派新品发布会将揭开酷派D550的神秘面纱
8
铭记留不住的青春
9
linux内核网络接收数据流程图【转】
10
没有#!的bash脚本的执行
1
Unix后记&寻找Shen Lin
23
2
压缩算法---以golang/snappy为例
18
3
链表问题汇集
15
4
MySQL长文本字段的选取
22
5
数组问题汇集
13
6
栈和队列问题汇集
13
7
Linux内核分析与应用9-设备驱动
23
8
Linux内核分析与应用8-文件系统
21
9
不用任何比较运算符找出两个整数中的较大的值
11
相关课程
更多
ElasticSearch 最新快速入门教程 ElasticSearch 入门精讲 Kibana Dashboard 入门
相关电子书
更多
阿里云Elasticsearch 2.0发布 开源与云：Elasticsearch应用剖析 阿里云数据智能-Elasticsearch分享
相关实验场景
更多
使用阿里云Elasticsearch快速搭建可观测系统
推荐文章
更多
重磅来袭！参与评测赢Iphone14 pro！ 文件存储NAS评测征集令！ 招募！寻找技术人的伯乐！ 乘风者计划邀您入驻社区，精彩权益即刻享
下一篇
部署并使用Docker（CentOS 8）