Elasticsearch分析聚合

简介: Elasticsearch不仅仅适合做全文检索,分析聚合功能也很好用。下面通过实例来学习。一、准备数据{"index":{ "_index": "books", "_type": "IT", "_id": "...

Elasticsearch不仅仅适合做全文检索,分析聚合功能也很好用。下面通过实例来学习。

一、准备数据

{"index":{ "_index": "books", "_type": "IT", "_id": "1" }}
{"id":"1","title":"Java编程思想","language":"java","author":"Bruce Eckel","price":70.20,"year":    2007,"description":"Java学习必读经典,殿堂级著作!赢得了全球程序员的广泛赞誉。"}

{"index":{ "_index": "books", "_type": "IT", "_id": "2" }}
{"id":"2","title":"Java程序性能优化","language":"java","author":"葛一鸣","price":46.50,"year":     2012,"description":"让你的Java程序更快、更稳定。深入剖析软件设计层面、代码层面、JVM虚拟机层面的优化方法"}

{"index":{ "_index": "books", "_type": "IT", "_id": "3" }}
{"id":"3","title":"Python科学计算","language":"python","author":"张若愚","price":81.40,"year":    2016,"description":"零基础学python,光盘中作者独家整合开发winPython运行环境,涵盖了Python各个扩展库"}

{"index":{ "_index": "books", "_type": "IT", "_id": "4" }}
{"id":"4","title":"Python基础教程","language":"python","author":"张若愚","price":54.50,"year": 2014,"description":"经典的Python入门教程,层次鲜明,结构严谨,内容翔实"}

{"index":{ "_index": "books", "_type": "IT", "_id": "5" }}
{"id":"5","title":"JavaScript高级程序设计","language":"javascript","author":"Nicholas C.Zakas","price":66.40,"year":2012,"description":"JavaScript技术经典名著"}
AI 代码解读

准备5条数据,保存着books.json中,批量导入:

curl -XPOST "http://localhost:9200/_bulk?pretty" --data-binary @books.json
AI 代码解读

二、Group By分组统计

执行命令:

curl -XPOST "http://localhost:9200/books/_search?pretty" -d '{
"size": 0,
  "aggs": {
    "per_count": {
      "terms": {
        "field": "language"
      }
    }
  }
}'
AI 代码解读

统计结果:

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 5,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "per_count" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [ {
        "key" : "java",
        "doc_count" : 2
      }, {
        "key" : "python",
        "doc_count" : 2
      }, {
        "key" : "javascript",
        "doc_count" : 1
      } ]
    }
  }
}
AI 代码解读

按编程语言分类,java类2本,python类1本,javascript类1本。

三、Max最大值

执行命令,统计price最大的:

curl -XPOST "http://localhost:9200/books/_search?pretty" -d '{
  "size": 0,
  "aggs": {
    "max_price": {
      "max": {
        "field": "price"
      }
    }
  }
}'
AI 代码解读

返回结果:

{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 5,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "max_price" : {
      "value" : 81.4
    }
  }
}
AI 代码解读

四、Min最小值

求价格最便宜的那本:

curl -XPOST "http://localhost:9200/books/_search?pretty" -d '{
  "size": 0,
  "aggs": {
    "max_price": {
      "max": {
        "field": "price"
      }
    }
  }
}'
AI 代码解读

统计结果:

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 5,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "max_price" : {
      "value" : 81.4
    }
  }
}
AI 代码解读

五、Average平均值

分组统计并求5本书的平均价格:

curl -XPOST "http://localhost:9200/books/_search?pretty" -d '{
"size": 0,
"aggs": {
    "per_count": {
        "terms": {
            "field": "language"
        },
        "aggs": {
            "avg_price": {
                "avg": {
                    "field": "price"
                }
            }
        }
    }
}
}
'
AI 代码解读

返回结果:

{
  "took" : 4,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 5,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "per_count" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [ {
        "key" : "java",
        "doc_count" : 2,
        "avg_price" : {
          "value" : 58.35 }
      }, {
        "key" : "python",
        "doc_count" : 2,
        "avg_price" : {
          "value" : 67.95 }
      }, {
        "key" : "javascript",
        "doc_count" : 1,
        "avg_price" : {
          "value" : 66.4 }
      } ]
    }
  }
}
AI 代码解读

六、Sum求和

求5本书总价:

curl -XPOST "http://localhost:9200/books/_search?pretty" -d '
{
  "size": 0,
  "aggs": {
    "sum_price": {
      "sum": {
        "field": "price"
      }
    }
  }
}'
AI 代码解读

返回结果:

{
  "took" : 6,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 5,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "sum_price" : {
      "value" : 319.0
    }
  }
}
AI 代码解读

七、基本统计

基本统计会返回字段的最大值、最小值、平均值、求和:

curl -XPOST "http://localhost:9200/books/_search?pretty" -d '{
"size": 0,
"aggs": {
    "grades_stats": {
        "stats": {
            "field": "price"
        }
    }
}
}'
AI 代码解读

返回结果:

{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 5,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "grades_stats" : {
      "count" : 5,
      "min" : 46.5,
      "max" : 81.4,
      "avg" : 63.8,
      "sum" : 319.0
    }
  }
}
AI 代码解读

八、高级统计

高级统计还会返回方差、标准差等:

curl -XPOST "http://localhost:9200/books/_search?pretty" -d'
{
  "size": 0,
  "aggs": {
    "grades_stats": {
      "extended_stats": {
        "field": "price"
      }
    }
  }
}
'
AI 代码解读

统计结果:

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 5,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "grades_stats" : {
      "count" : 5,
      "min" : 46.5,
      "max" : 81.4,
      "avg" : 63.8,
      "sum" : 319.0,
      "sum_of_squares" : 21095.46,
      "variance" : 148.65199999999967,
      "std_deviation" : 12.19229264740638,
      "std_deviation_bounds" : {
        "upper" : 88.18458529481276,
        "lower" : 39.41541470518724
      }
    }
  }
}
AI 代码解读

九、百分比统计

curl -XPOST "http://localhost:9200/books/_search?pretty" -d '
{
    "size": 0,
    "aggs": {
        "load_time_outlier": {
            "percentiles": {
                "field": "year"
            }
        }
    }
}
'
AI 代码解读

返回结果:

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 5,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "load_time_outlier" : {
      "values" : {
        "1.0" : 2007.2,
        "5.0" : 2008.0000000000002,
        "25.0" : 2012.0,
        "50.0" : 2012.0,
        "75.0" : 2014.0,
        "95.0" : 2015.6000000000001,
        "99.0" : 2015.92
      }
    }
  }
}
AI 代码解读

十、分段统计

统计价格小于50、50-80、大于80的百分比:

curl -XPOST "http://localhost:9200/books/_search?pretty" -d '{
    "size": 0,
    "aggs": {
        "price_ranges": {
            "range": {
                "field": "price",
                "ranges": [{
                    "to": 50
                }, {
                    "from": 50,
                    "to": 80
                }, {
                    "from": 80
                }]
            }
        }
    }
}
'
AI 代码解读

返回结果:

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 5,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "price_ranges" : {
      "buckets" : [ {
        "key" : "*-50.0",
        "to" : 50.0,
        "to_as_string" : "50.0",
        "doc_count" : 1
      }, {
        "key" : "50.0-80.0",
        "from" : 50.0,
        "from_as_string" : "50.0",
        "to" : 80.0,
        "to_as_string" : "80.0",
        "doc_count" : 3
      }, {
        "key" : "80.0-*",
        "from" : 80.0,
        "from_as_string" : "80.0",
        "doc_count" : 1
      } ]
    }
  }
}
AI 代码解读
相关实践学习
使用阿里云Elasticsearch体验信息检索加速
通过创建登录阿里云Elasticsearch集群,使用DataWorks将MySQL数据同步至Elasticsearch,体验多条件检索效果,简单展示数据同步和信息检索加速的过程和操作。
ElasticSearch 入门精讲
ElasticSearch是一个开源的、基于Lucene的、分布式、高扩展、高实时的搜索与数据分析引擎。根据DB-Engines的排名显示,Elasticsearch是最受欢迎的企业搜索引擎,其次是Apache Solr(也是基于Lucene)。 ElasticSearch的实现原理主要分为以下几个步骤: 用户将数据提交到Elastic Search 数据库中 通过分词控制器去将对应的语句分词,将其权重和分词结果一并存入数据 当用户搜索数据时候,再根据权重将结果排名、打分 将返回结果呈现给用户 Elasticsearch可以用于搜索各种文档。它提供可扩展的搜索,具有接近实时的搜索,并支持多租户。
目录
打赏
0
0
0
0
41
分享
相关文章
Elasticsearch 的DSL查询,聚合查询与多维度数据统计
Elasticsearch的DSL查询与聚合查询提供了强大的数据检索和统计分析能力。通过合理构建DSL查询,用户可以高效地搜索数据,并使用聚合查询对数据进行多维度统计分析。在实际应用中,灵活运用这些工具不仅能提高查询效率,还能为数据分析提供深入洞察。理解并掌握这些技术,将显著提升在大数据场景中的分析和处理能力。
41 20
超值选择:阿里云Elasticsearch Serverless在企业数据检索与分析中的高性能与灵活性
本文介绍了阿里云Elasticsearch Serverless服务的高性价比与高度弹性灵活性。
212 8
深度解析ElasticSearch:构建高效搜索与分析的基石
【9月更文挑战第8天】在数据爆炸的时代,如何快速、准确地从海量数据中检索出有价值的信息成为了企业面临的重要挑战。ElasticSearch,作为一款基于Lucene的开源分布式搜索和分析引擎,凭借其强大的实时搜索、分析和扩展能力,成为了众多企业的首选。本文将深入解析ElasticSearch的核心原理、架构设计及优化实践,帮助读者全面理解这一强大的工具。
425 7
ElasticSearch基础3——聚合、补全、集群。黑马旅游检索高亮+自定义分词器+自动补全+前后端消息同步
聚合、补全、RabbitMQ消息同步、集群、脑裂问题、集群分布式存储、黑马旅游实现过滤和搜索补全功能
ElasticSearch 实现分词全文检索 - 聚合查询 cardinality
ElasticSearch 实现分词全文检索 - 聚合查询 cardinality
222 1
Elasticsearch如何聚合查询多个统计值,如何嵌套聚合?并相互引用,统计索引中某一个字段的空值率?语法是怎么样的?
Elasticsearch聚合查询用于复杂数据分析,包括统计空值率。示例展示了如何计算字段`my_field`非空非零文档的百分比。查询分为三步:总文档数计数、符合条件文档数计数及计算百分比。聚合概念涵盖度量、桶和管道聚合。脚本在聚合中用于动态计算。常见聚合类型如`sum`、`avg`、`date_histogram`等。组合使用可实现多值统计、嵌套聚合和空值率计算。[阅读更多](https://zhangfeidezhu.com/?p=515)
383 0
Elasticsearch如何聚合查询多个统计值,如何嵌套聚合?并相互引用,统计索引中某一个字段的空值率?语法是怎么样的?
在大数据场景下,Elasticsearch作为分布式搜索与分析引擎,因其扩展性和易用性成为全文检索首选。
【7月更文挑战第1天】在大数据场景下,Elasticsearch作为分布式搜索与分析引擎,因其扩展性和易用性成为全文检索首选。本文讲解如何在Java中集成Elasticsearch,包括安装配置、使用RestHighLevelClient连接、创建索引和文档操作,以及全文检索查询。此外,还涉及高级查询、性能优化和故障排查,帮助开发者高效处理非结构化数据。
108 0

热门文章

最新文章

AI助理

你好,我是AI助理

可以解答问题、推荐解决方案等