0. ES相关推荐
首先,不要再使用curl,请安装sense(kibana5.x中默认包含sense)
1)ES官方向导
https://www.elastic.co/guide/en/elasticsearch/guide/master/index.html
2)ES官方文档(API相关)
https://www.elastic.co/guide/en/elasticsearch/reference/master/index.html
3)ES资源清单(全)
https://github.com/dzharii/awesome-elasticsearch
4)ES云
5)ES官方论坛(英文)
6)ES|stackOverflow论坛地址
https://stackoverflow.com/questions/tagged/elasticsearch
7)ES 性能测试相关(NB)
https://www.datadoghq.com/blog/monitor-elasticsearch-performance-metrics/
1. 查询操作
基本查询有两种语法:左边是一个简单的语法,你不能使用任何选项,
右边是一个扩展。 大多数初学者头痛的DSL来自于:
GET _search { "query": { "match": { "FIELD": "TEXT" } } } GET _search { "query": { "match": { "FIELD": { "query": "TEXT", "OPTION": "VALUE" } } } }• 1 • 2 • 3 • 4 • 5 • 6 • 7 • 8 • 9 • 10 • 11 • 12 • 13 • 14 • 15 • 16 • 17 • 18 • 19
1.1 包含高亮、聚合、过滤器的完整的检索示例
GET /_search { "query": { "bool": { "must": [ { "match": { "title": "smith" } } ], "must_not": [ { "match_phrase": { "title": "granny smith" } } ], "filter": [ { "exists": { "field": "title" } } ] } }, "aggs": { "my_agg": { "terms": { "field": "user", "size": 10 } } }, "highlight": { "pre_tags": [ "<em>" ], "post_tags": [ "</em>" ], "fields": { "body": { "number_of_fragments": 1, "fragment_size": 20 }, "title": {} } }, "size": 20, "from": 100, "_source": [ "title", "id" ], "sort": [ { "_id": { "order": "desc" } } ] }• 1 • 2 • 3 • 4 • 5 • 6 • 7 • 8 • 9 • 10 • 11 • 12 • 13 • 14 • 15 • 16 • 17 • 18 • 19 • 20 • 21 • 22 • 23 • 24 • 25 • 26 • 27 • 28 • 29 • 30 • 31 • 32 • 33 • 34 • 35 • 36 • 37 • 38 • 39 • 40 • 41 • 42 • 43 • 44 • 45 • 46 • 47 • 48 • 49 • 50 • 51 • 52 • 53 • 54 • 55 • 56 • 57 • 58 • 59 • 60 • 61 • 62 • 63 • 64
1.2 普通检索
多字段检索
"multi_match": { "query": "Elastic", "fields": ["user.*", "title^3"], "type": "best_fields" }• 1 • 2 • 3 • 4 • 5
bool检索
"bool": { "must": [], "must_not": [], "filter": [], "should": [], "minimum_should_match" : 1 }• 1 • 2 • 3 • 4 • 5 • 6 • 7
范围检索
"range": { "age": { "gte": 10, "lte": 20, "boost": 2 } }• 1 • 2 • 3 • 4 • 5 • 6 • 7
1.3 QueryString语法概述
1.3.1 检索所有的_all字段
GET /_search?q=pony• 1
1.3.2 包含运算符和包含boost精确检索的复杂检索
GET /_search?q=title:(joli OR code) AND author:"Damien Alexandre"^2• 1
1.3.3 使用通配符和特殊查询进行检索
GET /_search?q=_exists_:title OR title:singl? noneOrAnyChar*cter• 1
1.3.4 模糊搜素和范围检索
GET /_search?q=title:elastichurch~3 AND date:[2016-01-01 TO 2018-12-31]• 1
1.3.5 使用 DSL检索(不推荐用于用户搜索):
GET /_search { "query": { "query_string": { "default_field": "content", "query": "elastic AND (title:lucene OR title:solr)" } } }• 1 • 2 • 3 • 4 • 5 • 6 • 7 • 8 • 9
2. 索引操作
2.1 创建包含设置和mapping的索引
PUT /my_index_name { "settings": { "number_of_replicas": 1, "number_of_shards": 3, "analysis": {}, "refresh_interval": "1s" }, "mappings": { "my_type_name": { "properties": { "title": { "type": "text", "analyzer": "english" } } } } }• 1 • 2 • 3 • 4 • 5 • 6 • 7 • 8 • 9 • 10 • 11 • 12 • 13 • 14 • 15 • 16 • 17 • 18 • 19
2.2 动态的更新设置
PUT /my_index_name/_settings { "index": { "refresh_interval": "-1", "number_of_replicas": 0 } }• 1 • 2 • 3 • 4 • 5 • 6 • 7
2.3 通过向类型添加字段更新索引
PUT /my_index_name/_mapping/my_type_name { "my_type_name": { "properties": { "tag": { "type": "keyword" } } } }• 1 • 2 • 3 • 4 • 5 • 6 • 7 • 8 • 9 • 10
2.4 获取Mapping和设置
GET /my_index_name/_mapping GET /my_index_name/_settings• 1 • 2
2.5 创建document
POST /my_index_name/my_type_name { "title": "Elastic is funny", "tag": [ "lucene" ] }• 1 • 2 • 3 • 4 • 5 • 6 • 7
2.6 创建或更新document
PUT /my_index_name/my_type_name/12abc { "title": "Elastic is funny", "tag": [ "lucene" ] }• 1 • 2 • 3 • 4 • 5 • 6 • 7
2.7 删除文档
DELETE /my_index_name/my_type_name/12abc• 1
2.8 打开或关闭索引已节约内存和CPU
POST /my_index_name/_close POST /my_index_name/_open• 1 • 2
2.9 移除和创建别名
POST /_aliases { "actions": [ { "remove": { "index": "my_index_name", "alias": "foo" } }, { "add": { "index": "my_index_name", "alias": "bar", "filter" : { "term" : { "user" : "damien" } } } } ] }• 1 • 2 • 3 • 4 • 5 • 6 • 7 • 8 • 9 • 10 • 11 • 12 • 13 • 14 • 15 • 16 • 17 • 18
2.10 列举别名
GET /_aliases GET /my_index_name/_alias/* GET /*/_alias/* GET /*/_alias/foo• 1 • 2 • 3 • 4
2.11 索引监控和信息
GET /my_index_name/_stats GET /my_index_name/_segments GET /my_index_name/_recovery?pretty&human• 1 • 2 • 3
2.12 索引状态和管理
POST /my_index_name/_cache/clear POST /my_index_name/_refresh POST /my_index_name/_flush POST /my_index_name/_forcemerge POST /my_index_name/_upgrade GET /my_index_name/_upgrade?pretty&human• 1 • 2 • 3 • 4 • 5 • 6
3. 调试和部署
3.1 检索调试
3.1.1 获取query操作到底做了什么?
GET /blog/post/_validate/query?explain { "query": { "match": { "title": "Smith" } } }• 1 • 2 • 3 • 4 • 5 • 6 • 7 • 8
3.1.2 获取文档是否匹配?
GET /blog/post/1/_explain { "query": { "match": { "title": "Smith" } } }• 1 • 2 • 3 • 4 • 5 • 6 • 7 • 8
3.2 分析
3.2.1 测试内容如何在文档中被标记?
GET /blog/_analyze?field=title&text=powerful• 1
3.2.2 测试分析器输出?
GET /_analyze?analyzer=english&text=powerful• 1
3.3 集群管理和插件管理
3.3.1 集群和节点信息
GET /_cluster/health?pretty GET /_cluster/health?wait_for_status=yellow&timeout=50s GET /_cluster/state GET /_cluster/stats?human&pretty GET /_cluster/pending_tasks GET /_nodes GET /_nodes/stats GET /_nodes/nodeId1,nodeId2/stats• 1 • 2 • 3 • 4 • 5 • 6 • 7 • 8
3.3.2 手动移动分片
索引1的分片移动到索引2
POST /_cluster/reroute { "commands": [ { "move": { "index": "my_index_name", "shard": 0, "from_node": "node1", "to_node": "node2" } }, { "allocate": { "index": "my_index_name", "shard": 1, "node": "node3" } } ] }• 1 • 2 • 3 • 4 • 5 • 6 • 7 • 8 • 9 • 10 • 11 • 12 • 13 • 14 • 15 • 16 • 17 • 18 • 19 • 20
3.3.3 更新设置
动态更新最小节点数。
PUT /_cluster/settings { "persistent": { "discovery.zen.minimum_master_nodes": 3 } } PUT /_cluster/settings { "transient": { "discovery.zen.minimum_master_nodes": 2 } }• 1 • 2 • 3 • 4 • 5 • 6 • 7 • 8 • 9 • 10 • 11 • 12 • 13
使分片失效,在rolling重启前有效。
PUT /_cluster/settings { "transient" : { "cluster.routing.allocation.enable" : "none" } } PUT /_cluster/settings { "transient" : { "cluster.routing.allocation.enable" : "all" } }• 1 • 2 • 3 • 4 • 5 • 6 • 7 • 8 • 9 • 10 • 11 • 12 • 13
4.最有用的插件
ES5.X版本已不再支持站点插件,请查看kibana应用或类似Cerebro等其他独立的app做管理。
Cerebro地址:https://github.com/lmenezes/cerebro
Analysis ICU
从Unicode ICU库中添加有用的tokenizer和令牌过滤器。
bin/elasticsearch-plugin install analysis-icu• 1
AWS Cloud
允许在Amazon云(EC2和S3)中发现和存储。
bin / elasticsearch-plugin install discovery-ec2 bin / elasticsearch-plugin install repository-s3• 1
Azure Cloud
允许在微软云(EC2和S3)中发现和存储。
bin/elasticsearch-plugin install discovery-azure-classicbin/elasticsearch-plugin install repository-azure• 1
插件管理:
bin/elasticsearch-plugin install file:///path/to/plugin bin/elasticsearch-plugin list bin/elasticsearch-plugin remove [pluginname]• 1 • 2 • 3
5.其他信息
5.1 如何发现其他插件
RPM: /usr/share/elasticsearch/bin Debian: /usr/share/elasticsearch/bin• 1 • 2
5.2 缺省端口
Kibana: http://localhost:5601/. Elasticsearch: http://localhost:9200/.• 1 • 2 • 3
5.3 如何设置堆大小
单一Elasticsearch 服务器最大可用内存小于总内存的50%,且小于32GB;
假定你使用Ubuntu/Debian 服务器,你可以通过如下配置修改:
/etc/security/limits.conf elasticsearch - nofile 65535 elasticsearch - memlock unlimited• 1 • 2 • 3 • 4
对于Centos服务器,修改配置如下:
/etc/default/elasticsearch (on CentOS/RH: /etc/sysconfig/elasticsearch) ES_HEAP_SIZE=20g MAX_OPEN_FILES=65535 MAX_LOCKED_MEMORY=unlimited• 1 • 2 • 3 • 4 • 5
5.4 elasticsearch.yml中有用的需要改变的配置
cluster.name: jolicluster node.name: ${HOSTNAME} discovery.zen.ping.unicast.hosts: ["front01", "front02"] discovery.zen.minimum_master_nodes: 2 network.host: _site_ network.bind_host: [_site_, _local_] plugin.mandatory: analysis-icu node.data: true node.master: true bootstrap.memory_lock: true action.auto_create_index: +aaa*,-bbb*,+ccc*,-*• 1 • 2 • 3 • 4 • 5 • 6 • 7 • 8 • 9 • 10 • 11
6. 小结
本文是翻译的 http://elasticsearch-cheatsheet.jolicode.com/#es5。
Cheat sheet是作弊表的意思。可见源作者对文章寄托的意义就是最核心、最关键的ES知识点“小抄”。
我把它理解成ES学习中待翻看的字典。
尤其第0章节的相关链接,都是上乘ES佳作,待进一步深度剖析。