1. 环境说明
- elasticsearch7.9.3
- elasticsearch-analysis-ik-7.9.3
- kibana7.9.3(与此需求无关)
2. 分析思路
- 由于es在存储数据时如果使用ik分词器, 进行如下配置:
{ "settings": { "number_of_replicas": 1, "number_of_shards": 5 }, "mappings": { "properties": { "title": { "type": "text", "analyzer": "ik_max_word", "search_analyzer": "ik_smart", "index": true, "store": false } } }
- 在分词过程中, 默认IK分词器只会处理分词, 但是单个字是不会变成term储存进倒排表的
- 所以如果要做单个字的全文检索, 就需要增加额外字典
- 对检索进行优化, 对检索词会进行最大粒度分词, 比如: 在检索:"手机壳"的时候, 就不会将"手机壳"拆分为"手机"和"手机壳"等, 避免搜索手机壳的时候出现手机的结果
3. analysis-ik配置
- 修改配置文件elasticsearch-7.9.3\plugins\elasticsearch-analysis-ik-7.9.3\config
- 配置中相对路径都是以config下路径
<?xmlversion="1.0"encoding="UTF-8"?><!DOCTYPEpropertiesSYSTEM"http://java.sun.com/dtd/properties.dtd"><properties><comment>IKAnalyzer扩展配置</comment><!--用户可以在这里配置自己的扩展字典--><entrykey="ext_dict">extra_single_word.dic</entry><!--用户可以在这里配置自己的扩展停止词字典--><entrykey="ext_stopwords"></entry><!--用户可以在这里配置远程扩展字典--><!--<entrykey="remote_ext_dict">words_location</entry>--><!--用户可以在这里配置远程扩展停止词字典--><!--<entrykey="remote_ext_stopwords">words_location</entry>--></properties>
4. 重启ES并重建索引
再一次建立以下索引:
{ "settings": { "number_of_replicas": 1, "number_of_shards": 5 }, "mappings": { "properties": { "title": { "type": "text", "analyzer": "ik_max_word", "search_analyzer": "ik_smart", "index": true, "store": false } } }
此时, 再存入的数据会根据IK分词器+额外字典(单字字典)进行分词。
kibana测试分词
POST_analyze{ "analyzer": "ik_max_word", "text": ["我们是共产主义接班人。"] }
分词结果
{ "tokens" : [ { "token" : "我们", "start_offset" : 0, "end_offset" : 2, "type" : "CN_WORD", "position" : 0 }, { "token" : "我", "start_offset" : 0, "end_offset" : 1, "type" : "CN_WORD", "position" : 1 }, { "token" : "们", "start_offset" : 1, "end_offset" : 2, "type" : "CN_WORD", "position" : 2 }, { "token" : "是", "start_offset" : 2, "end_offset" : 3, "type" : "CN_WORD", "position" : 3 }, { "token" : "共产主义", "start_offset" : 3, "end_offset" : 7, "type" : "CN_WORD", "position" : 4 }, { "token" : "共产", "start_offset" : 3, "end_offset" : 5, "type" : "CN_WORD", "position" : 5 }, { "token" : "共", "start_offset" : 3, "end_offset" : 4, "type" : "CN_WORD", "position" : 6 }, { "token" : "产", "start_offset" : 4, "end_offset" : 5, "type" : "CN_WORD", "position" : 7 }, { "token" : "主义", "start_offset" : 5, "end_offset" : 7, "type" : "CN_WORD", "position" : 8 }, { "token" : "主", "start_offset" : 5, "end_offset" : 6, "type" : "CN_WORD", "position" : 9 }, { "token" : "义", "start_offset" : 6, "end_offset" : 7, "type" : "CN_WORD", "position" : 10 }, { "token" : "接班人", "start_offset" : 7, "end_offset" : 10, "type" : "CN_WORD", "position" : 11 }, { "token" : "接班", "start_offset" : 7, "end_offset" : 9, "type" : "CN_WORD", "position" : 12 }, { "token" : "接", "start_offset" : 7, "end_offset" : 8, "type" : "CN_WORD", "position" : 13 }, { "token" : "班", "start_offset" : 8, "end_offset" : 9, "type" : "CN_WORD", "position" : 14 }, { "token" : "人", "start_offset" : 9, "end_offset" : 10, "type" : "CN_WORD", "position" : 15 } ] }
参考文章: https://blog.csdn.net/nazeniwaresakini/article/details/104220237
