搜索引擎 _ Elasticsearch（二）-阿里云开发者社区

搜索引擎 _ Elasticsearch（一）https://developer.aliyun.com/article/1469573

指定字段的类型

获得这个规则！可以通过 GET 请求获取具体的信息！

查看默认的信息

如果自己的文档字段没有指定，那么es 就会给我们默认配置字段类型！

扩展：通过命令 elasticsearch 索引情况！通过get _cat/ 可以获得es的当前的很多信息！

修改提交还是使用PUT 即可！然后覆盖！最新办法！

曾经！

现在的方法！

删除索引！

通过DELETE 命令实现删除、根据你的请求来判断是删除索引还是删除文档记录！

使用RESTFUL 风格是我们ES推荐大家使用的！

8. 关于文档的基本操作（重点）

基本操作

添加数据

PUT /kuangshen/user/1
{
 "name": "狂神说",
 "age": 23,
 "desc": "一顿操作猛如虎，一看工资2500",
 "tags": ["技术宅","温暖","直男"]
}

获取数据 GET

更新数据 PUT

Post _update , 推荐使用这种更新方式！

简单地搜索！

GET kuangshen/user/1

简答的条件查询，可以根据默认的映射规则，产生基本的查询！

复杂操作搜索 select ( 排序，分页，高亮，模糊查询，精准查询！)

输出结果，不想要那么多！

我们之后使用Java操作es ，所有的方法和对象就是这里面的 key！

排序！

分页查询！

数据下标还是从0开始的，和学的所有数据结构是一样的！

/search/{current}/{pagesize}

布尔值查询

must （and），所有的条件都要符合 where id = 1 and name = xxx

should（or），所有的条件都要符合 where id = 1 or name = xxx

must_not （not）

过滤器 filter

gt 大于
gte 大于等于
lt 小于
lte 小于等于！

匹配多个条件！

精确查询！

term 查询是直接通过倒排索引指定的词条进程精确查找的！

关于分词：

term ，直接查询精确的
match，会使用分词器解析！（先分析文档，然后在通过分析的文档进行查询！）

两个类型 text keyword

多个值匹配精确查询

高亮查询！

使用的命令

PUT /test1/type1/1
{
  "name" : "小冷",
  "age" : 3
}
PUT /test2
{
  "mappings": {
    "properties": {
      "name": {
        "type": "text"
      },
      "age":{
        "type": "long"
      },
      "birthDay":{
        "type": "date"
      }
    }
  }
}
GET test2
PUT /test3/_doc/1
{
  "name": "",
  "age":8,
  "brith":"2004-02-08"
}
POST /test3/_doc/1/_update
{
  "doc":{
    "name": "小冷"
  }
}
GET test3
PUT /lhy/user/1
{
 "name": "狂神说",
 "age": 23,
 "desc": "一顿操作猛如虎，一看工资2500",
 "tags": ["技术宅","温暖","直男"]
}
PUT /lhy/user/2
{
 "name": "法外狂徒张三",
 "age": 30,
 "desc": "罗老师手下的得力干将",
 "tags": ["身体好","懂法律","难判刑"]
}
PUT /lhy/user/2
{
 "name": "法外狂徒张三",
 "age": 19,
 "desc": "罗老师手下的得力干将",
 "tags": ["身体好","懂法律","难判刑"]
}
POST /lhy/user/2/_update
{
  "doc":{
     "name": "张三"
  }
}
PUT /lhy/user/3
{
 "name": "狂神说前端",
 "age": 23,
 "desc": "前端特效大杀手",
 "tags": ["游戏强","抗压强","007"]
}
GET /lhy/user/2
GET lhy/user/_search?q=name:狂神说
GET lhy/user/_search
{
"query":{
  "match": {
    "name": "狂神"
  }
},
"sort": [
  {
    "age": {
      "order": "asc"
    }
  }
],
"from": 0,
"size": 1
}
#boolean
GET lhy/user/_search
{
"query":{
  "bool":{
    "should": [
      {
        "match": {
          "name": "狂神说"
        }
      },
      {
        "match": {
          "age": 23
        }
      }
    ]
  }
}
}
#没有什么，相当与 not 
GET lhy/user/_search
{
"query":{
  "bool":{
    "must_not": [
      {
        "match": {
          "name": "狂神说"
        }
      }
    ]
  }
}
}
#过滤器filter 筛选
GET lhy/user/_search
{
"query":{
  "bool":{
    "must": [
      {
        "match": {
          "name": "狂神说"
        }
      }
    ],
    "filter": [
      {
        "range": {
          "age": {
              "lt": 20
          }
        }
      }
    ]
  }
}
}
GET lhy/user/_search
{
"query":{
  "bool":{
    "must": [
      {
        "match": {
          "tags": "技术 男 身体 007"
        }
      }
    ]
  }
}
}
#精确查询和text keyword 两种类型的细节
PUT testdb
{
  "mappings": {
    "properties": {
      "name":{
        "type": "text"
      },
      "desc":{
        "type": "keyword"
      }
    }
  }
}
PUT testdb/_doc/1
{
  "name":"小冷学java",
  "desc":"java真的是个好玩的语言"
}
PUT testdb/_doc/2
{
  "name":"小冷学java",
  "desc":"java真的是个好玩的语言2"
}
GET _analyze
{
  "analyzer": "keyword" ,
  "text": "小冷"
}
  
GET _analyze
{
  "analyzer": "standard" ,
  "text": "小冷"
}
GET testdb/_search
{
  "query": {
    "term": {
        "desc": "java真的是个好玩的语言"
    }
  }
}
#高亮查询
GET lhy/user/_search
{
  "query": {
      "match": {
        "name":"狂神"
      }
  },
  "highlight": {
    "fields": {
      "name":{}
    }
  }
}

集成SpringBoot

找官方文档！

找到原生的依赖

<dependency>
  <groupId>org.elasticsearch.client</groupId>
  <artifactId>elasticsearch-rest-high-level-client</artifactId>
  <version>7.6.2</version>
</dependency>

找对象
分析这个类中的方法即可！

配置基本的项目

问题：一定要保证我们的导入的依赖和我们的es 版本一致

源码中提供对象！

虽然这里导入3个类，静态内部类，核心类就一个！

/**
* Elasticsearch rest client infrastructure configurations.
*
* @author Brian Clozel
* @author Stephane Nicoll
*/
class RestClientConfigurations {
    @Configuration(proxyBeanMethods = false)
    static class RestClientBuilderConfiguration {
        // RestClientBuilder
        @Bean
        @ConditionalOnMissingBean
        RestClientBuilder elasticsearchRestClientBuilder(RestClientProperties
                                                         properties,
                                                         ObjectProvider<RestClientBuilderCustomizer> builderCustomizers) {
            HttpHost[] hosts =
                properties.getUris().stream().map(HttpHost::create).toArray(HttpHost[]::new);
            RestClientBuilder builder = RestClient.builder(hosts);
            PropertyMapper map = PropertyMapper.get();
            map.from(properties::getUsername).whenHasText().to((username) -> {
                CredentialsProvider credentialsProvider = new
                    BasicCredentialsProvider();
                Credentials credentials = new
                    UsernamePasswordCredentials(properties.getUsername(),
                                                properties.getPassword());
                credentialsProvider.setCredentials(AuthScope.ANY, credentials);
                builder.setHttpClientConfigCallback(
                    (httpClientBuilder) ->
                    httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider));
            });
            builder.setRequestConfigCallback((requestConfigBuilder) -> {
                map.from(properties::getConnectionTimeout).whenNonNull().asInt(Duration::toMill
                                                                               is)
                    .to(requestConfigBuilder::setConnectTimeout);
                map.from(properties::getReadTimeout).whenNonNull().asInt(Duration::toMillis)
                    .to(requestConfigBuilder::setSocketTimeout);
                return requestConfigBuilder;
            });
            builderCustomizers.orderedStream().forEach((customizer) ->
                                                       customizer.customize(builder));
            return builder;
        }
    }
    @Configuration(proxyBeanMethods = false)
    @ConditionalOnClass(RestHighLevelClient.class)
    static class RestHighLevelClientConfiguration {
        // RestHighLevelClient 高级客户端，也是我们这里要讲，后面项目会用到的客户端
        @Bean
        @ConditionalOnMissingBean
        RestHighLevelClient elasticsearchRestHighLevelClient(RestClientBuilder
                                                             restClientBuilder) {
            return new RestHighLevelClient(restClientBuilder);
        }
        @Bean
        @ConditionalOnMissingBean
        RestClient elasticsearchRestClient(RestClientBuilder builder,
                                           ObjectProvider<RestHighLevelClient> restHighLevelClient) {
            RestHighLevelClient client = restHighLevelClient.getIfUnique();
            if (client != null) {
                return client.getLowLevelClient();
            }
            return builder.build();
        }
    }
    @Configuration(proxyBeanMethods = false)
    static class RestClientFallbackConfiguration {
        // RestClient 普通的客户端！
        @Bean
        @ConditionalOnMissingBean
        RestClient elasticsearchRestClient(RestClientBuilder builder) {
            return builder.build();
        }
    }
}

具体的Api测试！

创建索引
判断索引是否存在
删除索引
创建文档
crud文档！

@SpringBootTest
class KuangshenEsApiApplicationTests {
    // 面向对象来操作
    @Autowired
    @Qualifier("restHighLevelClient")
    private RestHighLevelClient client;
    // 测试索引的创建 Request PUT kuang_index
    @Test
    void testCreateIndex() throws IOException {
        // 1、创建索引请求
        CreateIndexRequest request = new CreateIndexRequest("kuang_index");
        // 2、客户端执行请求 IndicesClient,请求后获得响应
        CreateIndexResponse createIndexResponse =
            client.indices().create(request, RequestOptions.DEFAULT);
        System.out.println(createIndexResponse);
    }
    // 测试获取索引,判断其是否存在
    @Test
    void testExistIndex() throws IOException {
        GetIndexRequest request = new GetIndexRequest("kuang_index2");
        boolean exists = client.indices().exists(request,                                      RequestOptions.DEFAULT);
        System.out.println(exists);
    }
    // 测试删除索引
    @Test
    void testDeleteIndex() throws IOException {
        DeleteIndexRequest request = new DeleteIndexRequest("kuang_index");
        // 删除
        AcknowledgedResponse delete = client.indices().delete(request,
                                                              RequestOptions.DEFAULT);
        System.out.println(delete.isAcknowledged());
    }
    // 测试添加文档
    @Test
    void testAddDocument() throws IOException {
        // 创建对象
        User user = new User("狂神说", 3);
        // 创建请求
        IndexRequest request = new IndexRequest("kuang_index");
        // 规则 put /kuang_index/_doc/1
        request.id("1");
        request.timeout(TimeValue.timeValueSeconds(1));
        request.timeout("1s");
        // 将我们的数据放入请求 json
        request.source(JSON.toJSONString(user), XContentType.JSON);
        // 客户端发送请求 , 获取响应的结果
        IndexResponse indexResponse = client.index(request,
                                                   RequestOptions.DEFAULT);
        System.out.println(indexResponse.toString()); //
        System.out.println(indexResponse.status()); // 对应我们命令返回的状态
        CREATED
    }
    // 获取文档，判断是否存在 get /index/doc/1
    @Test
    void testIsExists() throws IOException {
        GetRequest getRequest = new GetRequest("kuang_index", "1");
        // 不获取返回的 _source 的上下文了
        getRequest.fetchSourceContext(new FetchSourceContext(false));
        getRequest.storedFields("_none_");
        boolean exists = client.exists(getRequest, RequestOptions.DEFAULT);
        System.out.println(exists);
    }
    // 获得文档的信息
    @Test
    void testGetDocument() throws IOException {
        GetRequest getRequest = new GetRequest("kuang_index", "1");
        GetResponse getResponse = client.get(getRequest,
                                             RequestOptions.DEFAULT);
        System.out.println(getResponse.getSourceAsString()); // 打印文档的内容
        System.out.println(getResponse); // 返回的全部内容和命令式一样的
    }
    // 更新文档的信息
    @Test
    void testUpdateRequest() throws IOException {
        UpdateRequest updateRequest = new UpdateRequest("kuang_index","1");
        updateRequest.timeout("1s");
        User user = new User("狂神说Java", 18);
        updateRequest.doc(JSON.toJSONString(user),XContentType.JSON);
        UpdateResponse updateResponse = client.update(updateRequest,
                                                      RequestOptions.DEFAULT);
        System.out.println(updateResponse.status());
    }
    // 删除文档记录
    @Test
    void testDeleteRequest() throws IOException {
        DeleteRequest request = new DeleteRequest("kuang_index","1");
        request.timeout("1s");
        DeleteResponse deleteResponse = client.delete(request,
                                                      RequestOptions.DEFAULT);
        System.out.println(deleteResponse.status());
    }
    // 特殊的，真的项目一般都会批量插入数据！
    @Test
    void testBulkRequest() throws IOException {
        BulkRequest bulkRequest = new BulkRequest();
        bulkRequest.timeout("10s");
        ArrayList<User> userList = new ArrayList<>();
        userList.add(new User("kuangshen1",3));
        userList.add(new User("kuangshen2",3));
        userList.add(new User("kuangshen3",3));
        userList.add(new User("qinjiang1",3));
        userList.add(new User("qinjiang1",3));
        userList.add(new User("qinjiang1",3));
        // 批处理请求
        for (int i = 0; i < userList.size() ; i++) {
            // 批量更新和批量删除，就在这里修改对应的请求就可以了
            bulkRequest.add(
                new IndexRequest("kuang_index")
                .id(""+(i+1))
                .source(JSON.toJSONString(userList.get(i)),XContentType.JSON));
        }
        BulkResponse bulkResponse = client.bulk(bulkRequest,
                                                RequestOptions.DEFAULT);
        System.out.println(bulkResponse.hasFailures()); // 是否失败，返回 false 代表
        成功！
    }
    // 查询
    // SearchRequest 搜索请求
    // SearchSourceBuilder 条件构造
    // HighlightBuilder 构建高亮
    // TermQueryBuilder 精确查询
    // MatchAllQueryBuilder
    // xxx QueryBuilder 对应我们刚才看到的命令！
    @Test
    void testSearch() throws IOException {
        SearchRequest searchRequest = new SearchRequest("kuang_index");
        // 构建搜索条件
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.highlighter()
            // 查询条件，我们可以使用 QueryBuilders 工具来实现
            // QueryBuilders.termQuery 精确
            // QueryBuilders.matchAllQuery() 匹配所有
            TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("name",
                                                                        "qinjiang1");
        //    MatchAllQueryBuilder matchAllQueryBuilder =
        QueryBuilders.matchAllQuery();
        sourceBuilder.query(termQueryBuilder);
        sourceBuilder.timeout(new TimeValue(60,TimeUnit.SECONDS));
        searchRequest.source(sourceBuilder);
        SearchResponse searchResponse = client.search(searchRequest,
                                                      RequestOptions.DEFAULT);
        System.out.println(JSON.toJSONString(searchResponse.getHits()));
        System.out.println("=================================");
        for (SearchHit documentFields : searchResponse.getHits().getHits()) {
            System.out.println(documentFields.getSourceAsMap());
        }
    }
}

实战

新建jd boot的项目

之后我们设置端口和把 thymeleaf的缓存关掉，之后访问一下 index 查看

爬虫

数据怎么来？从数据库获取，消息队列中获取，都可以成为数据源爬虫！

需要使用爬虫来爬取数据

public List<content> parseJD(String keywords) throws IOException {
        //    https://search.jd.com/Search?keyword=java
        // 前提需要联网
        String url = "https://search.jd.com/Search?keyword=" + keywords + "&enc=utf-8";
        //解析网页(jsoup返回document就是js,浏览器的Document对象)
        Document document = Jsoup.parse(new URL(url), 30000);
        //所有我们再js中可以操作的,在这里都可以
        Element element = document.getElementById("J_goodsList");
        //System.out.println(element.html());
        //获取所有的li标签
        Elements li_elements = document.getElementsByTag("li");
        ArrayList<content> goodsList = new ArrayList<>();
        for (Element el : li_elements) {
            if (el.attr("class").equalsIgnoreCase("gl-item")) {
                String img = el.getElementsByTag("img").eq(0).attr("data-lazy-img");
                String price = el.getElementsByClass("p-price").eq(0).text();
                String title = el.getElementsByClass("p-name").eq(0).text();
                content content = new content();
                content.setTitle(title);
                content.setImg(img);
                content.setPrice(price);
                goodsList.add(content);
            }
        }
        return goodsList;
    }

关键高亮

//获取数据实现搜索高亮功能
    public List<Map<String, Object>> getContentHighContent(String keywords, int pageNo, int pageSize) throws IOException {
        if (pageNo < 1) {
            pageNo = 1;
        }
        //条件搜索
        SearchRequest searchRequest = new SearchRequest("jd_goods");
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.from(pageNo);
        searchSourceBuilder.size(pageSize);
        //精确查询
        MatchBoolPrefixQueryBuilder queryBuilder = QueryBuilders.matchBoolPrefixQuery("title", keywords);
        searchSourceBuilder.query(queryBuilder);
        searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
        //配置高亮
        HighlightBuilder highlightBuilder = new HighlightBuilder();
        highlightBuilder.field("title");
        highlightBuilder.requireFieldMatch(false); //多个高亮关闭
        highlightBuilder.preTags("<span style='color:red '>");
        highlightBuilder.postTags("</span>");
        searchSourceBuilder.highlighter(highlightBuilder);
        //执行搜索
        searchRequest.source(searchSourceBuilder);
        SearchResponse searchResponse = Client.search(searchRequest, RequestOptions.DEFAULT);
        ArrayList<Map<String, Object>> list = new ArrayList<>();
        for (SearchHit documentFields : searchResponse.getHits().getHits()) {
            //解析高亮的字段
            Map<String, HighlightField> highlightFields = documentFields.getHighlightFields();
            HighlightField title = highlightFields.get("title");
            Map<String, Object> sourceAsMap = documentFields.getSourceAsMap();
            if (title != null) {
                //取出全部的高亮title
                Text[] texts = title.fragments();
                String name = "";
                //拼接成新字段
                for (Text text : texts) {
                    name += text;
                }
                //如果需要就替换原来获取到的title
                sourceAsMap.put("title", name);
            }
            list.add(sourceAsMap);
        }
        return list;
    }

之后我们需要用前端

vue 去解析html

<!--标题-->
<p class="productTitle">
 <a v-html="result.title"></a>
</p>

搜索引擎 _ Elasticsearch（二）