ElasticSearch7集群安装

本文涉及的产品
检索分析服务 Elasticsearch 版,2核4GB开发者规格 1个月
应用实时监控服务-应用监控,每月50GB免费额度
应用实时监控服务-用户体验监控,每月100OCU免费额度
简介: ElasticSearch7集群安装

ElasticSearch安装

下载安装包

下载地址:https://www.elastic.co/cn/downloads/past-releases#elasticsearch

安装版本:7.14.2

image-20230112094348092

基础环境安装

JDK安装

安装es的机器都需要安装

JDK版本11

  • 将安装包移动到/opt目录下
tar -zxvf jdk-11.0.17_linux-x64_bin.tar.gz
  • JDK加入环境变量

    修改/etc/profile,在尾部添加

    export JAVA_HOME=/opt/jdk-11.0.17
    export PATH=$PATH:$JAVA_HOME/bin
    

修改Linux配置

  • 编辑 /etc/security/limits.conf,追加以下内容
* soft nofile 65536
* hard nofile 65536
* hard memlock unlimited
* soft memlock unlimited
  • 在/etc/sysctl.conf文件最后添加一行
vm.max_map_count=262144

执行

sysctl -p

  • 修改 /etc/systemd/system.conf
vi /etc/systemd/system.conf
DefaultLimitMEMLOCK=infinity

安装ES

需要安装es的集群都要执行以下步骤

  • 将安装包移动到/opt目录
mv elasticsearch-7.14.2-linux-x86_64.tar.gz /opt
  • 修改ES使用内存

修改配置文件config/jvm.options

################################################################
## IMPORTANT: JVM heap size
################################################################
##
## The heap size is automatically configured by Elasticsearch
## based on the available memory in your system and the roles
## each node is configured to fulfill. If specifying heap is
## required, it should be done through a file in jvm.options.d,
## and the min and max should be set to the same value. For
## example, to set the heap to 4 GB, create a new file in the
## jvm.options.d directory containing these lines:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################
-Xms25g
-Xmx25g
  • 修改elasticsearch.yml配置
cluster.name: dev-cluster
node.name: node-2
node.master: true
node.data: true
node.ingest: true
bootstrap.memory_lock: true
bootstrap.system_call_filter: false
network.host: 172.16.24.192
http.port: 9200
transport.tcp.port: 9300
transport.tcp.compress: true
http.cors.enabled: true
http.cors.allow-origin: "*"
cluster.initial_master_nodes:  ["node-1","node-2","node-3"]
discovery.zen.ping.unicast.hosts: ["172.16.24.191:9300","172.16.24.192:9300","172.16.24.193:9300"]
discovery.zen.minimum_master_nodes: 1
# 启用脚本 默认painless
cluster.routing.allocation.same_shard.host: true
#超时时间
discovery.zen.ping_timeout: 5s
discovery.zen.fd.ping_timeout: 5s
#禁止自动创建索引
action.auto_create_index: true
# 一个集群中的N个节点启动后,才允许进行恢复处理 
gateway.recover_after_nodes: 1
# 设置初始化恢复过程的超时时间,超时时间从上一个配置中配置的N个节点启动后算起 
gateway.recover_after_time: 5m 
# 设置这个集群中期望有多少个节点.一旦这N个节点启动(并且recover_after_nodes也符合), 
# 立即开始恢复过程(不等待recover_after_time超时) 
gateway.expected_nodes: 2
# 线程池  
thread_pool.search.size: 100  
thread_pool.search.queue_size: 1000 
#xpack 插件的授权类型,basic是免费的,还有其他收费版本
xpack.license.self_generated.type: basic
#是否开启安全验证
xpack.security.enabled: true
#是否开启远程访问安全验证
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
# 证书位置,该文件在下面流程会生成
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
# 证书位置,该文件在下面流程会生成
xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12
  • 创建elasticsearch用户
useradd elasticsearch
passwd elasticsearch
  • 修改目录权限
chown elasticsearch.elasticsearch -R /opt/elasticsearch-7.14.2
  • 切换用户启动es
su elasticsearch
cd /opt/elasticsearch-7.14.2
  • 生成CA证书
bin/elasticsearch-certutil ca
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

The 'ca' mode generates a new 'certificate authority'
This will create a new X.509 certificate and private key that can be used
to sign certificate when running in 'cert' mode.

Use the 'ca-dn' option if you wish to configure the 'distinguished name'
of the certificate authority

By default the 'ca' mode produces a single PKCS#12 output file which holds:
    * The CA certificate
    * The CA's private key

If you elect to generate PEM format certificates (the -pem option), then the output will
be a zip file containing individual files for the CA certificate and private key

Please enter the desired output file [elastic-stack-ca.p12]:  #直接回车即可,不用设置密码
Enter password for elastic-stack-ca.p12 : #直接回车即可,不用设置密码

在当前目录可以看到elastic-stack-ca.p12文件

  • 生成密钥
bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

The 'cert' mode generates X.509 certificate and private keys.
    * By default, this generates a single certificate and key for use
       on a single instance.
    * The '-multiple' option will prompt you to enter details for multiple
       instances and will generate a certificate and key for each one
    * The '-in' option allows for the certificate generation to be automated by describing
       the details of each instance in a YAML file

    * An instance is any piece of the Elastic Stack that requires an SSL certificate.
      Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beats
      may all require a certificate and private key.
    * The minimum required value for each instance is a name. This can simply be the
      hostname, which will be used as the Common Name of the certificate. A full
      distinguished name may also be used.
    * A filename value may be required for each instance. This is necessary when the
      name would result in an invalid file or directory name. The name provided here
      is used as the directory name (within the zip) and the prefix for the key and
      certificate files. The filename is required if you are prompted and the name
      is not displayed in the prompt.
    * IP addresses and DNS names are optional. Multiple values can be specified as a
      comma separated string. If no IP addresses or DNS names are provided, you may
      disable hostname verification in your SSL configuration.

    * All certificates generated by this tool will be signed by a certificate authority (CA)
      unless the --self-signed command line option is specified.
      The tool can automatically generate a new CA for you, or you can provide your own with
      the --ca or --ca-cert command line options.

By default the 'cert' mode produces a single PKCS#12 output file which holds:
    * The instance certificate
    * The private key for the instance certificate
    * The CA certificate

If you specify any of the following options:
    * -pem (PEM formatted output)
    * -keep-ca-key (retain generated CA key)
    * -multiple (generate multiple certificates)
    * -in (generate certificates from an input file)
then the output will be be a zip file containing individual certificate/key files

Enter password for CA (elastic-stack-ca.p12) : # 直接回车
Please enter the desired output file [elastic-certificates.p12]: # 直接回车
Enter password for elastic-certificates.p12 : # 直接回车

Certificates written to /opt/elasticsearch-7.14.2/elastic-certificates.p12

This file should be properly secured as it contains the private key for 
your instance.

This file is a self contained file and can be copied and used 'as is'
For each Elastic product that you wish to configure, you should copy
this '.p12' file to the relevant configuration directory
and then follow the SSL configuration instructions in the product guide.

For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.

在当前目录会生成elastic-certificates.p12文件

  • 证书移动到指定目录
# 在config目录下创建certs
mkdir config/certs
# 证书移动到certs目录
 mv elastic-certificates.p12 config/certs/
  • 后台启动es
cd /opt/elasticsearch-7.14.2
bin/elasticsearch -d
  • 设置密码

这里可以复制输入

bin/elasticsearch-setup-passwords interactive
warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME
Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y


Enter password for [elastic]: 
Reenter password for [elastic]: 
Enter password for [apm_system]: 
Reenter password for [apm_system]: 
Enter password for [kibana_system]: 
Reenter password for [kibana_system]: 
Enter password for [logstash_system]: 
Reenter password for [logstash_system]: 
Enter password for [beats_system]: 
Reenter password for [beats_system]: 
Enter password for [remote_monitoring_user]: 
Reenter password for [remote_monitoring_user]: 

Changed password for user [apm_system]
Changed password for user [kibana_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]
  • 验证

浏览器访问http://ip:9200/,会提示输入账号密码,输入elastic账号与设置的密码登录即可

{
   
   
  "name" : "node-1",
  "cluster_name" : "dev-cluster",
  "cluster_uuid" : "J1_qbq7ZTcC7RC_9HIOJnw",
  "version" : {
   
   
    "number" : "7.14.2",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "6bc13727ce758c0e943c3c21653b3da82f627f75",
    "build_date" : "2021-09-15T10:18:09.722761972Z",
    "build_snapshot" : false,
    "lucene_version" : "8.9.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

启动报错

bootstrap check failure [1] of [1]: memory locking requested for elasticsearch process but memory is not locked

修改参数

> vi /etc/systemd/system.conf
DefaultLimitMEMLOCK=infinity

> /etc/security/limits.conf
* hard memlock unlimited
* soft memlock unlimited

Kibana安装

Kibana简介

Kibana是一个开源的基于浏览器的分析和可视化平台,可以用于搜索,查看,删除Elasticsearch索引并与存储在Elasticsearch索引中的数据进行交互。可以执行高级数据分析,并且以各种图标、表格和地图的形式可视化数据。

Kibana下载

下载地址:https://www.elastic.co/cn/downloads/past-releases#elasticsearch

安装版本:7.14.2

image-20230112095310874

Kibana安装

  • 安装包移动到/opt目录
 mv kibana-7.14.2-linux-x86_64 /opt
  • 解压
 tar -zxvf kibana-7.14.2-linux-x86_64.tar.gz
  • 修改配置文件

vi config/kibana.yml

server.host: "172.16.24.191"
elasticsearch.hosts: ["http://172.16.24.191:9200"]
# 设置支持中文
i18n.locale: "zh-CN"
# 在es中设置kibana密码
elasticsearch.username: "kibana"
elasticsearch.password: "设置的密码"
  • 修改权限
chown elasticsearch.elasticsearch -R kibana-7.14.2-linux-x86_64
  • 启动
nohup bin/kibana > kibana.log &
  • 查看进程
ps -ef|grep kibana
elastic+ 11336 11288 99 14:09 pts/0    00:00:12 /opt/kibana-7.14.2-linux-x86_64/node/bin/node --preserve-symlinks-main --preserve-symlinks /opt/kibana-7.14.2-linux-x86_64/src/cli/dist
  • 访问地址

http://ip:5601

输入账号elastic,与设置的密码登录

image-20230112144634650

相关实践学习
使用阿里云Elasticsearch体验信息检索加速
通过创建登录阿里云Elasticsearch集群,使用DataWorks将MySQL数据同步至Elasticsearch,体验多条件检索效果,简单展示数据同步和信息检索加速的过程和操作。
ElasticSearch 入门精讲
ElasticSearch是一个开源的、基于Lucene的、分布式、高扩展、高实时的搜索与数据分析引擎。根据DB-Engines的排名显示,Elasticsearch是最受欢迎的企业搜索引擎,其次是Apache Solr(也是基于Lucene)。 ElasticSearch的实现原理主要分为以下几个步骤: 用户将数据提交到Elastic Search 数据库中 通过分词控制器去将对应的语句分词,将其权重和分词结果一并存入数据 当用户搜索数据时候,再根据权重将结果排名、打分 将返回结果呈现给用户 Elasticsearch可以用于搜索各种文档。它提供可扩展的搜索,具有接近实时的搜索,并支持多租户。
目录
相关文章
|
15天前
|
存储 监控 安全
Elasticsearch 集群
【11月更文挑战第3天】
94 54
|
6月前
|
存储 数据可视化 Java
ElasticSearch单机版安装
ElasticSearch单机版安装
99 0
|
前端开发 Java 数据安全/隐私保护
【2022】Elasticsearch-7.17.6集群部署
【2022】Elasticsearch-7.17.6集群部署
842 0
|
域名解析 开发工具
elasticsearch 8.3版本安装和集群部署
elasticsearch 8.3版本安装和集群部署
528 1
Elasticsearch 集群相关命令
Elasticsearch 集群相关命令
|
存储 负载均衡 安全
Elasticsearch7.1.1的集群部署
Elasticsearch7.1.1的集群部署
262 0
Elasticsearch7.1.1的集群部署
|
存储 Docker 索引
elasticsearch学习四:elasticsearch集群
elasticsearch学习四:elasticsearch集群
265 0
elasticsearch学习四:elasticsearch集群
|
Java
elasticsearch 部署
elasticsearch
122 0
|
存储 Java 测试技术
Elasticsearch 安装与部署
Elasticsearch 安装与部署
561 0
|
存储 消息中间件 安全
Elasticsearch集群部署详解
#1、题记 之前应用都是Elasticsearch单节点,随着业务的拓展、数据量的增多,部署分布式Elasticsearch刻不容缓。 本文以Elaticsearch2.3.4版本为基础,讲解Elasticsearch三个节点的分布式部署、核心配置的含义以及分布式部署遇到的坑。 #2、三节点 Elasticsearch 分布式部署。 ##步骤1:配置好主节点Master信息。
435 1
Elasticsearch集群部署详解
下一篇
无影云桌面