开发者学堂课程【Elasticsearch 实战进阶营:实操讲解:微服务运行状态监测(二)】学习笔记,与课程紧密联系,让用户快速学习知识。
课程地址:https://developer.aliyun.com/learning/course/899/detail/14321
实操讲解:微服务运行状态监测(二)
三、Elastic Stack 技术栈
1.主要概述图
早期 Elastic Stack 名为 ELK,对应 Elasticsearch,Logstash,Kibana 这三个开源软件,但由于在做日志收集时 Logstash 过于重量级,因此官方又开发了各种 Beat 组件,比如 FileBeat,PackBeat 等等。
随着这些角色的加入 ELK 更名位 Elastic Stack。
Elastic Stack 组件的主要优点有如下几个:
(1)处理方式灵活:
Elasticsearch 是实时全文索引,具有强大的搜索功能。
(2)配置相对简单:
Elasticsearch 全部使用 JSON 接口,Logstash 使用模块配置,Kibana 的配置文件部分更简单。
(3)检索性能高效:
基于优秀的设计,虽然每次查询都是实时,但是也可以达到百万级数据的查询查询秒级响应。
(4)集群线性扩展:
Elasticsearch 和 Logstash 都可以灵活线性扩展。
(5)前端操作绚丽:
Kibana 的前端设计比较绚丽,而且操作简单。
为什么要用 Elastic Stack 组件?
Elastic Stack 组件在海量日志系统运维中,可以用于解决以下主要问题:
(1)分布式日志数据统一收集,实现集中式查询和管理;
(2)故障排除;
(3)安全信息和事件管理;
(4)报表功能;
Elastic Stack 在大数据运维系统中,主要可解决的问题如下:
(1)日志查询,问题排查,故障恢复,故障自愈;
(2)应用日志分析,错误报警;
(3)性能分析,用户行为分析;
四、Elastic+kibana 自有环境
1、动手配置自己的 ES 运行环境
(1)准备好环境
## ############并并Heartbeat Configuration Example #并# ## ##并####### # ## ##并##并井
is file is an example configuration
file highlighting only some common options。
# The heartbeat. reference .yml file in the same directory contains all the supported options
# with detailed com
ments. You can use it for reference。
#
# You can find the full configuration reference here ;
# https://www. elastic。co/ guide/ en/ beats/ heartbeat/ index。html
######## ###井林## ### ###井#井####井Heartbeat ###井 ### ##井并井
### ## # ####并##井井林##井#并##井并井
# Define a directory to load monitor definitions from. Definitions take the form
# of individual yaml files. .
heartbeat . confi
配置节点
be loaded.
type: http # monitor type http 。Connect via HTTP an optionally verify response
# Monitor name used for job name and document type
name: eureka
t Enable/Disable monitor
enabled: true
# Configure task schedule
schedule: ' @every 5s' # every 5 seconds from start of beat
# Configure URLS to ping
hosts: ["http: //1ocalhost:8020"]
# Configure IP protocol types to ping on if hostnames are confi gured。
# Ping all resolvable IPs if" mode is" all^ ,or only one IP if mode is" any.
ipv4: true
ipv6: true
I
mode: any
# Optional HTTP proxy url.
#proxy_ url: '
# Total test connection and data exchange t imeout
#timeout: 16s
# Optional Authentication Credentials
#uS ername :
g。monitors :
I
# Directory + glob pattern to search for configuration files
path: ${path . config}/ monitors .d/* . yml
# If enabled, heartbeat will periodically check the config . monitors path for c
hanges
reload. enabled: true
# How often to check for changes
reload.period: 5s
# Configure monitors inline
# heartbeat 。monitors ;
# - type: http
# List or urls to
query
# urls: ["http://localhost:9200" ]
# Configure task schedule
# schedule:" @every 10s
输出地要到 es 里面
2nd2no yse2s807 -
“unKTe@ypJeasoTaseT3. :pJomssed
DTiSeTan
:aweuuasn
wKey Tde:pTm :Kax τde#
”pJomssed/a
weuJasn JO Kay IdV JB42τa 一sTeTauapaJs uoTaesT2ua4any #
sd224u : To5020Jd#
、 sd224. JO (T
安全地址
nefep) d224、Ja47ta - TODO10Jd #
[.00Z6 :wO2. sounAtte "ypJeasotaseta'ottand. ZUMnTOOOUUMJATMa0-U2-sa.] :s2soy
02 23auUOS of S2Soy 4o KeJuy #
:ypJeasoTa
seTa "2nd2no
andzno ypueasoTset3 ---
# You can find the full configuration reference here :
# https:/ /www. elastic。co/ guide/ en/ beats/ heartbeat/ index. html
##########################并并并Heartbeat ###并 ###井#井# # ## ## #井# # # # ## ## ### # # #并# ####
# Define a directory to load monitor definitions from. Definitions take the form
# of individual yaml files。
heartbeat。config。monitors :
# Directory + glob pattern to search for configuration files
path: ${path. config}/monitors . dA . yml
# If enabled, heartbeat will periodically check the config 。monitors path for changes
reload .enabled: true
# How often to check for changes
reload. period: 5s
# Configure monitors inline
# heartbeat 。monitors ;
# - type: http
# List or urls to query
# urls: ["http://localhost:9200"]
# Configure task schedule
# schedule:" @every 10s
# Total test connection and data exchange t imeout
#timeout: 16s
# These files contain a list of monitor confi gurations identical
# to the heartbe
at。monitors section in heartbeat。yml
# The 。example extension on this file must be removed for it to
# be loaded 。
type: http # monitor type http^ 。Connect via HTTP an optionally verify res ponse
# Monitor nam
e used for job name and document type
name: eureka
# Enable/Disable monitor
enabled: true
# Configure task schedule
schedule:' @every 5s' # every 5 seconds from start of beat
# Configure URLS to ping
hosts:| [ "http://localhost:8020"]
并Configure IP protocol types to ping on if hostnames are
configured .
# Ping all
resolvable IPs if mode i S
all,or only one IP if" mode is" any.
ipv4: true
ipv6: true
mode: any
# Optional HTTP proxy url.
#proxy_ url:
并Total test connection and data exchange t imeout
#timeout: 16s
主要配置顶
# cluster 集群配置
cluster.name: dmall_ test_ es
#名字自己起,确保集群名不同
cluster. initial_ master_ nodes: ["ip:port"]
#初始主节点的ip和端口,填你本机ip,默认端口是9300
network.host: ip
#填你本机ip
http.port: 9200
#绑定到传入HTTP请求的端口默认是9200
transport. tcp.port: 9300
#端口绑定节点之间的通信默认是9300
#该配置十分重要,没有正确配置,可能无法构成集群
#这是一个集群中的主节点的初始列表,当节点(主节点或者数据节点)启动时使用这个列表进行探测
discovery. zen. ping. unicast.hosts: ["ip:port"]
#端口一般填 TCP 端口,如9300
#以下两个配置,可能不配置也可以
xpack. security . enabled: false
#没置为 true 则在节点上启用 X-Pack 安全特性,false 表 示禁用 X-Pack 安全:
xpack. security. transport. ssl.enabled: false
#xpack ssl 配置,设置成 false 就可以。
出现以下结果,代表运行启动正常
"name" : "6LUo_ Ih",
"cluster_ name" :
"elasticsearch",
"cluster_ uuid" : "tnCuoQs9RQOTkDevKI4y6Q" ,
"version”: {
"number" : "7.5.1".
"build_ flavor" : "default",
"build_ type”: "tar",
"build_ hash" : "1fad4e1",
"build_ date”: "2019-06-18T13:16:52.517138Z",
"build_ snapshot" : false,
"lucene_ version" : "7.7.0",
"minimum_ wire_ compatibility_ version" : "5.6.0",
"minimum_ index_ compatibility_ version" : "5.0.0"
},
"tagline" : "You Know, for Search"
启动 kibana
2nd2no yse2s807 -
“unKTe@ypJeasoTaseT3. :pJomssed
DTiSeTan
:aweuuasn
wKey Tde:pTm :Kax τde#
”pJomssed/a
weuJasn JO Kay IdV JB42τa 一sTeTauapaJs uoTaesT2ua4any #
sd224u : To5020Jd#
、 sd224. JO (T
安全地址
nefep) d224、Ja47ta - TODO10Jd #
[.00Z6 :wO2. sounAtte "ypJeasotaseta'ottand. ZUMnTOOOUUMJATMa0-U2-sa.] :s2soy
02 23auUOS of S2Soy 4o KeJuy #
:ypJeasoTa
seTa "2nd2no
andzno ypueasoTset3 ---
# You can find the full configuration reference here :
# https:/ /www. elastic。co/ guide/ en/ beats/ heartbeat/ index. html
path: ${path. config}/monitors . dA . yml
# If enabled, heartbeat will periodically check the config 。monitors path for changes
reload .enabled: true
# How often to check for changes
reload. period: 5s
# Configure monitors inline
# heartbeat 。monitors ;
# - type: http
# List or urls to query
# urls: ["http://localhost:9200"]
# Configure task schedule
# schedule:" @every 10s
# Total test connection and data exchange t imeout