目标：将json格式的两类日志输出到elasticsearch两类索引
1. 安装logstash。
2. 编写logstash处理配置文件，创建一个test.conf文件，内容如下：
input {
file {
path => "/home/vagrant/logstash/logstash-2.2.2/dbpool-logs/dev/common-sql-*.log"
start_position => "beginning"
type => "sql"
codec => json {
charset => "UTF-8"
}
}
file {
path => "/home/vagrant/logstash/logstash-2.2.2/dbpool-logs/dev/common-transaction-*.log"
start_position => "beginning"
type => "transaction"
codec => json {
charset => "UTF-8"
}
}
}
output {
if "_grokparsefailure" in [tags] {
}else{
if [type] == "sql"{
elasticsearch {
hosts => ["http://192.168.33.10:9200"]
index => "common-sql-%{+YYYY.MM.dd}"
}
}
if [type] == "transaction"{
elasticsearch {
hosts => ["http://192.168.33.10:9200"]
index => "common-transaction-%{+YYYY.MM.dd}"
}
}
}
}
或者
input {
file {
path => "/home/vagrant/logstash/logstash-2.2.2/dbpool-logs/dev/common-sql-*.log"
start_position => "beginning"
type => "sql"
codec => json {
charset => "UTF-8"
}
}
file {
path => "/home/vagrant/logstash/logstash-2.2.2/dbpool-logs/dev/common-transaction-*.log"
start_position => "beginning"
type => "transaction"
codec => json {
charset => "UTF-8"
}
}
}
output {
if "_grokparsefailure" in [tags] {
}else{
if [type] == "sql"{
elasticsearch {
hosts => ["http://192.168.33.10:9200"]
index => "common-%{type}-%{+YYYY.MM.dd}"
}
}
}
}
指定输入日志的路径，按通配符匹配。分为两类：sql和transaction。
根据type分别输出到elasticsearch不同的索引。
3. 安装elasticsearch。
4. 启动elasticsearch，./bin/elasticsearch -d ，默认端口为9200。
5. 启动logstash开始处理，./bin/logstash -f conf/test.conf。
6. 完成。
========广告时间========
鄙人的新书《Tomcat内核设计剖析》已经在京东销售了，有需要的朋友可以到 https://item.jd.com/12185360.html 进行预定。感谢各位朋友。
=========================
