CentOS7 部署ElastiFlow网络流量分析平台
本文参考如下链接完成
https://docs.elastiflow.com/docs/install_linux https://cloud.tencent.com/developer/article/1648854 https://blog.csdn.net/weixin_43838503/article/details/122432963 https://blog.51cto.com/coolsky/3190806
条件准备
- 1、host-sflow agent
https://github.com/sflow/host-sflow/releases/download/v2.0.25-3/hsflowd-centos7-2.0.25-3.x86_64.rpm
- 2、ELK的安装包
elasticsearch-7.17.2-x86_64.rpm kibana-7.17.2-x86_64.rpm logstash-7.17.2-x86_64.rpm
具体安装部署步骤如下
一、调整相关的内核参数并关闭防火墙
sed -i 's/enable/disabled/g' /etc/selinux/config setenforce 0 hostnamectl set-hostname elastiflow echo "vm.max_map_count=262144" | sudo tee /etc/sysctl.d/70-elasticsearch.conf > /dev/null echo -e "net.core.netdev_max_backlog=4096\nnet.core.rmem_default=262144\nnet.core.rmem_max=67108864\nnet.ipv4.udp_rmem_min=131072\nnet.ipv4.udp_mem=2097152 4194304 8388608" | tee /etc/sysctl.d/60-net.conf > /dev/null sysctl -w vm.max_map_count=262144 && sysctl -w net.core.netdev_max_backlog=4096 && sysctl -w net.core.rmem_default=262144 && sysctl -w net.core.rmem_max=67108864 && sysctl -w net.ipv4.udp_rmem_min=131072 && sysctl -w net.ipv4.udp_mem='2097152 4194304 8388608' systemctl stop firewalld.service && systemctl disable firewalld.service
二、安装JDK环境并安装ELK
yum install java-openjdk-devel java-openjdk
rpm -ivh logstash-7.17.2-x86_64.rpm rpm -ivh elasticsearch-7.17.2-x86_64.rpm rpm -ivh kibana-7.17.2-x86_64.rpm
systemctl daemon-reload systemctl enable elasticsearch.service systemctl enable kibana.service systemctl enable logstash.service
三、Elasticsearch和Kibana配置文件修改
vim /etc/elasticsearch/elasticsearch.yml
vim /etc/elasticsearch/jvm.options
vim /etc/kibana/kibana.yml
四、elastiflow安装包解压
https://codeload.github.com/robcowart/elastiflow
mv elastiflow-master elastiflow cp -a elastiflow/logstash/elastiflow/. /etc/logstash/elastiflow/ cp -a elastiflow/logstash.service.d/. /etc/systemd/system/logstash.service.d/
vim /etc/logstash/pipelines.yml 修改添加成如下 #- pipeline.id: main # path.config: "/etc/logstash/conf.d/*.conf" - pipeline.id: elastiflow path.config: "/etc/logstash/elastiflow/conf.d/*.conf" pipeline.workers: 4
chown -R logstash:logstash /etc/logstash/elastiflow chown -R logstash:logstash /etc/logstash/pipelines.yml
vi /etc/logstash/jvm.options vim /etc/logstash/startup.options
/usr/share/logstash/bin/logstash-plugin install logstash-codec-sflow /usr/share/logstash/bin/logstash-plugin install logstash-codec-netflow /usr/share/logstash/bin/logstash-plugin install logstash-input-udp /usr/share/logstash/bin/logstash-plugin install logstash-input-tcp /usr/share/logstash/bin/logstash-plugin install logstash-filter-dns /usr/share/logstash/bin/logstash-plugin install logstash-filter-geoip /usr/share/logstash/bin/logstash-plugin install logstash-filter-translate
/usr/share/logstash/bin/system-install
五、启动服务
systemctl restart elasticsearch.service systemctl restart kibana.service systemctl restart logstash.service
六、Kibana配置
导入elastiflow.kibana.7.8.x.ndjson到kibana
修改如下kibana中的配置
七、Linux服务器安装及配置hsflow
rpm -ivh hsflowd-centos7-2.0.25-3.x86_64.rpm vim /etc/hsflowd.conf 添加如下配置 sampling = 1 #截取包大小为256 headerBytes = 256 #设置收集器地址和端口 collector { ip=192.168.31.189 udpport=6343 } #设置采样的网卡 pcap { dev = ens33 } systemctl enable hsflowd systemctl start hsflowd systemctl status hsflowd
八、效果测试截图
九、Tips
1、交换机配置sflow暂未进行测试
后续有环境会可以进行测试
2、若logstash启动失败
需要/var/log/logstash/logstash-plain.log 和journalctl -xe -u logstash进行排错
3、上面的kibana与ES未配置安全认证 后续有时间再做调整
