1 下载安装
1.kafka-eagle-bin-1.2.4.tar.gz 安装包下载，官网 或者在我百度网盘中下载。点击下载
2.我们直接按照步骤详细说明下安装步骤，JDK一定要安装必须知道JDK的安装路径
2 环境配置
2.1.安装包拷贝至服务器解压安装包
tar -zxf kafka-eagle-bin-1.2.4.tar.gz
2.2设置全局变量，设置JDK的全局变量JAVA_HOME和本应用的KE_HOME
--使用root权限编辑环境变量 sudo vi /etc/profile --打开后加上 export JAVA_HOME=/kafka/jdk1.8.0_65 export KE_HOME=/kafka/kafka-eagle-bin-1.2.4/kafka-eagle-web-1.2.4 export PATH=$PATH:$KE_HOME/bin:$JAVA_HOME/bin
2.3进入kafka-eagle的conf目录下修改配置文件，配置基础配置
#修改配置文件 vim system-config.properties #配置文件详情介绍 #kafkazookeeper节点配置属性多个可以添加一个，cluster1 kafka.eagle.zk.cluster.alias=cluster1 cluster1.zk.list=192.168.163.131:2181 ###################################### # zk 线程数量 ###################################### kafka.zk.limit.size=25 ###################################### # kafka eagle 的端口 ###################################### kafka.eagle.webui.port=8048 ###################################### # kafka offset storage ###################################### cluster1.kafka.eagle.offset.storage=kafka ###################################### # enable kafka 开启图表 # 及开始sql查询 ###################################### kafka.eagle.metrics.charts=true kafka.eagle.sql.fix.error=true ###################################### # 提醒的email ###################################### kafka.eagle.mail.enable=true kafka.eagle.mail.sa=alert_sa kafka.eagle.mail.username=alert_sa@163.com kafka.eagle.mail.password=mqslimczkdqabbbh kafka.eagle.mail.server.host=smtp.163.com kafka.eagle.mail.server.port=25 ###################################### # 删除kafka topic 的token ###################################### kafka.eagle.topic.token=keadmin ###################################### # kafka sasl authenticate ###################################### kafka.eagle.sasl.enable=false kafka.eagle.sasl.protocol=SASL_PLAINTEXT kafka.eagle.sasl.mechanism=PLAIN ###################################### # kafka jdbc 地址注意可以自己安装数据mysql也可以自带的 ###################################### kafka.eagle.driver=org.sqlite.JDBC kafka.eagle.url=jdbc:sqlite:/kafka/kafka-eagle-bin-1.2.4/kafka-eagle-web-1.2.4/db/ke.db kafka.eagle.username=root kafka.eagle.password=smartloli
2.4文件配置完毕后开始启动 ，启动要进入bin目录下
#进入bin目录后会看到 ke.sh 文件先修改文件的权限 sudo chomd 777 ke.sh #启动命令 ./ke.sh start
启动成功后的控制台输出
部署完成结束
3 页面功能
3.1登陆界面
ttp://192.168.80.212:8048/ke/ 账号：admin 密码：123456
3.2功能界面
3.3我们来着重的说下kafkasql查询的功能其他的功能点点就知道什么意思和怎么使用了
4 KSQL的常用语句
1. SELECT COUNT(*) FROM wiseweb_crawler_foreignmedias WHERE site_id=90000 AND (gathertime BETWEEN '2017-05-01 00:00:01' AND '2017-05-10 23:59:59')
2、 SELECT site_id, COUNT(*) FROM wiseweb_crawler_foreignmedias WHERE (site_id BETWEEN 90000 and 99999) AND (gathertime BETWEEN '2017-05-11 00:00:01' AND '2017-05-11 23:59:59') GROUP BY site_id limit 3000
3 title:'Иртыш потоп' or content:'Иртыш потоп' AND gathertime:[ "2017-05-20 00:00:01" TO "2017-05-23 23:59:59"]
4 SELECT site_id,title,publishtime,gathertime FROM wiseweb_crawler_foreignmedias where (title = '%Водная%' OR content = '%Водная%') AND(title = '%сотрудничество%' OR content = '%сотрудничество%') AND(title = '%Китай%' OR content = '%Китай%')
5 SELECT site_name,title,publishtime,content,url FROM wiseweb_crawler_website20170718 WHERE (title = '舆情' OR content = '舆情') AND (gathertime BETWEEN '2017-07-21 00:00:00' AND '2017-07-28 23:59:59')AND (site_id BETWEEN 129494 AND 130357) 金融116886-117064 金融 129494- 130357 SELECT site_id,title,publishtime,site_name FROM wiseweb_crawler_website20171024 where (content='%舆情%'or content='%舆论引导%'or content='%脱敏%'or content='%数据脱敏%'or content='%精准搜索%'or content='%画像%'or content='%风控%'or content='%风险控制%'or content='%内容管理%'or content='%大数据分析%' or content='%大数据采集%'or content='%大数据挖掘%')and(contet='%招标公告%' or title='%招标公告%') SELECT site_id,title,publishtime,content,site_name,url FROM wiseweb_crawler_website20171201 where (content='%舆情%'or content='%舆论引导%'or content='%脱敏%'or content='%数据脱敏%'or content='%精准搜索%'or content='%画像%'or content='%风控%'or content='%风险控制%'or content='%内容管理%'or content='%大数据分析%' or content='%大数据采集%'or content='%大数据挖掘%')and(contet='%招标公告%' or title='%招标公告%')AND (publishtime BETWEEN '2017-12-01 00:00:00' AND '2017-12-08 23:59:59')
