需求：
针对GrayLog的安全告警日志中的一些攻击IP，经常需要手工去微步在线情报社区去查恶意IP，为了提高效率，探索是否可以对IP进行自动化关联查询
思路：
1、GrayLog的LookupTable有HTTP JSONPATH这种方式，可以研究一下怎么使用2、翻阅微步在线API接口文档以及一些常用的IP地址库接口API 3、结合GrayLog强大的可定制化的功能，探索是否可以实现想要的效果
解决过程与步骤：
1、微步在线API接口文档
2、curl命令测试
curl -v -X GET 'https://api.threatbook.cn/v3/scene/ip_reputation?apikey=API_KEY&resource=37.59.54.205'
yum install jq -y curl -X GET 'https://api.threatbook.cn/v3/scene/ip_reputation?apikey=API_KEY&resource=37.59.54.205'| jq
3、GrayLog配置Lookup Tables
https://api.threatbook.cn/v3/scene/ip_reputation?apikey=API_KEY&resource=${key}
{ "data": { "37.59.54.205": { "severity": "low", "judgments": [ "Scanner", "IDC" ], "tags_classes": [], "basic": { "carrier": "OVH SAS", "location": { "country": "France", "province": "Ile-de-France", "city": "Paris", "lng": "2.345101", "lat": "48.848483", "country_code": "FR" } }, "asn": { "rank": 2, "info": "OVH, FR", "number": 16276 }, "scene": "", "confidence_level": "low", "is_malicious": true, "update_time": "2020-11-25 09:11:44" } }, "response_code": 0, "verbose_msg": "OK" }
IPv4 归属地，经纬度查询可以使用如下API接口
接口地址：https://api.ipplus360.com/ip/geo/v1/city/ 请求方式：HTTPS GET/POST 请求示例：https://api.ipplus360.com/ip/geo/v1/city/?key=您申请的key&ip=您需要查询的ip&coordsys=WGS84
最后针对以上API接口配置的LookupTable如下
4、可以针对有公网IP字段调用以上接口并将查询结果保存为日志
例如对Linux服务器暴力破解攻击IP为例例如将Linux服务器接入到GrayLog后，使用正则表达式提取器提取出暴力破解IP的ssh_login_failed_ip字段
Failed password for .* from(.+?)\s.?
然后使用LookupTable提取器，将IP经纬度结果转成字段
ip_geo_latitude Lookup Table Trying to extract data from ssh_login_failed_ip into ssh_login_failed_ip_geo_latitude ip_geo_longitude Lookup Table Trying to extract data from ssh_login_failed_ip into ssh_login_failed_ip_geo_longitude
当然也可以调用IP威胁情报的LookupTable
ip_geo_longitude Lookup Table Trying to extract data from ssh_login_failed_ip into ssh_login_failed_ip_geo_longitude
5、Pipeline配置
使用Pipeline拼接出经纬度地址位置字段
rule "GeoIP lookup: ssh_login_failed_ip" when has_field("ssh_login_failed_ip_geo_latitude") and has_field("ssh_login_failed_ip_geo_longitude") then let ip_geo_latitude = to_string($message.ssh_login_failed_ip_geo_latitude); let ip_geo_longitude = to_string($message.ssh_login_failed_ip_geo_longitude); let ip_location_temp1 = concat(to_string(ip_geo_latitude),","); let ip_location_temp2 = concat(ip_location_temp1, ip_geo_longitude); set_field("ssh_login_failed_ip_location", ip_location_temp2); end
使用Pipeline配置生成IP威胁情报查询结果字段ssh_login_failed_ip_threatcheck_result
ip_threatcheck_result rule "IPThreatChecklookup: ssh_login_failed_ip" when has_field("ssh_login_failed_ip") then let ip_threatcheck_result = lookup("ipthreatcheck",to_string($message.ssh_login_failed_ip)); debug (ip_threatcheck_result) ; set_field("ssh_login_failed_ip_threatcheck_result", ip_threatcheck_result); end
6、配置DashBoard显示攻击IP的经纬度
最终的效果
如上图所示：Zombie关键字搜索可以判定该攻击IP为僵尸傀儡机
地理位置为荷兰
Tips:
附上提取器的相关语法配置文件
{ "extractors": [ { "title": "ip_geo_latitude", "extractor_type": "lookup_table", "converters": [], "order": 1, "cursor_strategy": "copy", "source_field": "ssh_login_failed_ip", "target_field": "ssh_login_failed_ip_geo_latitude", "extractor_config": { "lookup_table_name": "ipplus360_lat" }, "condition_type": "none", "condition_value": "" }, { "title": "ip_geo_longitude", "extractor_type": "lookup_table", "converters": [], "order": 2, "cursor_strategy": "copy", "source_field": "ssh_login_failed_ip", "target_field": "ssh_login_failed_ip_geo_longitude", "extractor_config": { "lookup_table_name": "ipplus360_lng" }, "condition_type": "none", "condition_value": "" }, { "title": "ip_threatcheck", "extractor_type": "lookup_table", "converters": [], "order": 3, "cursor_strategy": "copy", "source_field": "ssh_login_failed_ip", "target_field": "ssh_login_failed_ip_threatcheck", "extractor_config": { "lookup_table_name": "ipthreatcheck" }, "condition_type": "none", "condition_value": "" }, { "title": "json_extractor", "extractor_type": "json", "converters": [], "order": 4, "cursor_strategy": "copy", "source_field": "ssh_login_failed_ip_threatcheck_result", "target_field": "", "extractor_config": { "flatten": false, "list_separator": ", ", "kv_separator": "=", "key_prefix": "ip_threatcheck_result_", "key_separator": "_", "replace_key_whitespace": false, "key_whitespace_replacement": "_" }, "condition_type": "none", "condition_value": "" }, { "title": "ssh_login_failed_ip_extractor", "extractor_type": "regex", "converters": [], "order": 0, "cursor_strategy": "copy", "source_field": "message", "target_field": "ssh_login_failed_ip", "extractor_config": { "regex_value": "Failed password for .* from(.+?)\\s.?" }, "condition_type": "string", "condition_value": "Failed password " } ], "version": "4.2.10" }
