概述
通常我们要 silence 某个 AlertManager 的 alert 时，需要通过 UI 界面操作，如下图：
AlertManager silence
效率有点低，而且不够自动化，那么是否可以有一种办法快速创建 AlertManager silence 呢？
– 有的，通过 API.
API Payload
v1
如下：
{ "matchers": [ { "name": "alername1", "value": ".*", "isRegex": true } ], "startsAt": "2022-04-29T22:12:33.533Z", "endsAt": "2022-04-29T23:11:44.603Z", "createdBy": "api", "comment": "Silence", "status": { "state": "active" } } JSON |
v2
{ "matchers": [ { "name": "service", "value": "rancher", "isRegex": false, "isEqual": true }, { "name": "alertname", "value": "TargetDown", "isRegex": false, "isEqual": true } ], "startsAt": "2022-04-29T10:11:35.656Z", "endsAt": "2022-04-29T12:11:35.656Z", "createdBy": "Casey Cui", "comment": " 配置错误导致的误报 ", "id": null } JSON |
具体实现
curl 实现
📝 Notes:
以 v1 api 为例
如下：
curl https://alertmanager.ewhisper.cn/api/v1/silences -d '{ "matchers": [ { "name": "alername1", "value": ".*", "isRegex": true } ], "startsAt": "2022-04-29T22:12:33.533Z", "endsAt": "2022-04-29T23:11:44.603Z", "createdBy": "api", "comment": "Silence", "status": { "state": "active" } }' BASH |
📝Notes:
在 K8S 集群内，地址一般为：
http://alertmanager:9093
python 实现
如下：
#!/usr/bin/python3 import requests import socket import datetime import time res = requests.post("http://alertmanager:9093/api/v2/silences", json={ "matchers": [ {"name": "job", "value": "myjob", "isRegex": False}, {"name": "instance", "value": "{}:1234".format(socket.gethostname()), "isRegex": False}, ], "startsAt": datetime.datetime.utcfromtimestamp(time.time()).isoformat(), "endsAt": datetime.datetime.utcfromtimestamp(time.time() + 4*3600).isoformat(), "comment": "Backups on {}".format(socket.gethostname()), "createdBy": "My backup script", }, ) res.raise_for_status() silenceId = res.json()["silenceID"] PYTHON |
移除 silence 的脚本：
res = requests.delete("http://alertmanager:9093/api/v2/silence/{}".format(silenceId)) res.raise_for_status() PYTHON |
EOF
