简介
- Elasticsearch 拥有副本机制来保障集群的高可用，然而无法解决如下情况的数据丢失:
- 主副本所在机器存储全部损坏。
- 误删除索引数据。
- 升级失败，数据无法回滚。
- 定期对数据做备份，按需恢复可以很好的解决如上问题。
- Elasticsearch 提供了 Snapshot 和 Restore API 用于对集群数据完成备份与恢复。
- 数据备份的过程可以简单理解成将本地数据文件同步到远程仓库(repository) 的过程。
- 支持全量和增量备份。
- repository 常见有如下类型: fs / S3 / HDFS / Azure / Google Cloud Storage。
实战
1 创建原始索引并导入数据
PUT _template/blog-template { "index_patterns": [ "blog-*" ], "settings": { "number_of_shards": 3, "number_of_replicas": 1 }, "mappings": { "dynamic_templates": [ { "integers": { "match": "int_*", "mapping": { "type": "integer" } } }, { "strings": { "match_mapping_type": "string", "mapping": { "type": "keyword", "ignore_above": 256 } } } ], "properties": { "title": { "type": "text" } } } } PUT blog-test POST blog-test/_bulk {"index":{}} {"title":"elasticsearch best practice","author_firstname":"tony","author_lastname":"song","tags":"elasticsearch ,logstash","int_age":18,"locale":"zh,en"} {"index":{}} {"title":"elastic stack release V5","author_firstname":"shay","author_lastname":"banon","tags":"elk ,elasticsearch,release","int_age":28,"locale":"zhc,en"} {"index":{}} {"title":"kibana tutorial","author_firstname":"zen","author_lastname":"wei","tags":"","int_age":38,"locale":"zhc,en"} GET blog-test GET blog-test/_search
2 修改elasticsearch配置文件
修改 elasticsearch.yml 配置文件，增加如下配置:
path.repo: ["/home/elastic/backup"]
重启elasticsearch进程，查看创建的repo：
GET _cluster/settings?include_defaults&filter_path=*.path.repo #输出结果： { "defaults" : { "path" : { "repo" : [ "/home/elastic/backup" ] } } }
3 创建关联 repository
#创建关联 repository PUT /_snapshot/my_fs_backup { "type": "fs", "settings": { "location": "/home/elastic/backup/my_fs_backup", "compress": true } }
查看创建的关联：
GET _snapshot/my_fs_backup #输出结果: { "my_fs_backup" : { "type" : "fs", "settings" : { "compress" : "true", "location" : "/home/elastic/backup/my_fs_backup" } } }
查看在哪个node上创建了关联：
POST _snapshot/my_fs_backup/_verify #输出结果 { "nodes" : { "pMrJwVGSQcSgeTZdh61QRw" : { "name" : "node1" } } }
4 执行 snapshot
indices：做快照的索引。wait_for_completion=true：是否等待完成快照后再响应，如果为true会等快照完成后才响应。（默认为false，不等快照完成立即响应）ignore_unavailable: 设置为true时，当创建快照时忽略不存在的索引。include_global_state: 设置为false时，当某个索引所有的主分片不是全部的都可用时，可以完成快照。
#创建snapshot_1对blog-test索引做快照 PUT /_snapshot/my_fs_backup/snapshot_1?wait_for_completion=true { "indices": "blog-test", "ignore_unavailable": true, "include_global_state": false }
查看创建的快照：
GET /_snapshot/my_fs_backup/snapshot_* #GET /_snapshot/my_fs_backup/_all 看所有的 #输出结果： { "snapshots" : [ { "snapshot" : "snapshot_1", "uuid" : "FEbAt3BiR1SAiBkO7pfoZg", "version_id" : 7020199, "version" : "7.2.1", "indices" : [ "blog-test" ], "include_global_state" : false, "state" : "SUCCESS", "start_time" : "2021-02-06T03:28:40.001Z", "start_time_in_millis" : 1612582120001, "end_time" : "2021-02-06T03:28:40.213Z", "end_time_in_millis" : 1612582120213, "duration_in_millis" : 212, "failures" : [ ], "shards" : { "total" : 3, "failed" : 0, "successful" : 3 } } ] }
查看快照状态：
GET _snapshot/my_fs_backup/snapshot_1/_status #输出结果: { "snapshots" : [ { "snapshot" : "snapshot_1", "repository" : "my_fs_backup", "uuid" : "FEbAt3BiR1SAiBkO7pfoZg", "state" : "SUCCESS", "include_global_state" : false, "shards_stats" : { "initializing" : 0, "started" : 0, "finalizing" : 0, "done" : 3, "failed" : 0, "total" : 3 }, "stats" : { "incremental" : { "file_count" : 12, "size_in_bytes" : 15608 }, "total" : { "file_count" : 12, "size_in_bytes" : 15608 }, "start_time_in_millis" : 1612582120074, "time_in_millis" : 108 }, "indices" : { "blog-test" : { "shards_stats" : { "initializing" : 0, "started" : 0, "finalizing" : 0, "done" : 3, "failed" : 0, "total" : 3 }, "stats" : { "incremental" : { "file_count" : 12, "size_in_bytes" : 15608 }, "total" : { "file_count" : 12, "size_in_bytes" : 15608 }, "start_time_in_millis" : 1612582120074, "time_in_millis" : 108 }, "shards" : { "0" : { "stage" : "DONE", "stats" : { "incremental" : { "file_count" : 4, "size_in_bytes" : 5104 }, "total" : { "file_count" : 4, "size_in_bytes" : 5104 }, "start_time_in_millis" : 1612582120143, "time_in_millis" : 39 } }, "1" : { "stage" : "DONE", "stats" : { "incremental" : { "file_count" : 4, "size_in_bytes" : 5265 }, "total" : { "file_count" : 4, "size_in_bytes" : 5265 }, "start_time_in_millis" : 1612582120074, "time_in_millis" : 25 } }, "2" : { "stage" : "DONE", "stats" : { "incremental" : { "file_count" : 4, "size_in_bytes" : 5239 }, "total" : { "file_count" : 4, "size_in_bytes" : 5239 }, "start_time_in_millis" : 1612582120113, "time_in_millis" : 21 } } } } } } ] }
5 恢复 snapshot
rename_pattern: 正则匹配原索引名rename_replacement: 将匹配到的字段用于重命名新索引
POST _snapshot/my_fs_backup/snapshot_1/_restore { "indices": "blog-test", "ignore_unavailable": true, "rename_pattern": "(.+)", "rename_replacement": "restored_$1" }
查询新索引的数据：
GET restored_blog-test/_search
