"Flink CDC啥情况会导致checkpoint的scn不更新?
oracle cdc 的flink 同步任务,
flink checkpoint的配置如下:
execution.checkpointing.interval: 5min
execution.checkpointing.externalized-checkpoint-retention: DELETE_ON_CANCELLATION
execution.checkpointing.max-concurrent-checkpoints: 1
execution.checkpointing.min-pause: 500
execution.checkpointing.mode: EXACTLY_ONCE
execution.checkpointing.timeout: 10min
execution.checkpointing.tolerable-failed-checkpoints: 20
state.backend.type: filesystem
state.checkpoints.dir: file:///mnt/nfs/flink/checkpoints
state.savepoints.dir: file:///mnt/nfs/flink/savepoints
网络故障中断后,通过checkpoint的scn恢复续传时, 报错
Caused by: io.debezium.DebeziumException: Online REDO LOG files or archive log files do not contain the offset scn 23867001247. Please perform a new snapshot.
oracle 查scn的生成时间,
SQL> select THREAD#, SEQUENCE#, RESETLOGS_ID, FIRST_CHANGE#, NEXT_CHANGE#, FIRST_TIME, NEXTTIME from sys.v$archived_log where DEST_ID=1 and FIRST_CHANGE# <=23867001247 and NEXT_CHANGE#>=23867001247;
THREAD# SEQUENCE# RESETLOGS_ID FIRST_CHANGE# NEXT_CHANGE# FIRST_TIM
2 5756 1074621776 2.3448E+10 2.3479E+10 09-APR-24
10-APR-24
1 4577 1074621776 2.3465E+10 2.3479E+10 09-APR-24
10-APR-24
这个时间时我第一次启动该flink job 同步任务的时间, 说明配置的每5分钟checkpoint的scn号没有更新, 但是checkpoint的状态一直是正常的。"
版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。
实时计算Flink版是阿里云提供的全托管Serverless Flink云服务,基于 Apache Flink 构建的企业级、高性能实时大数据处理系统。提供全托管版 Flink 集群和引擎,提高作业开发运维效率。