有大佬做过oracle-cdc在日志文件定时删除的场景下的同步吗?

有大佬做过oracle-cdc在日志文件定时删除的场景下的同步吗?2023-05-12 17:23:45 com.ververica.cdc.connectors.shaded.org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped. at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:42) at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:252) at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:152) at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:119) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.sql.SQLException: ORA-00310: archived log contains sequence 39422; sequence 39417 required ORA-00334: archived log: '/oraredo/sybarch/redo02.log'

at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:509)
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:461)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1104)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:550)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:268)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:655)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:270)
at oracle.jdbc.driver.T4CPreparedStatement.fetch(T4CPreparedStatement.java:1079)
at oracle.jdbc.driver.OracleStatement.fetchMoreRows(OracleStatement.java:3456)
at oracle.jdbc.driver.InsensitiveScrollableResultSet.fetchMoreRows(InsensitiveScrollableResultSet.java:742)
at oracle.jdbc.driver.InsensitiveScrollableResultSet.absoluteInternal(InsensitiveScrollableResultSet.java:698)
at oracle.jdbc.driver.InsensitiveScrollableResultSet.next(InsensitiveScrollableResultSet.java:412)
at io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.hasNext(LogMinerQueryResultProcessor.java:245)
at io.debezium.connector.oracle.logminer.LogMinerQueryResultProcessor.processResult(LogMinerQueryResultProcessor.java:102)
at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:230)
... 7 more

Caused by: Error : 310, Position : 0, Sql = SELECT SCN, SQL_REDO, OPERATION_CODE, TIMESTAMP, XID, CSF, TABLE_NAME, SEG_OWNER, OPERATION, USERNAME, ROW_ID, ROLLBACK FROM V$LOGMNR_CONTENTS WHERE SCN > :1 AND SCN <= :2 AND ((OPERATION_CODE IN (5,34) AND USERNAME NOT IN ('SYS','SYSTEM','FLINKUSER')) OR (OPERATION_CODE IN (7,36)) OR (OPERATION_CODE IN (1,2,3) AND TABLE_NAME != 'LOG_MINING_FLUSH' AND SEG_OWNER NOT IN ('APPQOSSYS','AUDSYS','CTXSYS','DVSYS','DBSFWUSER','DBSNMP','GSMADMIN_INTERNAL','LBACSYS','MDSYS','OJVMSYS','OLAPSYS','ORDDATA','ORDSYS','OUTLN','SYS','SYSTEM','WMSYS','XDB') AND (REGEXP_LIKE(SEG_OWNER,'^SYBBAKUSER$','i')) AND (REGEXP_LIKE(SEG_OWNER || '.' || TABLE_NAME,'^SYBBAKUSER.KTRXINFO6$','i')) )), OriginalSql = SELECT SCN, SQL_REDO, OPERATION_CODE, TIMESTAMP, XID, CSF, TABLE_NAME, SEG_OWNER, OPERATION, USERNAME, ROW_ID, ROLLBACK FROM V$LOGMNR_CONTENTS WHERE SCN > ? AND SCN <= ? AND ((OPERATION_CODE IN (5,34) AND USERNAME NOT IN ('SYS','SYSTEM','FLINKUSER')) OR (OPERATION_CODE IN (7,36)) OR (OPERATION_CODE IN (1,2,3) AND TABLE_NAME != 'LOG_MINING_FLUSH' AND SEG_OWNER NOT IN ('APPQOSSYS','AUDSYS','CTXSYS','DVSYS','DBSFWUSER','DBSNMP','GSMADMIN_INTERNAL','LBACSYS','MDSYS','OJVMSYS','OLAPSYS','ORDDATA','ORDSYS','OUTLN','SYS','SYSTEM','WMSYS','XDB') AND (REGEXP_LIKE(SEG_OWNER,'^SYBBAKUSER$','i')) AND (REGEXP_LIKE(SEG_OWNER || '.' || TABLE_NAME,'^SYBBAKUSER.KTRXINFO6$','i')) )), Error Msg = ORA-00310: archived log contains sequence 39422; sequence 39417 required ORA-00334: archived log: '/oraredo/sybarch/redo02.log'

at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:513)
... 21 more

展开
收起
真的很搞笑 2023-05-14 21:37:25 149 分享 版权
阿里云 AI 助理回答

遇到这个问题,看起来是Oracle CDC(Change Data Capture)在使用Debezium进行日志挖掘(LogMiner)时,遇到了归档日志文件被删除或不完整的情况。具体来说,错误信息指出需要的归档日志序列号(Sequence 39417)与当前可访问的归档日志序列号(Sequence 39422)不匹配。

在处理Oracle CDC且日志文件可能被定时删除的场景下,可以考虑以下几个解决方案和建议:

  1. 保留足够的归档日志:确保你的归档日志策略能够保留足够长的时间,以覆盖到所有可能的CDC处理周期。这可能意味着需要调整数据库的归档日志保留策略,比如通过设置LOG_ARCHIVE_RETENTION_TARGET参数来延长归档日志的保留时间。

  2. 重新配置LogMiner以适应日志清理策略:如果日志定期清理不可避免,需要确保Debezium的LogMiner配置能够从最新的可用日志开始读取,而不是依赖于已被删除的日志。这可能涉及到在初始化Debezium连接器时指定正确的起始SCN或时间点。

  3. 处理异常并重试:在应用层面,捕获这类异常并实现逻辑以优雅地处理它们,例如记录错误、通知运维,并根据情况自动或手动重新配置Debezium以从一个已知的安全点继续同步。

  4. 监控和报警:实施有效的监控和报警机制,当发现日志序列不连续或缺失时立即通知维护人员,以便及时介入并采取措施。

  5. 使用Oracle Goldengate:如果你的环境允许,考虑使用Oracle Goldengate作为替代方案,它提供了更高级的日志处理能力,对于复杂或严格的数据同步需求更为健壮。

  6. 检查和修复归档日志链:如果可能,检查数据库的归档日志链是否完整,必要时使用Oracle的恢复工具如RMAN来尝试修复或补充缺失的日志。

记得在进行任何配置更改之前,详细规划并测试这些变更,以避免对生产环境造成不必要的影响。您可以复制页面截图提供更多信息,我可以进一步帮您分析问题原因。

有帮助
无帮助
AI 助理回答生成答案可能存在不准确,仅供参考
0 条回答
写回答
取消 提交回答

大数据领域前沿技术分享与交流,这里不止有技术干货、学习心得、企业实践、社区活动,还有未来。

还有其他疑问?
咨询AI助理