Flink CDC里为什么我的 flink job 一直卡在 DEPLOYING 不动啊?
2024-01-23 01:52:44,015 INFO org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Registered job manager 00000000000000000000000000000000@pekko.tcp://flink@flink-jobmanager:6123/user/rpc/jobmanager_2 for job 04ea74dd688e3908309b92f28207761a.
2024-01-23 01:52:44,018 INFO org.apache.flink.runtime.jobmaster.JobMaster [] - JobManager successfully registered at ResourceManager, leader id: 00000000000000000000000000000000.
2024-01-23 01:52:44,028 INFO org.apache.flink.runtime.resourcemanager.slotmanager.FineGrainedSlotManager [] - Received resource requirements from job 04ea74dd688e3908309b92f28207761a: [ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}]
2024-01-23 01:52:44,098 INFO org.apache.flink.runtime.resourcemanager.slotmanager.FineGrainedSlotManager [] - Matching resource requirements against available resources.
Missing resources:
Job 04ea74dd688e3908309b92f28207761a
ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN}, numberOfRequiredSlots=1}
Current resources:
TaskManager 10.42.0.120:38895-bfc5cd
Available: ResourceProfile{cpuCores=1, taskHeapMemory=384.000mb (402653174 bytes), taskOffHeapMemory=0 bytes, managedMemory=512.000mb (536870920 bytes), networkMemory=128.000mb (134217730 bytes)}
Total: ResourceProfile{cpuCores=1, taskHeapMemory=384.000mb (402653174 bytes), taskOffHeapMemory=0 bytes, managedMemory=512.000mb (536870920 bytes), networkMemory=128.000mb (134217730 bytes)}
2024-01-23 01:52:44,105 INFO org.apache.flink.runtime.resourcemanager.slotmanager.DefaultSlotStatusSyncer [] - Starting allocation of slot 678a4196ca38b40c863833981baeb169 from 10.42.0.120:38895-bfc5cd for job 04ea74dd688e3908309b92f28207761a with resource profile ResourceProfile{cpuCores=1, taskHeapMemory=384.000mb (402653174 bytes), taskOffHeapMemory=0 bytes, managedMemory=512.000mb (536870920 bytes), networkMemory=128.000mb (134217730 bytes)}.
2024-01-23 01:52:44,364 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Source: job[1] -> DropUpdateBefore[2] -> ConstraintEnforcer[3] -> Sink: skill_upp_table_job[3] (1/1) (0d41fb8db7f44cfb64adb97f8248b1c7_cbc357ccb763df2852fee8c4fc7d55f2_0_0) switched from SCHEDULED to DEPLOYING.
2024-01-23 01:52:44,374 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] - Deploying Source: job[1] -> DropUpdateBefore[2] -> ConstraintEnforcer[3] -> Sink: skill_upp_table_job[3] (1/1) (attempt #0) with attempt id 0d41fb8db7f44cfb64adb97f8248b1c7_cbc357ccb763df2852fee8c4fc7d55f2_0_0 and vertex id cbc357ccb763df2852fee8c4fc7d55f2_0 to 10.42.0.120:38895-bfc5cd @ ip-10-42-0-120.ap-southeast-1.compute.internal (dataPort=40729) with allocation id 678a4196ca38b40c863833981baeb169
2024-01-23 01:52:45,698 INFO com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner [] - Split table skill_upp.job into 1 chunks, time cost: 1701ms.
根据您提供的信息,Flink CDC作业在部署过程中卡住的原因可能是资源不足。从日志中可以看到,作业需要1个slot,但当前可用的资源只有1个TaskManager,且其可用资源与作业所需的资源不匹配。
要解决这个问题,您可以尝试以下方法:
版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。
实时计算Flink版是阿里云提供的全托管Serverless Flink云服务,基于 Apache Flink 构建的企业级、高性能实时大数据处理系统。提供全托管版 Flink 集群和引擎,提高作业开发运维效率。