Azkaban 任务一直处于运行状态,日志显示如下:
12-04-2023 10:12:04 CST test INFO - Cannot request memory (Xms 65536 kb, Xmx 270336 kb) from system for job test, sleep for 60 secs and retry, attempt 1 of 720
12-04-2023 10:13:04 CST test INFO - Cannot request memory (Xms 65536 kb, Xmx 270336 kb) from system for job test, sleep for 60 secs and retry, attempt 2 of 720
12-04-2023 10:14:04 CST test INFO - Cannot request memory (Xms 65536 kb, Xmx 270336 kb) from system for job test, sleep for 60 secs and retry, attempt 3 of 720
12-04-2023 10:15:04 CST test INFO - Cannot request memory (Xms 65536 kb, Xmx 270336 kb) from system for job test, sleep for 60 secs and retry, attempt 4 of 720
12-04-2023 10:16:04 CST test INFO - Cannot request memory (Xms 65536 kb, Xmx 270336 kb) from system for job test, sleep for 60 secs and retry, attempt 5 of 720
这是由于 Azkaban 在执行任务时,检测到所有的 executor
节点内存都小于 3GB
,其最低要求的内存是 3GB
,导致其无法进行任务的分配,一直处于等待的过程。
解决方法:
释放内存;
增加内存;
- 关闭检测内存的选项;
- 编辑
$AZKABAN_HOME/azkaban-exec-server/plugins/jobtypes/
目录下的commonprivate.properties
文件,在其中添加memCheck.enabled=false
配置项。
- 编辑