开发者社区 问答 正文

flink 1.12.0 kubernetes-session部署问题

环境:MacBook Pro 单机安装了 minkube v1.15.1 和 kubernetes v1.19.4 我在flink v1.11.3发行版下执行如下命令 kubectl create namespace flink-session-cluster

kubectl create serviceaccount flink -n flink-session-cluster

kubectl create clusterrolebinding flink-role-binding-flink \ --clusterrole=edit \ --serviceaccount=flink-session-cluster:flink

./bin/kubernetes-session.sh \ -Dkubernetes.namespace=flink-session-cluster \ -Dkubernetes.jobmanager.service-account=flink \ -Dkubernetes.cluster-id=session001 \ -Dtaskmanager.memory.process.size=8192m \ -Dkubernetes.taskmanager.cpu=1 \ -Dtaskmanager.numberOfTaskSlots=4 \ -Dresourcemanager.taskmanager-timeout=3600000

屏幕打印的结果显示flink web UI启在了 http://192.168.64.2:8081 而不是类似于 http://192.168.50.135:31753 这样的5位数端口,是哪里有问题?这里的host ip应该是minikube ip吗?我本地浏览器访问不了http://192.168.64.2:8081

2021-01-02 10:28:04,177 INFO org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils [] - The derived from fraction jvm overhead memory (160.000mb (167772162 bytes)) is less than its min value 192.000mb (201326592 bytes), min value will be used instead

2021-01-02 10:28:04,907 INFO org.apache.flink.kubernetes.KubernetesClusterDescriptor [] - Create flink session cluster session001 successfully, JobManager Web Interface: http://192.168.64.2:8081

查看了pods, service, deployment都正常启动好了,显示全绿色的

接下来提交任务 ./bin/flink run -d \ -e kubernetes-session \ -Dkubernetes.namespace=flink-session-cluster \ -Dkubernetes.cluster-id=session001 \ examples/streaming/WindowJoin.jar

Using windowSize=2000, data rate=3

To customize example, use: WindowJoin [--windowSize ] [--rate ]

2021-01-02 10:21:48,658 INFO org.apache.flink.kubernetes.KubernetesClusterDescriptor [] - Retrieve flink cluster session001 successfully, JobManager Web Interface: http://10.106.136.236:8081

这里显示的 http://10.106.136.236:8081 我是能够通过浏览器访问到的,打开显示作业正在运行,而且available slots一项显示的是 0,查看JM日志有如下error

Causedby: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Couldnot allocate the required slot within slot request timeout. Please make sure that the cluster has enough resources. at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeWrapWithNoResourceAvailableException(DefaultScheduler.java:441) ~[flink-dist_2.12-1.11.3.jar:1.11.3] ... 47 more Causedby: java.util.concurrent.CompletionException: java.util.concurrent.TimeoutException at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[?:1.8.0_275] at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[?:1.8.0_275] at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:607) ~[?:1.8.0_275] at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591) ~[?:1.8.0_275] ... 27 more Causedby: java.util.concurrent.TimeoutException ... 25 more

为什么会报这个资源配置不足的错?谢谢解答!*来自志愿者整理的flink邮件归档

展开
收起
毛毛虫雨 2021-12-08 11:20:59 925 分享 版权
1 条回答
写回答
取消 提交回答
  • native方式默认使用的是LoadBalancer的方式来暴露,所以会打印出来一个你无法访问的地址 你可以加一个-Dkubernetes.rest-service.exposed.type=NodePort的方式来使用NodePort来暴露 这样Flink Client端打印出来的地址就是正确的了

    另外你可以可以使用minikube ip来查看ip地址,同时用kubectl get svc获取你创建的Flink cluster svc的NodePort,拼起来就可以

    至于你说的NoResourceAvailableException,你可以看下是不是TaskManager的Pod已经创建出来了,但是pending状态 如果是,那就是你minikube资源不够了,可以把minikube资源调大或者把JobManager、TaskManager的Pod资源调小 如果不是,你可以把完整的JobManager日志发一下,这样方便查问题*来自志愿者整理的flink邮件归档

    2021-12-08 18:32:32
    赞同 展开评论