flink savepoint问题

Hi，您好，在使用flink的过程中遇到如下问题，个人未能解决，所以请求您指导一下，谢谢！ 

1. flink cancel -s $SAVEPOINT_DIR $job_id -yid $application_id 在程序有背压的时候停不掉 

The program finished with the following exception:  org.apache.flink.util.FlinkException: Could not cancel job 1f768e4ca9ad5792a4844a5d12163b73.  at org.apache.flink.client.cli.CliFrontend.lambda$cancel$6(CliFrontend.java:523)  at org.apache.flink.client.cli.CliFrontend.runClusterAction(CliFrontend.java:843)  at org.apache.flink.client.cli.CliFrontend.cancel(CliFrontend.java:515)  at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:904)  at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:968)  at java.security.AccessController.doPrivileged(Native Method)  at javax.security.auth.Subject.doAs(Subject.java:422)  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)  at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)  at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:968)  Caused by: java.util.concurrent.TimeoutException  at java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1771)  at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)  at org.apache.flink.client.cli.CliFrontend.lambda$cancel$6(CliFrontend.java:521)  ... 9 more  stop flink job failed!!! 

2.再用flink sql的ddl时候增加一个字段后，程序启动失败，需要删除savepoint才能启动(ddl后双流join的逻辑，其中一个流加了一个字段) 

The program finished with the following exception:  org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: org.apache.flink.runtime.concurrent.FutureUtils$RetryException: Could not complete the operation. Number of retries has been exhausted.  at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)  at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)  at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138)  at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:664)  at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)  at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:895)  at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:968)  at java.security.AccessController.doPrivileged(Native Method)  at javax.security.auth.Subject.doAs(Subject.java:422)  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)  at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)  at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:968)  Caused by: java.util.concurrent.ExecutionException: org.apache.flink.runtime.concurrent.FutureUtils$RetryException: Could not complete the operation. Number of retries has been exhausted.  at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)  at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)  at org.apache.flink.streaming.api.environment.StreamContextEnvironment.execute(StreamContextEnvironment.java:83)  at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1620)  at com.etl.DwdBaseCarlifeTIotBindGeoManage.main(DwdBaseCarlifeTIotBindGeoManage.java:265)  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  at java.lang.reflect.Method.invoke(Method.java:498)  at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)  ... 11 more  Caused by: org.apache.flink.runtime.concurrent.FutureUtils$RetryException: Could not complete the operation. Number of retries has been exhausted.  at org.apache.flink.runtime.concurrent.FutureUtils.lambda$retryOperationWithDelay$8(FutureUtils.java:284)  at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)  at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)  at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)  at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)  at org.apache.flink.runtime.rest.RestClient.lambda$submitRequest$1(RestClient.java:342)  at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:500)  at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:493)  at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:472)  at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:413)  at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:538)  at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:531)  at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:111)  at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:323)  at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:339)  at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:685)  at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)  at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)  at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)  at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)  at org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)  at java.lang.Thread.run(Thread.java:748)  Caused by: java.util.concurrent.CompletionException: org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused:/10...:42666  at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)  at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)  at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:943)  at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926)  ... 19 more  Caused by: org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused:centos/10..*:42666  Caused by: java.net.ConnectException: Connection refused  at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)  at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)  at org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:327)  at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:336)  at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:685)  at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:632)  at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:549)  at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:511)  at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)  at org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)  at java.lang.Thread.run(Thread.java:748) 

3.flink checkpoint超时问题，我之前的checkpoint间隔是3分钟，超时时间是5分钟，报了如下错误，而且job与taskmanager都不挂，很难监控。  现在checkpoint间隔调整成2分钟，超时时间是10分钟，不知是否会解决。  Checkpoint 298 of job 1f768e4ca9ad5792a4844a5d12163b73 expired before completing.*来自志愿者整理的flink邮件归档