Apache flink 在使用Apach Flink的节点上,我应该在防火墙中打开哪些端口?

Apache flink 在使用Apach Flink的节点上,我应该在防火墙中打开哪些端口?,apache-flink,Apache Flink,当我尝试在Apache Flink独立群集上运行流时,我看到以下异常: java.lang.IllegalStateException: Update task on instance aaa0859f6af25decf1f5fc1821ffa55d @ app-2 - 4 slots - URL: akka.tcp://flink@192.168.38.98:46369/user/taskmanager failed due to: at org.apache.flink.runtim

当我尝试在Apache Flink独立群集上运行流时,我看到以下异常:

java.lang.IllegalStateException: Update task on instance aaa0859f6af25decf1f5fc1821ffa55d @ app-2 - 4 slots - URL: akka.tcp://flink@192.168.38.98:46369/user/taskmanager failed due to:
    at org.apache.flink.runtime.executiongraph.Execution$6.onFailure(Execution.java:954)
    at akka.dispatch.OnFailure.internal(Future.scala:228)
    at akka.dispatch.OnFailure.internal(Future.scala:227)
    at akka.dispatch.japi$CallbackBridge.apply(Future.scala:174)
    at akka.dispatch.japi$CallbackBridge.apply(Future.scala:171)
    at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
    at scala.runtime.AbstractPartialFunction.applyOrElse(AbstractPartialFunction.scala:28)
    at scala.concurrent.Future$$anonfun$onFailure$1.apply(Future.scala:136)
    at scala.concurrent.Future$$anonfun$onFailure$1.apply(Future.scala:134)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
    at scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121)
    at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka.tcp://flink@192.168.38.98:46369/user/taskmanager#1804590378]] after [10000 ms]
    at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:333)
    at akka.actor.Scheduler$$anon$7.run(Scheduler.scala:117)
    at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:599)
    at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
    at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:597)
    at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(Scheduler.scala:467)
    at akka.actor.LightArrayRevolverScheduler$$anon$8.executeBucket$1(Scheduler.scala:419)
    at akka.actor.LightArrayRevolverScheduler$$anon$8.nextTick(Scheduler.scala:423)
    at akka.actor.LightArrayRevolverScheduler$$anon$8.run(Scheduler.scala:375)
    at java.lang.Thread.run(Thread.java:745)
似乎端口46369已被防火墙阻止。这是真的,因为我只读取和打开这些端口:

6121:
  comment: Apache Flink TaskManager (Data Exchange)
6122:
  comment: Apache Flink TaskManager (IPC)
6123:
  comment: Apache Flink JobManager
6130:
  comment: Apache Flink JobManager (BLOB Server)
8081:
  comment: Apache Flink JobManager (Web UI)
flink-conf.yaml中描述的相同端口:

因此,我有两个问题:

此异常与被阻止的端口有关。正当 我应该在防火墙上为独立Apache Flink群集打开哪些端口? 更新1


我在主文件和从文件中发现了配置问题,我跳过了这些文件中描述的主机之间的新行分隔符。我修复了它,现在我看到了其他异常:

我有两个节点:

app-1.stag.local,运行作业和任务管理器 app-2.stag.local,运行任务管理器 从这些日志中可以看到,app-1.stag.local任务管理器无法连接到其他任务管理器:

java.io.IOException:连接通道失败:连接到远程任务管理器+“app-2.stag.local/192.168.38.98:35806”失败。这可能表示远程任务管理器已丢失

但是app-2.stag.local有开放端口:

2016-03-18 16:24:14347 INFO org.apache.flink.runtime.io.network.netty.NettyServer-在SocketAddress/192.168.38.98:35806上侦听时,成功初始化花费了39毫秒

所以,我认为问题与防火墙有关,但我不知道在Apache Flink中可以在何处配置此端口或端口范围。

我发现了一个问题:taskmanager.data.port参数默认设置为0,但说明应将其设置为6121


因此,我在flink-conf.yaml中设置了这个端口,现在一切正常。

我不太确定这是否是防火墙配置问题,因为JobManager和TaskManager已经相互通信了。能否将taskmanager.log文件共享到taskmanager 192.168.18.98:46369的版本?我发现主文件和从文件中存在配置问题。我修复了它,现在我看到了其他异常,我将更新这个问题。
jobmanager.rpc.address: app-1.stag.local
jobmanager.rpc.port: 6123
jobmanager.heap.mb: 1024

taskmanager.heap.mb: 2048
taskmanager.numberOfTaskSlots: 4
taskmanager.memory.preallocate: false

blob.server.port: 6130

parallelism.default: 4

jobmanager.web.port: 8081

state.backend: jobmanager

restart-strategy: none
restart-strategy.fixed-delay.attempts: 2
restart-strategy.fixed-delay.delay: 60s