Apache spark 火花与纱线的资源配置
我正在使用Zeppelin 0.7.3和Spark 2.3的纱线客户端模式。 我的设置是: 火花:Apache spark 火花与纱线的资源配置,apache-spark,yarn,apache-zeppelin,Apache Spark,Yarn,Apache Zeppelin,我正在使用Zeppelin 0.7.3和Spark 2.3的纱线客户端模式。 我的设置是: 火花: spark.driver.memory 4096m spark.driver.memoryOverhead 3072m spark.executor.memory 4096m spark.executor.memoryOverhead 3072m spark.executor.cores
spark.driver.memory 4096m
spark.driver.memoryOverhead 3072m
spark.executor.memory 4096m
spark.executor.memoryOverhead 3072m
spark.executor.cores 3
spark.executor.instances 3
纱线:
分配的内存将减少22GB(22528MB)
Minimum allocation: memory:1024, vCores:2
Maximum allocation: memory:9216, vCores:6
The application started by Zeppelin gets the following resources:
Running Containers 4
Allocated CPU VCores 4
Allocated Memory MB 22528