Apache spark 停留在；洗牌主机丢失的文件“；在多个拼花文件和连接的EMR中

我正在尝试对多个数据帧执行查询，每个帧由大约4个拼花组成，其中一个由大约1800个拼花文件组成

EMR实例配置为自动缩放。 当我尝试运行一个包含3个以上联接的查询时，执行会受阻

我尽了一切可能，增加了超时时间，实现了洗牌、动态分配和广播。以下是spark配置：

spark.network.timeout=4800
spark.executor.heartbeatInterval=4200
spark.sql.broadcastTimeout=3600
spark.sql.autoBroadcastJoinThreshold=209715200
spark.shuffle.service.enabled=true
spark.dynamicAllocation.enabled=true

这是我在最后得到的输出日志，没有进一步的错误/异常

.Logging$class.logInfo(Logging.scala:54)ogger{39} : Shuffle files lost for host: ip-152-20-116-20.eu-central-2.compute.internal (epoch 7)
.Logging$class.logInfo(Logging.scala:54)ogger{39} : Shuffle files lost for host: ip-172-23-216-128.eu-central-1.compute.internal (epoch 8)
.Logging$class.logInfo(Logging.scala:54)ogger{39} : Shuffle files lost for host: ip-172-23-219-85.eu-central-1.compute.internal (epoch 9)
.Logging$class.logInfo(Logging.scala:54)ogger{39} : Shuffle files lost for host: ip-172-23-218-123.eu-central-1.compute.internal (epoch 4)
.Logging$class.logInfo(Logging.scala:54)ogger{39} : Shuffle files lost for host: ip-172-23-216-84.eu-central-1.compute.internal (epoch 5)
.Logging$class.logInfo(Logging.scala:54)ogger{39} : Shuffle files lost for host: ip-172-23-218-159.eu-central-1.compute.internal (epoch 6)
.Logging$class.logInfo(Logging.scala:54)ogger{39} : Shuffle files lost for host: ip-172-23-219-86.eu-central-1.compute.internal (epoch 7)