Apache spark 映射输出状态为N个字节，超过spark.akka.frameSize-spark中出现错误

我在一个简单的spark工作中遇到了这个错误。我不想在调查原因之前增加spark.akka.frameSize

火花减速器制动操作错误-

16/07/04 19:50:43 ERROR MapOutputTrackerMasterEndpoint: Map output statuses were 209098002 bytes which exceeds spark.akka.frameSize (104857600 bytes).
16/07/04 19:50:46 ERROR MapOutputTrackerMasterEndpoint: Map output statuses were 209098002 bytes which exceeds spark.akka.frameSize (104857600 bytes).
16/07/04 19:50:49 ERROR MapOutputTrackerMasterEndpoint: Map output statuses were 209098002 bytes which exceeds spark.akka.frameSize (104857600 bytes).
16/07/04 19:50:49 ERROR MapOutputTrackerMasterEndpoint: Map output statuses were 209098002 bytes which exceeds spark.akka.frameSize (104857600 bytes).

代码-

val events = inputFilesRdd.map {
      // Emit : (id, user, product) -> date
      splits => (splits(11), splits(4), splits(2)) -> splits(1)
    }.reduceByKey((left, right) => minString(left, right))

minString是一个比较字符串的简单函数。预期的数据大小是更高的GIG

输入文件被gzip压缩

---

有关此错误的任何提示？

您是否能够解决此问题？无法调查原因。增加akka帧大小可以解决此问题。您是否能够解决此问题？无法调查原因。增加akka帧大小可以修复此问题。