Scala Spark streaming在运行一段时间后引发堆栈溢出异常

Spark streaming在运行一段时间后开始出现堆栈溢出异常。堆栈跟踪对解决问题没有多大帮助，这就是我在这里发布的原因。请帮忙，例外情况如下

15/05/14 14:02:04 ERROR Executor: Exception in task 1.0 in stage 134161.0 (TID 7008)
java.lang.StackOverflowError
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1508)
    at java.io.ObjectInputStream.readClass(ObjectInputStream.java:1480)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1330)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
    at scala.collection.immutable.$colon$colon.readObject(List.scala:362)
    at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1004)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1866)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
    at scala.collection.immutable.$colon$colon.readObject(List.scala:362)
    at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1004)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1866)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)

---

它可能是一些致命的检查点组合：一系列的

RDD

s同时被设置为检查点。如果是这种情况，我的理解是，实际的检查点写入被延迟，对象增长，最终填满堆栈

---

更多信息如下：。

该错误与spark streaming无关，它是一个序列化错误。请看一看这个问题，看看你是否面临类似的问题：你能为这个问题添加代码吗？伙计们，我能够通过缩短检查点的持续时间来解决这个问题。如果检查点持续时间太长，Spark将无法读取该持续时间内写入的所有对象。虽然此链接可能会回答问题，但最好在此处包含答案的基本部分，并提供链接供参考。如果链接页面更改，只有链接的答案可能会无效。假设你不是机器人，谢谢，我对这里有点陌生。