Scala 为什么Spark作业会失败而不抛出运行时异常？

我遇到了一个问题，在这个问题中，我的作业在调用类的特定阶段失败

这是电话线

val stockDataFilteredRDD: RDD[stockPriceInfo] =
  lineMapToStockPriceInfoObjectRDD
  .map(new stockDataFilter(_).requirementsMet.get)

这是我看到的

15/10/0916:02:28INFOClient:ApplicationreportfromResourceManager:
applicationidentifier:application_1438798768056_0254
appId:254
clientToAMToken:null
appDiagnostics:
appMasterHost:ip-10-0-142-138.ec2.internal
appQueue:root.add_twitter_user
appMasterRpcPort:0
appStartTime:1444421062565
yarnAppState:RUNNING
distributedFinalState:UNDEFINED
appTrackingUrl:http://myIP.ip

15/10/0916:02:29INFOClient:ApplicationreportfromResourceManager:
applicationidentifier:application_1438798768056_0254
appId:254
clientToAMToken:null
appDiagnostics:
appMasterHost:ip-10-0-142-138.ec2.internal
appQueue:root.add_twitter_user
appMasterRpcPort:0
appStartTime:1444421062565
yarnAppState:FINISHED
distributedFinalState:FAILED
appTrackingUrl:http://myIP.ip
appUser:add_twitter_user

调用的类

class stockDataFilter(val s:stockPriceInfo){

val dateDelim="-"
val timeDelim=":"
val dateAndTime=s.dateTime
val splitDateTime=dateAndTime.split("@")
val dateStamp=splitDateTime(0)
val time=splitDateTime(1)
val splitDate=dateStamp.split(dateDelim)
val splitTime=time.split(timeDelim)
val year=splitDate(0);
val month=splitDate(1);
val day=splitDate(2)
val hour=splitTime(0);
val minute=splitTime(1);
val second=splitTime(2)

val openingBell:LocalTime=newLocalTime(9,30)
val closingBell:LocalTime=newLocalTime(16,0)
val currentTime:LocalTime=newLocalTime(hour.toInt,minute.toInt)

//NonTradingSessions
val newYearsDay:Date=newDate(year.toInt-1900,0,1)
val weekends:List[String]=List("Saturday","Sunday")

val date=new Date(year.toInt-1900,month.toInt-1,day.toInt)
val currentDate=new LocalDate(year.toInt-1990,month.toInt-1,day.toInt)

//NewYorkStockExchangeoperatesfrom9:30a.m.to4:00p.m
def isWithinTradingSession:Boolean={
    val isAfterOpen:Boolean=currentTime.isAfter(openingBell)
    val isBeforeClose:Boolean=currentTime.isBefore(closingBell)
    isAfterOpen&&isBeforeClose
}//return Trueifitiswithingtradingtime

def requirementsMet:Option[stockPriceInfo]=isWithinTradingSessionmatch{
   case true=>Some(s)
   case false=>None
 }
}

在此之前，我可以显示（存储在HDFS中）任何内容，但一旦添加这一行，它就会失败。我看过日志，没有明显的问题。也没有编译时或运行时异常。这件事我已经坚持了好几天了，没办法了。你的帮助将不胜感激。。问候

日志：

LogType: stderr
LogLength: 6638
Log Contents:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.2.1-1.cdh5.2.1.p0.12/jars/spark-assembly-1.1.0-cdh5.2.1-hadoop2.5.0-cdh5.2.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.2.1-1.cdh5.2.1.p0.12/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data01/yarn/nm/usercache/add_twitter_user/filecache/62/twitteryahoofinanceanalytics.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
15/10/10 11:27:24 INFO executor.CoarseGrainedExecutorBackend: Registered signal handlers for [TERM, HUP, INT]
15/10/10 11:27:25 INFO spark.SecurityManager: Changing view acls to: yarn,add_twitter_user
15/10/10 11:27:25 INFO spark.SecurityManager: Changing modify acls to: yarn,add_twitter_user
15/10/10 11:27:25 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, add_twitter_user); users with modify permissions: Set(yarn, add_twitter_user)
15/10/10 11:27:25 INFO slf4j.Slf4jLogger: Slf4jLogger started
15/10/10 11:27:25 INFO Remoting: Starting remoting
15/10/10 11:27:26 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://driverPropsFetcher@IP]
15/10/10 11:27:26 INFO Remoting: Remoting now listens on addresses: [akka.tcp://driverPropsFetcher@IP]
15/10/10 11:27:26 INFO util.Utils: Successfully started service 'driverPropsFetcher' on port port.
15/10/10 11:27:26 INFO spark.SecurityManager: Changing view acls to: yarn,add_twitter_user
15/10/10 11:27:26 INFO spark.SecurityManager: Changing modify acls to: yarn,add_twitter_user
15/10/10 11:27:26 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, add_twitter_user); users with modify permissions: Set(yarn, add_twitter_user)
15/10/10 11:27:26 INFO remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
15/10/10 11:27:26 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
15/10/10 11:27:26 INFO slf4j.Slf4jLogger: Slf4jLogger started
15/10/10 11:27:26 INFO Remoting: Starting remoting
15/10/10 11:27:26 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkExecutor@myIP]
15/10/10 11:27:26 INFO Remoting: Remoting now listens on addresses: [akka.tcp://sparkExecutor@myIP]
15/10/10 11:27:26 INFO util.Utils: Successfully started service 'sparkExecutor' on port 54841.
15/10/10 11:27:26 INFO Remoting: Remoting shut down
15/10/10 11:27:26 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
15/10/10 11:27:26 INFO executor.CoarseGrainedExecutorBackend: Connecting to driver: akka.tcp://sparkDriver@IP:port/user/CoarseGrainedScheduler
15/10/10 11:27:26 INFO executor.CoarseGrainedExecutorBackend: Successfully registered with driver
15/10/10 11:27:26 INFO spark.SecurityManager: Changing view acls to: yarn,add_twitter_user
15/10/10 11:27:26 INFO spark.SecurityManager: Changing modify acls to: yarn,add_twitter_user
15/10/10 11:27:26 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(yarn, add_twitter_user); users with modify permissions: Set(yarn, add_twitter_user)
15/10/10 11:27:26 INFO slf4j.Slf4jLogger: Slf4jLogger started
15/10/10 11:27:26 INFO Remoting: Starting remoting
15/10/10 11:27:26 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkExecutor@myIP]
15/10/10 11:27:26 INFO Remoting: Remoting now listens on addresses: [akka.tcp://sparkExecutor@myIP]
15/10/10 11:27:26 INFO util.Utils: Successfully started service 'sparkExecutor' on port 36011.
15/10/10 11:27:26 INFO util.AkkaUtils: Connecting to MapOutputTracker: akka.tcp://sparkDriver@IP:PORT/user/MapOutputTracker
15/10/10 11:27:26 INFO util.AkkaUtils: Connecting to BlockManagerMaster: akka.tcp://sparkDriver@IP/user/BlockManagerMaster
15/10/10 11:27:26 INFO storage.DiskBlockManager: Created local directory at /data01/yarn/nm/usercache/add_twitter_user/appcache/application_1438798768056_0263/spark-local-20151010112726-839d
15/10/10 11:27:26 INFO storage.DiskBlockManager: Created local directory at /data02/yarn/nm/usercache/add_twitter_user/appcache/application_1438798768056_0263/spark-local-20151010112726-4371
15/10/10 11:27:26 INFO util.Utils: Successfully started service 'Connection manager for block manager' on port port.
15/10/10 11:27:26 INFO network.ConnectionManager: Bound socket to port port with id = ConnectionManagerId(myIP,port)
15/10/10 11:27:26 INFO storage.MemoryStore: MemoryStore started with capacity 530.3 MB
15/10/10 11:27:26 INFO storage.BlockManagerMaster: Trying to register BlockManager
15/10/10 11:27:26 INFO storage.BlockManagerMaster: Registered BlockManager
15/10/10 11:27:26 INFO util.AkkaUtils: Connecting to HeartbeatReceiver: akka.tcp://sparkDriver@IP/user/HeartbeatReceiver
15/10/10 11:27:27 INFO executor.CoarseGrainedExecutorBackend: Driver commanded a shutdown
15/10/10 11:27:27 INFO network.ConnectionManager: Selector thread was interrupted!
15/10/10 11:27:27 INFO network.ConnectionManager: ConnectionManager stopped
15/10/10 11:27:27 INFO storage.MemoryStore: MemoryStore cleared
15/10/10 11:27:27 INFO storage.BlockManager: BlockManager stopped
15/10/10 11:27:27 INFO remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
15/10/10 11:27:27 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
15/10/10 11:27:27 INFO remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
15/10/10 11:27:27 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
15/10/10 11:27:27 INFO Remoting: Remoting shut down
15/10/10 11:27:27 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
15/10/10 11:27:27 INFO Remoting: Remoting shut down
15/10/10 11:27:27 INFO remote.RemoteActorRefProvider$RemotingTerminator: Remoting shut down.

LogType: stdout
LogLength: 0
Log Contents:

---

这似乎只是驱动程序的日志，对于调试来说几乎毫无用处。我假设您使用的是纱线客户端模式。尝试使用

纱线日志--aplicationId

Hello@DanielL从纱线获取完整日志。我已经看过那些日志了。那里的一切看起来都很好。我想我会把它们寄出去的。。我错过了一些东西。你不应该得到每个节点的STDRR和STDUT的一个PAR值吗？抱歉，不知道还有什么可以帮助您：-哪些是

部分。获取
？据我所知，这与返回类型不匹配。。。那么它真的编译了吗？谢谢铍。那是我的错误。。我只是想看看这个选择是否是工作失败的原因。但如果没有它，它就可以编译，我可以用sbt构建。。