Twitter 推特流错误
我正在使用spark streaming编写一个twitter连接器Twitter 推特流错误,twitter,apache-spark,streaming,twitter4j,spark-streaming,Twitter,Apache Spark,Streaming,Twitter4j,Spark Streaming,我正在使用spark streaming编写一个twitter连接器 我面临以下例外情况 错误ReceiverTracker:取消注册流0的接收器:正在重新启动 延迟2000ms的接收器:启动Twitter流时出错- java.lang.NullPointerException 位于org.apache.spark.streaming.twitter.TwitterReceiver.onStart(TwitterInputDStream.scala:89) 在org.apache.spark.s
我面临以下例外情况 错误ReceiverTracker:取消注册流0的接收器:正在重新启动 延迟2000ms的接收器:启动Twitter流时出错- java.lang.NullPointerException 位于org.apache.spark.streaming.twitter.TwitterReceiver.onStart(TwitterInputDStream.scala:89) 在org.apache.spark.streaming.receiver.ReceiverSupervisor.startReceiver(ReceiverSupervisor.scala:121) 在org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply$mcV$sp(ReceiverSupervisor.scala:159) 在org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:152) 在org.apache.spark.streaming.receiver.ReceiverSupervisor$$anonfun$restartReceiver$1.apply(ReceiverSupervisor.scala:152) 在scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24) 在scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24) 在scala.concurrent.impl.ExecutionContextImpl$$anon$3.exec(ExecutionContextImpl.scala:107) 位于scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) 位于scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) 位于scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) 在scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)中 下面是相关的代码片段
val config = new twitter4j.conf.ConfigurationBuilder()
.setOAuthConsumerKey("*********************")
.setOAuthConsumerSecret("**********************************************")
.setOAuthAccessToken("****************************************************")
.setOAuthAccessTokenSecret("**********************************************************")
.build
val twitter_auth = new TwitterFactory(config)
val a = new twitter4j.auth.OAuthAuthorization(config)
val atwitter : Option[twitter4j.auth.Authorization] = Some(twitter_auth.getInstance(a).getAuthorization())
val sparkConf = new SparkConf().setAppName("TwitterPopularTags").setMaster("local[*]")
val ssc = new StreamingContext(sparkConf, Seconds(2))
// ssc.checkpoint("D:/test")
val stream = TwitterUtils.createStream(ssc, atwitter, null, StorageLevel.MEMORY_AND_DISK_2)
val hashTags = stream.map(status => status.getUser().getName())
hashTags.foreachRDD(rdd => {
rdd.foreach(println)
})
ssc.start()
ssc.awaitTermination()
有人能帮我解决这个问题吗?谢谢:)转到引发异常的行,我们可以看到: 要使该行抛出NPE,过滤器必须为null,这正是TwitterStream实例化时发生的情况: val stream=TwitterUtils.createStream(ssc、atwitter、null、存储级别、内存和磁盘2) 作为一个序列,用
Seq()
而不是null
初始化它