Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/scala/19.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Postgresql 将地理定位Twitter4J写入Postgres_Postgresql_Scala_Geolocation_Twitter4j_Akka Stream - Fatal编程技术网

Postgresql 将地理定位Twitter4J写入Postgres

Postgresql 将地理定位Twitter4J写入Postgres,postgresql,scala,geolocation,twitter4j,akka-stream,Postgresql,Scala,Geolocation,Twitter4j,Akka Stream,我正在使用Twitter4J和Akka流提取推文。我选择了一些字段,比如userId、tweetId、tweet文本等等。此Tweet实体将写入数据库: class Counter extends StatusAdapter with Databases{ implicit val system = ActorSystem("TweetsExtractor") implicit val materializer = ActorMaterializer() implicit val e

我正在使用Twitter4J和Akka流提取推文。我选择了一些字段,比如userId、tweetId、tweet文本等等。此Tweet实体将写入数据库:

class Counter extends StatusAdapter with Databases{
  implicit val system = ActorSystem("TweetsExtractor")
  implicit val materializer = ActorMaterializer()
  implicit val executionContext = system.dispatcher
  implicit val LoggingAdapter =
    Logging(system, classOf[Counter])

  val overflowStrategy = OverflowStrategy.backpressure
  val bufferSize = 1000
  val statusSource = Source.queue[Status](
    bufferSize,
    overflowStrategy
  )

  val insertFlow: Flow[Status, Tweet, NotUsed] =
    Flow[Status].map(status => Tweet(status.getId, status.getUser.getId, status.getText, status.getLang,
      status.getFavoriteCount, status.getRetweetCount))
  val insertSink: Sink[Tweet, Future[Done]] = Sink.foreach(tweetRepository.create)
  val insertGraph = statusSource via insertFlow to insertSink
  val queueInsert = insertGraph.run()

  override def onStatus(status: Status) = 
    Await.result(queueInsert.offer(status), Duration.Inf)
}
Flow[Status].map(status => Tweet(status.getId, status.getUser.getId, status.getText, status.getLang, status.getFavoriteCount, status.getRetweetCount, status.getGeoLocation.getLatitude, status.getGeoLocation.getLongitude))
我的意图是添加位置字段。Twitter4J中有一个特定的地理定位类型,它包含双类型的纬度和经度。但是,当我尝试通过流直接提取纬度和经度时,不会向数据库写入任何内容:

class Counter extends StatusAdapter with Databases{
  implicit val system = ActorSystem("TweetsExtractor")
  implicit val materializer = ActorMaterializer()
  implicit val executionContext = system.dispatcher
  implicit val LoggingAdapter =
    Logging(system, classOf[Counter])

  val overflowStrategy = OverflowStrategy.backpressure
  val bufferSize = 1000
  val statusSource = Source.queue[Status](
    bufferSize,
    overflowStrategy
  )

  val insertFlow: Flow[Status, Tweet, NotUsed] =
    Flow[Status].map(status => Tweet(status.getId, status.getUser.getId, status.getText, status.getLang,
      status.getFavoriteCount, status.getRetweetCount))
  val insertSink: Sink[Tweet, Future[Done]] = Sink.foreach(tweetRepository.create)
  val insertGraph = statusSource via insertFlow to insertSink
  val queueInsert = insertGraph.run()

  override def onStatus(status: Status) = 
    Await.result(queueInsert.offer(status), Duration.Inf)
}
Flow[Status].map(status => Tweet(status.getId, status.getUser.getId, status.getText, status.getLang, status.getFavoriteCount, status.getRetweetCount, status.getGeoLocation.getLatitude, status.getGeoLocation.getLongitude))

这种行为的原因可能是什么?我该如何解决它?

正如对问题的评论所确认的,这里发生的事情是,大多数推特没有附带地理位置数据,使这些字段为空并导致错误行为


几个简单的空值检查应该可以解决这个问题。

可能是个愚蠢的问题:您是否检查了
GeoLocation
字段对于您想要保存的tweet是否有一些值?这不是强制性的,它没有任何价值,这是有道理的。@stefanobaghino这不是一个愚蠢的问题,你实际上帮助我找到了问题的解决方案。据我所知,没有多少tweet是用位置发布的,我会实时获取这些tweet,所以我需要等待很长一段时间才能收到这些tweet,或者选择一些其他参数,比如用户位置,如果我需要的话。非常感谢你的帮助!很好,我将此作为答复发布,如果您能将其标记为已接受,我将不胜感激。:)