Scala 用Akka流插入卡桑德拉
我正在学习Akka Streams,作为练习,我想将日志插入Cassandra。问题是我无法使流将日志插入数据库 我天真地尝试了以下方法:Scala 用Akka流插入卡桑德拉,scala,cassandra,akka-stream,Scala,Cassandra,Akka Stream,我正在学习Akka Streams,作为练习,我想将日志插入Cassandra。问题是我无法使流将日志插入数据库 我天真地尝试了以下方法: object Application extends AkkaApp with LogApacheDao { // The log file is read line by line val source: Source[String, Unit] = Source.fromIterator(() => scala.io.Source.fro
object Application extends AkkaApp with LogApacheDao {
// The log file is read line by line
val source: Source[String, Unit] = Source.fromIterator(() => scala.io.Source.fromFile(filename).getLines())
// Each line is converted to an ApacheLog object
val flow: Flow[String, ApacheLog, Unit] = Flow[String]
.map(rawLine => {
rawLine.split(",") // implicit conversion Array[String] -> ApacheLog
})
// Log objects are inserted to Cassandra
val sink: Sink[ApacheLog, Future[Unit]] = Sink.foreach[ApacheLog] { log => saveLog(log) }
source.via(flow).to(sink).run()
}
saveLog()在LogApacheDao中的定义如下(为了代码更清晰,我省略了列值):
当进入接收器时,从Array[String]到ApacheLog的转换不会出现问题(通过println验证)。此外,键空间和表都是创建的,但是当执行到saveLog时,似乎有什么东西被阻塞了,没有插入
我没有收到任何错误,但Cassandra driver core(3.0.0)一直给我:
Connection[/172.17.0.2:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
Connection[/172.17.0.2:9042-2, inFlight=0, closed=false] heartbeat query succeeded
我应该补充一点,我使用的是Docked Cassandra。尝试使用。我只读过关于akka流的文章,所以这只是一个猜测,你确定
saveLog
没有抛出一个被吞没的异常吗?我会尝试/捕捉它,并在例外情况上打印一个标记,以确保安全。这些cassandra日志显示连接在30秒内处于非活动状态,之后会发送心跳信号以保持其打开。可能这与此相关,我将尝试从saveLog捕获最终异常,感谢您的建议即使在处理异常时(使用onComplete(…),我也无法获得任何错误消息。在我看来,执行似乎无法执行bind()
方法。
Connection[/172.17.0.2:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
Connection[/172.17.0.2:9042-2, inFlight=0, closed=false] heartbeat query succeeded