Scala 迭代不同类型的集合并写入拼花地板
我有很多桌子,我需要以拼花地板的形式保存它们。我试着使用这个库,结果却出了问题Scala 迭代不同类型的集合并写入拼花地板,scala,akka,parquet,alpakka,Scala,Akka,Parquet,Alpakka,我有很多桌子,我需要以拼花地板的形式保存它们。我试着使用这个库,结果却出了问题 [error] found : akka.stream.scaladsl.Sink[shapeless.HNil,scala.concurrent.Future[akka.Done]] [error] required: akka.stream.Graph[akka.stream.SinkShape[Product with Serializable],?] 是否可以使用表对列表进行迭代 import co
[error] found : akka.stream.scaladsl.Sink[shapeless.HNil,scala.concurrent.Future[akka.Done]]
[error] required: akka.stream.Graph[akka.stream.SinkShape[Product with Serializable],?]
是否可以使用表对列表进行迭代
import com.github.mjakubowski84.parquet4s.{ParquetStreams, ParquetWriter}
import akka.actor.ActorSystem
import akka.stream.{ActorMaterializer, Materializer}
import akka.stream.scaladsl.Source
import scala.concurrent.Await
import scala.concurrent.duration._
object Main extends App {
case class Table1(Id: Int, name: String)
case class Table2(Id: Int, age: Option[Int])
val table1_entities = List(Table1(1, "Foo"))
val table2_entities = List(Table2(2, Some(9999)))
//assume here is a lot of tables
val all_tables = List(
("file:///tmp/table1.parquet", table1_entities),
("file:///tmp/table2.parquet", table2_entities)
)
implicit val system: ActorSystem = ActorSystem()
implicit val materializer: Materializer = ActorMaterializer()
//for single tables it works perfectly
//val future = Source(table1_entities).runWith(ParquetStreams.toParquetSingleFile(path = "file:///tmp/table1.parquet"))
//Await.result(future, 2.second)
all_tables.foreach{
case (path, table) => {
val future = Source(table).runWith(ParquetStreams.toParquetSingleFile(path = path))
Await.result(future, 2.second)
}
}
}```