Akka Streams:如何将未来[Seq[T]]转换为源[T,未使用]
我有一个方法:Akka Streams:如何将未来[Seq[T]]转换为源[T,未使用],akka,akka-stream,Akka,Akka Stream,我有一个方法:def-Sighting(from:YearMonth):Future[Seq[Sighting]] 另一个:def目击(from:YearMonth,to:YearMonth):Source[signing,NotUsed] 我想为每个YearMonth调用第一个,从from开始,到结束,然后合并/合并结果。我似乎无法在Source上找到合适的方法来实现这一点。我现在看到的是: val months = from.until(to, ChronoUnit.MONTHS) + 1
def-Sighting(from:YearMonth):Future[Seq[Sighting]]
另一个:def目击(from:YearMonth,to:YearMonth):Source[signing,NotUsed]
我想为每个YearMonth
调用第一个,从from
开始,到结束,然后合并/合并结果。我似乎无法在Source
上找到合适的方法来实现这一点。我现在看到的是:
val months = from.until(to, ChronoUnit.MONTHS) + 1
Source.fromIterator(() => Iterator.range(0, months.toInt))
.map(from.plusMonths(_))
.mapAsyncUnordered(1)(sightings)
trait Client {
def sightings(yearMonth: YearMonth): Source[(HttpResponse, YearMonth), NotUsed]
}
trait HttpClient extends Client {
implicit def system: ActorSystem
private val yearMonthFormatter = DateTimeFormatter.ofPattern("yyyyMM")
override def sightings(yearMonth: YearMonth): Source[(HttpResponse, YearMonth), NotUsed] = {
Source.fromGraph(GraphDSL.create() { implicit b: GraphDSL.Builder[NotUsed] =>
import GraphDSL.Implicits._
// prepare graph elements
val uri = "whatever.html"
val src = Source.single(RequestBuilding.Get(uri))
lazy val conn = Http().outgoingConnection("www.doesnotexist.com")
.map((_, yearMonth))
val flow = b.add(conn)
// connect the graph
src ~> flow
// expose port
SourceShape(flow.out)
})
}
}
trait Crawler {
self: Client =>
implicit def executionContext: ExecutionContext
implicit def materializer: Materializer
final def sightings(from: YearMonth, to: YearMonth): Source[Sighting, NotUsed] = {
val months = from.until(to, ChronoUnit.MONTHS) + 1
Source.fromIterator(() => Iterator.range(0, months.toInt))
.map(x => from.plusMonths(x.toLong))
.flatMapConcat(self.sightings)
.mapAsyncUnordered(1)(t => {
val (response, yearMonth) = (t._1, t._2)
val body = Unmarshal(response.entity).to[String]
val status = response.status
responseMapper(body, yearMonth)(status)
})
.mapConcat(_.to[collection.immutable.Seq])
}
import scala.collection.JavaConversions._
private def responseMapper(body: Future[String], yearMonth: YearMonth):
PartialFunction[StatusCode, Future[Seq[Sighting]]] = {...}
}
这会产生一个源[Int,NotUsed]
,而不是我正在寻找的源[Sighting,NotUsed]
编辑:
我的结论如下:
val months = from.until(to, ChronoUnit.MONTHS) + 1
Source.fromIterator(() => Iterator.range(0, months.toInt))
.map(from.plusMonths(_))
.mapAsyncUnordered(1)(sightings)
trait Client {
def sightings(yearMonth: YearMonth): Source[(HttpResponse, YearMonth), NotUsed]
}
trait HttpClient extends Client {
implicit def system: ActorSystem
private val yearMonthFormatter = DateTimeFormatter.ofPattern("yyyyMM")
override def sightings(yearMonth: YearMonth): Source[(HttpResponse, YearMonth), NotUsed] = {
Source.fromGraph(GraphDSL.create() { implicit b: GraphDSL.Builder[NotUsed] =>
import GraphDSL.Implicits._
// prepare graph elements
val uri = "whatever.html"
val src = Source.single(RequestBuilding.Get(uri))
lazy val conn = Http().outgoingConnection("www.doesnotexist.com")
.map((_, yearMonth))
val flow = b.add(conn)
// connect the graph
src ~> flow
// expose port
SourceShape(flow.out)
})
}
}
trait Crawler {
self: Client =>
implicit def executionContext: ExecutionContext
implicit def materializer: Materializer
final def sightings(from: YearMonth, to: YearMonth): Source[Sighting, NotUsed] = {
val months = from.until(to, ChronoUnit.MONTHS) + 1
Source.fromIterator(() => Iterator.range(0, months.toInt))
.map(x => from.plusMonths(x.toLong))
.flatMapConcat(self.sightings)
.mapAsyncUnordered(1)(t => {
val (response, yearMonth) = (t._1, t._2)
val body = Unmarshal(response.entity).to[String]
val status = response.status
responseMapper(body, yearMonth)(status)
})
.mapConcat(_.to[collection.immutable.Seq])
}
import scala.collection.JavaConversions._
private def responseMapper(body: Future[String], yearMonth: YearMonth):
PartialFunction[StatusCode, Future[Seq[Sighting]]] = {...}
}
根据第一个Sighting
函数(def Sighting(from:YearMonth):Future[Seq[Sighting]]
)的类型签名,听起来您想要一个具有以下类型的函数:
def fToS[A, B](f: A => Future[Seq[T]]): Flow[A, B, NotUsed]
然后您可以这样做:
val yearMonths: Source[YearMonth, NotUsed] = ??? // whatever you want
val toSightings: Flow[YearMonth, Sighting, NotUsed] = fToS(sightings)
val source: Source[Sighting, NotUsed] = yearMonths.via(toSightings)
FTO
看起来像这样:
def fToS[A, B](f: A => Future[Seq[T]]): Flow[A, B, NotUsed] =
Flow[A].mapAsync(1)(f).mapConcat(identity)
这应该行。根据第一个观测
函数的类型签名(def sightings(from:YearMonth):Future[Seq[Sighting]]
)听起来您需要具有此类型的函数:
def fToS[A, B](f: A => Future[Seq[T]]): Flow[A, B, NotUsed]
然后您可以这样做:
val yearMonths: Source[YearMonth, NotUsed] = ??? // whatever you want
val toSightings: Flow[YearMonth, Sighting, NotUsed] = fToS(sightings)
val source: Source[Sighting, NotUsed] = yearMonths.via(toSightings)
FTO
看起来像这样:
def fToS[A, B](f: A => Future[Seq[T]]): Flow[A, B, NotUsed] =
Flow[A].mapAsync(1)(f).mapConcat(identity)
这应该行。你也可以使用Source.fromFuture(fut).mapcont(identity)
。你也可以使用Source.fromFuture(fut).mapcont(identity)
。