Multithreading 无功并行化不'；行不通_Multithreading_Concurrency_Rx Java_Reactive Programming_Project Reactor

Multithreading 无功并行化不'；行不通

multithreading concurrency rx-java

Multithreading 无功并行化不'；行不通,multithreading,concurrency,rx-java,reactive-programming,project-reactor,Multithreading,Concurrency,Rx Java,Reactive Programming,Project Reactor,使用项目反应堆3.0.4.1版本。从概念上讲，在RxJava中也应该是相同的 public Mono<Map<String, Boolean>> refreshPods(List<String> apps) { return pods(apps) .filter(this::isRunningAndNotThisApp) .groupBy(Item::getName) .flatM

使用项目反应堆3.0.4.1版本。从概念上讲，在RxJava中也应该是相同的

public Mono<Map<String, Boolean>> refreshPods(List<String> apps) {
    return pods(apps)
            .filter(this::isRunningAndNotThisApp)
            .groupBy(Item::getName)
            .flatMap(g -> g
                    .distinct(Item::getIp)
                    .collectList()
                    // TODO: This doesn't seem to be working as expected
                    .subscribeOn(Schedulers.newParallel("par-grp"))
                    .flatMap(client::refreshPods))
            .flatMap(m -> Flux.fromIterable(m.entrySet()))
            .collectMap(Map.Entry::getKey, Map.Entry::getValue);
}

问题在于，尝试刷新pod的log语句

打印在每个组的同一线程上。我错过了什么
来自测试运行的日志：
2017-02-07 10:g12:55.348  INFO 33905 --- [        timer-1] c.n.d.cloud.config.MyServiceClientImpl  : Attempting to refresh pod: Item(name=news, ip=127.0.0.1, port=8888, podPhase=Running).
2017-02-07 10:12:55.357  INFO 33905 --- [        timer-1] c.n.d.cloud.config.MyServiceClientImpl  : Successfully refreshed pod: Item(name=news, ip=127.0.0.1, port=8888, podPhase=Running).
2017-02-07 10:12:55.358  INFO 33905 --- [        timer-1] c.n.d.cloud.config.MyServiceClientImpl  : Attempting to refresh pod: Item(name=parking, ip=127.0.0.1, port=8888, podPhase=Running).
2017-02-07 10:12:55.363  INFO 33905 --- [        timer-1] c.n.d.cloud.config.MyServiceClientImpl  : Successfully refreshed pod: Item(name=parking, ip=127.0.0.1, port=8888, podPhase=Running).
2017-02-07 10:12:55.364  INFO 33905 --- [        timer-1] c.n.d.cloud.config.MyServiceClientImpl  : Attempting to refresh pod: Item(name=localsearch, ip=127.0.0.1, port=8888, podPhase=Running).
2017-02-07 10:12:55.368  INFO 33905 --- [        timer-1] c.n.d.cloud.config.MyServiceClientImpl  : Successfully refreshed pod: Item(name=localsearch, ip=127.0.0.1, port=8888, podPhase=Running).
2017-02-07 10:12:55.369  INFO 33905 --- [        timer-1] c.n.d.cloud.config.MyServiceClientImpl  : Attempting to refresh pod: Item(name=auth, ip=127.0.0.1, port=8888, podPhase=Running).
2017-02-07 10:12:55.372  INFO 33905 --- [        timer-1] c.n.d.cloud.config.MyServiceClientImpl  : Successfully refreshed pod: Item(name=auth, ip=127.0.0.1, port=8888, podPhase=Running).
2017-02-07 10:12:55.373  INFO 33905 --- [        timer-1] c.n.d.cloud.config.MyServiceClientImpl  : Attempting to refresh pod: Item(name=log, ip=127.0.0.1, port=8888, podPhase=Running).
2017-02-07 10:12:55.377  INFO 33905 --- [        timer-1] c.n.d.cloud.config.MyServiceClientImpl  : Successfully refreshed pod: Item(name=log, ip=127.0.0.1, port=8888, podPhase=Running).
2017-02-07 10:12:55.378  INFO 33905 --- [        timer-1] c.n.d.cloud.config.MyServiceClientImpl  : Attempting to refresh pod: Item(name=fuel, ip=127.0.0.1, port=8888, podPhase=Running).
2017-02-07 10:12:55.381  INFO 33905 --- [        timer-1] c.n.d.cloud.config.MyServiceClientImpl  : Successfully refreshed pod: Item(name=fuel, ip=127.0.0.1, port=8888, podPhase=Running).

zipWith(Flux.interval(Duration.ofSeconds(groupMemberDelaySeconds)),
    (x, delay) -> x)
.publishOn(Schedulers.newParallel("par-grp"))
.flatMap(this:: refreshWithRetry)

zipWith(Flux.intervalMillis(1000 * groupMemberDelaySeconds, Schedulers.newTimer("par-grp")),
    (x, delay) -> x)
.flatMap(this:: refreshWithRetry)

编辑：由于您新提供的日志，以及David在您创建的问题中提到的，根本原因是您在此处使用了间隔
。这将把上下文切换到默认的TimedScheduler
（对于所有组都是相同的）。这就是为什么在调用refreshPods
之前所做的任何事情似乎都会被忽略（工作在interval线程上完成），但在interval操作符之后的publishOn/subscribeOn应该会起作用。简而言之我的建议是在create
之后直接使用subscribeOn

触发一个阻塞行为（refresh（pod）
），在refreshWithRetry
中将其包装为Mono

除非您非常需要在这个级别上不考虑并发性，否则我建议您立即将subscribeOn
链接到create
旁边
这样，在使用Mono时就不足为奇了：它尊重合同，不会阻塞。像这样：
return Mono.<Boolean>create(emitter -> {
        //...
    })
.subscribeOn(Schedulers.newParallel("par-grp"))
.map(b -> Tuples.of(pod.getIp(), b))

在代码中，您将publishOn
放在flatMap
之前。当使用异步源时，方法结合观察对象，如flatMap
或zip
进行自己的重新调度<代码>间隔

在您的案例中就是这样一个异步源。这就是为什么在“计时器”线程上得到所有结果
1）在您希望并行执行的操作之前使用
publishOn
。操作本身不应涉及重新调度。例如，
map
是好的，
flatMap
是坏的
2）使用紧跟其后的另一个
publishOn
重新安排结果。否则订阅服务器的线程可能会干扰

Flux.range(1, 100) .groupBy(i -> i % 5) .flatMap(group -> group .publishOn(Schedulers.newParallel("grp", 8)) .map(v -> { // processing here String threadName = Thread.currentThread().getName(); logger.info("processing {} from {} on {}", v, group.key(), threadName); return v; }) .publishOn(Schedulers.single()) ) .subscribe(v -> logger.info("got {}", v));

如果您想确保所有组的项目都在同一个线程上运行，请参见以下答案：
为了完整性，我自己发布了一个答案。在@simon baslé和@akarnokd的帮助下，我做对了。以下两项工作。有关详细信息，请参阅
解决方案1：

2017-02-07 10:g12:55.348 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Attempting to refresh pod: Item(name=news, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.357 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Successfully refreshed pod: Item(name=news, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.358 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Attempting to refresh pod: Item(name=parking, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.363 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Successfully refreshed pod: Item(name=parking, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.364 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Attempting to refresh pod: Item(name=localsearch, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.368 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Successfully refreshed pod: Item(name=localsearch, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.369 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Attempting to refresh pod: Item(name=auth, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.372 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Successfully refreshed pod: Item(name=auth, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.373 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Attempting to refresh pod: Item(name=log, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.377 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Successfully refreshed pod: Item(name=log, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.378 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Attempting to refresh pod: Item(name=fuel, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.381 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Successfully refreshed pod: Item(name=fuel, ip=127.0.0.1, port=8888, podPhase=Running).

zipWith(Flux.interval(Duration.ofSeconds(groupMemberDelaySeconds)), (x, delay) -> x) .publishOn(Schedulers.newParallel("par-grp")) .flatMap(this:: refreshWithRetry)

zipWith(Flux.intervalMillis(1000 * groupMemberDelaySeconds, Schedulers.newTimer("par-grp")), (x, delay) -> x) .flatMap(this:: refreshWithRetry)
解决方案2：

2017-02-07 10:g12:55.348 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Attempting to refresh pod: Item(name=news, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.357 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Successfully refreshed pod: Item(name=news, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.358 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Attempting to refresh pod: Item(name=parking, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.363 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Successfully refreshed pod: Item(name=parking, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.364 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Attempting to refresh pod: Item(name=localsearch, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.368 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Successfully refreshed pod: Item(name=localsearch, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.369 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Attempting to refresh pod: Item(name=auth, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.372 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Successfully refreshed pod: Item(name=auth, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.373 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Attempting to refresh pod: Item(name=log, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.377 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Successfully refreshed pod: Item(name=log, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.378 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Attempting to refresh pod: Item(name=fuel, ip=127.0.0.1, port=8888, podPhase=Running). 2017-02-07 10:12:55.381 INFO 33905 --- [ timer-1] c.n.d.cloud.config.MyServiceClientImpl : Successfully refreshed pod: Item(name=fuel, ip=127.0.0.1, port=8888, podPhase=Running).

zipWith(Flux.interval(Duration.ofSeconds(groupMemberDelaySeconds)), (x, delay) -> x) .publishOn(Schedulers.newParallel("par-grp")) .flatMap(this:: refreshWithRetry)

zipWith(Flux.intervalMillis(1000 * groupMemberDelaySeconds, Schedulers.newTimer("par-grp")), (x, delay) -> x) .flatMap(this:: refreshWithRetry)

refreshPods
方法中不需要
subscribeOn
或
publishOn
。
你有一个
日志，你能分享它的输出吗？您是否可以确认可以有多个项目具有相同的名称（即分组确实会产生包含多个元素的组）？您是否尝试过publishOn 而不是subscribeOn ，因为您只对在单独的线程中执行下游操作感兴趣？您需要在TODO行之后执行publishOn ，而不是subscribeOn @akarnokd，这也是我的理解，因为我想在单独的线程中发出消息，但不起作用。请参阅编辑。有一个错误导致flatMap（->observeOn（s）.X）无法在“s”上正常运行。我不确定哪个反应堆版本包含修复，但为了快速解决问题，flatMap中的最后一个操作符应该是hide（）：flatMap（->observeOn（）.X（）.hide（）） @akarnokd我想你的意思是publishOn ，因为反应堆中没有observeOn 。但这并不奏效：（您的第二个解决方案不起作用，请参见编辑。如果我没有错，您的第一个解决方案将在一个单独的线程上运行组中的每个项目，对吗？这与在单独的线程上运行组中的每个项目不同，但每个项目都在同一个线程上。我不确定第二个publishOn 是否有意义，并且它无论如何都不起作用。@AbhijitSarkar请尝试我的示例。）充足-我已经在3.0.4上测试了它，它可以工作-然后逐渐在上面构建代码。尝试使用和不使用secondpublishOn 并查看。实际上，我建议您在ticket中尝试这个示例。您的示例的问题是我称之为“hello world”症候群；它只处理问题的一部分，有时一部分不适合更大的难题。只是为了好玩，我在罚单中运行了示例的代码，就像我之前所说的，它对我的用例不起作用。你自己试试，让我知道。但不了解“hello world”是怎么回事有效，你几乎无法构建更大的内容。除非你想建议一些对所发布的问题有效的内容，否则我们不要扩展此对话。其他任何内容都是无关的。