持续向ExecutorService提交可运行任务,直到工作完成,获取java.util.concurrent.RejectedExecutionException
我的多线程类应该在类持续向ExecutorService提交可运行任务,直到工作完成,获取java.util.concurrent.RejectedExecutionException,java,threadpool,producer-consumer,java.util.concurrent,blockingqueue,Java,Threadpool,Producer Consumer,Java.util.concurrent,Blockingqueue,我的多线程类应该在类ClassA的许多对象上执行三个操作—operation1、operation2和operation3,其中每种类型的操作都依赖于前面的操作。为此,我尝试使用大量的BlockingQueues和ExecutorService实现生产者-消费者模式 final ExecutorService executor = ForkJoinPool.commonPool(); final BlockingQueue<ClassA> operationOneQueue = ne
ClassA
的许多对象上执行三个操作—operation1
、operation2
和operation3
,其中每种类型的操作都依赖于前面的操作。为此,我尝试使用大量的BlockingQueue
s和ExecutorService
实现生产者-消费者模式
final ExecutorService executor = ForkJoinPool.commonPool();
final BlockingQueue<ClassA> operationOneQueue = new ArrayBlockingQueue<>(NO_OF_CLASS_A_OBJECTS);
final BlockingQueue<ClassA> operationTwoQueue = new ArrayBlockingQueue<>(NO_OF_CLASS_A_OBJECTS);
final BlockingQueue<ClassA> operationThreeQueue = new ArrayBlockingQueue<>(NO_OF_CLASS_A_OBJECTS);
final BlockingQueue<ClassA> resultQueue = new ArrayBlockingQueue<>(NO_OF_CLASS_A_OBJECTS);
其中,每种类型的操作都有自己相应的方法,包括“自己的”入队列和出队列。每个操作方法调用ClassA
对象上的相应方法。方法doOperationThree
将ClassA
对象放入resultQueue
中,这意味着它们已被完全处理
首先,我用要操作的所有ClassA
对象填充operationOneQueue
。然后,我尝试将可执行任务分配给ExecutorService
,如下所示:
void doOperationOne() throws InterruptedException {
ClassA objectA = operationOneQueue.take();
objectA.operationOne();
operationTwoQueue.put(objectA);
}
while (resultQueue.size() < NO_OF_CLASS_A_OBJECTS) {
executor.execute(() -> {
try {
doOperationOne();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
});
executor.execute(() -> {
try {
doOperationTwo();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
});
executor.execute(() -> {
try {
doOperationThree();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
});
}
executor.shutdown();
我做错了什么?如何在不使线程池过度饱和的情况下实现这一点
编辑:完全公开——这是有一些要求的家庭作业。1.我必须使用
ForkJoinPool.commonPool()
并且不能自己设置线程数,2。我必须使用消费者-生产者模式,以及3。我不能修改ClassA
我真的很喜欢做并发的东西,所以我试着写了它。我确实使用了CompletableFuture
,a)默认情况下在ForkJoinPool.commonPool
中运行,b)使实际处理非常简单:
while (true) {
final ClassA nextOperation = queue.take();
CompletableFuture.runAsync(nextOperation::operationOne)
.thenRun(nextOperation::operationTwo)
.thenRun(nextOperation::operationThree)
.thenRun(() -> resultQueue.add(nextOperation));
}
这将从队列中获取ClassA
对象,并按顺序并发执行其所有操作
您确实遗漏了任务的来源,以及是否需要使用者终止。一般来说,你不想这样做,这确实让事情变得更复杂了
private static final int COUNT = 10;
private static final Random RANDOM = new Random();
public static void main(String[] args) throws ExecutionException, InterruptedException {
BlockingQueue<ClassA> runnables = new ArrayBlockingQueue<>(COUNT);
BlockingQueue<ClassA> finished = new ArrayBlockingQueue<>(COUNT);
// start producer
ExecutorService createTaskExecutor = Executors.newSingleThreadExecutor();
createTaskExecutor.submit(() -> fillQueue(runnables));
// wait for all consumer tasks to finish
while (finished.size() != COUNT) {
try {
// we need to poll instead of waiting forever
// because the last tasks might still be running
// while there are no others to add anymore
// so we need to check again if all have finished in the meantime
final ClassA nextOperation = runnables.poll(2, TimeUnit.SECONDS);
if (nextOperation != null) {
CompletableFuture.runAsync(nextOperation::operationOne)
.thenRun(nextOperation::operationTwo)
.thenRun(nextOperation::operationThree)
.thenRun(() -> finished.add(nextOperation));
}
} catch (InterruptedException e) {
System.err.println("exception while retrieving next operation");
// we will actually need to terminate now, or probably never will
throw e;
}
}
System.out.printf("finished tasks (%d):%n", finished.size());
for (ClassA classA : finished) {
System.out.printf("finished task %d%n", classA.designator);
}
createTaskExecutor.shutdown();
}
private static void fillQueue(BlockingQueue<ClassA> runnables) {
// start thread filling the queue at random
for (int i = 0; i < COUNT; i++) {
runnables.add(new ClassA(i));
try {
Thread.sleep(RANDOM.nextInt(1_000));
} catch (InterruptedException e) {
System.err.println("failed to add runnable");
}
}
}
为什么要使用
ForkJoinPool.commonPool()
?尝试执行者。fixedThreadPool()。这是家庭作业。约束包括我必须使用ForkJoinPool.commonPool(),而不是自己设置线程数。在这种情况下,您可能需要设置完整的约束。你所做的看起来并没有错,只是在某个时候提交任务会超过完成任务。如果给我你现在已经放弃的约束条件,我会使用CompletableFuture
s,它们非常适合并在通用池IIUC中运行。我刚刚在本地尝试了你的代码,它似乎成功地完成了。您确定创建的ClassA
实例数等于NO\u of_CLASS\u A\u对象的值吗?再次检查您的doOperationX
方法,检查它们是否从队列中获取并将放入正确的队列中。谢谢,@sp00m,我已经仔细检查了这两个方法。当我的计算机上有超过40个ClassA
实例时,程序就会中断。
private static final int COUNT = 10;
private static final Random RANDOM = new Random();
public static void main(String[] args) throws ExecutionException, InterruptedException {
BlockingQueue<ClassA> runnables = new ArrayBlockingQueue<>(COUNT);
BlockingQueue<ClassA> finished = new ArrayBlockingQueue<>(COUNT);
// start producer
ExecutorService createTaskExecutor = Executors.newSingleThreadExecutor();
createTaskExecutor.submit(() -> fillQueue(runnables));
// wait for all consumer tasks to finish
while (finished.size() != COUNT) {
try {
// we need to poll instead of waiting forever
// because the last tasks might still be running
// while there are no others to add anymore
// so we need to check again if all have finished in the meantime
final ClassA nextOperation = runnables.poll(2, TimeUnit.SECONDS);
if (nextOperation != null) {
CompletableFuture.runAsync(nextOperation::operationOne)
.thenRun(nextOperation::operationTwo)
.thenRun(nextOperation::operationThree)
.thenRun(() -> finished.add(nextOperation));
}
} catch (InterruptedException e) {
System.err.println("exception while retrieving next operation");
// we will actually need to terminate now, or probably never will
throw e;
}
}
System.out.printf("finished tasks (%d):%n", finished.size());
for (ClassA classA : finished) {
System.out.printf("finished task %d%n", classA.designator);
}
createTaskExecutor.shutdown();
}
private static void fillQueue(BlockingQueue<ClassA> runnables) {
// start thread filling the queue at random
for (int i = 0; i < COUNT; i++) {
runnables.add(new ClassA(i));
try {
Thread.sleep(RANDOM.nextInt(1_000));
} catch (InterruptedException e) {
System.err.println("failed to add runnable");
}
}
}
class ClassA {
private static final Random RANDOM = new Random();
public final int designator;
public ClassA(int i) {
designator = i;
}
public void operationOne() {
System.out.printf("%d: operation 1%n", designator);
sleep();
}
public void operationTwo() {
System.out.printf("%d: operation 2%n", designator);
sleep();
}
public void operationThree() {
System.out.printf("%d: operation 3%n", designator);
sleep();
}
private static void sleep() {
try {
Thread.sleep(RANDOM.nextInt(5_000));
} catch (InterruptedException e) {
System.err.println("interrupted while executing task");
}
}
}