Java 队列已满时ThreadPoolExecutor块?
我正在尝试使用ThreadPoolExecutor执行许多任务。下面是一个假设的例子:Java 队列已满时ThreadPoolExecutor块?,java,multithreading,concurrency,executorservice,executor,Java,Multithreading,Concurrency,Executorservice,Executor,我正在尝试使用ThreadPoolExecutor执行许多任务。下面是一个假设的例子: def workQueue = new ArrayBlockingQueue<Runnable>(3, false) def threadPoolExecutor = new ThreadPoolExecutor(3, 3, 1L, TimeUnit.HOURS, workQueue) for(int i = 0; i < 100000; i++) threadPoolExecuto
def workQueue = new ArrayBlockingQueue<Runnable>(3, false)
def threadPoolExecutor = new ThreadPoolExecutor(3, 3, 1L, TimeUnit.HOURS, workQueue)
for(int i = 0; i < 100000; i++)
threadPoolExecutor.execute(runnable)
def workQueue=new ArrayBlockingQueue(3,false)
def threadPoolExecutor=新的threadPoolExecutor(3,3,1L,TimeUnit.HOURS,workQueue)
对于(int i=0;i<100000;i++)
threadPoolExecutor.execute(可运行)
问题是我很快得到了一个
java.util.concurrent.RejectedExecutionException
,因为任务的数量超过了工作队列的大小。然而,我所期望的行为是拥有主线程块,直到队列中有空间为止。实现这一点的最佳方法是什么?在一些非常狭窄的环境中,您可以实现一个java.util.concurrent.RejectedExecutionHandler来完成您需要的任务
RejectedExecutionHandler block = new RejectedExecutionHandler() {
rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
executor.getQueue().put( r );
}
};
ThreadPoolExecutor pool = new ...
pool.setRejectedExecutionHandler(block);
现在。这是一个非常糟糕的主意,原因如下
- 它很容易死锁,因为池中的所有线程都可能在您放入队列的内容可见之前死亡。通过设置合理的保持活动时间来缓解此问题
- 任务没有按照执行者预期的方式包装。许多执行器实现在执行之前将其任务包装在某种跟踪对象中。看看你的来源
- API强烈反对通过getQueue()进行添加,在某些情况下可能会被禁止
- 只有一个线程调用execute()
- 您必须(或希望)有一个非常小的队列长度
- 您绝对需要限制运行这项工作的线程数量(通常是出于外部原因),调用方运行策略将打破这一限制
- 您的任务大小不可预测,因此,如果池暂时忙于4个短任务,而调用execute的单线程被一个大任务卡住,则调用程序运行可能会导致饥饿
祝你好运。这是我在本例中的代码片段:
public void executeBlocking( Runnable command ) {
if ( threadPool == null ) {
logger.error( "Thread pool '{}' not initialized.", threadPoolName );
return;
}
ThreadPool threadPoolMonitor = this;
boolean accepted = false;
do {
try {
threadPool.execute( new Runnable() {
@Override
public void run() {
try {
command.run();
}
// to make sure that the monitor is freed on exit
finally {
// Notify all the threads waiting for the resource, if any.
synchronized ( threadPoolMonitor ) {
threadPoolMonitor.notifyAll();
}
}
}
} );
accepted = true;
}
catch ( RejectedExecutionException e ) {
// Thread pool is full
try {
// Block until one of the threads finishes its job and exits.
synchronized ( threadPoolMonitor ) {
threadPoolMonitor.wait();
}
}
catch ( InterruptedException ignored ) {
// return immediately
break;
}
}
} while ( !accepted );
}
threadPool是已经初始化的java.util.concurrent.ExecutorService的本地实例。我使用一个自定义的RejectedExecutionHandler解决了这个问题,它只是暂时阻止调用线程,然后再次尝试提交任务:
public class BlockWhenQueueFull implements RejectedExecutionHandler {
public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
// The pool is full. Wait, then try again.
try {
long waitMs = 250;
Thread.sleep(waitMs);
} catch (InterruptedException interruptedException) {}
executor.execute(r);
}
}
该类只能在线程池执行器中用作RejectedExecutionHandler,就像其他任何类一样。在本例中:
executorPool = new def threadPoolExecutor = new ThreadPoolExecutor(3, 3, 1L, TimeUnit.HOURS, workQueue, new BlockWhenQueueFull())
我看到的唯一缺点是调用线程的锁定时间可能比严格需要的时间稍长(高达250ms)。对于许多短时间运行的任务,可能会将等待时间减少到10ms左右。此外,由于该执行器被有效地递归调用,因此等待线程变得可用的时间过长(小时)可能会导致堆栈溢出
然而,我个人喜欢这种方法。它结构紧凑,易于理解,运行良好。我遗漏了什么重要的东西吗?您可以使用
信号量来阻止线程进入池中
执行器服务=新的线程池执行器(
3.
3.
1.
时间单位:小时,
新建ArrayBlockingQueue(6,false)
);
信号量锁=新信号量(6);//等于队列容量
对于(int i=0;i<100000;i++){
试一试{
lock.acquire();
服务提交(()->{
试一试{
task.run();
}最后{
锁定。释放();
}
});
}捕捉(中断异常e){
抛出新的运行时异常(e);
}
}
一些陷阱:
- 仅在固定线程池中使用此模式。队列不太可能经常满,因此不会创建新线程。查看ThreadPoolExecutor上的java文档了解更多详细信息:有一种方法可以解决这个问题,但它超出了这个答案的范围
- 队列大小应大于核心线程数。如果我们将队列大小设为3,最终会发生以下情况:
- T0:所有三个线程都在工作,队列为空,没有可用的许可证
- T1:螺纹1完成,释放许可证
- T2:线程1轮询队列中的新工作,未找到任何工作,等待
- T3:主线程将工作提交到池中,线程1开始工作
private static class BlockingExecutor implements Executor {
final Semaphore semaphore;
final Executor delegate;
private BlockingExecutor(final int concurrentTasksLimit, final Executor delegate) {
semaphore = new Semaphore(concurrentTasksLimit);
this.delegate = delegate;
}
@Override
public void execute(final Runnable command) {
try {
semaphore.acquire();
} catch (InterruptedException e) {
e.printStackTrace();
return;
}
final Runnable wrapped = () -> {
try {
command.run();
} finally {
semaphore.release();
}
};
delegate.execute(wrapped);
}
}
您可以将concurrentTasksLimit调整为您的委托执行者的threadPoolSize+queueSize,这将基本上解决您的问题
int NUM_THREADS = 6;
Semaphore lock = new Semaphore(NUM_THREADS);
ExecutorService pool = Executors.newCachedThreadPool();
for (int i = 0; i < 100000; i++) {
try {
lock.acquire();
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
pool.execute(() -> {
try {
// Task logic
} finally {
lock.release();
}
});
}
int NUM_THREADS=6;
信号量锁=新信号量(NUM_线程);
ExecutorService池=Executors.newCachedThreadPool();
对于(int i=0;i<100000;i++){
试一试{
lock.acquire();
}捕捉(中断异常e){
抛出新的运行时异常(e);
}
pool.execute(()->{
试一试{
//任务逻辑
}最后{
锁定。释放();
}
});
}
一个相当简单的选择是,在调用offer(…)
时,使用调用put(…)
的实现包装您的BlockingQueue
:
public class BlockOnOfferAdapter<T> implements BlockingQueue<T> {
(..)
public boolean offer(E o) {
try {
delegate.put(o);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
return false;
}
return true;
}
(.. implement all other methods simply by delegating ..)
}
没有捕获被拒绝的执行异常或需要复杂的锁定。好的,旧线程,但这是我在搜索时发现的
/**
* Inserts the specified element into this queue, waiting if necessary
* for space to become available.
*
* @param e the element to add
* @throws InterruptedException if interrupted while waiting
* @throws ClassCastException if the class of the specified element
* prevents it from being added to this queue
* @throws NullPointerException if the specified element is null
* @throws IllegalArgumentException if some property of the specified
* element prevents it from being added to this queue
*/
void put(E e) throws InterruptedException;
class BlockingThreadPoolTaskExecutor(concurrency: Int) : ThreadPoolTaskExecutor() {
companion object {
lateinit var semaphore: Semaphore
}
init {
semaphore = Semaphore(concurrency)
val semaphoreTaskDecorator = SemaphoreTaskDecorator()
this.setTaskDecorator(semaphoreTaskDecorator)
}
override fun <T> submit(task: Callable<T>): Future<T> {
log.debug("submit")
semaphore.acquire()
return super.submit(task)
}
}
private class SemaphoreTaskDecorator : TaskDecorator {
override fun decorate(runnable: Runnable): Runnable {
log.debug("decorate")
return Runnable {
try {
runnable.run()
} finally {
log.debug("decorate done")
semaphore.release()
}
}
}
}