Java SpringBoot@SqsListener-不工作-出现异常-TaskRejectedException

我有一个AWS SQS，队列中已经有5000条消息（示例消息看起来像‘Hello@1’） 我创建了一个SpringBoot应用程序，并在其中一个组件类中创建了一个从SQS读取消息的方法

package com.example.aws.sqs.service;

import org.springframework.cloud.aws.messaging.listener.SqsMessageDeletionPolicy;
import org.springframework.cloud.aws.messaging.listener.annotation.SqsListener;
import org.springframework.stereotype.Component;
import lombok.extern.slf4j.Slf4j;

@Component
@Slf4j
public class MessageReceiverService {   

@SqsListener(value = { "${cloud.aws.sqs.url}" }, deletionPolicy = SqsMessageDeletionPolicy.ALWAYS)
public void readMessage(String message){
    log.info("Reading Message... {}", message);
}

}

我的主要春靴课

@SpringBootApplication 
public class AwsSqsApplicationConsumer {
public static void main(String[] args) {
    SpringApplication.run(AwsSqsApplicationConsumer.class, args);
}
}

应用程序运行时出现的异常：

s.c.a.m.l.SimpleMessageListenerContainer : An Exception occurred while polling queue '<my sqs name>'. The failing operation will be retried in 10000 milliseconds
org.springframework.core.task.TaskRejectedException: Executor [java.util.concurrent.ThreadPoolExecutor@7c1594a5[Running, pool size = 3, active threads = 3, queued tasks = 0, completed tasks = 20]] did not accept task: org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer$SignalExecutingRunnable@1cbd9ef2
at org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor.execute(ThreadPoolTaskExecutor.java:309) ~[spring-context-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer$AsynchronousMessageListener.run(SimpleMessageListenerContainer.java:286) ~[spring-cloud-aws-messaging-2.0.0.RELEASE.jar:2.0.0.RELEASE]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_65]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_65]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_65]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_65]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
Caused by: java.util.concurrent.RejectedExecutionException: Task org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer$SignalExecutingRunnable@1cbd9ef2 rejected from java.util.concurrent.ThreadPoolExecutor@7c1594a5[Running, pool size = 3, active threads = 2, queued tasks = 0, completed tasks = 20]
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047) ~[na:1.8.0_65]
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823) [na:1.8.0_65]
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369) [na:1.8.0_65]
at org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor.execute(ThreadPoolTaskExecutor.java:306) ~[spring-context-5.0.7.RELEASE.jar:5.0.7.RELEASE]
... 6 common frames omitted

s.c.a.m.l.SimpleMessageListenerContainer:轮询队列“”时发生异常。失败的操作将在10000毫秒后重试
org.springframework.core.task.TaskRejectedException:Executor[java.util.concurrent。ThreadPoolExecutor@7c1594a5[正在运行，池大小=3，活动线程=3，排队任务=0，已完成任务=20]]未接受任务：org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer$SignalExecutingRunnable@1cbd9ef2
在org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor.execute（ThreadPoolTaskExecutor.java:309）~[spring-context-5.0.7.RELEASE.jar:5.0.7.RELEASE]
在org.springframework.cloud.aws.messaging.listener.SimpleMessageListener$AsynchronousMessageListener.run（simpleMessageListener.java:286）~[spring-cloud-aws-messaging-2.0.0.RELEASE.jar:2.0.0.RELEASE]
在java.util.concurrent.Executors$RunnableAdapter.call（Executors.java:511）[na:1.8.0\u 65]
在java.util.concurrent.FutureTask.run（FutureTask.java:266）[na:1.8.0_65]
位于java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1142）[na:1.8.0_65]
在java.util.concurrent.ThreadPoolExecutor$Worker.run（ThreadPoolExecutor.java:617）[na:1.8.0_65]
在java.lang.Thread.run（Thread.java:745）[na:1.8.0_65]
原因：java.util.concurrent.RejectedExecutionException:Task org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer$SignalExecutingRunnable@1cbd9ef2已从java.util.concurrent中拒绝。ThreadPoolExecutor@7c1594a5[正在运行，池大小=3，活动线程=2，排队任务=0，已完成任务=20]
在java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution（ThreadPoolExecutor.java:2047）~[na:1.8.0\u 65]
在java.util.concurrent.ThreadPoolExecutor.reject（ThreadPoolExecutor.java:823）[na:1.8.0＠
在java.util.concurrent.ThreadPoolExecutor.execute（ThreadPoolExecutor.java:1369）[na:1.8.0_65]
在org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor.execute（ThreadPoolTaskExecutor.java:306）~[spring-context-5.0.7.RELEASE.jar:5.0.7.RELEASE]
... 省略了6个公共框架

我没有配置任何自定义执行器服务。使用预配置的SpringBean。 springBootVersion='2.0.3.RELEASE'

---

springCloudVersion='Finchley.RELEASE'

问题在于侦听器线程配置。见下文

...
ThreadPoolExecutor@7c1594a5[Running, pool size = 3, active threads = 3, queued tasks = 0, completed tasks = 20]]
...

默认线程池大小小于所需大小

将以下配置添加到Spring应用程序中

@Configuration
public class TasksConfiguration implements SchedulingConfigurer {

    @Override
    public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
        ThreadPoolTaskScheduler taskScheduler = new ThreadPoolTaskScheduler();
        taskScheduler.setPoolSize(5); // TODO: Load this from configuration
        taskScheduler.initialize();
        taskRegistrar.setTaskScheduler(taskScheduler);
    }
}

现在，您应该能够处理这些任务了

另外，无论之前被拒绝的任务是什么，它们都将在一定时间后被接受

---

编辑：我认为人们被

行中的数字吓坏了。setPoolSize（5000）

。这是一个可配置的数字，您可以选择任何适合您需求的数字。对于答案，我将它减少到一个较小的数字

设置最大消息数似乎可以解决问题：

@Bean
public SimpleMessageListenerContainerFactory simpleMessageListenerContainerFactory(AmazonSQSAsync amazonSQS){
    SimpleMessageListenerContainerFactory factory = new SimpleMessageListenerContainerFactory();
    factory.setAmazonSqs(amazonSQS);
    factory.setMaxNumberOfMessages(10);
    return factory;
}

---

嘿，我用Spring Listener解决了这个问题。下面是代码，希望对您有所帮助

在下面的解决方案中，一旦所有bean初始化完成，就会分配一个池大小更大的新任务执行器

@Component
public class PostBeansConstructionListener{

    @EventListener
    public void handleContextRefreshedEvent(ContextRefreshedEvent event){
        final ApplicationContext applicationContext = event.getApplicationContext();
        final SimpleMessageListenerContainer simpleMessageListenerContainer = applicationContext.getBean(SimpleMessageListenerContainer.class);
        setAsyncTaskExecutor(simpleMessageListenerContainer);
    }

    private void setAsyncTaskExecutor(SimpleMessageListenerContainer simpleMessageListenerContainer) {
        try{
            simpleMessageListenerContainer.setTaskExecutor(getAsyncExecutor());
        }catch(Exception ex){
            throw new RuntimeException("Not able to create Async Task Executor for SimpleMessageListenerContainer.", ex);
        }
    }

    public AsyncTaskExecutor getAsyncExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(7);
        executor.setMaxPoolSize(42);
        executor.setQueueCapacity(11);
        executor.setThreadNamePrefix("threadPoolExecutor-SimpleMessageListenerContainer-");
        executor.initialize();
        return executor;
    }
}

---

无法在前面的回答中添加注释，以进一步解释问题发生的原因以及解决方案设置MaxNumberOfMessages of messages的工作原理。希望以下内容有助于澄清一切

SimpleMessageListenerContainer

的

ThreadPoolTaskExecutor

配置为核心池大小为2个线程，最大池大小为3个线程，队列容量为0。但是，轮询Amazon SQS时返回的默认最大消息数设置为10。这意味着如果在一次轮询中有10条消息可用，那么将没有足够的线程来处理它们。因此，将抛出

RejectedExecutionException

在

SimpleMessageListenerContainerFactory

上将

setMaxNumberOfMessages

配置为10将最大线程池大小设置为11，这将允许足够的线程可用。它不设置队列容量

要设置队列容量，可以初始化一个单独的TaskExecutor，并在

SimpleMessageListenerContainerFactory

bean上进行设置，如下所示：

@Bean(name = "sqsAsyncTaskExecutor")
public AsyncTaskExecutor asyncTaskExecutor(@Value("${external.aws.sqs.core-thread-count}") int coreThreadCount,
                                           @Value("${external.aws.sqs.max-thread-count}") int maxThreadCount,
                                           @Value("${external.aws.sqs.queue-capacity}") int queueCapacity) {
    ThreadPoolTaskExecutor asyncTaskExecutor = new ThreadPoolTaskExecutor();
    asyncTaskExecutor.setCorePoolSize(coreThreadCount);
    asyncTaskExecutor.setMaxPoolSize(maxThreadCount);
    asyncTaskExecutor.setQueueCapacity(queueCapacity);
    asyncTaskExecutor.setThreadNamePrefix("threadPoolExecutor-SimpleMessageListenerContainer-");
    asyncTaskExecutor.initialize();
    return asyncTaskExecutor;
}

@Bean
public SimpleMessageListenerContainerFactory simpleMessageListenerContainerFactory(AmazonSQSAsync amazonSQS, @Qualifier("sqsAsyncTaskExecutor") AsyncTaskExecutor asyncTaskExecutor) {
    SimpleMessageListenerContainerFactory simpleMessageListenerContainerFactory = new SimpleMessageListenerContainerFactory();
    simpleMessageListenerContainerFactory.setTaskExecutor(asyncTaskExecutor);
    return simpleMessageListenerContainerFactory;
}

我使用的值是coreThreadCount=5、maxThreadCount=20、queueCapacity=10

---

如前所述，我认为在SimpleMessageListenerContainerFactory上将setMaxNumberOfMessages配置为10应该足以处理从单个请求获取的所有批处理消息。但是，如果您觉得需要对TaskExecutor进行更精确的控制，那么此配置也可以工作。

我认为这是Spring中的一个bug或疏忽。该问题源于以下默认值：

public class SimpleMessageListenerContainer extends AbstractMessageListenerContainer {

    private static final int DEFAULT_WORKER_THREADS = 2;

及

如果未设置maxNumberOfMessages，则它使用10作为要从SQS提取的消息数，使用2作为任务执行器中的工作进程数。这意味着，如果它一次提取3条或更多消息，您将得到该异常。如果您手动将maxNumberOfMessages设置为一个值（任意值），它将在两个位置使用它来同步值，我相信这是意料之中的：

    @Bean
    public SimpleMessageListenerContainer simpleMessageListenerContainer(
            SimpleMessageListenerContainerFactory factory, QueueMessageHandler messageHandler)
    {
        SimpleMessageListenerContainer container = factory.createSimpleMessageListenerContainer();
        container.setMaxNumberOfMessages(5);
        container.setMessageHandler(messageHandler);
        return container;
    }

---

错误似乎是

SimpleMessageListenerContainer

-此代码在哪里？org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer是一个Spring类，spring-cloud-aws-messaging-2.0.0.RELEASE.jarHi随附，您是如何解决此问题的issue@AnkitaAgrawal-我已经停止使用'@SqsListener'，并从Spring开始使用带有'@Scheduled'和fixedRate系统的AmazonSqs客户端。因此，根据您的解决方案，我需要

    @Bean
    public SimpleMessageListenerContainer simpleMessageListenerContainer(
            SimpleMessageListenerContainerFactory factory, QueueMessageHandler messageHandler)
    {
        SimpleMessageListenerContainer container = factory.createSimpleMessageListenerContainer();
        container.setMaxNumberOfMessages(5);
        container.setMessageHandler(messageHandler);
        return container;
    }