Multithreading 1000+；Tomcat响应进程上的线程块

问题第一：

我的服务响应时间随着并发性的增加而增加。我在tomcat响应客户端的步骤中发现了1000多个线程块。但我不知道怎么解决它

背景：

我的服务是一个SpringBoot项目，它嵌入了一个tomcat。（tomcat-embed-core-8.5.23）

我的实验是使用JMeter作为客户端，创建2000个线程向我的服务发送10轮请求（总共20000个请求）。在我的服务之后，有一个存根服务对每个请求在2秒延迟内作出响应。（即JMeter（2000线程）=>我的服务=>存根服务器）

实验数据

• 在我的服务中，我设置了“server.tomcat.max threads=2000”。因此有2000多个线程名为“http-nio-8080*”
• 结果是，最大响应时间>30秒。这是从CLI复制的
  汇总=20000英寸00:01:23=241.7/s平均值：6890最小值：2288最大值：31158误差：0（0.00%）

最小响应时间（2288ms）与预期一致（因为存根服务器的成本为2000ms）。但最大响应时间（31158ms）太长。我分析了jstack日志。在同一个锁上阻塞了1121个线程。（我附加了一个堆栈内容。我认为这是我对JMeter的服务响应的时间点。但不幸的是，有一个SynchronizedStack阻止了1000多个线程）

我有证据表明锁是响应时间的真正杀手。在9秒后的另一个jstack日志中，仍然有1058个线程锁定在同一个锁上

"http-nio-8080-exec-3988" #4038 daemon prio=5 os_prio=0 tid=0x00007fdec045c800 nid=0x1050 waiting for monitor entry [0x00007fdb6c94e000]
   java.lang.Thread.State: BLOCKED (on object monitor)
                at org.apache.tomcat.util.collections.SynchronizedStack.pop(SynchronizedStack.java:75)
                - waiting to lock <0x00000000dac7afd0> (a org.apache.tomcat.util.collections.SynchronizedStack)
                at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:89)
                at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:157)
                at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.doWrite(NioEndpoint.java:1267)
                at org.apache.tomcat.util.net.SocketWrapperBase.doWrite(SocketWrapperBase.java:670)
                at org.apache.tomcat.util.net.SocketWrapperBase.flushBlocking(SocketWrapperBase.java:607)
                at org.apache.tomcat.util.net.SocketWrapperBase.flush(SocketWrapperBase.java:597)
                at org.apache.coyote.http11.Http11OutputBuffer.flushBuffer(Http11OutputBuffer.java:581)
                at org.apache.coyote.http11.Http11OutputBuffer.flush(Http11OutputBuffer.java:272)
                at org.apache.coyote.http11.Http11Processor.flush(Http11Processor.java:1560)
                at org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:283)
                at org.apache.coyote.Response.action(Response.java:173)
                at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:317)
                at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:284)
                at org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:118)
                at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:297)
                at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
                - locked <0x00000000fe000000> (a java.io.OutputStreamWriter)
                at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
                at org.springframework.util.StreamUtils.copy(StreamUtils.java:119)
                at org.springframework.http.converter.StringHttpMessageConverter.writeInternal(StringHttpMessageConverter.java:106)
                at org.springframework.http.converter.StringHttpMessageConverter.writeInternal(StringHttpMessageConverter.java:41)
                at org.springframework.http.converter.AbstractHttpMessageConverter.write(AbstractHttpMessageConverter.java:227)
                at org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodProcessor.writeWithMessageConverters(AbstractMessageConverterMethodProcessor.java:247)
                at org.springframework.web.servlet.mvc.method.annotation.RequestResponseBodyMethodProcessor.handleReturnValue(RequestResponseBodyMethodProcessor.java:174)
                at org.springframework.web.method.support.HandlerMethodReturnValueHandlerComposite.handleReturnValue(HandlerMethodReturnValueHandlerComposite.java:81)
                at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:113)
                at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
                at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
                at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
                at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
                at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
                at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
                at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
                at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
                at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
                at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
                at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
                at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55)
                at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
                at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:110)
                at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
                at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
                at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
                at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:108)
                at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
                at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:81)
                at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
                at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
                at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
                at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:106)
                at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
                at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
                at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
                at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:478)
                at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
                at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
                at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
                at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
                at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
                at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
                at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
                at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459)
                at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
                - locked <0x00000000e1a59188> (a org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
                at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
                - <0x00000000df55e210> (a java.util.concurrent.ThreadPoolExecutor$Worker)

“http-nio-8080-exec-3988”#4038守护进程prio=5 os_prio=0 tid=0x00007fdec045c800 nid=0x1050等待监视器条目[0x00007fdb6c94e000]
java.lang.Thread.State:阻塞（在对象监视器上）
位于org.apache.tomcat.util.collections.SynchronizedStack.pop（SynchronizedStack.java:75）
-等待锁定（org.apache.tomcat.util.collections.SynchronizedStack）
位于org.apache.tomcat.util.net.NioBlockingSelector.write（NioBlockingSelector.java:89）
位于org.apache.tomcat.util.net.NioSelectorPool.write（NioSelectorPool.java:157）
位于org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.doWrite（NioEndpoint.java:1267）
位于org.apache.tomcat.util.net.SocketWrapperBase.doWrite（SocketWrapperBase.java:670）
位于org.apache.tomcat.util.net.SocketWrapperBase.flushBlocking（SocketWrapperBase.java:607）
位于org.apache.tomcat.util.net.SocketWrapperBase.flush（SocketWrapperBase.java:597）
位于org.apache.coyote.http11.Http11OutputBuffer.flushBuffer（Http11OutputBuffer.java:581）
位于org.apache.coyote.http11.Http11OutputBuffer.flush（Http11OutputBuffer.java:272）
位于org.apache.coyote.http11.Http11Processor.flush（Http11Processor.java:1560）
位于org.apache.coyote.AbstractProcessor.action（AbstractProcessor.java:283）
位于org.apache.coyote.Response.action（Response.java:173）
位于org.apache.catalina.connector.OutputBuffer.doFlush（OutputBuffer.java:317）
位于org.apache.catalina.connector.OutputBuffer.flush（OutputBuffer.java:284）
位于org.apache.catalina.connector.CoyoteOutputStream.flush（CoyoteOutputStream.java:118）
位于sun.nio.cs.streamncoder.implFlush（streamncoder.java:297）
位于sun.nio.cs.streamncoder.flush（streamncoder.java:141）
-锁定（java.io.OutputStreamWriter）
位于java.io.OutputStreamWriter.flush（OutputStreamWriter.java:229）
位于org.springframework.util.StreamUtils.copy（StreamUtils.java:119）
位于org.springframework.http.converter.StringHttpMessageConverter.writeInternal（StringHttpMessageConverter.java:106）
位于org.springframework.http.converter.StringHttpMessageConverter.writeInternal（StringHttpMessageConverter.java:41）
位于org.springframework.http.converter.AbstractHttpMessageConverter.write（AbstractHttpMessageConverter.java:227）
位于org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodProcessor.WriteWithMessageConverterMethodProcessor（AbstractMessageConverterMethodProcessor.java:247）
位于org.springframework.web.servlet.mvc.method.annotation.RequestResponseBodyMethodProcessor.handleReturnValue（RequestResponseBodyMethodProcessor.java:174）
位于org.springframework.web.method.support.HandlerMethodReturnValueHandlerComposite.HandlerReturnValue（HandlerMethodReturnValueHandlerComposite.java:81）
位于org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle（ServletInvocableHandlerMethod.java:113）
位于org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod（RequestMappingHandlerAdapter.java:827）
位于org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal（RequestMappingHandlerAdapter.java:738）
位于org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle（AbstractHandlerMethodAdapter.java:85）
位于org.springframework.web.servlet.DispatcherServlet.doDispatch（DispatcherServlet.java:967）
位于org.springframework.web.servlet.DispatcherServlet.doService（DispatcherServlet.java:901）
位于org.springframework.web.servlet.FrameworkServlet.processRequest（FrameworkServlet.java:970）
位于org.springframework.web.servlet.FrameworkServlet.doPost（FrameworkServlet.java:872）
位于javax.servlet.http.HttpServlet.service（HttpServlet.java:661）
位于org.springframework.web.servlet.FrameworkServlet.service（FrameworkServlet.java:846）
位于javax.servlet.http.HttpServlet.service（HttpServlet.java:742）