Java 单线程挂起在Tomcat7上

Java 单线程挂起在Tomcat7上,java,multithreading,spring-mvc,tomcat7,Java,Multithreading,Spring Mvc,Tomcat7,有人遇到过tomcat7中挂线的问题吗?在工作时间内,我们在生产服务器上收到数以百万计的HTTP请求。Tomcat对于所有请求都表现得非常好,并在指定的时间限制内响应。一些线程一周只暂停一到两次超过10秒,这会导致响应延迟 环境详情如下: 爪哇7 雄猫7 分配的最大内存为8GB springmvc实现 CentOS 6.5 不确定问题是否与Tomcat或Java有关,或者与虚拟机有关。请注意,已启用GC日志记录,它显示每分钟不到一秒钟的暂停。在10秒钟内,我没有看到gc日志中有任何暂停 应用

有人遇到过tomcat7中挂线的问题吗?在工作时间内,我们在生产服务器上收到数以百万计的HTTP请求。Tomcat对于所有请求都表现得非常好,并在指定的时间限制内响应。一些线程一周只暂停一到两次超过10秒,这会导致响应延迟

环境详情如下:

  • 爪哇7
  • 雄猫7
  • 分配的最大内存为8GB
  • springmvc实现
  • CentOS 6.5
不确定问题是否与Tomcat或Java有关,或者与虚拟机有关。请注意,已启用GC日志记录,它显示每分钟不到一秒钟的暂停。在10秒钟内,我没有看到gc日志中有任何暂停

应用程序日志

2015-12-01 14:28:41,696|lvl=INFO|cls=identity.service.IdentityService|thd=http-apr-8080-exec-6|mod=URS|opn=[modifyPostionAssignment]|ssoTicket=ESR - N/A|requestID=C6232437-0116-0401-1201-142827450001|deviceId=0.481455058954918820af268b0.6793986896475551|sessionId=JSESSIONID0.5594741341655929|tid=HG-INT-01-0GRZI7BhTreRniamQm0esA|msg=[maintainUserMasterData] START
2015-12-01 14:28:28,226|lvl=INFO|cls=identity.service.controller.IdentityServiceController|thd=http-apr-8080-exec-6|mod=URS|opn=[modifyPostionAssignment]|ssoTicket=ESR - N/A|requestID=C6232437-0116-0401-1201-142827450001|deviceId=0.481455058954918820af268b0.6793986896475551|sessionId=JSESSIONID0.5594741341655929|tid=HG-INT-01-0GRZI7BhTreRniamQm0esA|msg=[maintainUserMasterData] START
GC日志

2015-12-01T14:28:41.461+0000: 500601.110: Total time for which application threads were stopped: 0.2703630 seconds
2015-12-01T14:28:40.705+0000: 500600.354: Total time for which application threads were stopped: 0.2837290 seconds
2015-12-01T14:28:39.473+0000: 500599.122: Total time for which application threads were stopped: 0.0011780 seconds
2015-12-01T14:28:39.123+0000: 500598.771: Total time for which application threads were stopped: 0.0866600 seconds
2015-12-01T14:28:35.918+0000: 500595.566: Total time for which application threads were stopped: 0.0001030 seconds
2015-12-01T14:28:35.918+0000: 500595.566: Total time for which application threads were stopped: 0.0001520 seconds
2015-12-01T14:28:35.917+0000: 500595.566: Total time for which application threads were stopped: 0.0005330 seconds
2015-12-01T14:28:35.914+0000: 500595.562: Total time for which application threads were stopped: 0.0001030 seconds
2015-12-01T14:28:35.914+0000: 500595.562: Total time for which application threads were stopped: 0.0002980 seconds
2015-12-01T14:28:35.913+0000: 500595.562: Total time for which application threads were stopped: 0.0012430 seconds
2015-12-01T14:28:34.468+0000: 500594.117: Total time for which application threads were stopped: 0.0001340 seconds
2015-12-01T14:28:34.468+0000: 500594.117: Total time for which application threads were stopped: 0.0001210 seconds
2015-12-01T14:28:34.468+0000: 500594.116: Total time for which application threads were stopped: 0.0001500 seconds
2015-12-01T14:28:34.467+0000: 500594.116: Total time for which application threads were stopped: 0.0012510 seconds
2015-12-01T14:28:29.479+0000: 500589.127: Total time for which application threads were stopped: 0.0012900 seconds
2015-12-01T14:28:27.979+0000: 500587.628: Total time for which application threads were stopped: 0.0607270 seconds
2015-12-01T14:28:27.101+0000: 500586.750: Total time for which application threads were stopped: 0.0982840 seconds
Tomcat访问日志:

[01/Dec/2015:14:28:41 +0000] remotehost=192.168.62.2 x-forwarded-for=192.168.61.11 method=POST uri=/ursbusservice/maintainUserMasterData/ qs= status=200 bytes=41365 respTime=4766 method = POST status = 200
[01/Dec/2015:14:28:41 +0000] remotehost=192.168.62.2 x-forwarded-for=192.168.61.26 method=POST uri=/ursbusservice/maintainUserMasterData/ qs= status=200 bytes=4041 respTime=13580 method = POST status = 200
[01/Dec/2015:14:28:40 +0000] remotehost=192.168.62.2 x-forwarded-for=192.168.61.13 method=POST uri=/ursbusservice/maintainUserMasterData/ qs= status=200 bytes=14204 respTime=1159 method = POST status = 200
[01/Dec/2015:14:28:39 +0000] remotehost=192.168.62.2 x-forwarded-for=192.168.61.12 method=POST uri=/ursbusservice/maintainUserMasterData/ qs= status=200 bytes=59554 respTime=13990 method = POST status = 200
[01/Dec/2015:14:28:35 +0000] remotehost=192.168.62.2 x-forwarded-for=192.168.61.10 method=POST uri=/ursbusservice/maintainUserMasterData/ qs= status=200 bytes=37664 respTime=186 method = POST status = 200
[01/Dec/2015:14:28:33 +0000] remotehost=192.168.62.2 x-forwarded-for=192.168.61.12 method=POST uri=/ursbusservice/maintainUserMasterData/ qs= status=200 bytes=42983 respTime=426 method = POST status = 200
代码段,IdentityServiceController.java:

@RequestMapping(value = RequestMappings.MAINTAINUSER_MASTER_DATA, method = RequestMethod.POST, consumes = IdentityServiceConstants.APPLICATION_JSON)
@ResponseBody
public UserMasterVO maintainUserMasterData(@RequestBody UserMasterVO userMasterVO)
{
    ContextInfo contextInfo = userMasterVO.getContextInfo();
    ServiceContext.set(contextInfo);
    dLogger.info(IdentityServiceConstants.START, contextInfo, new Object[] { IdentityServiceConstants.Methods.MAINTAIN_USERMASTER_DATA});
    if (userMasterVO.getUuid() != null && userMasterVO.getOperation() != null)
    {
        identityService.maintainUserMasterData(userMasterVO);
    }
    else
    {
        throw new IdentityServiceException();
    }
    return userMasterVO;
}
IdentityService.java:

public void maintainUserMasterData(UserMasterVO userMasterVO) {
    ContextInfo contextInfo = userMasterVO.getContextInfo();
    ServiceContext.set(contextInfo);
    dLogger.info(IdentityServiceConstants.START, contextInfo, new Object[] {IdentityServiceConstants.Methods.MAINTAIN_USERMASTER_DATA});
    //do some operations
    dLogger.info(IdentityServiceConstants.END, contextInfo, new Object[] {IdentityServiceConstants.Methods.MAINTAIN_USERMASTER_DATA});
}

如果它像你说的那样稀有,那么它几乎可以是任何东西

有些地方值得一看:

  • GC正在发生吗
  • 有模式吗
  • 如果存在计时模式,请查看当时发生了什么
  • 您的反病毒软件多久进行一次完整扫描
  • 如果有一个数据库连接,那么它当时的性能如何
  • 当时是否有人登录系统
  • 当时是否记录了任何系统错误
  • 你的日志文件有多大?日志文件夹中有多少日志文件

    • 如果它像你说的那样罕见,那么它几乎可以是任何东西

      有些地方值得一看:

      • GC正在发生吗
      • 有模式吗
      • 如果存在计时模式,请查看当时发生了什么
      • 您的反病毒软件多久进行一次完整扫描
      • 如果有一个数据库连接,那么它当时的性能如何
      • 当时是否有人登录系统
      • 当时是否记录了任何系统错误
      • 你的日志文件有多大?日志文件夹中有多少日志文件


      尝试在发生这种情况时转储所有线程,并找出慢线程在等待什么for@Jan你能给我提供一个线程转储的详细信息吗?请注意,线程不会永远暂停。它会在10-15秒后做出响应。令人担忧的是,您的生产服务器有数百万个请求,但不知道如何进行线程转储。请参阅以获取参考。请尝试记录每个响应的开始和结束时间以及其他数据。这可能有助于分类。此外,您应该有可用的gc日志。我认为您的服务器可能正在进行垃圾收集。尝试在垃圾收集发生时转储所有线程,并找出慢线程正在等待什么for@Jan你能给我提供一个线程转储的详细信息吗?请注意,线程不会永远暂停。它会在10-15秒后做出响应。令人担忧的是,您的生产服务器有数百万个请求,但不知道如何进行线程转储。请参阅以获取参考。请尝试记录每个响应的开始和结束时间以及其他数据。这可能有助于分类。此外,您应该有可用的gc日志。我认为您的服务器很可能正在进行垃圾收集。GC几乎每秒都在发生。线程进入一个方法并在第一条语句中打印日志。调用第二个方法时,记录第一条语句所用的时间超过10秒。这似乎不是数据库的问题。它在执行数据库操作时不会暂停。在此期间未报告任何错误。@KrishnaKuntala-记录第一条语句所用的时间超过10秒-可能是记录问题。添加更多的建议。@KrishnaKuntala您应该添加这些方法的内容以及问题。谢谢@eis。更新了我的问题。这似乎不是一个问题,只是记录。正如您在Tomcat访问日志中看到的那样,在这个线程响应期间,响应时间也会出现峰值。GC几乎每秒都在发生。线程进入一个方法并在第一条语句中打印日志。调用第二个方法时,记录第一条语句所用的时间超过10秒。这似乎不是数据库的问题。它在执行数据库操作时不会暂停。在此期间未报告任何错误。@KrishnaKuntala-记录第一条语句所用的时间超过10秒-可能是记录问题。添加更多的建议。@KrishnaKuntala您应该添加这些方法的内容以及问题。谢谢@eis。更新了我的问题。这似乎不是一个问题,只是记录。正如您在Tomcat访问日志中看到的,在这个线程响应期间,我们的响应时间也出现了峰值。