Java Wildfly 10.1.0最终远程处理端点任务线程持续增长

我们正在使用Wildfly 10.1.0 Final

我们遇到了一个OutOfMemoryError，它是由线程不断增长引起的

在检查线程转储之后

我们发现有数千个远程处理“端点”任务-N线程

远程处理“端点”任务-N线程的作用是什么

它们是通过作业远程处理创建的吗

重新启动服务器后，我们发现一开始只有16个线程：

远程“端点”任务1~远程“端点”任务16

服务器运行数天或数月后，可能会有数百或数千个远程处理线程：

下面列出了线程转储的一个片段

在这个线程转储中，有几个不同编号的远程处理端点task-11

其他任务（如任务1到任务16）也是如此

所有这些线程都在等待，什么也不做

"Remoting "endpoint" task-11" #55415 daemon prio=5 os_prio=0 tid=0x00007f2b8c0a8000 nid=0x276e waiting on condition [0x00007f280a36c000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006eabee2b8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


"Remoting "endpoint" task-11" #55417 daemon prio=5 os_prio=0 tid=0x00007f2ba003f800 nid=0x276d waiting on condition [0x00007f2794818000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006eabecf40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


"Remoting "endpoint" task-11" #55414 daemon prio=5 os_prio=0 tid=0x00007f2b98023800 nid=0x276b waiting on condition [0x00007f2792bfc000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006eabeda50> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


"Remoting "endpoint" task-10" #55411 daemon prio=5 os_prio=0 tid=0x00007f2ba003e000 nid=0x276a waiting on condition [0x00007f27926f7000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006eabecf40> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


"Remoting "endpoint" task-10" #55413 daemon prio=5 os_prio=0 tid=0x00007f2b8c0a7000 nid=0x2769 waiting on condition [0x00007f27927f8000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006eabee2b8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


"Remoting "endpoint" task-10" #55412 daemon prio=5 os_prio=0 tid=0x00007f2b98022800 nid=0x2768 waiting on condition [0x00007f27c4815000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006eabeda50> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


"Remoting "endpoint" task-9" #55372 daemon prio=5 os_prio=0 tid=0x00007f2c7408f000 nid=0x41df waiting on condition [0x00007f27907d8000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006eabece88> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


"Remoting "endpoint" task-8" #55369 daemon prio=5 os_prio=0 tid=0x00007f2c7408d000 nid=0x41dd waiting on condition [0x00007f27909da000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006eabece88> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


"Remoting "endpoint" task-7" #55368 daemon prio=5 os_prio=0 tid=0x00007f2c7408b000 nid=0x41dc waiting on condition [0x00007f2790adb000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006eabece88> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


"Remoting "endpoint" task-6" #55367 daemon prio=5 os_prio=0 tid=0x00007f2c74089000 nid=0x41db waiting on condition [0x00007f2790bdc000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006eabece88> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


"Remoting "endpoint" task-5" #55366 daemon prio=5 os_prio=0 tid=0x00007f2c74087000 nid=0x41da waiting on condition [0x00007f2790cdd000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006eabece88> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


"Remoting "endpoint" task-4" #55365 daemon prio=5 os_prio=0 tid=0x00007f2c74085000 nid=0x41d9 waiting on condition [0x00007f2790dde000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006eabece88> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


"Remoting "endpoint" task-9" #55364 daemon prio=5 os_prio=0 tid=0x00007f2bd813c000 nid=0x41d8 waiting on condition [0x00007f2790edf000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006eabed500> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


"Remoting "endpoint" task-9" #55363 daemon prio=5 os_prio=0 tid=0x00007f2bf4044000 nid=0x41d7 waiting on condition [0x00007f2790fe0000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00000006eabee3c0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

....

“远程处理”端点“任务-11”#55415守护进程优先级=5 os#u优先级=0 tid=0x00007f2b8c0a8000 nid=0x276e等待条件[0x00007f280a36c000]
java.lang.Thread.State:等待（停车）
在sun.misc.Unsafe.park（本机方法）
-停车等待（java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject）
位于java.util.concurrent.locks.LockSupport.park（LockSupport.java:175）
位于java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await（AbstractQueuedSynchronizer.java:2039）
位于java.util.concurrent.LinkedBlockingQueue.take（LinkedBlockingQueue.java:442）
位于java.util.concurrent.ThreadPoolExecutor.getTask（ThreadPoolExecutor.java:1074）
位于java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1134）
位于java.util.concurrent.ThreadPoolExecutor$Worker.run（ThreadPoolExecutor.java:624）
运行（Thread.java:748）
“远程处理”端点“任务-11”#55417守护进程优先级=5 os_优先级=0 tid=0x00007f2ba003f800 nid=0x276d等待条件[0x00007f2794818000]
java.lang.Thread.State:等待（停车）
在sun.misc.Unsafe.park（本机方法）
-停车等待（java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject）
位于java.util.concurrent.locks.LockSupport.park（LockSupport.java:175）
位于java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await（AbstractQueuedSynchronizer.java:2039）
位于java.util.concurrent.LinkedBlockingQueue.take（LinkedBlockingQueue.java:442）
位于java.util.concurrent.ThreadPoolExecutor.getTask（ThreadPoolExecutor.java:1074）
位于java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1134）
位于java.util.concurrent.ThreadPoolExecutor$Worker.run（ThreadPoolExecutor.java:624）
运行（Thread.java:748）
“远程处理”端点“任务-11”#55414守护进程优先级=5 os_优先级=0 tid=0x00007f2b98023800 nid=0x276b等待条件[0x00007f2792bfc000]
java.lang.Thread.State:等待（停车）
在sun.misc.Unsafe.park（本机方法）
-停车等待（java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject）
位于java.util.concurrent.locks.LockSupport.park（LockSupport.java:175）
位于java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await（AbstractQueuedSynchronizer.java:2039）
位于java.util.concurrent.LinkedBlockingQueue.take（LinkedBlockingQueue.java:442）
位于java.util.concurrent.ThreadPoolExecutor.getTask（ThreadPoolExecutor.java:1074）
位于java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1134）
位于java.util.concurrent.ThreadPoolExecutor$Worker.run（ThreadPoolExecutor.java:624）
运行（Thread.java:748）
“远程处理”端点“任务-10”#55411守护进程优先级=5 os_优先级=0 tid=0x00007f2ba003e000 nid=0x276a等待条件[0x00007f27926f7000]
java.lang.Thread.State:等待（停车）
在sun.misc.Unsafe.park（本机方法）
-停车等待（java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject）
位于java.util.concurrent.locks.LockSupport.park（LockSupport.java:175）
位于java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await（AbstractQueuedSynchronizer.java:2039）
位于java.util.concurrent.LinkedBlockingQueue.take（LinkedBlockingQueue.java:442）
位于java.util.concurrent.ThreadPoolExecutor.getTask（ThreadPoolExecutor.java:1074）
位于java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1134）
位于java.util.concurrent.ThreadPoolExecutor$Worker.run（ThreadPoolExecutor.java:624）
运行（Thread.java:748）
“远程处理”端点“任务-10”#55413守护进程优先级=5 os_优先级=0 tid=0x00007f2b8c0a7000 nid=0x2769等待条件[0x00007f27927f8000]
java.lang.Thread.State:等待（停车）
在sun.misc.Unsafe.park（本机方法）
-停车等待（java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject）
位于java.util.concurrent.locks.LockSupport.park（LockSupport.java:175）
位于java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await（AbstractQueuedSynchronizer.java:2039）
位于java.util.concurrent.LinkedBlockingQueue.take（LinkedBlockingQueue.java:442）
位于java.util.concurrent.ThreadPoolExecutor.getTask（ThreadPoolExecutor.java:1074）
位于java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1134）
位于java.util.concurrent.ThreadPoolExecutor$Worker.run（ThreadPoolExecutor.java:624）
运行（Thread.java:748）
“远程处理”端点“任务-10”#55412守护进程优先级=5 os_优先级=0 tid=0x00007F2B9802800NID=0x2768等待条件[0x00007f27c4815000]
java.lang.Thread.State:等待（停车）
在sun.misc.Unsafe.park（本机方法）
-停车等待（java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject）
位于java.util.concurrent.locks.LockSupport.park（LockSupport.java:175）
位于java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await（AbstractQueuedSynchronizer.java:2039）
位于java.util.concurrent.LinkedBlockingQueue.take（LinkedBlockingQueue.java:442）
位于java.util.concurrent.ThreadPoolExecutor.g
// JNDI context "A"
Context jndiCtx = new InitialContext(props);
// Now let's lookup a EJB
MyBean bean = jndiCtx.lookup("ejb:app/module/distinct/bean!interface");

// Remember, the ejb: is backed by a URL context factory which returns a Context for the ejb: URL (that's why it's called a context factory)
final Context ejbNamingContext = (Context) jndiCtx.lookup("ejb:");
// Use the returned EJB naming context to lookup the rest of the JNDI string for EJB
final MyBean bean = ejbNamingContext.lookup("app/module/distinct/bean!interface");

final Properties props = new Properties();
// mark it for scoped EJB client context
props.put("org.jboss.ejb.client.scoped.context","true");
// add other properties
props.put(....);
...
Context jndiCtx = new InitialContext(props);
try {
      final MyBean bean = jndiCtx.lookup("ejb:app/module/distinct/bean!interface");
      bean.doSomething();
} finally {
  jndiCtx.close();
}

final Properties props = new Properties();
// mark it for scoped EJB client context
props.put("org.jboss.ejb.client.scoped.context","true");
// add other properties
props.put(....);
...
Context jndiCtx = new InitialContext(props);
Context ejbRootNamingContext = (Context) jndiCtx.lookup("ejb:");
try {
    final MyBean bean = ejbRootNamingContext.lookup("app/module/distinct/bean!interface"); // the rest of the EJB jndi string
    bean.doSomething();
} finally {
    try {
        // close the EJB naming JNDI context
        ejbRootNamingContext.close();
    } catch (Throwable t) {
        // log and ignore
    }
    try {
        // also close our other JNDI context since we are done with it too
        jndiCtx.close();
    } catch (Throwable t) {
        // log and ignore
    }

}