缓慢的应用程序,频繁的JVM挂起,单CPU设置和Java 12+;
我们有一个客户端应用程序(经过10多年的开发)。其JDK最近从OpenJDK11升级到OpenJDK14。在单CPU(禁用超线程)Windows 10设置(以及在只有一个可用CPU的VirtualBox机器中)上,与Java 11相比,应用程序的启动速度非常慢。此外,它大部分时间使用100%的CPU。我们还可以将处理器关联设置为仅一个CPU(缓慢的应用程序,频繁的JVM挂起,单CPU设置和Java 12+;,java,freeze,java-14,Java,Freeze,Java 14,我们有一个客户端应用程序(经过10多年的开发)。其JDK最近从OpenJDK11升级到OpenJDK14。在单CPU(禁用超线程)Windows 10设置(以及在只有一个可用CPU的VirtualBox机器中)上,与Java 11相比,应用程序的启动速度非常慢。此外,它大部分时间使用100%的CPU。我们还可以将处理器关联设置为仅一个CPU(c:\windows\system32\cmd.exe/c start/affinity 1…)来重现这个问题 在我的VirtualBox机器中,启动应用程
c:\windows\system32\cmd.exe/c start/affinity 1…
)来重现这个问题
在我的VirtualBox机器中,启动应用程序并以最少的手动交互执行查询时的一些度量:
- OpenJDK 11.0.2:36秒
- OpenJDK 13.0.2:~1.5分钟
- OpenJDK 13.0.2与
:46秒-XX:-UseBiasedLocking
- OpenJDK 13.0.2,带有
:40秒-XX:-threadLocalHandshake
- OpenJDK 14:5-6分钟
- OpenJDK 14与
:3-3,5分钟-XX:-使用biasedlocking
- OpenJDK 15 EA构建20:~4,5分钟
-XX:-ThreadLocalHandshakes
在Java 14中不可用。)
我们已经尝试记录JDK14对-Xlog:all=debug:file=app.txt的操作:正常运行时间、tid、级别、标记:filecount=50
使用OpenJDK 11.0.2,每秒计算一次日志行似乎相当顺利:
$ cat jdk11-log/app* | grep "^\[" | cut -d. -f 1 | cut -d[ -f 2 | sort | uniq -c | sort -k 2 -n
30710 0
44012 1
55461 2
55974 3
27182 4
41292 5
43796 6
51889 7
54170 8
58850 9
51422 10
44378 11
41405 12
53589 13
41696 14
29526 15
2350 16
50228 17
62623 18
42684 19
45045 20
另一方面,OpenJDK 14似乎有着有趣的安静时期:
$ cat jdk14-log/app* | grep "^\[" | cut -d. -f 1 | cut -d[ -f 2 | sort | uniq -c | sort -k 2 -n
7726 0
1715 5
10744 6
4341 11
42792 12
45979 13
38783 14
17253 21
34747 22
1025 28
2079 33
2398 39
3016 44
那么,在1-4秒、7-10秒和14-20秒之间发生了什么
...
[0.350s][7248][debug][class,resolve ] jdk.internal.ref.CleanerFactory$1 java.lang.Thread CleanerFactory.java:45
[0.350s][7248][debug][class,resolve ] jdk.internal.ref.CleanerImpl java.lang.Thread CleanerImpl.java:117
[0.350s][7248][info ][biasedlocking ] Aligned thread 0x000000001727e010 to 0x000000001727e800
[0.350s][7248][info ][os,thread ] Thread started (tid: 2944, attributes: stacksize: default, flags: CREATE_SUSPENDED STACK_SIZE_PARAM_IS)
[0.350s][6884][info ][os,thread ] Thread is alive (tid: 6884).
[0.350s][6884][debug][os,thread ] Thread 6884 stack dimensions: 0x00000000175b0000-0x00000000176b0000 (1024k).
[0.350s][6884][debug][os,thread ] Thread 6884 stack guard pages activated: 0x00000000175b0000-0x00000000175b4000.
[0.350s][7248][debug][thread,smr ] tid=7248: Threads::add: new ThreadsList=0x0000000017254500
[0.350s][7248][debug][thread,smr ] tid=7248: ThreadsSMRSupport::free_list: threads=0x0000000017253d50 is freed.
[0.350s][2944][info ][os,thread ] Thread is alive (tid: 2944).
[0.350s][2944][debug][os,thread ] Thread 2944 stack dimensions: 0x00000000177b0000-0x00000000178b0000 (1024k).
[0.350s][2944][debug][os,thread ] Thread 2944 stack guard pages activated: 0x00000000177b0000-0x00000000177b4000.
[0.351s][2944][debug][class,resolve ] java.lang.Thread java.lang.Runnable Thread.java:832
[0.351s][2944][debug][class,resolve ] jdk.internal.ref.CleanerImpl jdk.internal.misc.InnocuousThread CleanerImpl.java:135
[0.351s][2944][debug][class,resolve ] jdk.internal.ref.CleanerImpl jdk.internal.ref.PhantomCleanable CleanerImpl.java:138
[0.351s][2944][info ][biasedlocking,handshake] JavaThread 0x000000001727e800 handshaking JavaThread 0x000000000286d800 to revoke object 0x00000000c0087f78
[0.351s][2944][debug][vmthread ] Adding VM operation: HandshakeOneThread
[0.351s][6708][debug][vmthread ] Evaluating non-safepoint VM operation: HandshakeOneThread
[0.351s][6708][debug][vmoperation ] begin VM_Operation (0x00000000178af250): HandshakeOneThread, mode: no safepoint, requested by thread 0x000000001727e800
# no log until 5.723s
[5.723s][7248][info ][biasedlocking ] Revoked bias of currently-unlocked object
[5.723s][7248][debug][handshake,task ] Operation: RevokeOneBias for thread 0x000000000286d800, is_vm_thread: false, completed in 94800 ns
[5.723s][7248][debug][class,resolve ] java.util.zip.ZipFile$CleanableResource java.lang.ref.Cleaner ZipFile.java:715
[5.723s][7248][debug][class,resolve ] java.lang.ref.Cleaner jdk.internal.ref.CleanerImpl$PhantomCleanableRef Cleaner.java:220
[5.723s][7248][debug][class,resolve ] java.util.zip.ZipFile$CleanableResource java.util.WeakHashMap ZipFile.java:716
...
第二次暂停稍晚一点:
...
[6.246s][7248][info ][class,load ] java.awt.Graphics source: jrt:/java.desktop
[6.246s][7248][debug][class,load ] klass: 0x0000000100081a00 super: 0x0000000100001080 loader: [loader data: 0x0000000002882bd0 of 'bootstrap'] bytes: 5625 checksum: 0025818f
[6.246s][7248][debug][class,resolve ] java.awt.Graphics java.lang.Object (super)
[6.246s][7248][info ][class,loader,constraints] updating constraint for name java/awt/Graphics, loader 'bootstrap', by setting class object
[6.246s][7248][debug][jit,compilation ] 19 4 java.lang.Object::<init> (1 bytes) made not entrant
[6.246s][7248][debug][vmthread ] Adding VM operation: HandshakeAllThreads
[6.246s][6708][debug][vmthread ] Evaluating non-safepoint VM operation: HandshakeAllThreads
[6.246s][6708][debug][vmoperation ] begin VM_Operation (0x000000000203ddf8): HandshakeAllThreads, mode: no safepoint, requested by thread 0x000000000286d800
[6.246s][6708][debug][handshake,task ] Operation: Deoptimize for thread 0x00000000026b0800, is_vm_thread: true, completed in 1400 ns
[6.246s][6708][debug][handshake,task ] Operation: Deoptimize for thread 0x00000000026bb800, is_vm_thread: true, completed in 700 ns
[6.246s][6708][debug][handshake,task ] Operation: Deoptimize for thread 0x00000000026ef800, is_vm_thread: true, completed in 100 ns
[6.246s][6708][debug][handshake,task ] Operation: Deoptimize for thread 0x00000000026f0800, is_vm_thread: true, completed in 100 ns
[6.246s][6708][debug][handshake,task ] Operation: Deoptimize for thread 0x00000000026f1800, is_vm_thread: true, completed in 100 ns
[6.246s][6708][debug][handshake,task ] Operation: Deoptimize for thread 0x00000000026f4800, is_vm_thread: true, completed in 100 ns
[6.247s][6708][debug][handshake,task ] Operation: Deoptimize for thread 0x0000000002768800, is_vm_thread: true, completed in 100 ns
[6.247s][6708][debug][handshake,task ] Operation: Deoptimize for thread 0x000000000276e000, is_vm_thread: true, completed in 100 ns
[6.247s][6708][debug][handshake,task ] Operation: Deoptimize for thread 0x0000000017268800, is_vm_thread: true, completed in 100 ns
[6.247s][6708][debug][handshake,task ] Operation: Deoptimize for thread 0x000000001727e800, is_vm_thread: true, completed in 800 ns
# no log until 11.783s
[11.783s][6708][debug][handshake,task ] Operation: Deoptimize for thread 0x000000000286d800, is_vm_thread: true, completed in 6300 ns
[11.783s][6708][info ][handshake ] Handshake "Deoptimize", Targeted threads: 11, Executed by targeted threads: 0, Total completion time: 5536442500 ns
[11.783s][6708][debug][vmoperation ] end VM_Operation (0x000000000203ddf8): HandshakeAllThreads, mode: no safepoint, requested by thread 0x000000000286d800
[11.783s][7248][debug][protectiondomain ] Checking package access
[11.783s][7248][debug][protectiondomain ] class loader: a 'jdk/internal/loader/ClassLoaders$AppClassLoader'{0x00000000c0058628} protection domain: a 'java/security/ProtectionDomain'{0x00000000c058b948} loading: 'java/awt/Graphics'
[11.783s][7248][debug][protectiondomain ] granted
[11.783s][7248][debug][class,resolve ] sun.launcher.LauncherHelper java.awt.Graphics LauncherHelper.java:816 (reflection)
[11.783s][7248][debug][class,resolve ] jdk.internal.reflect.Reflection [Ljava.lang.reflect.Method; Reflection.java:300
[11.783s][7248][debug][class,preorder ] java.lang.PublicMethods$MethodList source: C:\Users\example\AppData\Local\example\stable\jdk\lib\modules
...
以下两行似乎很有趣:
[11.783s][6708][info ][handshake ] Handshake "Deoptimize", Targeted threads: 11, Executed by targeted threads: 0, Total completion time: 5536442500 ns
[21.455s][6708][info ][handshake ] Handshake "Deoptimize", Targeted threads: 11, Executed by targeted threads: 0, Total completion time: 6876829000 ns
这些握手需要5.5秒和6.8秒,这正常吗
我在使用以下命令运行时(与我们的应用程序完全无关)也经历了相同的减速(以及类似的日志):
Z:\swing>\jdk-14\bin\java -Xlog:all=debug:file=app.txt:uptime,tid,level,tags:filecount=50 \
-jar update4j-1.4.5.jar --remote http://docs.update4j.org/demo/setup.xml
在单CPU Windows 10设置上,我应该寻找什么使我们的应用程序再次更快?我可以通过更改应用程序中的某些内容或添加JVM参数来解决这个问题吗
这是一个JDK错误,我应该报告它吗
更新2020-04-25:
据我所见,日志文件还包含GC日志。以下是第一批GC日志:
$ cat app.txt.00 | grep "\[gc"
[0.016s][7248][debug][gc,heap ] Minimum heap 8388608 Initial heap 60817408 Maximum heap 1073741824
[0.017s][7248][info ][gc,heap,coops ] Heap address: 0x00000000c0000000, size: 1024 MB, Compressed Oops mode: 32-bit
[0.018s][7248][info ][gc ] Using Serial
[22.863s][6708][info ][gc,start ] GC(0) Pause Young (Allocation Failure)
[22.863s][6708][debug][gc,heap ] GC(0) Heap before GC invocations=0 (full 0): def new generation total 17856K, used 15936K [0x00000000c0000000, 0x00000000c1350000, 0x00000000d5550000)
...
不幸的是,它似乎不相关,因为它在第三次暂停后开始
更新2020-04-26:
使用OpenJDK 14,应用程序在我的(单CPU)VirtualBox机器(在i7-6600U CPU上运行)中使用100%的CPU。虚拟机有3.5 GB RAM。根据TaskManager,40%以上是空闲的,磁盘活动是0%(我想这意味着没有交换)。向虚拟机添加另一个CPU(并为物理机启用超线程)使应用程序再次足够快。我只是想知道,在JDK开发过程中,为了让JVM在多核/超线程CPU上运行得更快,在(罕见的)单CPU机器上损失性能是否是一种有意的权衡?根据我的经验,JDK的性能问题主要与以下之一有关:
- JIT编译
- VM配置(堆大小)
- GC算法
- JVM/JDK中的更改破坏了已知良好运行的应用程序
- (哦,我忘了提到类加载…)
我是否正确理解您的应用程序在运行的第一秒钟内(在OpenJDK11下)将大约30710行写入日志?为什么它在第一秒内“只”在OpenJDK14下写大约7k行?对我来说,对于刚在不同JVM上启动的应用程序来说,这似乎是一个巨大的差异。。。例如,您确定没有大量异常堆栈跟踪转储到日志中吗 其他数字有时甚至更高,所以可能减速与异常日志记录有关?如果RAM变低,甚至交换 实际上我在想,如果一个应用程序没有在日志中写入任何内容,这是一个平稳运行的标志,没有问题(除非它在这段时间完全冻结)。从第12秒到第22秒(在这里的OpenJDK14案例中)发生的事情更让我担心。。。记录的管线穿过屋顶。。。为什么? 之后,记录下降到所有时间的最低点,大约1-2k行。。。原因是什么??(好吧,也许是GC在第22秒踢进来,做了一个表格,解决了一些问题…?) 另一件事可能是你关于“单CPU”机器的陈述。这是否也意味着“单核”(Idk,也许你的软件是在传统硬件或其他东西上定制的) “单CPU”虚拟机在这些机器上运行? 但是我假设,我对这些假设是错误的,因为现在几乎所有的CPU都是多核的。。。但我可能会研究一个多线程问题(死锁)。因为它“大部分时间”都使用100%的CPU,而且Java 14需要10倍的时间(!),这意味着Java 14浪费了90%的CPU 耗尽堆空间可以做到这一点,因为您在GC中花费了大量时间,但您似乎已经排除了这一点 我注意到,您正在调整偏置锁定选项,这会产生显著的差异。这告诉我,您的程序可能在多个线程中执行大量并发工作。您的程序可能存在Java 14中出现的并发错误,但Java 10中没有。这也可以解释为什么添加另一个CPU会使速度提高一倍以上 并发错误通常只在
$ cat app.txt.00 | grep "\[gc"
[0.016s][7248][debug][gc,heap ] Minimum heap 8388608 Initial heap 60817408 Maximum heap 1073741824
[0.017s][7248][info ][gc,heap,coops ] Heap address: 0x00000000c0000000, size: 1024 MB, Compressed Oops mode: 32-bit
[0.018s][7248][info ][gc ] Using Serial
[22.863s][6708][info ][gc,start ] GC(0) Pause Young (Allocation Failure)
[22.863s][6708][debug][gc,heap ] GC(0) Heap before GC invocations=0 (full 0): def new generation total 17856K, used 15936K [0x00000000c0000000, 0x00000000c1350000, 0x00000000d5550000)
...
public class Main {
public static void main(String[] args) {
System.out.println("Hello world");
}
}
c:\windows\system32\cmd.exe /C start /affinity 1 \
\jdk-14\bin\java \
-Xlog:all=trace:file=app-1cpu.txt:uptime,tid,level,tags:filecount=50 \
Main
c:\windows\system32\cmd.exe /C start /affinity FF \
\jdk-14\bin\java \
-Xlog:all=trace:file=app-full.txt:uptime,tid,level,tags:filecount=50 \
Main
PS Z:\main> Measure-Command { .\main-1cpu.bat }
...
TotalSeconds : 7.0203455
...
PS Z:\main> Measure-Command { .\main-full.bat }
...
TotalSeconds : 1.5751352
...
PS Z:\main> Measure-Command { .\main-full.bat }
...
TotalSeconds : 1.5585384
...
PS Z:\main> Measure-Command { .\main-1cpu.bat }
...
TotalSeconds : 23.6482685
...