Java 使用BufferedReader读取大型文本文件时,打印每一行的速度会有多慢?
我创建了一个程序来读取一个非常大的文本文件(2.5 GB),然后将每一行重新写入一个新的文本文件,格式如我所愿。一切都进行得很顺利,但当我编写它时,我在其中放了一个小“println”,用于测试目的,以查看它是否正常工作。一切都进行得很顺利,所以我启动了它,离开了计算机,3小时后回来,看到我在那里留下了调试消息。我的问题是,如果我正在读/写大约46000000行,这会让我的速度慢到什么程度?如果要将其删除到目前为止,是否明智?好吧,我想这大概是ETA的两倍,因为您基本上是在日志中创建另一个文件副本;-) 检查您已经完成了多少,然后考虑终止进程……调试的额外成本Java 使用BufferedReader读取大型文本文件时,打印每一行的速度会有多慢?,java,performance,bufferedreader,bufferedwriter,Java,Performance,Bufferedreader,Bufferedwriter,我创建了一个程序来读取一个非常大的文本文件(2.5 GB),然后将每一行重新写入一个新的文本文件,格式如我所愿。一切都进行得很顺利,但当我编写它时,我在其中放了一个小“println”,用于测试目的,以查看它是否正常工作。一切都进行得很顺利,所以我启动了它,离开了计算机,3小时后回来,看到我在那里留下了调试消息。我的问题是,如果我正在读/写大约46000000行,这会让我的速度慢到什么程度?如果要将其删除到目前为止,是否明智?好吧,我想这大概是ETA的两倍,因为您基本上是在日志中创建另一个文件副
println
非常依赖于平台
如果您的PrintStream
被路由到控制台,则与控制台相关的额外处理可能会降低您的速度。在Windows上,默认控制台的速度非常慢,在Linux上则非常慢
此外,如果您使用的是System.out
/System.err
您的PrintStream
将配置为在每次println
后刷新,从而增加额外的开销
3小时处理磁盘中的2.5 GiB非常慢。因此,要么你的代码很复杂,额外的调试行对整体性能影响很小,要么你在Windows上写入默认控制台(在以后的情况下,调试
println
是一个巨大的问题)。整个过程有两个原因-
Time took by CPU to copy a line = 1 microsecond
Time took by IO to paste a line = 1 microsecond
Time took by CPU to paste a line = 1 microsecond
Waiting time of CPU for every line = 1 microsecond
Total time taken by CPU for copying 46,000,000 lines = 46,000,000 microsecond = 46 sec
Total time taken by IO for copying 46,000,000 lines = 46,000,000 microsecond = 46 sec
Waiting time of CPU = 46,000,000 microsecond = 46 sec
Total time by CPU for pasting 46,000,000 lines = 46,000,000 microsecond = 46 sec
Total time of the entire process = 184 sec.
希望这能回答你的疑问
Time took by CPU to copy a line = 1 microsecond
Time took by IO to paste a line = 1 microsecond
Time took by CPU to paste a line = 1 microsecond
Waiting time of CPU for every line = 1 microsecond
Total time taken by CPU for copying 46,000,000 lines = 46,000,000 microsecond = 46 sec
Total time taken by IO for copying 46,000,000 lines = 46,000,000 microsecond = 46 sec
Waiting time of CPU = 46,000,000 microsecond = 46 sec
Total time by CPU for pasting 46,000,000 lines = 46,000,000 microsecond = 46 sec
Total time of the entire process = 184 sec.