使用Java进行硬盘基准测试,获得不合理的快速结果

使用Java进行硬盘基准测试,获得不合理的快速结果,java,benchmarking,fileinputstream,fileoutputstream,hard-drive,Java,Benchmarking,Fileinputstream,Fileoutputstream,Hard Drive,我写了一段代码来测试硬盘。它相当简单:使用BufferedOutputStream将大量字节(2500*10M)写入磁盘,然后使用BufferedInputStream读取。为了模拟我编写的另一个程序中的条件,我每次写入2500字节,持续10万次。还有另一个全局变量“无意义”,它是用读取的字节计算的-它绝对没有意义,只用于强制执行实际读取和使用字节(以避免由于某些优化而无法读取字节的情况) 代码运行4次并输出结果 这是: import java.io.*; public class Drive

我写了一段代码来测试硬盘。它相当简单:使用BufferedOutputStream将大量字节(2500*10M)写入磁盘,然后使用BufferedInputStream读取。为了模拟我编写的另一个程序中的条件,我每次写入2500字节,持续10万次。还有另一个全局变量“无意义”,它是用读取的字节计算的-它绝对没有意义,只用于强制执行实际读取和使用字节(以避免由于某些优化而无法读取字节的情况)

代码运行4次并输出结果

这是:

import java.io.*;

public class DriveTest
{
    public static long meaningless = 0;

    public static String path = "C:\\test";

    public static int chunkSize = 2500;

    public static int iterations = 10000000;

    public static void main(String[] args)
    {
        try
        {
            for(int i = 0; i < 4; i++)
            {
                System.out.println("Test " + (i + 1) + ":");
                System.out.println("==================================");

                write();
                read();

                new File(path).delete();

                System.out.println("==================================");
            }
        }
        catch(Exception e)
        {
            e.printStackTrace();
        }
    }

    private static void write() throws Exception
    {
        BufferedOutputStream bos = new BufferedOutputStream(
                                   new FileOutputStream(new File(path)));

        long t1 = System.nanoTime();

        for(int i = 0; i < iterations; i++)
        {
            byte[] data = new byte[chunkSize];

            for(int j = 0; j < data.length; j++)
            {
                data[j] = (byte)(j % 127);
            }

            bos.write(data);
        }

        bos.close();

        long t2 = System.nanoTime();

        double seconds = ((double)(t2 - t1) / 1000000000.0);

        System.out.println("Writing took " + (t2 - t1) + 
                           " ns (" + seconds + " seconds).");

        System.out.println("Write rate " + (((double)chunkSize * 
                           iterations / seconds) / 
                           (1024.0 * 1024.0)) + " MB/s.");
    }

    private static void read() throws Exception
    {
        BufferedInputStream bis = new BufferedInputStream(
                                  new FileInputStream(new File(path)));

        long t1 = System.nanoTime();

        byte[] data;

        for(int i = 0; i < iterations; i++)
        {
            data = new byte[chunkSize];

            bis.read(data);

            meaningless += data[i % chunkSize];
        }

        bis.close();

        long t2 = System.nanoTime();

        System.out.println("meaningless is: " + meaningless + ".");

        double seconds = ((double)(t2 - t1) / 1000000000.0);

        System.out.println("Reading Took " + (t2 - t1) + 
                           " ns, which is " + 
                           seconds + " seconds.");

        System.out.println("Read rate " + (((double)chunkSize * 
                           iterations / seconds) / 
                           (1024.0 * 1024.0)) + " MB/s.");
    }
}
这似乎很奇怪-磁盘真的能达到这样的速度吗?我严重怀疑,考虑到经过测试的规范(甚至不是在java输出/输入流下,在我的新手看来,java输出/输入流不应该是最优的!)较低:

  • 当迭代次数设置为1M(1000000)时,数字会变得非常疯狂:
  • _

    这是什么样的魔法??(什么样的缓存会导致写入速度加快??)如何撤消?我已经读写了2.3GB的文件!如果这确实是个问题的话,这是一个巨大的缓存


    谢谢

    您的测试可能只是读取和写入操作系统页面缓存。小数据量将完全适合。较大的一个不会,但会被操作系统异步刷新。您应该尝试使用OpenOptions.DSYNC和SYNC选项

    您的顺序写入速率看起来很合理。我有一个5900rpm的外部硬盘,通常性能约为120MB/s。您检查过系统的FS缓存吗?我假设您的系统的容量在>2.3到16GB之间:),但缓存不能解释写入结果,对吗?您需要定义。在进行写入测量之前需要关闭流,因为除非您关闭(或刷新)它,否则它可能不会将其刷新到磁盘。不过,这并不能解释读取性能……谢谢,我只是修改了它(在计算第二个时间点t2之前将关闭流移动到),但结果保持不变。对于普通硬盘和2.3GB文件,我的写入速度约为325 MB/s,读取速度约为1150 MB/s。真奇怪,谢谢你。这似乎确实是一个缓存问题(异步写入可能解释了加速的原因)。我正在监控RAM的使用情况,事实上,随着程序的每次迭代,RAM的“修改”量达到峰值,约为1.5GB,然后再次下降。如何将StandardOpenOptions枚举与缓冲流一起使用?我应该把它放在哪里,或者静态地将它设置在哪里?@amirkr:
    write()
    系统调用可以并且确实会在数据进入页面缓存时返回。他们不会等到数据到达磁盘,因为有很好的理由不立即启动对磁盘的写入。(例如,可能它是一个即将被删除的tmp文件,在这种情况下,如果它的数据从未到达磁盘就可以了。)@PeterCordes但这种情况下的写入速度不是很快吗?如果数据只是在高速缓存中,即在超快的主内存中,写操作将在几分之一秒内结束。这里有太多的变量(缓冲、缓存、操作系统限制、硬盘缓存、硬盘吞吐量、java对象缓冲区……几乎不可能估计速度:)),我仍然不清楚DSYNC和SYNC参数(java.nio的一部分)如何与缓冲流(java.io)一起使用。有什么想法吗?@amirkr他们不能。您必须使用文件通道。即使这样,也只能解决你的写作问题。我认为,除非先刷新页面缓存,否则读取仍会命中页面缓存。
    Test 1:
    Writing took 148738975163 ns (148.738975163 seconds).
    Write rate 160.29327810029918 MB/s.
    meaningless is: 1246080000.
    Reading Took 139143051529 ns, which is 139.143051529 seconds.
    Read rate 171.34781541848795 MB/s.
    
    Test 2:
    Writing took 146591885655 ns (146.591885655 seconds).
    Write rate 162.64104799270686 MB/s.
    meaningless is: 1869120000.
    Reading Took 139845492688 ns, which is 139.845492688 seconds.
    Read rate 170.48713871206587 MB/s.
    
    Test 3:
    Writing took 152049678671 ns (152.049678671 seconds).
    Write rate 156.8030798785472 MB/s.
    meaningless is: 2492160000.
    Reading Took 140152776858 ns, which is 140.152776858 seconds.
    Read rate 170.11334662539255 MB/s.
    
    Test 4:
    Writing took 151363950081 ns (151.363950081 seconds).
    Write rate 157.51344951950355 MB/s.
    meaningless is: 3115200000.
    Reading Took 139176911081 ns, which is 139.176911081 seconds.
    Read rate 171.30612919179143 MB/s.
    
    Test 1:
    Writing took 6918084976 ns (6.918084976 seconds).
    Write rate 344.6308912490619 MB/s.
    meaningless is: 62304000.
    Reading Took 2060226375 ns, which is 2.060226375 seconds.
    Read rate 1157.244572706543 MB/s.
    
    Test 2:
    Writing took 6970893036 ns (6.970893036 seconds).
    Write rate 342.0201369756931 MB/s.
    meaningless is: 124608000.
    Reading Took 2013661185 ns, which is 2.013661185 seconds.
    Read rate 1184.0054368508995 MB/s.
    
    Test 3:
    Writing took 7140592101 ns (7.140592101 seconds).
    Write rate 333.89188981705496 MB/s.
    meaningless is: 186912000.
    Reading Took 2011346987 ns, which is 2.011346987 seconds.
    Read rate 1185.367719456367 MB/s.
    
    Test 4:
    Writing took 7140064035 ns (7.140064035 seconds).
    Write rate 333.91658384694375 MB/s.
    meaningless is: 249216000.
    Reading Took 2041787713 ns, which is 2.041787713 seconds.
    Read rate 1167.6952387535623 MB/s.