Java 在大型迭代中划分循环

Java 在大型迭代中划分循环,java,performance,optimization,arraylist,nested-loops,Java,Performance,Optimization,Arraylist,Nested Loops,我的问题的目标是通过在一个大数组列表上拆分循环迭代的范围来提高算法的性能。 例如:我有一个数组列表,大小约为100亿个长值条目,我试图实现的目标是从0到1亿个条目开始循环,输出循环中任何计算的1亿个条目的结果;然后开始,1亿到2亿做上一步并输出结果,然后3亿到4亿,4亿到5亿,依此类推。 在我得到所有1000亿/1亿个结果之后,我可以在循环外汇总它们,从循环输出并行收集结果 我曾尝试使用一个范围,通过尝试使用动态范围转换方法,可能可以实现类似的效果,但我似乎无法像我希望的那样完全实现逻辑 pub

我的问题的目标是通过在一个大数组列表上拆分循环迭代的范围来提高算法的性能。
例如:我有一个数组列表,大小约为100亿个长值条目,我试图实现的目标是从0到1亿个条目开始循环,输出循环中任何计算的1亿个条目的结果;然后开始,1亿到2亿做上一步并输出结果,然后3亿到4亿,4亿到5亿,依此类推。 在我得到所有1000亿/1亿个结果之后,我可以在循环外汇总它们,从循环输出并行收集结果

我曾尝试使用一个范围,通过尝试使用动态范围转换方法,可能可以实现类似的效果,但我似乎无法像我希望的那样完全实现逻辑

public static void tt4() {
    long essir2 = 0;
    long essir3 = 0;

    List cc = new ArrayList<>();  
    List<Long> range = new ArrayList<>();  

    // break point is a method that returns list values, it was converted to 
    // string because of some concatenations and would be converted back to long here
    for (String ari1 : Breakpoint()) {
        cc.add(Long.valueOf(ari1));
    }  

    // the size of the List is huge about 1 trillion entries at the minimum
    long hy = cc.size() - 1;

    for (long k = 0; k < hy; k++) { 
        long t1 = (long) cc.get((int) k);
        long t2 = (long) cc.get((int) (k + 1)); 

        // My main question: I am trying to iterate the entire list in a dynamic way 
        // which would exclude repeated endpoints on each iteration. 

        range = LongStream.rangeClosed(t1 + 1, t2)
                          .boxed()
                          .collect(Collectors.toList());  

        for (long i : range) {
            // Hard is another method call on the iteration
            // complexcalc is a method as well

            essir2 = complexcalc((int) i, (int) Hard(i)); 
            essir3 += essir2;
        } 
    }

    System.out.println("\n" + essir3);  
}
publicstaticvoidtt4(){
长径比r2=0;
长ESIR3=0;
List cc=new ArrayList();
列表范围=新的ArrayList();
//断点是一个返回列表值的方法,它被转换为
//字符串,因为有一些连接,在这里会转换回long
for(字符串ari1:Breakpoint()){
cc.add(长期价值(ari1));
}  

//这个列表的规模是巨大的,至少有1万亿个条目 长hy=cc.size()-1; 对于(长k=0;k

我没有任何错误,我只是在寻找提高性能和时间的方法。我可以在一秒钟内直接输入一百万个条目,但当我输入所需的大小时,它将永远运行。我给出的尺寸是用来说明尺寸大小的摘要,我不想看到像1000亿这样的观点,如果我能在一秒钟内完成100万次,我要说的是大量的数字,我需要迭代完成复杂的任务和调用,如果可以的话,我只需要获得我试图实现的逻辑方面的帮助。

我建议立即将
断点
返回值存储在一个简单数组中,而不是使用
列表
。这将大大缩短您的执行时间:

    List<Long> cc = new ArrayList<>();
    for (String ari1 : Breakpoint()) {
        cc.add(Long.valueOf(ari1));
    }
    Long[] ccArray = cc.toArray(new Long[0]);
List cc=new ArrayList();
for(字符串ari1:Breakpoint()){
cc.add(长期价值(ari1));
}
Long[]ccArray=cc.toArray(新Long[0]);
我相信你想要的是将你的任务分成多个线程。您可以通过“简化异步模式下任务的执行”来实现这一点

请注意,我对整个概念不太熟悉,但最近对它进行了一些实验,并为您提供了如何实现这一点的快速草案

我欢迎那些更有多线程经验的人来纠正这篇文章,或者在评论中提供更多信息来帮助改进这个答案

可运行任务类

public class CompartmentalizationTask implements Runnable {

    private final ArrayList<Long> cc;
    private final long index;

    public CompartmentalizationTask(ArrayList<Long> list, long index) {

        this.cc = list;
        this.index = index;
    }

    @Override
    public void run() {
        Main.compartmentalize(cc, index);
    }
}
private static ExecutorService exeService = Executors.newCachedThreadPool();
private static List<Future> futureTasks = new ArrayList<>();

public static void tt4() throws ExecutionException, InterruptedException 
{
    long essir2 = 0;
    long essir3 = 0;

    ArrayList<Long> cc = new ArrayList<>();
    List<Long> range = new ArrayList<>();

    // break point is a method that returns list values, it was converted to
    // string because of some concatenations and would be converted back to long here
    for (String ari1 : Breakpoint()) {
        cc.add(Long.valueOf(ari1));
    }

    // the size of the List is huge about 1 trillion entries at the minimum
    long hy = cc.size() - 1;

    for (long k = 0; k < hy; k++) {
        futureTasks.add(Main.exeService.submit(new CompartmentalizationTask(cc, k)));
    }
    for (int i = 0; i < futureTasks.size(); i++) {
        futureTasks.get(i).get();
    }
    exeService.shutdown();
}

public static void compartmentalize(ArrayList<Long> cc, long index)
{
    long t1 = (long) cc.get((int) index);
    long t2 = (long) cc.get((int) (index + 1));

    // My main question: I am trying to iterate the entire list in a dynamic way
    // which would exclude repeated endpoints on each iteration.

    range = LongStream.rangeClosed(t1 + 1, t2)
            .boxed()
            .collect(Collectors.toList());

    for (long i : range) {
        // Hard is another method call on the iteration
        // complexcalc is a method as well

        essir2 = complexcalc((int) i, (int) Hard(i));
        essir3 += essir2;
    }
}
公共类划分任务实现可运行{
私人最终ArrayList cc;
私人最终长期指数;
公共分区任务(ArrayList列表,长索引){
this.cc=列表;
这个指数=指数;
}
@凌驾
公开募捐{
主要。划分(cc,索引);
}
}
主类

public class CompartmentalizationTask implements Runnable {

    private final ArrayList<Long> cc;
    private final long index;

    public CompartmentalizationTask(ArrayList<Long> list, long index) {

        this.cc = list;
        this.index = index;
    }

    @Override
    public void run() {
        Main.compartmentalize(cc, index);
    }
}
private static ExecutorService exeService = Executors.newCachedThreadPool();
private static List<Future> futureTasks = new ArrayList<>();

public static void tt4() throws ExecutionException, InterruptedException 
{
    long essir2 = 0;
    long essir3 = 0;

    ArrayList<Long> cc = new ArrayList<>();
    List<Long> range = new ArrayList<>();

    // break point is a method that returns list values, it was converted to
    // string because of some concatenations and would be converted back to long here
    for (String ari1 : Breakpoint()) {
        cc.add(Long.valueOf(ari1));
    }

    // the size of the List is huge about 1 trillion entries at the minimum
    long hy = cc.size() - 1;

    for (long k = 0; k < hy; k++) {
        futureTasks.add(Main.exeService.submit(new CompartmentalizationTask(cc, k)));
    }
    for (int i = 0; i < futureTasks.size(); i++) {
        futureTasks.get(i).get();
    }
    exeService.shutdown();
}

public static void compartmentalize(ArrayList<Long> cc, long index)
{
    long t1 = (long) cc.get((int) index);
    long t2 = (long) cc.get((int) (index + 1));

    // My main question: I am trying to iterate the entire list in a dynamic way
    // which would exclude repeated endpoints on each iteration.

    range = LongStream.rangeClosed(t1 + 1, t2)
            .boxed()
            .collect(Collectors.toList());

    for (long i : range) {
        // Hard is another method call on the iteration
        // complexcalc is a method as well

        essir2 = complexcalc((int) i, (int) Hard(i));
        essir3 += essir2;
    }
}
private static ExecutorService exeService=Executors.newCachedThreadPool();
私有静态列表futureTasks=newArrayList();
public static void tt4()引发ExecutionException、InterruptedException
{
长径比r2=0;
长ESIR3=0;
ArrayList cc=新的ArrayList();
列表范围=新的ArrayList();
//断点是一个返回列表值的方法,它被转换为
//字符串,因为有一些连接,在这里会转换回long
for(字符串ari1:Breakpoint()){
cc.add(长期价值(ari1));
}

//这个列表的规模是巨大的,至少有1万亿个条目 长hy=cc.size()-1; 对于(长k=0;k
我建议将代码的格式设置得更好一点,这样更具可读性。这样会更容易发现潜在的未优化代码。“列表的大小非常大,至少有1万亿个条目”-1万亿个元素?我不这么认为……这是15-20次方的值。也就是说>=x^151万亿盒装原语需要至少20 TB的堆空间