Java：将N个2d数组附加到单个2d数组中的有效方法
提前谢谢你的帮助 我有N个二维数组，它们的维数完全相同。我想把它们组合成一个二维数组。下面是一个仅包含2个二维阵列的示例Java：将N个2d数组附加到单个2d数组中的有效方法,java,arrays,performance,multidimensional-array,Java,Arrays,Performance,Multidimensional Array,提前谢谢你的帮助 我有N个二维数组，它们的维数完全相同。我想把它们组合成一个二维数组。下面是一个仅包含2个二维阵列的示例 array1 = [[1 2] [3 4] [5 6]] array2 = [[7 8] [9 1] [2 3]] result = [[1 2 7 8] [3 4 9 1] [5 6 2 3]] 最有效的方法是什么？这些阵列可能相当大，在某些情况下约
array1 = [[1 2]
[3 4]
[5 6]]
array2 = [[7 8]
[9 1]
[2 3]]
result = [[1 2 7 8]
[3 4 9 1]
[5 6 2 3]]
最有效的方法是什么？这些阵列可能相当大，在某些情况下约为20x10000。最简单的方法是用于循环，但这肯定是最低效的，特别是因为我希望相当频繁地执行此操作。我怀疑我也可能使用java的一些内置方法数组类？。然而，可能有许多不同的方法来做到这一点。记住这一点，最有效的方法是什么？在我看来，将for循环与此类数组一起使用比使用流或列表更好 但这只是我的意见 我建议使用多线程，这对于小数组来说是一种过度杀伤力，因为线程创建开销太大了。即使使用大小为20*10000的数组，多线程也可能是一种过度杀伤力；但是，如果需要多次执行，则可以使用executorService。但这取决于你的需要 下面是一个例子：
public static void main(String[] args) {
test(100, 20, 3);
test(7, 3, 4);
test(3,7,4);
}
private static void test(int outerSize, int innerSize, int numberOfArrays) {
int[][][] arrays = new int[numberOfArrays][outerSize][innerSize];
int[][] resultArray;
int counter = 0;
System.out.println("Testing " + numberOfArrays + " arrays, " + outerSize + " by " + innerSize);
for (int arrayIndex = 0; arrayIndex < numberOfArrays; arrayIndex++)
for (int outerIndex = 0; outerIndex < outerSize; outerIndex++) {
for (int innerIndex = 0; innerIndex < innerSize; innerIndex++) {
arrays[arrayIndex][outerIndex][innerIndex] = counter++;
}
}
// Change number of threads here;
resultArray = new ArrayCombiner(5, arrays).combine();
System.out.println(Arrays.deepToString(resultArray));
}
static class ArrayCombiner {
private final int[][][] sources;
private final int[][] resultArray;
private final int innerSourceLength, outerSourceLength, numberOfThreads;
public ArrayCombiner(int numberOfThreads, int[][]... sources) {
this.sources = sources;
this.numberOfThreads = numberOfThreads;
resultArray = new int[outerSourceLength = sources[0].length][(innerSourceLength = sources[0][0].length)
* sources.length];
}
public int[][] combine() {
if (numberOfThreads <= 1) {
combinePortion(0, outerSourceLength);
} else {
Thread[] threads = new Thread[numberOfThreads];
for (int i = 0; i < numberOfThreads; i++) {
(threads[(int) i] = new Thread(runnableToCombinePortion(i))).start();
}
for (int i = 0; i < numberOfThreads; i++) {
try {
threads[i].join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
return resultArray;
}
private Runnable runnableToCombinePortion(int threadIndex) {
int outerFrom = (int) ((float) threadIndex / numberOfThreads * outerSourceLength),
outerTo = (int) ((float) (1 + threadIndex) / numberOfThreads * outerSourceLength);
return () -> {
combinePortion(outerFrom, outerTo);
};
}
private void combinePortion(int outerFrom, int outerTo) {
for (int outerIndex = outerFrom; outerIndex < outerTo; outerIndex++) {
for (int sourceIndex = 0; sourceIndex < sources.length; sourceIndex++) {
System.arraycopy(sources[sourceIndex][outerIndex], 0, resultArray[outerIndex],
sourceIndex * innerSourceLength, innerSourceLength);
}
}
}
}
static int[][] append(int[][]... matrices) {
int size = matrices.length;
int rows = matrices[0].length;
int cols = matrices[0][0].length;
int[][] result = new int[rows][cols * size];
for (int i = 0; i < rows; ++i)
for (int j = 0, k = 0; j < size; ++j, k += cols)
System.arraycopy(matrices[j][i], 0, result[i], k, cols);
return result;
}
[1, 2, 7, 8]
[3, 4, 9, 1]
[5, 6, 2, 3]
数组可以解释为具有行和列的矩阵。目标是创建一个结果矩阵，其中每一行是所有输入矩阵的对应行的串联 对于每一行，基本上可以分为两个步骤： 从所有输入数组中选择相应的行 将这些行合并为一个结果行 所以问题的核心是：将多个数组连接成一个数组最有效的方法是什么？这反过来又可以被看作是这个问题的一个概括：连接两个数组最有效的方法是什么 对于基本数组，例如int[]数组，我可以想到三种基本方法： 使用System.arraycopy 使用IntStream 凭直觉，我会把赌注押在System.arraycopy上。它基本上没有开销，可以归结为计算机可以执行的最基本的操作之一，即：将内存从这里复制到那里 旁注：在您的特定情况下，还有另一个可能的优化选项。即，对所有并行行调用此方法。但是，由于操作仅限于内存，并且内存传输速度在很大程度上与CPU的数量无关，因此这可能没有明显的影响 下面是一个比较这三种方法的示例 这不是一个完全可靠的基准 但它考虑了一些微基准的最佳实践，并给出了人们可以预期的性能的粗略估计：
import java.nio.IntBuffer;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Locale;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.function.Function;
import java.util.stream.IntStream;
import java.util.stream.Stream;
public class ArraycopyStreamPerformance
{
public static void main(String[] args)
{
basicTest();
int runs = 100;
int minNum = 2;
int maxNum = 8;
int minRows = 2;
int maxRows = 20;
int minCols = 100;
int maxCols = 10000;
for (int num = minNum; num <= maxNum; num *= 2)
{
for (int rows = minRows; rows <= maxRows; rows += 2)
{
for (int cols = minCols; cols <= maxCols; cols *= 10)
{
runTest(num, rows, cols, runs);
}
}
}
}
private static void runTest(int num, int rows, int cols, int runs)
{
int arrays[][][] = new int[num][rows][cols];
long before = 0;
long after = 0;
int blackHole = 0;
// arraycopy
before = System.nanoTime();
for (int i = 0; i < runs; i++)
{
int resultA[][] = combineRows(
ArraycopyStreamPerformance::combineWithArraycopy, arrays);
blackHole += resultA[0][0];
}
after = System.nanoTime();
System.out.printf(Locale.ENGLISH,
"%2d arrays, %3d rows, %6d cols, arraycopy : %8.3fms\n",
num, rows, cols, (after - before) / 1e6);
// arraycopy parallel
before = System.nanoTime();
for (int i = 0; i < runs; i++)
{
int resultA[][] = combineRowsParallel(
ArraycopyStreamPerformance::combineWithArraycopy, arrays);
blackHole += resultA[0][0];
}
after = System.nanoTime();
System.out.printf(Locale.ENGLISH,
"%2d arrays, %3d rows, %6d cols, arraycopy parallel: %8.3fms\n",
num, rows, cols, (after - before) / 1e6);
// buffer
before = System.nanoTime();
for (int i = 0; i < runs; i++)
{
int resultB[][] = combineRows(
ArraycopyStreamPerformance::combineWithBuffer, arrays);
blackHole += resultB[0][0];
}
after = System.nanoTime();
System.out.printf(Locale.ENGLISH,
"%2d arrays, %3d rows, %6d cols, buffer : %8.3fms\n",
num, rows, cols, (after - before) / 1e6);
// buffer parallel
before = System.nanoTime();
for (int i = 0; i < runs; i++)
{
int resultB[][] = combineRowsParallel(
ArraycopyStreamPerformance::combineWithBuffer, arrays);
blackHole += resultB[0][0];
}
after = System.nanoTime();
System.out.printf(Locale.ENGLISH,
"%2d arrays, %3d rows, %6d cols, buffer parallel: %8.3fms\n",
num, rows, cols, (after - before) / 1e6);
// streams
before = System.nanoTime();
for (int i = 0; i < runs; i++)
{
int resultC[][] = combineRows(
ArraycopyStreamPerformance::combineWithStreams, arrays);
blackHole += resultC[0][0];
}
after = System.nanoTime();
System.out.printf(Locale.ENGLISH,
"%2d arrays, %3d rows, %6d cols, stream : %8.3fms (" +
blackHole + ")\n", num, rows, cols, (after - before) / 1e6);
}
private static void basicTest()
{
int array1[][] =
{
{ 1, 2 },
{ 3, 4 },
{ 5, 6 }
};
int array2[][] =
{
{ 7, 8 },
{ 9, 1 },
{ 2, 3 }
};
int result[][] =
{
{ 1, 2, 7, 8 },
{ 3, 4, 9, 1 },
{ 5, 6, 2, 3 }
};
System.out.println(Arrays.deepToString(result));
int resultA[][] = combineRows(
ArraycopyStreamPerformance::combineWithArraycopy, array1, array2);
System.out.println(Arrays.deepToString(resultA));
int resultB[][] = combineRows(
ArraycopyStreamPerformance::combineWithBuffer, array1, array2);
System.out.println(Arrays.deepToString(resultB));
int resultC[][] = combineRows(
ArraycopyStreamPerformance::combineWithStreams, array1, array2);
System.out.println(Arrays.deepToString(resultC));
}
private static int[][] selectRows(int row, int[][]... arrays)
{
int result[][] = new int[arrays.length][];
for (int j = 0; j < arrays.length; j++)
{
result[j] = arrays[j][row];
}
return result;
}
private static int[][] combineRows(
Function<int[][], int[]> mergeFunction, int[][]... arrays)
{
int rows = arrays[0].length;
int result[][] = new int[rows][];
for (int i = 0; i < rows; i++)
{
result[i] = mergeFunction.apply(selectRows(i, arrays));
}
return result;
}
private static int[] combineWithArraycopy(int[]... arrays)
{
// Assuming the same length for all arrays!
int length = arrays[0].length;
int result[] = new int[arrays.length * length];
for (int i = 0; i < arrays.length; i++)
{
System.arraycopy(arrays[i], 0, result, i * length, length);
}
return result;
}
private static int[] combineWithBuffer(int[]... arrays)
{
// Assuming the same length for all arrays!
int length = arrays[0].length;
int result[] = new int[arrays.length * length];
IntBuffer buffer = IntBuffer.wrap(result);
for (int i = 0; i < arrays.length; i++)
{
buffer.put(arrays[i]);
}
return result;
}
private static int[] combineWithStreams(int[] ... arrays)
{
return Stream.of(arrays).flatMapToInt(IntStream::of).toArray();
}
private static final ExecutorService EXECUTOR_SERVICE =
createFixedTimeoutExecutorService(
Runtime.getRuntime().availableProcessors(), 5, TimeUnit.SECONDS);
public static ExecutorService createFixedTimeoutExecutorService(
int poolSize, long keepAliveTime, TimeUnit timeUnit)
{
ThreadPoolExecutor e =
new ThreadPoolExecutor(poolSize, poolSize,
keepAliveTime, timeUnit, new LinkedBlockingQueue<Runnable>());
e.allowCoreThreadTimeOut(true);
return e;
}
private static int[][] combineRowsParallel(
Function<int[][], int[]> mergeFunction, int[][]... arrays)
{
int rows = arrays[0].length;
int result[][] = new int[rows][];
List<Callable<Object>> tasks = new ArrayList<Callable<Object>>();
for (int i = 0; i < rows; i++)
{
int index = i;
tasks.add(Executors.callable(() ->
{
result[index] = mergeFunction.apply(selectRows(index, arrays));
}));
}
try
{
EXECUTOR_SERVICE.invokeAll(tasks);
}
catch (InterruptedException e)
{
Thread.currentThread().interrupt();
}
return result;
}
}
YMMV。如果有人有耐心做一个JMH跑步，我会很感激的 您可以使用一些索引技巧来访问初始数组并完全避免数组复制您仍然需要一个外部循环，但是System.arraycopy，或者如您所述，array类的任何方法都应该被证明是干净和高效的。一个很好的方法是将每个数组转换为一个流，将它们连接成一个大流，如果需要数组作为输出，只需使用StreamtoArray即可。@JacobG。这是否比使用System.arraycopy或Array.copyofrange和for循环更有效？这取决于具体情况。如果你愿意牺牲秩序，手术可以并行进行。阵列是否总是2x3？回答得很好！非常感谢你
private static int[] combineWithArraycopy(int[]... arrays)
{
// Assuming the same length for all arrays!
int length = arrays[0].length;
int result[] = new int[arrays.length * length];
for (int i = 0; i < arrays.length; i++)
{
System.arraycopy(arrays[i], 0, result, i * length, length);
}
return result;
}
private static int[] combineWithBuffer(int[]... arrays)
{
// Assuming the same length for all arrays!
int length = arrays[0].length;
int result[] = new int[arrays.length * length];
IntBuffer buffer = IntBuffer.wrap(result);
for (int i = 0; i < arrays.length; i++)
{
buffer.put(arrays[i]);
}
return result;
}
private static int[] combineWithStreams(int[] ... arrays)
{
return Stream.of(arrays).flatMapToInt(IntStream::of).toArray();
}
import java.nio.IntBuffer;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Locale;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.function.Function;
import java.util.stream.IntStream;
import java.util.stream.Stream;
public class ArraycopyStreamPerformance
{
public static void main(String[] args)
{
basicTest();
int runs = 100;
int minNum = 2;
int maxNum = 8;
int minRows = 2;
int maxRows = 20;
int minCols = 100;
int maxCols = 10000;
for (int num = minNum; num <= maxNum; num *= 2)
{
for (int rows = minRows; rows <= maxRows; rows += 2)
{
for (int cols = minCols; cols <= maxCols; cols *= 10)
{
runTest(num, rows, cols, runs);
}
}
}
}
private static void runTest(int num, int rows, int cols, int runs)
{
int arrays[][][] = new int[num][rows][cols];
long before = 0;
long after = 0;
int blackHole = 0;
// arraycopy
before = System.nanoTime();
for (int i = 0; i < runs; i++)
{
int resultA[][] = combineRows(
ArraycopyStreamPerformance::combineWithArraycopy, arrays);
blackHole += resultA[0][0];
}
after = System.nanoTime();
System.out.printf(Locale.ENGLISH,
"%2d arrays, %3d rows, %6d cols, arraycopy : %8.3fms\n",
num, rows, cols, (after - before) / 1e6);
// arraycopy parallel
before = System.nanoTime();
for (int i = 0; i < runs; i++)
{
int resultA[][] = combineRowsParallel(
ArraycopyStreamPerformance::combineWithArraycopy, arrays);
blackHole += resultA[0][0];
}
after = System.nanoTime();
System.out.printf(Locale.ENGLISH,
"%2d arrays, %3d rows, %6d cols, arraycopy parallel: %8.3fms\n",
num, rows, cols, (after - before) / 1e6);
// buffer
before = System.nanoTime();
for (int i = 0; i < runs; i++)
{
int resultB[][] = combineRows(
ArraycopyStreamPerformance::combineWithBuffer, arrays);
blackHole += resultB[0][0];
}
after = System.nanoTime();
System.out.printf(Locale.ENGLISH,
"%2d arrays, %3d rows, %6d cols, buffer : %8.3fms\n",
num, rows, cols, (after - before) / 1e6);
// buffer parallel
before = System.nanoTime();
for (int i = 0; i < runs; i++)
{
int resultB[][] = combineRowsParallel(
ArraycopyStreamPerformance::combineWithBuffer, arrays);
blackHole += resultB[0][0];
}
after = System.nanoTime();
System.out.printf(Locale.ENGLISH,
"%2d arrays, %3d rows, %6d cols, buffer parallel: %8.3fms\n",
num, rows, cols, (after - before) / 1e6);
// streams
before = System.nanoTime();
for (int i = 0; i < runs; i++)
{
int resultC[][] = combineRows(
ArraycopyStreamPerformance::combineWithStreams, arrays);
blackHole += resultC[0][0];
}
after = System.nanoTime();
System.out.printf(Locale.ENGLISH,
"%2d arrays, %3d rows, %6d cols, stream : %8.3fms (" +
blackHole + ")\n", num, rows, cols, (after - before) / 1e6);
}
private static void basicTest()
{
int array1[][] =
{
{ 1, 2 },
{ 3, 4 },
{ 5, 6 }
};
int array2[][] =
{
{ 7, 8 },
{ 9, 1 },
{ 2, 3 }
};
int result[][] =
{
{ 1, 2, 7, 8 },
{ 3, 4, 9, 1 },
{ 5, 6, 2, 3 }
};
System.out.println(Arrays.deepToString(result));
int resultA[][] = combineRows(
ArraycopyStreamPerformance::combineWithArraycopy, array1, array2);
System.out.println(Arrays.deepToString(resultA));
int resultB[][] = combineRows(
ArraycopyStreamPerformance::combineWithBuffer, array1, array2);
System.out.println(Arrays.deepToString(resultB));
int resultC[][] = combineRows(
ArraycopyStreamPerformance::combineWithStreams, array1, array2);
System.out.println(Arrays.deepToString(resultC));
}
private static int[][] selectRows(int row, int[][]... arrays)
{
int result[][] = new int[arrays.length][];
for (int j = 0; j < arrays.length; j++)
{
result[j] = arrays[j][row];
}
return result;
}
private static int[][] combineRows(
Function<int[][], int[]> mergeFunction, int[][]... arrays)
{
int rows = arrays[0].length;
int result[][] = new int[rows][];
for (int i = 0; i < rows; i++)
{
result[i] = mergeFunction.apply(selectRows(i, arrays));
}
return result;
}
private static int[] combineWithArraycopy(int[]... arrays)
{
// Assuming the same length for all arrays!
int length = arrays[0].length;
int result[] = new int[arrays.length * length];
for (int i = 0; i < arrays.length; i++)
{
System.arraycopy(arrays[i], 0, result, i * length, length);
}
return result;
}
private static int[] combineWithBuffer(int[]... arrays)
{
// Assuming the same length for all arrays!
int length = arrays[0].length;
int result[] = new int[arrays.length * length];
IntBuffer buffer = IntBuffer.wrap(result);
for (int i = 0; i < arrays.length; i++)
{
buffer.put(arrays[i]);
}
return result;
}
private static int[] combineWithStreams(int[] ... arrays)
{
return Stream.of(arrays).flatMapToInt(IntStream::of).toArray();
}
private static final ExecutorService EXECUTOR_SERVICE =
createFixedTimeoutExecutorService(
Runtime.getRuntime().availableProcessors(), 5, TimeUnit.SECONDS);
public static ExecutorService createFixedTimeoutExecutorService(
int poolSize, long keepAliveTime, TimeUnit timeUnit)
{
ThreadPoolExecutor e =
new ThreadPoolExecutor(poolSize, poolSize,
keepAliveTime, timeUnit, new LinkedBlockingQueue<Runnable>());
e.allowCoreThreadTimeOut(true);
return e;
}
private static int[][] combineRowsParallel(
Function<int[][], int[]> mergeFunction, int[][]... arrays)
{
int rows = arrays[0].length;
int result[][] = new int[rows][];
List<Callable<Object>> tasks = new ArrayList<Callable<Object>>();
for (int i = 0; i < rows; i++)
{
int index = i;
tasks.add(Executors.callable(() ->
{
result[index] = mergeFunction.apply(selectRows(index, arrays));
}));
}
try
{
EXECUTOR_SERVICE.invokeAll(tasks);
}
catch (InterruptedException e)
{
Thread.currentThread().interrupt();
}
return result;
}
}
...
8 arrays, 20 rows, 10000 cols, arraycopy : 354.977ms
8 arrays, 20 rows, 10000 cols, arraycopy parallel: 327.749ms
8 arrays, 20 rows, 10000 cols, buffer : 328.717ms
8 arrays, 20 rows, 10000 cols, buffer parallel: 312.522ms
8 arrays, 20 rows, 10000 cols, stream : 2044.017ms (0)