Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark 给出错误矩阵乘法结果的火花码_Apache Spark_Matrix Multiplication - Fatal编程技术网

Apache spark 给出错误矩阵乘法结果的火花码

Apache spark 给出错误矩阵乘法结果的火花码,apache-spark,matrix-multiplication,Apache Spark,Matrix Multiplication,我有两个矩阵{1,2,3;4,5,6;7,8,9}和{1,4;2,5;3,6}。 下面的代码是ApacheSpark中的矩阵乘法。但是它给了我错误的输出{15.0,29.0;36.0,71.0;57.0113.0}。我想知道我哪里做错了 JavaRDD<String> lines = ctx .textFile( "/home/hduser/Desktop/interpolation/Kriging/Matrix

我有两个矩阵{1,2,3;4,5,6;7,8,9}和{1,4;2,5;3,6}。 下面的代码是ApacheSpark中的矩阵乘法。但是它给了我错误的输出{15.0,29.0;36.0,71.0;57.0113.0}。我想知道我哪里做错了

    JavaRDD<String> lines = ctx
            .textFile(
                    "/home/hduser/Desktop/interpolation/Kriging/MatrixMultiplication/MatrixA.csv")
            .cache();

    JavaRDD<String> lines1 = ctx
            .textFile(
                    "/home/hduser/Desktop/interpolation/Kriging/MatrixMultiplication/MatrixB.csv")
            .cache();

    JavaRDD<Vector> rows = lines.map(new Function<String, Vector>() {

        @Override
        public Vector call(String line) throws Exception {

            String[] lineSplit = line.split(",");
            double[] arr = new double[lineSplit.length];
            for (int i = 0; i < lineSplit.length; i++) {
                arr[i] = Double.parseDouble(lineSplit[i]);
            }
            Vector dv = Vectors.dense(arr);
            return dv;
        }

    });

    //rows.saveAsTextFile("/home/hduser/Desktop/interpolation/Kriging/MatrixMultiplication/MatrixA_output");

    RowMatrix A = new RowMatrix(rows.rdd());



    JavaRDD<Vector> rows1 = lines1.map(new Function<String, Vector>() {

        @Override
        public Vector call(String line) throws Exception {

            String[] lineSplit = line.split(",");
            double[] arr = new double[lineSplit.length];
            for (int i = 0; i < lineSplit.length; i++) {
                arr[i] = Double.parseDouble(lineSplit[i]);
            }
            Vector dv = Vectors.dense(arr);
            return dv;
        }

    });

    List<Vector> arrList = new ArrayList<Vector>();
    arrList = rows1.toArray();


    double[] arr1 = new double[(int) rows1.count() * arrList.get(0).size()];
    int k=0;
    for (int i = 0; i < arrList.size(); i++) {
        for (int j = 0; j < arrList.get(i).size(); j++) {
            arr1[k] = arrList.get(i).apply(j);
            //System.out.println(arr1[k]);
            k++;
        }
    }
    Matrix B = Matrices.dense((int) rows1.count(), arrList.get(0)
            .size(), arr1);

    RowMatrix C = A.multiply(B);


    RDD<Vector> rows2 = C.rows();
    rows2.saveAsTextFile("/home/hduser/Desktop/interpolation/Kriging/MatrixMultiplication/Result");
JavaRDD行=ctx
.textFile(
“/home/hduser/Desktop/interpolation/Kriging/MatrixMultiplication/MatrixA.csv”)
.cache();
JavaRDD lines1=ctx
.textFile(
“/home/hduser/Desktop/interpolation/Kriging/MatrixMultiplication/MatrixB.csv”)
.cache();
javarddrows=lines.map(新函数(){
@凌驾
公共向量调用(字符串行)引发异常{
String[]lineSplit=line.split(“,”);
double[]arr=新的double[lineSplit.length];
对于(int i=0;i

提前感谢…

矩阵。dense
构造了一个列主矩阵(),并且您以错误的顺序遍历了行数组

我无法查看您的CSV文件,但我猜您也有输入错误。为什么?

为了产生错误的输出,B必须是
[1 3;4 5;2 6]
,因此数组必须是
{1,4,2,3,5,6}
,因此MatrixB.csv可能包含:

1,4
2,3
5,6

(3和5被切换)

如果您在调试器中运行代码,
arr1
看起来如何?请给出一个更完整的例子。数组中元素的顺序可能是错误的。在我们的示例中,可以使用数组文本。