Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
需要使用java将apache spark中的Dataframe 1数据推送到空Dataframe 2数据_Java_Apache Spark_Apache Spark Sql_Apache Spark Dataset - Fatal编程技术网

需要使用java将apache spark中的Dataframe 1数据推送到空Dataframe 2数据

需要使用java将apache spark中的Dataframe 1数据推送到空Dataframe 2数据,java,apache-spark,apache-spark-sql,apache-spark-dataset,Java,Apache Spark,Apache Spark Sql,Apache Spark Dataset,在这里,我需要将特定数据从dataframe1推送到空dataframe2,但我面临着麻烦。下面是代码 public class PrintValue { public static void main(String[] args) { System.setProperty("hadoop.home.dir", "C:\\winutils"); JavaSparkContext sc = new J

在这里,我需要将特定数据从dataframe1推送到空dataframe2,但我面临着麻烦。下面是代码

 public class PrintValue {
                public static void main(String[] args) {
                System.setProperty("hadoop.home.dir", "C:\\winutils");
                JavaSparkContext sc = new JavaSparkContext(newSparkConf().setAppName("JoinFunctions").setMaster("local[*]"));
                SQLContext sqlContext = new SQLContext(sc);
                SparkSession spark = SparkSession.builder().appName("JavaTokenizerExample").getOrCreate();

                JaroWinkler jw = new JaroWinkler();

                List<Row> data = Arrays.asList(
                RowFactory.create(1,"Hi I heard about Spark"),
                RowFactory.create(2,"I wish Java could use case classes"),
                RowFactory.create(3,"Logistic,regression,models,are,neat"));

                StructType schema = new StructType(new StructField[] {
                new StructField("label", DataTypes.IntegerType, false,
                  Metadata.empty()),
                new StructField("sentence", DataTypes.StringType, false,
                Metadata.empty()) });
                Dataset<Row> DataFrame1 = spark.createDataFrame(data, schema);
                sentenceDataFrame.show();


                List<Row> data1 = Arrays.asList();
                StructType schema2 = new StructType(new StructField[] {
                new StructField("label2", DataTypes.IntegerType, false,Metadata.empty()),
                new StructField("sentence2", DataTypes.StringType, false,Metadata.empty()) });
                Dataset<Row> DataFrame1 = spark.createDataFrame(sc.emptyRDD(), schema2);
                sentenceDataFrame1.show();
DataFrame1
+-----+--------------------+
|label|            sentence|
+-----+--------------------+
|    1|Hi I heard about ...|
|    2|I wish Java could...|
|    3|Logistic,regressi...|
+-----+--------------------+

DataFrame2
+-----+--------------------+
|label2|         sentence2  |
+-----+--------------------+
|     |                    |
|    2|I wish Java could...|
|     |                    |
+-----+--------------------+