Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Java 使用Spark消费时,如何在Zookeeper内保存偏移id?_Java_Apache Spark_Apache Kafka_Apache Zookeeper - Fatal编程技术网

Java 使用Spark消费时,如何在Zookeeper内保存偏移id?

Java 使用Spark消费时,如何在Zookeeper内保存偏移id?,java,apache-spark,apache-kafka,apache-zookeeper,Java,Apache Spark,Apache Kafka,Apache Zookeeper,我正在使用Spark来使用Kafka的数据。我正在用Java编写程序。我希望能够在Zookeeper中保存偏移id,并能够从上次使用的偏移id重新启动。如何在Zookeeper中保存偏移id SparkConf sparkConf = new SparkConf().setAppName("name"); JavaStreamingContext jssc = new JavaStreamingContext(sparkConf, new Duration(2000)); Map<Stri

我正在使用Spark来使用Kafka的数据。我正在用Java编写程序。我希望能够在Zookeeper中保存偏移id,并能够从上次使用的偏移id重新启动。如何在Zookeeper中保存偏移id

SparkConf sparkConf = new SparkConf().setAppName("name");
JavaStreamingContext jssc = new JavaStreamingContext(sparkConf, new Duration(2000));
Map<String,String> kafkaParams = new HashMap<>();
kafkaParams.put("zookeeper.connect", "127.0.0.1");
kafkaParams.put("group.id", App.GROUP);
JavaPairReceiverInputDStream<String, EventLog> messages =
  KafkaUtils.createStream(jssc, String.class, EventLog.class, StringDecoder.class, EventLogDecoder.class,
    kafkaParams, topicMap, StorageLevel.MEMORY_AND_DISK_SER_2());
SparkConf SparkConf=new SparkConf().setAppName(“name”);
JavaStreamingContext jssc=新的JavaStreamingContext(sparkConf,新的持续时间(2000));
Map kafkaParams=新HashMap();
卡夫卡帕拉姆斯.普特(“zookeeper.connect”,“127.0.0.1”);
kafkaParams.put(“group.id”,App.group);
JavaPairReceiverInputStream消息=
KafkaUtils.createStream(jssc、String.class、EventLog.class、StringDecoder.class、EventLogDecoder.class、,
kafkaParams、topicMap、StorageLevel.MEMORY_和_DISK_SER_2());