Java 使用Spark消费时,如何在Zookeeper内保存偏移id?
我正在使用Spark来使用Kafka的数据。我正在用Java编写程序。我希望能够在Zookeeper中保存偏移id,并能够从上次使用的偏移id重新启动。如何在Zookeeper中保存偏移idJava 使用Spark消费时,如何在Zookeeper内保存偏移id?,java,apache-spark,apache-kafka,apache-zookeeper,Java,Apache Spark,Apache Kafka,Apache Zookeeper,我正在使用Spark来使用Kafka的数据。我正在用Java编写程序。我希望能够在Zookeeper中保存偏移id,并能够从上次使用的偏移id重新启动。如何在Zookeeper中保存偏移id SparkConf sparkConf = new SparkConf().setAppName("name"); JavaStreamingContext jssc = new JavaStreamingContext(sparkConf, new Duration(2000)); Map<Stri
SparkConf sparkConf = new SparkConf().setAppName("name");
JavaStreamingContext jssc = new JavaStreamingContext(sparkConf, new Duration(2000));
Map<String,String> kafkaParams = new HashMap<>();
kafkaParams.put("zookeeper.connect", "127.0.0.1");
kafkaParams.put("group.id", App.GROUP);
JavaPairReceiverInputDStream<String, EventLog> messages =
KafkaUtils.createStream(jssc, String.class, EventLog.class, StringDecoder.class, EventLogDecoder.class,
kafkaParams, topicMap, StorageLevel.MEMORY_AND_DISK_SER_2());
SparkConf SparkConf=new SparkConf().setAppName(“name”);
JavaStreamingContext jssc=新的JavaStreamingContext(sparkConf,新的持续时间(2000));
Map kafkaParams=新HashMap();
卡夫卡帕拉姆斯.普特(“zookeeper.connect”,“127.0.0.1”);
kafkaParams.put(“group.id”,App.group);
JavaPairReceiverInputStream消息=
KafkaUtils.createStream(jssc、String.class、EventLog.class、StringDecoder.class、EventLogDecoder.class、,
kafkaParams、topicMap、StorageLevel.MEMORY_和_DISK_SER_2());