Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark 卡夫卡充满了火花。依赖性问题_Apache Spark_Apache Kafka - Fatal编程技术网

Apache spark 卡夫卡充满了火花。依赖性问题

Apache spark 卡夫卡充满了火花。依赖性问题,apache-spark,apache-kafka,Apache Spark,Apache Kafka,我尝试将Spark Streaming 2.0.0与Kafka 0.10结合使用。我正在使用集成测试,但启动服务器时遇到一些问题。当我尝试使用Spark 2.2.0时,它工作正常 <dependency> <groupId>net.manub</groupId> <artifactId>scalatest-embedded-kafka_2.11</artifactId> <ve

我尝试将Spark Streaming 2.0.0与Kafka 0.10结合使用。我正在使用集成测试,但启动服务器时遇到一些问题。当我尝试使用Spark 2.2.0时,它工作正常

   <dependency>
        <groupId>net.manub</groupId>
        <artifactId>scalatest-embedded-kafka_2.11</artifactId>
        <version>${embedded-kafka.version}</version> -->I tried many versions.
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
        <version>2.0.2</version>
    </dependency>

An exception or error caused a run to abort: kafka.server.KafkaServer$.$lessinit$greater$default$2()Lorg/apache/kafka/common/utils/Time; 
java.lang.NoSuchMethodError: kafka.server.KafkaServer$.$lessinit$greater$default$2()Lorg/apache/kafka/common/utils/Time;
    at net.manub.embeddedkafka.EmbeddedKafkaSupport$class.startKafka(EmbeddedKafka.scala:467)
    at net.manub.embeddedkafka.EmbeddedKafka$.startKafka(EmbeddedKafka.scala:38)
    at net.manub.embeddedkafka.EmbeddedKafka$.start(EmbeddedKafka.scala:55)
    at iris.orange.ScalaTest$$anonfun$1.apply$mcV$sp(ScalaTest.scala:10)

net.manub
scalatest-embedded-kafka_2.11
${embedded kafka.version}-->我尝试了许多版本。
测验
org.apache.spark
spark-streaming-kafka-0-10_2.11
2.0.2
异常或错误导致运行中止:kafka.server.KafkaServer$.$lessinit$greater$default$2()Lorg/apache/kafka/common/utils/Time;
java.lang.NoSuchMethodError:kafka.server.KafkaServer$.$lessinit$greater$default$2()Lorg/apache/kafka/common/utils/Time;
位于net.manub.embeddedkafka.EmbeddedKafkaSupport$class.startKafka(embeddedkafka.scala:467)
位于net.manub.embeddedkafka.embeddedkafka$.startKafka(embeddedkafka.scala:38)
位于net.manub.embeddedkafka.embeddedkafka$.start(embeddedkafka.scala:55)
在iris.orange.ScalaTest$$anonfun$1.apply$mcV$sp(ScalaTest.scala:10)

这似乎是一个关于依赖性的问题,但我没有开始工作。我选择了一个使用相同卡夫卡版本的嵌入式卡夫卡

您需要使用正确版本的
spark streaming kafka


org.apache.spark
spark-streaming-kafka-0-10_2.10
2.0.0

您需要获得spark版本的特定版本
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming-kafka-0-10 -->
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-streaming-kafka-0-10_2.10</artifactId>
    <version>2.0.0</version>
</dependency>