Spring boot 将spring boot与spark最新版本集成时出现的问题

我尝试了类似于下面链接中给出的示例，但使用了最新版本spark 2.3.1（Scala 2.11）和hadoop 2.7.7。

下面给出了pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.sg.spark</groupId>
  <artifactId>SparkJobServer</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>
  <name>spark-job-server</name>
  <description>SparkYarn tasklet</description>
  <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.3.1.RELEASE</version>
        <relativePath /> <!-- lookup parent from repository -->
    </parent>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>
    </properties>

  <dependencies>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-batch</artifactId>
      <exclusions>
        <exclusion>
          <groupId>org.springframework.boot</groupId>
          <artifactId>spring-boot-starter-logging</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-log4j</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.data</groupId>
      <artifactId>spring-data-hadoop-boot</artifactId>
      <version>2.5.0.RELEASE</version>
    </dependency>
    <dependency>
      <groupId>org.springframework.data</groupId>
      <artifactId>spring-data-hadoop-batch</artifactId>
      <version>2.5.0.RELEASE</version>
    </dependency>
    <dependency>
      <groupId>org.springframework.data</groupId>
      <artifactId>spring-data-hadoop-spark</artifactId>
      <version>2.5.0.RELEASE</version>
    </dependency>
    <dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-yarn_2.11</artifactId>
      <version>2.3.1</version>
    </dependency>
    <dependency>
        <groupId>jdk.tools</groupId>
        <artifactId>jdk.tools</artifactId>
        <version>1.8.0_172</version>
        <scope>system</scope>
        <systemPath>${JAVA_HOME}/lib/tools.jar</systemPath>
    </dependency>
    <dependency>
    <groupId>org.hibernate</groupId>
    <artifactId>hibernate-validator</artifactId>
    <version>5.2.1.Final</version>
</dependency>
<dependency>
    <groupId>org.hibernate</groupId>
    <artifactId>hibernate-validator-annotation-processor</artifactId>
    <version>5.2.1.Final</version>
</dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
      </plugin>
    </plugins>
  </build>
</project>

4.0.0
com.sg.spark
SparkJobServer
0.0.1-快照
罐子
spark作业服务器
SparkYarn微珠
org.springframework.boot
spring启动程序父级
1.3.1.发布
UTF-8
UTF-8
1.8
org.springframework.boot
弹簧靴起动器批次
org.springframework.boot
spring启动程序日志记录
org.springframework.boot
spring-boot-starter-log4j
org.springframework.data
spring数据hadoop启动
2.5.0.1发布
org.springframework.data
spring数据hadoop批处理
2.5.0.1发布
org.springframework.data
spring数据hadoop spark
2.5.0.1发布
org.apache.spark
火花丝2.11
2.3.1
jdk.tools
jdk.tools
1.8.0_172
系统
${JAVA_HOME}/lib/tools.jar
org.hibernate
休眠验证器
5.2.1.最终版本
org.hibernate
hibernate验证程序注释处理器
5.2.1.最终版本
org.springframework.boot
springbootmaven插件

我得到以下异常NoSuchMethodError

[2018-08-23 06:19:16.921] boot - 11694 ERROR [main] --- AbstractStep: Encountered an error executing step sparkTopHashtags in job TweetTopHashtags
java.lang.NoSuchMethodError: org.apache.spark.deploy.yarn.ClientArguments.<init>([Ljava/lang/String;Lorg/apache/spark/SparkConf;)V
    at org.springframework.data.hadoop.batch.spark.SparkYarnTasklet.execute(SparkYarnTasklet.java:117)
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:406)
    at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:330)
    at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
    at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:271)
    at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:81)
    at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:374)
    at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215)

[2018-08-23 06:19:16.921]启动-11694错误[main]--抽象步骤：在作业TweetTopHashtags中执行步骤sparkTopHashtags时遇到错误
java.lang.NoSuchMethodError:org.apache.spark.deploy.warn.ClientArguments.（[Ljava/lang/String；Lorg/apache/spark/SparkConf；）V
位于org.springframework.data.hadoop.batch.SparkYarnTasklet.execute（SparkYarnTasklet.java:117）
位于org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction（TaskletStep.java:406）
位于org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction（TaskletStep.java:330）
位于org.springframework.transaction.support.TransactionTemplate.execute（TransactionTemplate.java:133）
位于org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext（TaskletStep.java:271）
位于org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInitation（StepContextRepeatCallback.java:81）
位于org.springframework.batch.repeat.support.RepeatTemplate.getNextResult（RepeatTemplate.java:374）
位于org.springframework.batch.repeat.support.RepeatTemplate.executeInternal（RepeatTemplate.java:215）

你能帮忙吗