Scala 我的maven项目有什么配置错误?不拾取依赖项或类结构

Scala 我的maven项目有什么配置错误?不拾取依赖项或类结构,scala,maven,apache-spark,intellij-idea,Scala,Maven,Apache Spark,Intellij Idea,我的项目似乎存在一些错误配置,导致maven错过了我的base pom.xml中的依赖项。当我尝试使用intellij的构建选项构建模块,或者使用mvn clean package构建项目时,我得到一个错误,即“object apache不是package org的成员”。我的pom文件包含以下内容: <dependency> <groupId>org.apache.spark</groupId> <artifactId&

我的项目似乎存在一些错误配置,导致maven错过了我的base pom.xml中的依赖项。当我尝试使用intellij的构建选项构建模块,或者使用mvn clean package构建项目时,我得到一个错误,即“object apache不是package org的成员”。我的pom文件包含以下内容:

<dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql_${scala.major.version}.${scala.minor.version}</artifactId>
        <version>${spark.version}</version>
        <scope>provided</scope>
        <exclusions>
          ...
        </exclusions>
      </dependency>

org.apache.spark
spark-sql{scala.major.version}.${scala.minor.version}
${spark.version}
假如
...
我已经确认spark-sql_2.11(我的版本)jar存在于我的本地.m2存储库中,并且我的设置已配置为自动导入maven项目

当我删除对org.apache的引用时,jar使用mvn clean package和intellij的build module菜单选项成功构建,但是这两个jar的功能不同。通过intellij(/out/artifacts/[project]\u jar/[project].jar)构建模块创建的jar可以与spark submit配合使用,但是使用从mvn clean package(/target/[project]-test SNAPSHOT.jar)创建的jar会抛出一个错误“jar中没有设置主类”,尽管事实上存在“jar tvf”显示了这个类,并且MANIFEST.MF将其正确地列为主类

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>2.6</version>
                <executions>
                    <execution>
                        <id>jar-with-dependencies</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                        <configuration>
                            <descriptorRefs>
                                <descriptorRef>jar-with-dependencies</descriptorRef>
                            </descriptorRefs>
                            <archive>
                                <manifest>
                                    <mainClass>***com.abc.mainclass***</mainClass>
                                </manifest>
                            </archive>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
我的scala类和maven项目之间似乎存在脱节,但我不知道在哪里。任何关于如何诊断为什么这不起作用的提示都将不胜感激,如有必要,我绝对可以提供额外的细节。提前谢谢

编辑:添加pom.xml的完整内容

    <?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <artifactId>basepom</artifactId>
    <groupId>com.cdtk.maven.basepom</groupId>
    <version>1.0-SNAPSHOT</version>
    <relativePath />
  </parent>

  <groupId>com.[company].testproject2</groupId>
  <artifactId>swtest2</artifactId>
  <version>test-SNAPSHOT</version>
  <packaging>jar</packaging>

  <description>
    This parent pom holds all the necessary version
    and configuration information for all submodules.
  </description>

  <properties>
    <required.maven.version>3.5.2</required.maven.version>
    <upper.maven.version>3.5.999</upper.maven.version>
    <required.java.update>20</required.java.update>

    <scala.major.version>2</scala.major.version>
    <scala.minor.version>11</scala.minor.version>
    <scala.patch.version>0</scala.patch.version>

    <spark.version>2.3.0</spark.version>
  </properties>

  <dependencyManagement>
    <dependencies>
      <!-- =================================== -->
      <!--        Core Spark BOM               -->
      <!-- =================================== -->
      <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-sql_${scala.major.version}.${scala.minor.version}</artifactId>
        <version>${spark.version}</version>
        <scope>provided</scope>
        <exclusions>
          <exclusion>
            <artifactId>commons-beanutils-core</artifactId>
            <groupId>commons-beanutils</groupId>
          </exclusion>
          <exclusion>
            <artifactId>jersey-server</artifactId>
            <groupId>com.sun.jersey</groupId>
          </exclusion>
          <exclusion>
            <artifactId>javax.inject</artifactId>
            <groupId>javax.inject</groupId>
          </exclusion>
          <exclusion>
            <artifactId>javax.ws.rs-api</artifactId>
            <groupId>javax.ws.rs</groupId>
          </exclusion>
          <exclusion>
            <artifactId>stax-api</artifactId>
            <groupId>javax.xml.stream</groupId>
          </exclusion>
          <exclusion>
            <artifactId>aopalliance</artifactId>
            <groupId>aopalliance</groupId>
          </exclusion>
          <exclusion>
            <artifactId>commons-logging</artifactId>
            <groupId>commons-logging</groupId>
          </exclusion>
          <exclusion>
            <artifactId>paranamer</artifactId>
            <groupId>com.thoughtworks.paranamer</groupId>
          </exclusion>
        </exclusions>
      </dependency>
      <dependency>
        <groupId>org.bouncycastle</groupId>
        <artifactId>bcprov-jdk15on</artifactId>
        <version>1.60</version>
      </dependency>
      <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-mapreduce-client-core</artifactId>
        <version>2.6.5</version>
      </dependency>
      <dependency>
        <groupId>org.scala-lang</groupId>
        <artifactId>scala-library</artifactId>
        <version>${scala.major.version}.${scala.minor.version}.${scala.patch.version}</version>
      </dependency>
      <dependency>
        <groupId>org.apache.avro</groupId>
        <artifactId>avro</artifactId>
        <version>1.7.7</version>
      </dependency>
      <dependency>
        <groupId>org.xerial.snappy</groupId>
        <artifactId>snappy-java</artifactId>
        <version>1.1.2.6</version>
      </dependency>
      <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-math3</artifactId>
        <version>3.4.1</version>
      </dependency>
      <dependency>
        <groupId>commons-lang</groupId>
        <artifactId>commons-lang</artifactId>
        <version>2.6</version>
      </dependency>
      <dependency>
        <groupId>com.google.code.findbugs</groupId>
        <artifactId>jsr305</artifactId>
        <version>3.0.2</version>
      </dependency>
      <dependency>
        <groupId>org.scala-lang.modules</groupId>
        <artifactId>scala-parser-combinators_${scala.major.version}.${scala.minor.version}</artifactId>
        <version>1.0.4</version>
      </dependency>
      <dependency>
        <groupId>org.codehaus.janino</groupId>
        <artifactId>commons-compiler</artifactId>
        <version>3.0.8</version>
      </dependency>
      <dependency>
        <groupId>org.glassfish.hk2.external</groupId>
        <artifactId>javax.inject</artifactId>
        <version>2.5.0-b32</version>
      </dependency>
      <dependency>
        <groupId>org.glassfish.jersey.core</groupId>
        <artifactId>jersey-common</artifactId>
        <version>2.25.1</version>
      </dependency>
      <dependency>
        <groupId>io.netty</groupId>
        <artifactId>netty</artifactId>
        <version>3.9.9.Final</version>
      </dependency>
      <dependency>
        <groupId>javax.activation</groupId>
        <artifactId>activation</artifactId>
        <version>1.1.1</version>
      </dependency>
      <dependency>
        <groupId>org.glassfish.jersey.core</groupId>
        <artifactId>jersey-client</artifactId>
        <version>2.25.1</version>
      </dependency>
      <dependency>
        <groupId>org.codehaus.jackson</groupId>
        <artifactId>jackson-mapper-asl</artifactId>
        <version>1.9.13</version>
      </dependency>
      <dependency>
        <groupId>commons-net</groupId>
        <artifactId>commons-net</artifactId>
        <version>3.1</version>
      </dependency>
      <dependency>
        <groupId>org.codehaus.jackson</groupId>
        <artifactId>jackson-xc</artifactId>
        <version>1.9.13</version>
      </dependency>
      <dependency>
        <groupId>org.codehaus.jackson</groupId>
        <artifactId>jackson-jaxrs</artifactId>
        <version>1.9.13</version>
      </dependency>
      <dependency>
        <groupId>org.codehaus.jackson</groupId>
        <artifactId>jackson-core-asl</artifactId>
        <version>1.9.13</version>
      </dependency>
      <dependency>
        <groupId>org.scala-lang</groupId>
        <artifactId>scala-reflect</artifactId>
        <version>${scala.major.version}.${scala.minor.version}.${scala.patch.version}</version>
      </dependency>

      <!-- =================================== -->
      <!--        Top level dependencies       -->
      <!-- =================================== -->

      <dependency>
        <groupId>com.typesafe</groupId>
        <artifactId>config</artifactId>
        <version>1.3.0</version>
      </dependency>
      <dependency>
        <groupId>com.oracle</groupId>
        <artifactId>ojdbc6</artifactId>
        <version>11.2.0.3</version>
      </dependency>

      <dependency>
        <groupId>com.databricks</groupId>
        <artifactId>spark-avro_${scala.major.version}.${scala.minor.version}</artifactId>
        <version>4.0.0</version>
      </dependency>

      <!-- =================================== -->
      <!--        Test dependencies            -->
      <!-- =================================== -->
      <dependency>
        <groupId>org.scalatest</groupId>
        <artifactId>scalatest_${scala.major.version}.${scala.minor.version}</artifactId>
        <version>3.0.5</version>
        <scope>test</scope>
      </dependency>
      <dependency>
        <groupId>org.scalatest</groupId>
        <artifactId>scalatest-maven-plugin</artifactId>
        <version>2.0.0</version>
        <scope>test</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <dependencies>
    <dependency>
      <groupId>org.scalatest</groupId>
      <artifactId>scalatest_${scala.major.version}.${scala.minor.version}</artifactId>
      <exclusions>
        <exclusion>
          <artifactId>scala-library</artifactId>
          <groupId>org.scala-lang</groupId>
        </exclusion>
        <exclusion>
          <artifactId>scala-xml_${scala.major.version}.${scala.minor.version}</artifactId>
          <groupId>org.scala-lang.modules</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
    </dependency>
  </dependencies>

  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>net.alchim31.maven</groupId>
          <artifactId>scala-maven-plugin</artifactId>
          <version>3.2.2</version>
        </plugin>
        <plugin>
          <groupId>org.scalatest</groupId>
          <artifactId>scalatest-maven-plugin</artifactId>
          <version>2.0.0</version>
        </plugin>
      </plugins>
    </pluginManagement>
    <plugins>
      <plugin>
        <groupId>net.alchim31.maven</groupId>
        <artifactId>scala-maven-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>compile</goal>
              <goal>testCompile</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <scalaCompatVersion>${scala.major.version}.${scala.minor.version}</scalaCompatVersion>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.basepom.maven</groupId>
        <artifactId>duplicate-finder-maven-plugin</artifactId>
        <configuration>
          <ignoredClassPatterns combine.self="append">
            <ignoredClassPattern>org.apache.spark.unused.UnusedStubClass</ignoredClassPattern>
          </ignoredClassPatterns>
          <ignoredResourcePatterns combine.self="append">
            <!-- mercury internal file (should provide link to code that proves its ok to have more than one on the classpath) -->
            <ignoredResource>overrides.conf</ignoredResource>
            <ignoredResource>git.properties</ignoredResource>
          </ignoredResourcePatterns>
        </configuration>
      </plugin>
    </plugins>
  </build>

</project>

4.0.0
basepom
com.cdtk.maven.basepom
1.0-快照
com.[company].testproject2
swtest2
测试快照
罐子
此父pom包含所有必需的版本
以及所有子模块的配置信息。
3.5.2
3.5.999
20
2.
11
0
2.3.0
org.apache.spark
spark-sql{scala.major.version}.${scala.minor.version}
${spark.version}
假如
公地豆核
公地小海狸
泽西服务器
泽西岛
javax.inject
javax.inject
javax.ws.rs-api
javax.ws.rs
斯塔克斯api
javax.xml.stream
奥帕林
奥帕林
公用记录
公用记录
帕拉纳默
com.thoughtworks.paranamer
org.bouncycastle
bcprov-jdk15on
1.60
org.apache.hadoop
hadoop mapreduce客户端核心
2.6.5
org.scala-lang
scala图书馆
${scala.major.version}.${scala.minor.version}.${scala.patch.version}
org.apache.avro
阿夫罗
1.7.7
org.xerial.snapy
轻快的java
1.1.2.6
org.apache.commons
commons-math3
3.4.1
公地郎
公地郎
2.6
com.google.code.findbugs
jsr305
3.0.2
org.scala-lang.modules
scala-parser-combinators_${scala.major.version}.${scala.minor.version}
1.0.4
org.codehaus.janino
通用编译器
3.0.8
org.glassfish.hk2.external
javax.inject
2.5.0-b32
org.glassfish.jersey.core
泽西公地
2.25.1
伊奥·内蒂
内蒂
3.9.9.4最终版本
javax.activation
激活
1.1.1
org.glassfish.jersey.core
泽西岛客户
2.25.1
org.codehaus.jackson
杰克逊地图绘制者
1.9.13
公用网络
公用网络
3.1
org.codehaus.jackson
杰克逊xc
1.9.13
org.codehaus.jackson
杰克逊·贾克斯
1.9.13
org.codehaus.jackson
jackson core asl
1.9.13
org.scala-lang
斯卡拉反射
${scala.major.version}.${scala.minor.version}.${scala.patch.version}
com.typesafe
配置
1.3.0
com.oracle
ojdbc6
11.2.0.3
com.databricks
spark-avro{scala.major.version}.${scala.minor.version}
4.0.0
org.scalatest
scalatest{scala.major.version}.${scala.minor.version}
3.0.5
测试
org.scalatest
scalatest maven插件
2.0.0
测试
org.scalatest
scalatest{scala.major.version}.${scala.minor.version}
scala图书馆
org.scala-lang
scala-xml{scala.major.version}.${scala.minor.version}
org.scala-lang.modules
朱尼特
朱尼特
net.alchim31.maven
scala maven插件
3.2.2
org.scalatest
scalatest maven插件
2.0.0
net.alchim31.maven
scala maven插件
编译
测试编译
${scala.major.version}.${scala.minor.version}
org.basepom.maven
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>2.6</version>
                <executions>
                    <execution>
                        <id>jar-with-dependencies</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                        <configuration>
                            <descriptorRefs>
                                <descriptorRef>jar-with-dependencies</descriptorRef>
                            </descriptorRefs>
                            <archive>
                                <manifest>
                                    <mainClass>***com.abc.mainclass***</mainClass>
                                </manifest>
                            </archive>
                        </configuration>
                    </execution>
                </executions>
            </plugin>