Java 解决spark core和azure key vault依赖项之间的guava冲突

我正在试用java中的spark应用程序，该应用程序需要连接到Azure Key vault，在从blob存储读取数据并推送到SQL Server表后获取一些机密并处理数据。这是我的pom：

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.my.package</groupId>
    <artifactId>hdinsightSample</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>hdinsightSample</name>
    <url>http://maven.apache.org</url>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>3.8.1</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.12</artifactId>
            <version>2.4.7</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.12</artifactId>
            <version>2.4.7</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-azure</artifactId>
            <version>2.10.1</version>
        </dependency>
        <dependency>
            <groupId>com.microsoft.sqlserver</groupId>
            <artifactId>mssql-jdbc</artifactId>
            <version>6.2.2.jre8</version>
        </dependency>
        <dependency>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>azure-keyvault</artifactId>
            <version>1.0.0</version>
        </dependency>
        <dependency>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>azure</artifactId>
            <version>1.3.0</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <transformers>
                                <transformer
                                    implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                    <mainClass>com.my.package.hdinsightSample.App</mainClass>
                                </transformer>
                            </transformers>
                            <filters>
                                <filter>
                                    <artifact>*:*</artifact>
                                    <excludes>
                                        <exclude>META-INF/maven/**</exclude>
                                        <exclude>META-INF/*.SF</exclude>
                                        <exclude>META-INF/*.DSA</exclude>
                                        <exclude>META-INF/*.RSA</exclude>
                                    </excludes>
                                </filter>
                            </filters>
                            <relocations>
                                <relocation>
                                    <pattern>com.google.common</pattern>
                                    <shadedPattern>repackaged.com.google.common</shadedPattern>
                                    <includes>
                                        <include>com.google.guava:*</include>
                                    </includes>
                                </relocation>
                            </relocations>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

这是因为要从密钥库获取机密，它使用

guava

jar，但是

guava

jar默认值为11.0.2（存在于spark core dependecny中），而它应该使用的正确版本是20（存在于Azure密钥库依赖项中）。我试着在上面的pom中显示罐子的阴影，但问题仍然是一样的

---

请帮忙。

我创建了一个具有相同依赖关系的小项目，依赖关系树显示Guava依赖关系实际上来自

org.apache.hadoop:hadoop azure:jar:2.10.1

。不管怎样，解决此问题的最简单方法是明确说明您希望项目在POM中使用哪个Guava版本（理想情况下与本例中的Key Vault中的版本相同）：

番石榴
对于具有改进的API和附加功能的Azure Key Vault，它也不依赖于Guava，因此如果您还想更新代码库，这是一种替代方法：）


com.azure
azure安全密钥库证书
4.1.3/版本>
com.azure
azure security keyvault密钥/artifactId>
4.2.3/版本>
com.azure
azure安全密钥库机密
4.2.3/版本>
我创建了一个具有相同依赖关系的小项目，依赖关系树显示Guava依赖关系实际上来自
org.apache.hadoop:hadoop azure:jar:2.10.1
。不管怎样，解决此问题的最简单方法是明确说明您希望项目在POM中使用哪个Guava版本（理想情况下与本例中的Key Vault中的版本相同）：


番石榴
对于具有改进的API和附加功能的Azure Key Vault，它也不依赖于Guava，因此如果您还想更新代码库，这是一种替代方法：）


com.azure
azure安全密钥库证书
4.1.3/版本>
com.azure
azure security keyvault密钥/artifactId>
4.2.3/版本>
com.azure
azure安全密钥库机密
4.2.3/版本>

Exception in thread "main" java.lang.NoClassDefFoundError: com.google.common.reflect.TypeToken
    at java.lang.ClassLoader.defineClassImpl(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:379)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:682)
    at java.net.URLClassLoader.access$400(URLClassLoader.java:89)
    at java.net.URLClassLoader$ClassFinder.run(URLClassLoader.java:1086)
    at java.security.AccessController.doPrivileged(AccessController.java:739)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:589)
    at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:934)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:879)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:862)
    at com.microsoft.azure.keyvault.KeyVaultClientImpl.getSecretDelegate(KeyVaultClientImpl.java:2938)
    at com.microsoft.azure.keyvault.KeyVaultClientImpl.access$1800(KeyVaultClientImpl.java:92)
    at com.microsoft.azure.keyvault.KeyVaultClientImpl$82.call(KeyVaultClientImpl.java:2928)
    at com.microsoft.azure.keyvault.KeyVaultClientImpl$82.call(KeyVaultClientImpl.java:2924)
    at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:69)
    at retrofit2.adapter.rxjava.CallArbiter.deliverResponse(CallArbiter.java:120)
    at retrofit2.adapter.rxjava.CallArbiter.emitResponse(CallArbiter.java:102)
    at retrofit2.adapter.rxjava.CallExecuteOnSubscribe.call(CallExecuteOnSubscribe.java:46)
    at retrofit2.adapter.rxjava.CallExecuteOnSubscribe.call(CallExecuteOnSubscribe.java:24)
    at rx.Observable.unsafeSubscribe(Observable.java:10142)
    at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48)
    at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33)
    at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
    at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
    at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
    at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
    at rx.Observable.subscribe(Observable.java:10238)
    at rx.Observable.subscribe(Observable.java:10205)
    at rx.observables.BlockingObservable.blockForSingle(BlockingObservable.java:444)
    at rx.observables.BlockingObservable.single(BlockingObservable.java:341)
    at com.microsoft.azure.keyvault.KeyVaultClientImpl.getSecret(KeyVaultClientImpl.java:2868)
    at com.microsoft.azure.keyvault.KeyVaultClient.getSecret(KeyVaultClient.java:862)
    at com.my.package.hdsinsightSample.utils.KeyVaultUtils.getSecret(KeyVaultUtils.java:12)
    at com.my.package.hdinsightSample.App.main(App.java:23)
Caused by: java.lang.ClassNotFoundException: com.google.common.reflect.TypeToken
    at java.net.URLClassLoader.findClass(URLClassLoader.java:591)
    at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:934)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:879)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:862)
    ... 36 more