Java ApacheFlink-外部Jar

我正在尝试使用maven和一个额外的依赖项创建Flink应用程序：

 <dependency>
      <groupId>com.microsoft.sqlserver</groupId>
      <artifactId>mssql-jdbc</artifactId>
      <version>8.2.2.jre11</version>
  </dependency>

com.microsoft.sqlserver

因此，我最初的想法是依赖项没有使用maven shadow插件加载到胖jar上，但我认为是这样的。

我的Flink在docker容器中运行，我在容器上打开端口1433:1433（在compose中）

我的目标是将SQL数据添加到具有用户自定义规则的警报系统的流中

以下是my pom.xml的内容：

-->
http://maven.apache.org/xsd/maven-4.0.0.xsd">
4.0.0

<groupId>org.qscale</groupId>
<artifactId>FlinkPrototype</artifactId>
<version>1.0</version>
<packaging>jar</packaging>

<name>Flink Quickstart Job</name>
<url>http://www.myorganization.org</url>

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <flink.version>1.10.0</flink.version>
    <java.version>1.8</java.version>
    <scala.binary.version>2.11</scala.binary.version>
    <maven.compiler.source>${java.version}</maven.compiler.source>
    <maven.compiler.target>${java.version}</maven.compiler.target>
</properties>

<repositories>
    <repository>
        <id>apache.snapshots</id>
        <name>Apache Development Snapshot Repository</name>
        <url>https://repository.apache.org/content/repositories/snapshots/</url>
        <releases>
            <enabled>false</enabled>
        </releases>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
    </repository>
</repositories>

<dependencies>
    <!-- Apache Flink dependencies -->
    <!-- These dependencies are provided, because they should not be packaged into the JAR file. -->
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-java</artifactId>
        <version>${flink.version}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
        <version>${flink.version}</version>
        <scope>provided</scope>
    </dependency>

    <!-- Add connector dependencies here. They must be in the default scope (compile). -->

    <!-- Example:

    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-connector-kafka-0.10_${scala.binary.version}</artifactId>
        <version>${flink.version}</version>
    </dependency>
    -->

    <!-- Project dependencies -->
    <dependency>
        <groupId>com.microsoft.sqlserver</groupId>
        <artifactId>mssql-jdbc</artifactId>
        <version>8.2.2.jre11</version>
    </dependency>

    <!-- Add logging framework, to produce console output when running in the IDE. -->
    <!-- These dependencies are excluded from the application JAR by default. -->
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-log4j12</artifactId>
        <version>1.7.7</version>
        <scope>runtime</scope>
    </dependency>
    <dependency>
        <groupId>log4j</groupId>
        <artifactId>log4j</artifactId>
        <version>1.2.17</version>
        <scope>runtime</scope>
    </dependency>
</dependencies>

<build>
    <plugins>

        <!-- Java Compiler -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.1</version>
            <configuration>
                <source>${java.version}</source>
                <target>${java.version}</target>
            </configuration>
        </plugin>

        <!-- We use the maven-shade plugin to create a fat jar that contains all necessary dependencies. -->
        <!-- Change the value of <mainClass>...</mainClass> if your program entry point changes. -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>3.1.1</version>
            <executions>
                <!-- Run shade goal on package phase -->
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>
                        <shadedArtifactAttached>true</shadedArtifactAttached>
                        <shadedClassifierName>shaded</shadedClassifierName>
                        <artifactSet>
                            <excludes>
                                <exclude>org.apache.flink:force-shading</exclude>
                                <exclude>com.google.code.findbugs:jsr305</exclude>
                                <exclude>org.slf4j:*</exclude>
                                <exclude>log4j:*</exclude>
                            </excludes>
                        </artifactSet>
                        <filters>
                            <filter>
                                <!-- Do not copy the signatures in the META-INF folder.
                                Otherwise, this might cause SecurityExceptions when using the JAR. -->
                                <artifact>*:*</artifact>
                                <excludes>
                                    <exclude>META-INF/*.SF</exclude>
                                    <exclude>META-INF/*.DSA</exclude>
                                    <exclude>META-INF/*.RSA</exclude>
                                </excludes>
                            </filter>
                        </filters>
                        <transformers>
                            <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                <mainClass>org.qscale.StreamingJob</mainClass>
                            </transformer>
                        </transformers>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>

    <pluginManagement>
        <plugins>

            <!-- This improves the out-of-the-box experience in Eclipse by resolving some warnings. -->
            <plugin>
                <groupId>org.eclipse.m2e</groupId>
                <artifactId>lifecycle-mapping</artifactId>
                <version>1.0.0</version>
                <configuration>
                    <lifecycleMappingMetadata>
                        <pluginExecutions>
                            <pluginExecution>
                                <pluginExecutionFilter>
                                    <groupId>org.apache.maven.plugins</groupId>
                                    <artifactId>maven-shade-plugin</artifactId>
                                    <versionRange>[3.1.1,)</versionRange>
                                    <goals>
                                        <goal>shade</goal>
                                    </goals>
                                </pluginExecutionFilter>
                                <action>
                                    <ignore/>
                                </action>
                            </pluginExecution>
                            <pluginExecution>
                                <pluginExecutionFilter>
                                    <groupId>org.apache.maven.plugins</groupId>
                                    <artifactId>maven-compiler-plugin</artifactId>
                                    <versionRange>[3.1,)</versionRange>
                                    <goals>
                                        <goal>testCompile</goal>
                                        <goal>compile</goal>
                                    </goals>
                                </pluginExecutionFilter>
                                <action>
                                    <ignore/>
                                </action>
                            </pluginExecution>
                        </pluginExecutions>
                    </lifecycleMappingMetadata>
                </configuration>
            </plugin>
        </plugins>
    </pluginManagement>
</build>

org.qscale
FlinkPrototype
1
罐子
Flink快速启动作业
http://www.myorganization.org
UTF-8
1.10.0
1.8
2.11
${java.version}
${java.version}
apache.snapshots
Apache开发快照存储库
https://repository.apache.org/content/repositories/snapshots/
假的
真的
org.apache.flink
弗林克爪哇
${flink.version}
假如
org.apache.flink
flink-streaming-java_${scala.binary.version}
${flink.version}
假如
com.microsoft.sqlserver
mssql jdbc
8.2.2.jre11
org.slf4j
slf4j-log4j12
1.7.7
运行时
log4j
log4j
1.2.17
运行时
org.apache.maven.plugins
maven编译器插件
3.1
${java.version}
${java.version}
org.apache.maven.plugins
maven阴影插件
3.1.1
包裹
阴凉处
真的
遮住的
flink:force shading
com.google.code.findbugs:jsr305
org.slf4j：*
log4j：*
*:*
META-INF/*.SF
META-INF/*.DSA
META-INF/*.RSA
org.qscale.StreamingJob
org.eclipse.m2e
生命周期映射
1.0.0
org.apache.maven.plugins
maven阴影插件
[3.1.1,)
阴凉处
org.apache.maven.plugins
maven编译器插件
[3.1,)
测试编译
编译


感谢您的帮助！
您如何在flink？IDE？Docker？EC2中运行它
您可能需要将依赖项添加到类路径中

您还可以尝试更改要在pom中编译的范围依赖项

在main方法中编译添加
Class.forName（“com.microsoft.sqlserver.jdbc.SQLServerDriver”）
对您很有用，因为着色看起来是正确的

另一个问题是，您正在使用Java1.8在pom中编译，但添加了一个使用Java11编译的依赖项

<dependency>
      <groupId>com.microsoft.sqlserver</groupId>
      <artifactId>mssql-jdbc</artifactId>
      <version>8.2.2.jre8</version>
  </dependency>


com.microsoft.sqlserver
有关更多详细信息，请参见图片中的db字符串以及凭据。这被视为敏感信息。只需进行一次简单的检查。您是否添加了
Class.forName
子句？@chuckskull作为依赖项？对于我的项目，请使用org.qscale.StreamingJob in the shade plugin->transformerssection@chuckskull哈，我想我找到公关了问题。我在使用您的建议运行时出现此错误：原因：java.lang.UnsupportedClassVersionError:com/microsoft/sqlserver/jdbc/SQLServerDriver已由java运行时的最新版本（类文件版本55.0）编译，此版本的Java运行时仅识别高达52.0的类文件版本。容器中的Java版本是OpenJdk8。我想这就是问题所在，也可能是它之前无法工作的原因。将其添加为答案。您好，感谢您的答案。因此，它在docker中运行。默认情况下，如果未指定范围，则将其设置为c我如何将依赖性添加到类路径？