Apache flink 将作业jar添加到$FLINK_HOME/lib时出现以下类强制转换异常

Apache flink 将作业jar添加到$FLINK_HOME/lib时出现以下类强制转换异常,apache-flink,flink-streaming,Apache Flink,Flink Streaming,根据flink文档,为了避免动态类加载,可以执行以下操作。 当运行Flink JobManager和TaskManager是某个特定作业专用的安装程序时,可以将JAR文件直接放入/lib文件夹,以确保它们是类路径的一部分,而不是动态类加载 但是,当JAR添加到/lib文件夹时,会收到以下异常。 是否有解决此错误的方法 org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot instantiate user f

根据flink文档,为了避免动态类加载,可以执行以下操作。 当运行Flink JobManager和TaskManager是某个特定作业专用的安装程序时,可以将JAR文件直接放入/lib文件夹,以确保它们是类路径的一部分,而不是动态类加载

但是,当JAR添加到/lib文件夹时,会收到以下异常。 是否有解决此错误的方法

   org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot instantiate user function.
        at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:235) ~[iot-mirror-device.jar:na]
        at org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:95) ~[iot-mirror-device.jar:na]
        at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:231) ~[iot-mirror-device.jar:na]
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718) ~[iot-mirror-device.jar:na]
        at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_91]
Caused by: java.lang.ClassCastException: cannot assign instance of org.apache.commons.collections.map.LinkedMap to field org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.pendingOffsetsToCommit of type org.apache.commons.collections.map.LinkedMap in instance of org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010
        at java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2133) ~[na:1.8.0_91]
        at java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1305) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2024) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373) ~[na:1.8.0_91]
        at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:290) ~[iot-mirror-device.jar:na]
        at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:248) ~[iot-mirror-device.jar:na]
        at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:220) ~[iot-mirror-device.jar:na]
        ... 4 common frames omitted
Timestamp=2018-03-26 13:46:42,433 LogLevel=INFO  ThreadId=[flink-akka.actor.default-dispatcher-6] Class=o.a.f.r.e.ExecutionGraph Msg=Source: Custom Source -> Sink: Unnamed (1/1) (3f12f6953a235eb43f07cdf7966b5fcf) switched from RUNNING to FAILED.
org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot instantiate user function.
        at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:235) ~[iot-mirror-device.jar:na]
        at org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:95) ~[iot-mirror-device.jar:na]
        at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:231) ~[iot-mirror-device.jar:na]
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718) ~[iot-mirror-device.jar:na]
        at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_91]
Caused by: java.lang.ClassCastException: cannot assign instance of org.apache.commons.collections.map.LinkedMap to field org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.pendingOffsetsToCommit of type org.apache.commons.collections.map.LinkedMap in instance of org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010
        at java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2133) ~[na:1.8.0_91]
        at java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1305) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2024) ~[na:1.8.0_91]
        at java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2133) ~[na:1.8.0_91]
        at java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1305) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2024) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353) ~[na:1.8.0_91]
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373) ~[na:1.8.0_91]
        at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:290) ~[iot-mirror-device.jar:na]
        at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:248) ~[iot-mirror-device.jar:na]
        at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:220) ~[iot-mirror-device.jar:na]
        ... 4 common frames omitted
org.apache.flink.streaming.runtime.tasks.StreamTaskException:无法实例化用户函数。
在org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:235)~[iot mirror device.jar:na]
在org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:95)~[iot mirror device.jar:na]
在org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:231)~[iot mirror device.jar:na]
在org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)~[iot mirror device.jar:na]
在java.lang.Thread.run(Thread.java:745)~[na:1.8.091]
原因:java.lang.ClassCastException:无法将org.apache.commons.collections.map.LinkedMap实例分配给org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.PendingOffsetToCommit类型的字段org.apache.commons.collections.map.LinkedMap,该字段位于org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010实例中
在java.io.ObjectStreamClass$FieldReflector.SetObjJFieldValues(ObjectStreamClass.java:2133)~[na:1.8.091]
在java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1305)~[na:1.8.091]
在java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2024)~[na:1.8.091]
在java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)~[na:1.8.091]
在java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)~[na:1.8.091]
在java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)~[na:1.8.091]
在java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)~[na:1.8.091]
在java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)~[na:1.8.091]
在java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)~[na:1.8.091]
在java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)~[na:1.8.091]
在java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)~[na:1.8.091]
在org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:290)~[iot mirror device.jar:na]
在org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:248)~[iot mirror device.jar:na]
在org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:220)~[iot mirror device.jar:na]
... 省略4个公共框架
Timestamp=2018-03-26 13:46:42433 LogLevel=INFO ThreadId=[flink akka.actor.default-dispatcher-6]Class=o.a.f.r.e.ExecutionGraph Msg=Source:Custom Source->Sink:Unnamed(1/1)(3f12f6953a235eb43f07cdf7966b5fcf)从运行切换到失败。
org.apache.flink.streaming.runtime.tasks.StreamTaskException:无法实例化用户函数。
在org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:235)~[iot mirror device.jar:na]
在org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:95)~[iot mirror device.jar:na]
在org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:231)~[iot mirror device.jar:na]
在org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)~[iot mirror device.jar:na]
在java.lang.Thread.run(Thread.java:745)~[na:1.8.091]
原因:java.lang.ClassCastException:无法将org.apache.commons.collections.map.LinkedMap实例分配给org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.PendingOffsetToCommit类型的字段org.apache.commons.collections.map.LinkedMap,该字段位于org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010实例中
在java.io.ObjectStreamClass$FieldReflector.SetObjJFieldValues(ObjectStreamClass.java:2133)~[na:1.8.091]
在java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1305)~[na:1.8.091]
在java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2024)~[na:1.8.091]
在java.io.ObjectStreamClass$FieldReflector.SetObjJFieldValues(ObjectStreamClass.java:2133)~[na:1.8.091]
在java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1305)~[na:1.8.091]
在java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2024)~[na:1.8.091]
在java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)~[na:1.8.091]
在java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)~[na:1.8.091]
在java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)~[na:1.8.091]
在java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)~[na:1.8.091]
在java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)~[na:1.8.091]
在java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)~[na:1.8.091]
在java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)~[na:1.8.091]
在java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)~[na:1.8.091]
在org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:290)~[iot mirror device.jar:na]
在org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:248)~[iot mirror device.jar:na]
在org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:220)~[iot mirror device.jar:n
/** Data for pending but uncommitted offsets. */
private final LinkedMap pendingOffsetsToCommit = new LinkedMap();
classloader.resolve-order: parent-first