Scala 配置单元仓库连接器-读取包含类型为array的列的配置单元表
当我尝试对包含类型为Array的列的配置单元表执行简单的选择时,会出现以下错误:Scala 配置单元仓库连接器-读取包含类型为array的列的配置单元表,scala,hadoop,hive,Scala,Hadoop,Hive,当我尝试对包含类型为Array的列的配置单元表执行简单的选择时,会出现以下错误: User class threw exception: java.lang.NoSuchMethodError: org.apache.hadoop.hive.llap.FieldDesc.getTypeInfo()Lshadehive/org/apache/hadoop/hive/serde2/typeinfo/TypeInfo; at com.hortonworks.spark.sql.hive.lla
User class threw exception: java.lang.NoSuchMethodError: org.apache.hadoop.hive.llap.FieldDesc.getTypeInfo()Lshadehive/org/apache/hadoop/hive/serde2/typeinfo/TypeInfo;
at com.hortonworks.spark.sql.hive.llap.util.SchemaUtil.convertSchema(SchemaUtil.java:25)
at com.hortonworks.spark.sql.hive.llap.HiveWarehouseDataSourceReader.getTableSchema(HiveWarehouseDataSourceReader.java:113)
at com.hortonworks.spark.sql.hive.llap.HiveWarehouseDataSourceReader.readSchema(HiveWarehouseDataSourceReader.java:124)
at org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation$.apply(DataSourceV2Relation.scala:56)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:224)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:164)
at com.hortonworks.spark.sql.hive.llap.HiveWarehouseSessionImpl.executeQuery(HiveWarehouseSessionImpl.java:62)
at com.orange.vodcastview.datainjector.MetaProcessorMain$.main(MetaProcessorMain.scala:34)
at com.orange.vodcastview.datainjector.MetaProcessorMain.main(MetaProcessorMain.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:721)
尝试查看这篇文章- 通常,如果类路径在编译时和运行时不同,就会出现此错误。检查在构建项目时使用的spark版本,以及在部署项目的环境中运行的版本