“线程中的异常”;“主要”;java.lang.NoClassDefFoundError:org/apache/hadoop/mapreduce/RecordReader
我正在尝试将Json文件转换为拼花地板格式 下面是我的pom文件“线程中的异常”;“主要”;java.lang.NoClassDefFoundError:org/apache/hadoop/mapreduce/RecordReader,java,hadoop,kite-sdk,Java,Hadoop,Kite Sdk,我正在尝试将Json文件转换为拼花地板格式 下面是我的pom文件 <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://mav
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.mypackage</groupId>
<artifactId>JSONToParquet</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<repositories>
<repository>
<id>wso2</id>
<url>http://dist.wso2.org/maven2/</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.kitesdk</groupId>
<artifactId>kite-data-core</artifactId>
<version>1.1.0</version>
</dependency>
<dependency>
<groupId>org.kitesdk</groupId>
<artifactId>kite-morphlines-all</artifactId>
<version>1.0.0</version> <!-- or whatever the latest version is -->
<type>pom</type>
</dependency>
<!-- https://mvnrepository.com/artifact/ua_parser/ua-parser -->
<dependency>
<groupId>ua_parser</groupId>
<artifactId>ua-parser</artifactId>
<version>1.3.0</version>
<type>pom</type>
</dependency>
</dependencies>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
</project>
我使用的是JDK:8
我没有任何hadoop的背景,所以我无法理解它的根本原因
问题是什么?基于Kite SDK文档,
JSonfilerReader
,ParquetWriter
和AvroParquetWriter
使用Hadoop库工作。需要在pom中添加Hadoop依赖项。您至少需要以下依赖项。将它们添加到pom.xml中:
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-core</artifactId>
<version>2.6.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.6.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-jobclient</artifactId>
<version>2.6.0</version>
</dependency>
org.apache.hadoop
hadoop mapreduce客户端核心
2.6.0
org.apache.hadoop
hadoop通用
2.6.0
org.apache.hadoop
hadoop mapreduce客户端jobclient
2.6.0
基于Kite SDK文档,JSonfilerReader
,ParquetWriter
和AvroParquetWriter
使用Hadoop库工作。需要在pom中添加Hadoop依赖项。您至少需要以下依赖项。将它们添加到pom.xml中:
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-core</artifactId>
<version>2.6.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.6.0</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-mapreduce-client-jobclient</artifactId>
<version>2.6.0</version>
</dependency>
org.apache.hadoop
hadoop mapreduce客户端核心
2.6.0
org.apache.hadoop
hadoop通用
2.6.0
org.apache.hadoop
hadoop mapreduce客户端jobclient
2.6.0
你的风筝不见了
在某些情况下,您可能需要自己提供相关的Hadoop组件依赖项,而Kite具有用于此目的的分组依赖项
对于Haddop2(默认值),请将以下内容添加到pom中:
<dependency>
<groupId>org.kitesdk</groupId>
<artifactId>kite-hadoop2-dependencies</artifactId>
<version>1.0.0</version>
<type>pom</type>
<scope>compile</scope>
</dependency>
org.kitesdk
kite-hadoop2-dependencies
1.0.0
聚甲醛
编译
你的风筝不见了
在某些情况下,您可能需要自己提供相关的Hadoop组件依赖项,而Kite具有用于此目的的分组依赖项
对于Haddop2(默认值),请将以下内容添加到pom中:
<dependency>
<groupId>org.kitesdk</groupId>
<artifactId>kite-hadoop2-dependencies</artifactId>
<version>1.0.0</version>
<type>pom</type>
<scope>compile</scope>
</dependency>
org.kitesdk
kite-hadoop2-dependencies
1.0.0
聚甲醛
编译