错误:java.lang.NoClassDefFoundError:org/apache/spark/internal/Logging
我试图在spark中创建SparkSession(运行NaiveBayes模型),但出现以下错误:错误:java.lang.NoClassDefFoundError:org/apache/spark/internal/Logging,java,maven,Java,Maven,我试图在spark中创建SparkSession(运行NaiveBayes模型),但出现以下错误: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/internal/Logging at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(Unknown Source) a
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/internal/Logging
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(Unknown Source)
at java.security.SecureClassLoader.defineClass(Unknown Source)
at java.net.URLClassLoader.defineClass(Unknown Source)
at java.net.URLClassLoader.access$100(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at com.bnym.spark.SparkML.App.main(App.java:20)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.internal.Logging
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
... 13 more
我的java代码非常简单:
package com.bnym.spark.SparkML;
import org.apache.spark.ml.classification.NaiveBayes;
import org.apache.spark.ml.classification.NaiveBayesModel;
import org.apache.spark.ml.evaluation.MulticlassClassificationEvaluator;
import org.apache.spark.sql.Dataset;
import org.apache.spark.sql.Row;
import org.apache.spark.sql.SparkSession;
/**
* Naive Bayes Example
*
*/
public class App
{
public static void main( String[] args )
{
//System.out.println( "Hello World!" );
SparkSession spark = SparkSession
.builder()
.appName("NB")
.getOrCreate();
// Load training data
//Dataset<Row> dataFrame = spark.read().format("libsvm").load("data/mllib/sample_libsvm_data.txt");
}
}
包com.bnym.spark.SparkML;
导入org.apache.spark.ml.classification.bayes;
导入org.apache.spark.ml.classification.bayesmodel;
导入org.apache.spark.ml.evaluation.MultiClassificationEvaluator;
导入org.apache.spark.sql.Dataset;
导入org.apache.spark.sql.Row;
导入org.apache.spark.sql.SparkSession;
/**
*朴素贝叶斯例子
*
*/
公共类应用程序
{
公共静态void main(字符串[]args)
{
//System.out.println(“你好,世界!”);
火花会话火花=火花会话
.builder()
.appName(“NB”)
.getOrCreate();
//负荷训练数据
//Dataset dataFrame=spark.read().format(“libsvm”).load(“data/mllib/sample_libsvm_data.txt”);
}
}
我的pom.xml文件如下:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.bnym.spark</groupId>
<artifactId>SparkML</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>SparkML</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-mllib_2.10 -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-mllib_2.10</artifactId>
<version>1.6.0</version>
<scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql_2.10 -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.10</artifactId>
<version>2.0.0</version>
</dependency>
</dependencies>
</project>
4.0.0
com.bnym.spark
斯巴克米尔
0.0.1-快照
罐子
斯巴克米尔
http://maven.apache.org
UTF-8
朱尼特
朱尼特
3.8.1
测试
org.apache.spark
spark-mllib_2.10
1.6.0
假如
org.apache.spark
spark-sql_2.10
2.0.0
如果您能帮我找出错误原因,我将不胜感激。请参阅1.5.2版。无法导入:org.apache.spark.sql.SparkSession;org.apache.spark.sql.Dataset;