Fatal编程技术网
  • hadoop/
  • 在HDFS中使用-addMount时未找到类org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem

在HDFS中使用-addMount时未找到类org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem

hadoop

在HDFS中使用-addMount时未找到类org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem,hadoop,hdfs,azure-blob-storage,Hadoop,Hdfs,Azure Blob Storage,我有以下设置： $ export |grep HADOOP declare -x HADOOP_HOME="/home/jesaremi/hadoop-3.1.3-bin" declare -x HADOOP_OPTIONAL_TOOLS="hadoop-azure" $ echo $PATH /home/jesaremi/hadoop-3.1.3-bin/bin:/home/jesaremi/hadoop-3.1.3-bin/sbin:/opt/protobuf/bin:/usr/local

我有以下设置：

$ export |grep HADOOP
declare -x HADOOP_HOME="/home/jesaremi/hadoop-3.1.3-bin"
declare -x HADOOP_OPTIONAL_TOOLS="hadoop-azure"

$ echo $PATH
/home/jesaremi/hadoop-3.1.3-bin/bin:/home/jesaremi/hadoop-3.1.3-bin/sbin:/opt/protobuf/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/jesaremi/.dotnet/tools:/opt/HP_Fortify/HP_Fortify_SCA_and_Apps_4.30/bin:/opt/HP_Fortify/HP_Fortify_SCA_and_Apps_4.30/bin


$ ls -l /home/jesaremi/hadoop-3.1.3-bin/share/hadoop/common/
total 7452
-rw-rw-r-- 1 jesaremi jesaremi  303043 Dec 19 17:30 hadoop-azure-3.1.3.jar
-rw-rw-r-- 1 jesaremi jesaremi 4092533 Dec 17 09:42 hadoop-common-3.1.3-SNAPSHOT.jar
-rw-rw-r-- 1 jesaremi jesaremi 2877294 Dec 17 09:42 hadoop-common-3.1.3-SNAPSHOT-tests.jar
-rw-rw-r-- 1 jesaremi jesaremi  130001 Dec 17 09:42 hadoop-kms-3.1.3-SNAPSHOT.jar
-rw-rw-r-- 1 jesaremi jesaremi  201637 Dec 17 09:42 hadoop-nfs-3.1.3-SNAPSHOT.jar
drwxrwxr-x 2 jesaremi jesaremi    4096 Dec 17 09:42 jdiff
drwxrwxr-x 2 jesaremi jesaremi    4096 Dec 17 09:42 lib
drwxrwxr-x 2 jesaremi jesaremi    4096 Dec 17 09:42 sources
drwxrwxr-x 3 jesaremi jesaremi    4096 Dec 17 09:42 webapps
然而，当我尝试对ABFS文件系统进行计数时，我得到以下结果：

$ hdfs dfsadmin -addMount abfs://main@test1.dfs.core.windows.net/ /mnt/abfs/main "fs.azure.abfs.account.name=test1.dfs.core.windows.net,fs.azure.account.key.test1.dfs.core.windows.net=<key removed>"
addMount: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem not found

$hdfs dfsadmin-addMountabfs://main@test1.dfs.core.windows.net//mnt/abfs/main“fs.azure.abfs.account.name=test1.dfs.core.windows.net，fs.azure.account.key.test1.dfs.core.windows.net=”
addMount:java.lang.ClassNotFoundException:Class org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem未找到
担心HADOOP\u可选工具env var不够；您需要将hadoop azure JAR和其他一些应用程序放到common/lib中

从share/hadoop/tools/lib将hadoop-azure-jar、azure-*和wildfly-openssl.jar复制到share/hadoop/common/lib中

JAR具有诊断功能，因为它会告诉您缺少哪个JAR，例如

bin/hadoop jar cloudstore-0.1-SNAPSHOT.jar storediag abfs://c@account.dfs.core.windows.net/