Warning: file_get_contents(/data/phpspider/zhask/data//catemap/4/maven/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Java 在MAPR中找不到Hadoop FS API文件问题_Java_Maven_Hadoop_Hdfs - Fatal编程技术网

Java 在MAPR中找不到Hadoop FS API文件问题

Java 在MAPR中找不到Hadoop FS API文件问题,java,maven,hadoop,hdfs,Java,Maven,Hadoop,Hdfs,我在运行hadoop fs api删除目录时遇到问题。即使我有配置,程序也会抛出异常。在解决问题时需要帮助 我正在使用下面的maven依赖项 hadoop common 2.4.1-mapr-1408 hadoop核心2.4.1-mapr-1408 hadoop客户端2.7.1 回购: 堆栈跟踪: 16/02/19 23:04:33 ERROR cldbutils.CLDBRpcCommonUtils: File is not found: /opt/mapr/conf/mapr-cluster

我在运行hadoop fs api删除目录时遇到问题。即使我有配置,程序也会抛出异常。在解决问题时需要帮助

我正在使用下面的maven依赖项

hadoop common 2.4.1-mapr-1408 hadoop核心2.4.1-mapr-1408 hadoop客户端2.7.1

回购:

堆栈跟踪:

16/02/19 23:04:33 ERROR cldbutils.CLDBRpcCommonUtils: File is not found: /opt/mapr/conf/mapr-clusters.conf
java.io.FileNotFoundException: \opt\mapr\conf\mapr-clusters.conf (The system cannot find the path specified)
    at java.io.FileInputStream.open(Native Method)
    at java.io.FileInputStream.<init>(FileInputStream.java:146)
    at java.io.FileInputStream.<init>(FileInputStream.java:101)
    at java.io.FileReader.<init>(FileReader.java:58)
    at com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils.init(CLDBRpcCommonUtils.java:144)
    at com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils.<init>(CLDBRpcCommonUtils.java:72)
    at com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils.<clinit>(CLDBRpcCommonUtils.java:63)
    at org.apache.hadoop.conf.CoreDefaultProperties.<clinit>(CoreDefaultProperties.java:68)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:274)
    at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1847)
    at org.apache.hadoop.conf.Configuration.getProperties(Configuration.java:2062)
    at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2272)
    at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2224)
    at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2141)
    at org.apache.hadoop.conf.Configuration.get(Configuration.java:1081)
    at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
    at com.cisco.installbase.hiveconnector.ShellUtilities.DeleteDirectory(ShellUtilities.java:24)
    at com.cisco.installbase.hiveconnector.MainApp.importTables(MainApp.java:66)
    at com.cisco.installbase.hiveconnector.MainApp.startTimeLogger(MainApp.java:51)
    at com.cisco.installbase.hiveconnector.MainApp.main(MainApp.java:40)
16/02/19 23:04:35 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/02/19 23:04:35 ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
    at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318)
    at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333)
    at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326)
    at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76)
    at org.apache.hadoop.security.Groups.parseStaticMapping(Groups.java:93)
    at org.apache.hadoop.security.Groups.<init>(Groups.java:77)
    at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:240)
    at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:279)
    at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:185)
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:621)
    at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:606)
    at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:519)
    at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2590)
    at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2582)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2448)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:404)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
    at com.cisco.installbase.hiveconnector.ShellUtilities.DeleteDirectory(ShellUtilities.java:24)
    at com.cisco.installbase.hiveconnector.MainApp.importTables(MainApp.java:66)
    at com.cisco.installbase.hiveconnector.MainApp.startTimeLogger(MainApp.java:51)
    at com.cisco.installbase.hiveconnector.MainApp.main(MainApp.java:40)
16/02/19 23:04:36 WARN fs.MapRFileSystem: Could not find any cluster, defaulting to localhost
Some error on socket 1472
2016-02-19 23:04:37,3423 ERROR Client fs/client/fileclient/cc/client.cc:394 Thread: 4332 Failed to initialize client for cluster 127.0.0.1:7222, error Cannot send after transport endpoint shutdown(108)
16/02/19 23:04:37 ERROR hiveconnector.ShellUtilities: Could not create FileClient
16/02/19 23:04:37 INFO hiveconnector.ShellUtilities: Deleting the HDFS directory /app/dev/SmartAnalytics/sqoop_temp/XXCCS_DS_CVDPRDLINE_DETAIL
Some error on socket 1488
2016-02-19 23:04:38,3434 ERROR Client fs/client/fileclient/cc/client.cc:394 Thread: 4332 Failed to initialize client for cluster 127.0.0.1:7222, error Cannot send after transport endpoint shutdown(108)
16/02/19 23:04:38 ERROR hiveconnector.ShellUtilities: Could not create FileClient
16/02/19 23:04:38 INFO hiveconnector.ShellUtilities: Deleting the HDFS directory /app/dev/SmartAnalytics/sqoop_temp/XXCCS_DS_INSTANCE_DETAIL
Some error on socket 1488
2016-02-19 23:04:39,3424 ERROR Client fs/client/fileclient/cc/client.cc:394 Thread: 4332 Failed to initialize client for cluster 127.0.0.1:7222, error Cannot send after transport endpoint shutdown(108)
16/02/19 23:04:39 ERROR hiveconnector.ShellUtilities: Could not create FileClient
16/02/19 23:04:39 INFO hiveconnector.ShellUtilities: Deleting the HDFS directory /app/dev/SmartAnalytics/sqoop_temp/XXCCS_DS_CVDPRDLINE_DETAIL
Some error on socket 1488
2016-02-19 23:04:40,3445 ERROR Client fs/client/fileclient/cc/client.cc:394 Thread: 4332 Failed to initialize client for cluster 127.0.0.1:7222, error Cannot send after transport endpoint shutdown(108)
16/02/19 23:04:40 ERROR hiveconnector.ShellUtilities: Could not create FileClient
16/02/19 23:04:40 INFO hiveconnector.ShellUtilities: Deleting the HDFS directory /app/dev/SmartAnalytics/sqoop_temp/XXCCS_DS_SAHDR_CORE
Some error on socket 1488
2016-02-19 23:04:41,3525 ERROR Client fs/client/fileclient/cc/client.cc:394 Thread: 4332 Failed to initialize client for cluster 127.0.0.1:7222, error Cannot send after transport endpoint shutdown(108)
16/02/19 23:04:41 ERROR hiveconnector.ShellUtilities: Could not create FileClient
16/02/19 23:04:33错误cldbutils.CLDBRpcCommonUtils:找不到文件:/opt/mapr/conf/mapr-clusters.conf
java.io.FileNotFoundException:\opt\mapr\conf\mapr-clusters.conf(系统找不到指定的路径)
在java.io.FileInputStream.open(本机方法)
位于java.io.FileInputStream。(FileInputStream.java:146)
位于java.io.FileInputStream。(FileInputStream.java:101)
位于java.io.FileReader。(FileReader.java:58)
位于com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils.init(CLDBRpcCommonUtils.java:144)
位于com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils。(CLDBRpcCommonUtils.java:72)
位于com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils。(CLDBRpcCommonUtils.java:63)
位于org.apache.hadoop.conf.CoreDefaultProperties。(CoreDefaultProperties.java:68)
位于java.lang.Class.forName0(本机方法)
位于java.lang.Class.forName(Class.java:274)
位于org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1847)
位于org.apache.hadoop.conf.Configuration.getProperties(Configuration.java:2062)
位于org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2272)
位于org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2224)
位于org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2141)
位于org.apache.hadoop.conf.Configuration.get(Configuration.java:1081)
位于org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
位于org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
位于com.cisco.installbase.hiveconnector.ShellUtilities.DeleteDirectory(ShellUtilities.java:24)
位于com.cisco.installbase.hiveconnector.MainApp.importables(MainApp.java:66)
位于com.cisco.installbase.hiveconnector.MainApp.startTimeLogger(MainApp.java:51)
位于com.cisco.installbase.hiveconnector.MainApp.main(MainApp.java:40)
16/02/19 23:04:35警告util.NativeCodeLoader:无法为您的平台加载本机hadoop库。。。在适用的情况下使用内置java类
16/02/19 23:04:35错误util.Shell:未能在hadoop二进制路径中找到winutils二进制文件
java.io.IOException:在Hadoop二进制文件中找不到可执行文件null\bin\winutils.exe。
位于org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318)
位于org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333)
位于org.apache.hadoop.util.Shell(Shell.java:326)
位于org.apache.hadoop.util.StringUtils。(StringUtils.java:76)
位于org.apache.hadoop.security.Groups.parseStaticMapping(Groups.java:93)
位于org.apache.hadoop.security.Groups.(Groups.java:77)
位于org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:240)
位于org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:279)
位于org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:185)
位于org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:621)
位于org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:606)
位于org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:519)
位于org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:2590)
位于org.apache.hadoop.fs.FileSystem$Cache$Key.(FileSystem.java:2582)
位于org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2448)
位于org.apache.hadoop.fs.FileSystem.get(FileSystem.java:404)
位于org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
位于com.cisco.installbase.hiveconnector.ShellUtilities.DeleteDirectory(ShellUtilities.java:24)
位于com.cisco.installbase.hiveconnector.MainApp.importables(MainApp.java:66)
位于com.cisco.installbase.hiveconnector.MainApp.startTimeLogger(MainApp.java:51)
位于com.cisco.installbase.hiveconnector.MainApp.main(MainApp.java:40)
16/02/19 23:04:36警告fs.MapRFileSystem:找不到任何群集,默认为localhost
套接字1472上出现一些错误
2016-02-19 23:04:373423错误客户端fs/Client/fileclient/cc/Client.cc:394线程:4332未能为群集127.0.0.1:7222初始化客户端,传输终结点关闭后无法发送错误(108)
16/02/19 23:04:37错误hiveconnector.ShellUtilities:无法创建FileClient
16/02/19 23:04:37信息hiveconnector.ShellUtilities:删除HDFS目录/app/dev/SmartAnalytics/sqoop\u temp/XXCCS\u DS\u cvdpridline\u详细信息
套接字1488上有一些错误
2016-02-19 23:04:383434错误客户端fs/Client/fileclient/cc/Client.cc:394线程:4332未能为群集127.0.0.1:7222初始化客户端,传输终结点关闭后无法发送错误(108)
16/02/19 23:04:38错误hiveconnector.ShellUtilities:无法创建FileClient
16/02/19 23:04:38信息hiveconnector.ShellUtilities:删除HDFS目录/app/dev/SmartAnalytics/sqoop\u temp/XXCCS\u DS\u实例\u详细信息
套接字1488上有一些错误
2016-02-19 23:04:393424错误客户端fs/Client/fileclient/cc/Client.cc:394线程:4332未能为群集127.0.0.1:7222初始化客户端,传输终结点关闭后无法发送错误(108)
16/02/19 23:04:39错误hiveconnector.ShellUtilities:无法创建FileClient
16/02/19 23:04:39信息hiveconnector.ShellUtilities:删除HDFS目录/app/dev/SmartAnalytics/sqoop\u temp/XXCCS\u DS\u cvdpridline\u详细信息
套接字1488上有一些错误
2016-02-19 23:04:403445错误客户端fs/Client/fileclient/cc/Client.cc:394线程:4332未能为群集127.0.0.1:7222初始化客户端,传输终结点关闭后无法发送错误(108)
16/02/19 23:04:40错误hiveconnector.ShellUtilities:无法创建FileClient
16/
16/02/19 23:04:33 ERROR cldbutils.CLDBRpcCommonUtils: File is not found: /opt/mapr/conf/mapr-clusters.conf
java.io.FileNotFoundException: \opt\mapr\conf\mapr-clusters.conf (The system cannot find the path specified)
    at java.io.FileInputStream.open(Native Method)
    at java.io.FileInputStream.<init>(FileInputStream.java:146)
    at java.io.FileInputStream.<init>(FileInputStream.java:101)
    at java.io.FileReader.<init>(FileReader.java:58)
    at com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils.init(CLDBRpcCommonUtils.java:144)
    at com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils.<init>(CLDBRpcCommonUtils.java:72)
    at com.mapr.baseutils.cldbutils.CLDBRpcCommonUtils.<clinit>(CLDBRpcCommonUtils.java:63)
    at org.apache.hadoop.conf.CoreDefaultProperties.<clinit>(CoreDefaultProperties.java:68)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:274)
    at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1847)
    at org.apache.hadoop.conf.Configuration.getProperties(Configuration.java:2062)
    at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2272)
    at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2224)
    at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2141)
    at org.apache.hadoop.conf.Configuration.get(Configuration.java:1081)
    at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:177)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
    at com.cisco.installbase.hiveconnector.ShellUtilities.DeleteDirectory(ShellUtilities.java:24)
    at com.cisco.installbase.hiveconnector.MainApp.importTables(MainApp.java:66)
    at com.cisco.installbase.hiveconnector.MainApp.startTimeLogger(MainApp.java:51)
    at com.cisco.installbase.hiveconnector.MainApp.main(MainApp.java:40)
16/02/19 23:04:35 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/02/19 23:04:35 ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
    at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318)
    at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333)
    at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326)
    at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76)
    at org.apache.hadoop.security.Groups.parseStaticMapping(Groups.java:93)
    at org.apache.hadoop.security.Groups.<init>(Groups.java:77)
    at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:240)
    at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:279)
    at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:185)
    at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:621)
    at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:606)
    at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:519)
    at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2590)
    at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2582)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2448)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:404)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
    at com.cisco.installbase.hiveconnector.ShellUtilities.DeleteDirectory(ShellUtilities.java:24)
    at com.cisco.installbase.hiveconnector.MainApp.importTables(MainApp.java:66)
    at com.cisco.installbase.hiveconnector.MainApp.startTimeLogger(MainApp.java:51)
    at com.cisco.installbase.hiveconnector.MainApp.main(MainApp.java:40)
16/02/19 23:04:36 WARN fs.MapRFileSystem: Could not find any cluster, defaulting to localhost
Some error on socket 1472
2016-02-19 23:04:37,3423 ERROR Client fs/client/fileclient/cc/client.cc:394 Thread: 4332 Failed to initialize client for cluster 127.0.0.1:7222, error Cannot send after transport endpoint shutdown(108)
16/02/19 23:04:37 ERROR hiveconnector.ShellUtilities: Could not create FileClient
16/02/19 23:04:37 INFO hiveconnector.ShellUtilities: Deleting the HDFS directory /app/dev/SmartAnalytics/sqoop_temp/XXCCS_DS_CVDPRDLINE_DETAIL
Some error on socket 1488
2016-02-19 23:04:38,3434 ERROR Client fs/client/fileclient/cc/client.cc:394 Thread: 4332 Failed to initialize client for cluster 127.0.0.1:7222, error Cannot send after transport endpoint shutdown(108)
16/02/19 23:04:38 ERROR hiveconnector.ShellUtilities: Could not create FileClient
16/02/19 23:04:38 INFO hiveconnector.ShellUtilities: Deleting the HDFS directory /app/dev/SmartAnalytics/sqoop_temp/XXCCS_DS_INSTANCE_DETAIL
Some error on socket 1488
2016-02-19 23:04:39,3424 ERROR Client fs/client/fileclient/cc/client.cc:394 Thread: 4332 Failed to initialize client for cluster 127.0.0.1:7222, error Cannot send after transport endpoint shutdown(108)
16/02/19 23:04:39 ERROR hiveconnector.ShellUtilities: Could not create FileClient
16/02/19 23:04:39 INFO hiveconnector.ShellUtilities: Deleting the HDFS directory /app/dev/SmartAnalytics/sqoop_temp/XXCCS_DS_CVDPRDLINE_DETAIL
Some error on socket 1488
2016-02-19 23:04:40,3445 ERROR Client fs/client/fileclient/cc/client.cc:394 Thread: 4332 Failed to initialize client for cluster 127.0.0.1:7222, error Cannot send after transport endpoint shutdown(108)
16/02/19 23:04:40 ERROR hiveconnector.ShellUtilities: Could not create FileClient
16/02/19 23:04:40 INFO hiveconnector.ShellUtilities: Deleting the HDFS directory /app/dev/SmartAnalytics/sqoop_temp/XXCCS_DS_SAHDR_CORE
Some error on socket 1488
2016-02-19 23:04:41,3525 ERROR Client fs/client/fileclient/cc/client.cc:394 Thread: 4332 Failed to initialize client for cluster 127.0.0.1:7222, error Cannot send after transport endpoint shutdown(108)
16/02/19 23:04:41 ERROR hiveconnector.ShellUtilities: Could not create FileClient