Java HDFS写入导致“错误”;CreateSymbolicLink错误(1314):客户端未持有所需的权限;
尝试从执行示例映射还原程序。运行map reduce作业时出现以下异常。尝试了Java HDFS写入导致“错误”;CreateSymbolicLink错误(1314):客户端未持有所需的权限;,java,hadoop,mapreduce,hdfs,Java,Hadoop,Mapreduce,Hdfs,尝试从执行示例映射还原程序。运行map reduce作业时出现以下异常。尝试了hdfs-dfs-chmod 777/,但没有解决问题 15/03/10 13:13:10 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy th
hdfs-dfs-chmod 777/
,但没有解决问题
15/03/10 13:13:10 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with
ToolRunner to remedy this.
15/03/10 13:13:10 WARN mapreduce.JobSubmitter: No job jar file set. User classes may not be found. See Job or Job#setJar(String).
15/03/10 13:13:10 INFO input.FileInputFormat: Total input paths to process : 2
15/03/10 13:13:11 INFO mapreduce.JobSubmitter: number of splits:2
15/03/10 13:13:11 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1425973278169_0001
15/03/10 13:13:12 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar to the list of resources.
15/03/10 13:13:12 INFO impl.YarnClientImpl: Submitted application application_1425973278169_0001
15/03/10 13:13:12 INFO mapreduce.Job: The url to track the job: http://B2ML10803:8088/proxy/application_1425973278169_0001/
15/03/10 13:13:12 INFO mapreduce.Job: Running job: job_1425973278169_0001
15/03/10 13:13:18 INFO mapreduce.Job: Job job_1425973278169_0001 running in uber mode : false
15/03/10 13:13:18 INFO mapreduce.Job: map 0% reduce 0%
15/03/10 13:13:18 INFO mapreduce.Job: Job job_1425973278169_0001 failed with state FAILED due to: Application application_1425973278169_0001 failed 2 times due
to AM Container for appattempt_1425973278169_0001_000002 exited with exitCode: 1
For more detailed output, check application tracking page:http://B2ML10803:8088/proxy/application_1425973278169_0001/Then, click on links to logs of each attemp
t.
Diagnostics: Exception from container-launch.
Container id: container_1425973278169_0001_02_000001
Exit code: 1
Exception message: CreateSymbolicLink error (1314): A required privilege is not held by the client.
堆栈跟踪:
ExitCodeException exitCode=1: CreateSymbolicLink error (1314): A required privilege is not held by the client.
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
外壳输出:
1 file(s) moved.
Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
15/03/10 13:13:18 INFO mapreduce.Job: Counters: 0
我不知道错误的原因,但重新格式化NameNode有助于我在Windows 8中解决此问题
在管理员模式下使用调用命令重新格式化NameNode:c:\hadoop\bin>hdfs NameNode-format
我最近遇到了完全相同的问题。我尝试重新格式化namenode,但它不起作用,我相信这无法永久解决问题。在@aoetalks的引用下,我通过研究本地组策略在Windows Server 2012 R2上解决了这个问题 总之,请尝试以下步骤:
Win+R
打开“运行…”-键入gpedit.msc
)用户帐户控制:在同一目录下以管理员批准模式运行所有管理员(即组策略中的用户权限分配),然后重新启动计算机以使其生效
参考资料:我遇到了与您相同的问题。我们通过检查java环境解决了这个问题
检查java版本
和javac版本
确保集群中的每台计算机都具有相同的java环境
在Windows中,将hdfs-site.xml中的配置更改为
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:///C:/hadoop-2.7.2/data/namenode</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:///C:/hadoop-2.7.2/data/datanode</value>
</property>
</configuration>
dfs.replication
1.
dfs.namenode.name.dir
file:///C:/hadoop-2.7.2/data/namenode
dfs.datanode.data.dir
file:///C:/hadoop-2.7.2/data/datanode
在管理模式下打开cmd并运行命令:-
- stop-all.cmd
- hdfs namenode–格式
- start-all.cmd
然后在管理模式下运行最后一个jar
hadoop jar C:\hadoop\u Demo\wordCount\target\wordCount-0.0.1-SNAPSHOT.jarfile:///C:/Hadoop/input.txt file:///C:/Hadoop/output我解决了同样的问题。当您运行“命令提示符”时,让我们“以管理员身份运行”。在windows7下也有同样的问题,我不认为它的hdfs问题,更像是在windows下运行它导致了问题。使其在mac os下工作,而无需麻烦Win 10+hadoop 2.7.3。它对我有用。至少工作开始了。但我遇到了错误:java.lang.RuntimeException:在org.apache.hadoop.xxx配置对象时出错
我应该在admin提示符下运行所有作业吗?