以其他用户身份运行Hadoop作业
我正在运行Hadoop1.2.1,它是为用户hadoopuser配置的。我正在尝试从我的windows计算机提交作业,这里有一个问题,因为在我的windows计算机上,我不再是Hadoop用户,因此我得到:以其他用户身份运行Hadoop作业,hadoop,mapreduce,Hadoop,Mapreduce,我正在运行Hadoop1.2.1,它是为用户hadoopuser配置的。我正在尝试从我的windows计算机提交作业,这里有一个问题,因为在我的windows计算机上,我不再是Hadoop用户,因此我得到: security.UserGroupInformation: PriviledgedActionException 有没有办法从我的windows机器上为hadoop用户提交作业 更新:堆栈跟踪: ERROR security.UserGroupInformation: Priviled
security.UserGroupInformation: PriviledgedActionException
有没有办法从我的windows机器上为hadoop用户提交作业
更新:堆栈跟踪:
ERROR security.UserGroupInformation: PriviledgedActionException as:windowsuser cause:org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied: user=windowsuser, access=WRITE, inode="mapred":hadoopuser:supergroup:rwxr-xr-x
org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied: user=windowsuser, access=WRITE, inode="mapred":hadoopuser:supergroup:rwxr-xr-x
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:57)
at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:1459)
at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:362)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:126)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:942)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1353)
at com.rf.mapreduce.WordCount.main(WordCount.java:40)
你能发布整个错误日志吗 我认为您的问题是因为在更改用户时没有清理输出文件 尝试:
路径取决于您的HDFS实现。我在原始问题中添加了堆栈跟踪。由于尚未执行作业且尚未创建输出文件夹,因此没有输出文件夹。问题通常是mapreduce试图在用户目录中创建某些内容,例如/user/windowsuser。由于用户“windowsuser”没有对/mapred的写入权限,因此此操作失败。如果您在hdfs中创建用户的主目录并将其发送给正确的用户,您可能会克服此错误。
hdfs dfs -rmr /home/hadoop/your-output-dir