Warning: file_get_contents(/data/phpspider/zhask/data//catemap/0/hadoop/6.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Hadoop 使用Ooize时重置TableMapReduceUtil的Kerberos登录上下文_Hadoop_Mapreduce_Hbase_Kerberos_Oozie - Fatal编程技术网

Hadoop 使用Ooize时重置TableMapReduceUtil的Kerberos登录上下文

Hadoop 使用Ooize时重置TableMapReduceUtil的Kerberos登录上下文,hadoop,mapreduce,hbase,kerberos,oozie,Hadoop,Mapreduce,Hbase,Kerberos,Oozie,我尝试在受Kerberos保护的CDH群集上运行mapreduce作业。该作业从hdfs(映射器)读取文件并写入hbase表(reducer/TableMapReduceUtil)类 我做的作业配置是 Configuration hbaseConfigurarion = HBaseConfiguration.create(); hbaseConfigurarion.set("hbase.zookeeper.quorum", args[1]); hbaseConfigur

我尝试在受Kerberos保护的CDH群集上运行mapreduce作业。该作业从hdfs(映射器)读取文件并写入hbase表(reducer/TableMapReduceUtil)类

我做的作业配置是

Configuration hbaseConfigurarion =  HBaseConfiguration.create();            
hbaseConfigurarion.set("hbase.zookeeper.quorum", args[1]);
hbaseConfigurarion.set("hbase.zookeeper.property.clientPort", "2181");
hbaseConfigurarion.set("hadoop.security.authentication", "kerberos");
hbaseConfigurarion.set("hbase.security.authentication", "kerberos");
hbaseConfigurarion.addResource(new Path("/etc/hadoop/conf/core-site.xml"));
hbaseConfigurarion.addResource(new Path("/etc/hadoop/conf/hdfs-site.xml"));
hbaseConfigurarion.addResource(new Path("/etc/hadoop/conf/yarn-site.xml"));
hbaseConfigurarion.addResource(new Path("/etc/hadoop/conf/mapred-site.xml"));
hbaseConfigurarion.addResource(new Path("/etc/hbase/conf/hbase-site.xml"));
hbaseConfigurarion.addResource(new Path("/etc/oozie/conf/oozie-site.xml"));

Job job = new Job(hbaseConfigurarion, "hdfs_hbase_test");
job.setJarByClass(HdfsHbaseTestDriver.class);

job.setInputFormatClass(TextInputFormat.class);
TextInputFormat.addInputPath(job, new Path(args[0]));

job.setOutputFormatClass(TableOutputFormat.class);
TableMapReduceUtil.initTableReducerJob(args[2], null , job);

此MapReduce作业在使用Hadoop JAR命令运行时执行良好,它自动将Kerberos身份验证与存储在缓存中的TGT证书进行有效更新。ID@COMPANY.COM.

但是,当我使用oozie workflow运行此作业时,它失败了,行TableMapReduceUtil.initTableReducerJob()中出现以下异常

我真的不知道为什么登录的用户名是我的_ID@COMPANY.COM当使用oozie workflow运行时,TableMapReduceUtil.initTableReducerJob()行被执行时,成为我的_ID,KERBEROS身份验证方法变得简单


是否还有我缺少的配置…请建议添加TableMapReduceUtil.initCredentials(作业)

2014-09-25 20:19:21,611 WARN [main] org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:MY_ID (auth:SIMPLE) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]

2014-09-25 20:19:21,612 WARN [main] org.apache.hadoop.ipc.RpcClient: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
2014-09-25 20:19:21,613 FATAL [main] org.apache.hadoop.ipc.RpcClient: SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'.
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:136)
at org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupSaslConnection(RpcClient.java:762)
at org.apache.hadoop.hbase.ipc.RpcClient$Connection.access$600(RpcClient.java:354)
at org.apache.hadoop.hbase.ipc.RpcClient$Connection$2.run(RpcClient.java:883)
at org.apache.hadoop.hbase.ipc.RpcClient$Connection$2.run(RpcClient.java:880)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupIOstreams(RpcClient.java:880)
at org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1535)
at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1424)
.
.
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)