Hive Sqoop在色调工作流中失败
在命令shell中运行以下sqoop导入时,效果良好Hive Sqoop在色调工作流中失败,hive,sqoop,hue,cloudera-quickstart-vm,oozie-workflow,Hive,Sqoop,Hue,Cloudera Quickstart Vm,Oozie Workflow,在命令shell中运行以下sqoop导入时,效果良好 import --connect "jdbc:mysql://quickstart.cloudera:3306/retail_db" --username retail_dba --password cloudera -m 1 --table categories --hive-database retail_stage --hive-table categories --fields-terminated-by "|" --hive-imp
import --connect "jdbc:mysql://quickstart.cloudera:3306/retail_db" --username retail_dba --password cloudera -m 1 --table categories --hive-database retail_stage --hive-table categories --fields-terminated-by "|" --hive-import
但是同一条语句在工作流中运行失败,出现以下错误
>>> Invoking Sqoop command line now >>>
2019-02-04 11:46:18,411 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
2019-02-04 11:46:18,609 [main] INFO org.apache.sqoop.Sqoop - Running Sqoop version: 1.4.6-cdh5.13.0
2019-02-04 11:46:18,664 [main] WARN org.apache.sqoop.tool.BaseSqoopTool - Setting your password on the command-line is insecure. Consider using -P instead.
2019-02-04 11:46:18,696 [main] WARN org.apache.sqoop.ConnFactory - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
2019-02-04 11:46:18,936 [main] INFO org.apache.sqoop.manager.MySQLManager - Preparing to use a MySQL streaming resultset.
2019-02-04 11:46:18,951 [main] INFO org.apache.sqoop.tool.CodeGenTool - Beginning code generation
2019-02-04 11:46:20,510 [main] INFO org.apache.sqoop.manager.SqlManager - Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1
2019-02-04 11:46:20,555 [main] INFO org.apache.sqoop.manager.SqlManager - Executing SQL statement: SELECT t.* FROM `categories` AS t LIMIT 1
2019-02-04 11:46:20,565 [main] INFO org.apache.sqoop.orm.CompilationManager - HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce
2019-02-04 11:46:25,907 [main] ERROR org.apache.sqoop.tool.ImportTool - Import failed: java.io.IOException: Error returned by javac
at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:222)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:494)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81)
at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
<<< Invocation of Sqoop command completed <<<
No child hadoop job is executed.
Intercepting System.exit(1)
<<< Invocation of Main class completed <<<
现在调用Sqoop命令行>>>
2019-02-04 11:46:18411[main]WARN org.apache.sqoop.tool.SqoopTool-$sqoop_CONF_DIR未在环境中设置。无法检查其他配置。
2019-02-04 11:46:18609[main]INFO org.apache.sqoop.sqoop-运行sqoop版本:1.4.6-cdh5.13.0
2019-02-04 11:46:18664[main]WARN org.apache.sqoop.tool.BaseSqoopTool-在命令行上设置密码是不安全的。考虑使用-P代替。
2019-02-04 11:46:18696[main]WARN org.apache.sqoop.ConnFactory-$sqoop_CONF_DIR尚未在环境中设置。无法检查其他配置。
2019-02-04 11:46:18936[main]INFO org.apache.sqoop.manager.MySQLManager-准备使用MySQL流媒体结果集。
2019-02-04 11:46:18951[main]INFO org.apache.sqoop.tool.CodeGenTool-开始生成代码
2019-02-04 11:46:20510[main]INFO org.apache.sqoop.manager.SqlManager-执行SQL语句:从'categories'中选择t.*作为t限制1
2019-02-04 11:46:20555[main]INFO org.apache.sqoop.manager.SqlManager-正在执行SQL语句:从'categories'中选择t.*作为t限制1
2019-02-04 11:46:20565[main]INFO org.apache.sqoop.orm.compilementmanager-HADOOP\u MAPRED\u HOME是/usr/lib/HADOOP mapreduce
2019-02-04 11:46:25907[main]ERROR org.apache.sqoop.tool.ImportTool-导入失败:java.io.IOException:javac返回的错误
位于org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:222)
位于org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
位于org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:494)
位于org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
位于org.apache.sqoop.sqoop.run(sqoop.java:147)
位于org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
位于org.apache.sqoop.sqoop.runSqoop(sqoop.java:183)
位于org.apache.sqoop.sqoop.runTool(sqoop.java:234)
位于org.apache.sqoop.sqoop.runTool(sqoop.java:243)
位于org.apache.sqoop.sqoop.main(sqoop.java:252)
位于org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:187)
位于org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:170)
位于org.apache.oozie.action.hadoop.launchemain.run(launchemain.java:81)
位于org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51)
在sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)处
在sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)中
在sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)中
位于java.lang.reflect.Method.invoke(Method.java:606)
在org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235)上
位于org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
位于org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
位于org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
位于org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
位于java.security.AccessController.doPrivileged(本机方法)
位于javax.security.auth.Subject.doAs(Subject.java:415)
位于org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
位于org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
你检查过命令shell中的命令吗?@GaurangShah,是的,它从命令shell中不流动地工作,但从色调工作流中运行时,它总是以终止状态结束。你解决过这个问题吗?