Hadoop 导入时发生Sqoop错误
我是sqoop新手,正在尝试从hadoopguide数据库导入MYSQL表widgets表中的表 我使用的是Hadoop版本0.20 我的Sqoop是Sqoop-1.4.4.bin_uuhadoop-0.20 我正在运行命令:Hadoop 导入时发生Sqoop错误,hadoop,sqoop,Hadoop,Sqoop,我是sqoop新手,正在尝试从hadoopguide数据库导入MYSQL表widgets表中的表 我使用的是Hadoop版本0.20 我的Sqoop是Sqoop-1.4.4.bin_uuhadoop-0.20 我正在运行命令: sqoop import --connect jdbc:mysql://localhost/hadoopguide --table widgets -m 1 这是我收到的错误日志 Warning: /usr/lib/hcatalog does not exist! HC
sqoop import --connect jdbc:mysql://localhost/hadoopguide --table widgets -m 1
这是我收到的错误日志
Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
13/09/25 15:29:41 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
13/09/25 15:29:41 INFO tool.CodeGenTool: Beginning code generation
13/09/25 15:29:41 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `widgets` AS t LIMIT 1
13/09/25 15:29:41 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `widgets` AS t LIMIT 1
13/09/25 15:29:41 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/local/hadoop
13/09/25 15:29:41 INFO orm.CompilationManager: Found hadoop core jar at: /usr/local/hadoop/hadoop-0.20.2-core.jar
Note: /tmp/sqoop-ubuntu/compile/348861f092b25aac3fae4089da9abdf0/widgets.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
13/09/25 15:29:42 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop- ubuntu/compile/348861f092b25aac3fae4089da9abdf0/widgets.jar
13/09/25 15:29:42 WARN manager.MySQLManager: It looks like you are importing from mysql.
13/09/25 15:29:42 WARN manager.MySQLManager: This transfer can be faster! Use the -- direct
13/09/25 15:29:42 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
13/09/25 15:29:42 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
13/09/25 15:29:42 INFO mapreduce.ImportJobBase: Beginning import of widgets
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.mapred.JobConf.getCredentials()Lorg/apache/hadoop/security/Credentials;
at org.apache.sqoop.mapreduce.db.DBConfiguration.getPassword(DBConfiguration.java:304)
at org.apache.sqoop.mapreduce.db.DBConfiguration.getConnection(DBConfiguration.java:272)
at org.apache.sqoop.mapreduce.db.DBInputFormat.getConnection(DBInputFormat.java:187)
at org.apache.sqoop.mapreduce.db.DBInputFormat.setConf(DBInputFormat.java:162)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:882)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:186)
at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:159)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:239)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:600)
at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:118)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:413)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
有人知道吗。Hadoop 0.20版是一个非常旧的版本,缺少很多功能。Sqoop需要的一个特性是在1.x中添加的安全添加。因此,Sqoop无法在bare 0.20上工作,至少需要CDH3u1或Hadoop 1.x。我强烈建议您升级Hadoop群集。Hadoop 0.20版是非常旧的版本,缺少很多功能。Sqoop需要的一个特性是在1.x中添加的安全添加。因此,Sqoop无法在bare 0.20上工作,至少需要CDH3u1或Hadoop 1.x。我强烈建议您升级Hadoop群集。如果您已经安装了hive,则会随它一起安装hcatalog。现在在
.bashrc
中设置HCAT\u HOME
,如下所示
cd ~
gedit .bashrc
export HCAT_HOME=${HIVE_HOME}/hcatalog/
export PATH=$HCAT_HOME/bin:$PATH
source .bashrc //to refresh the .bashrc file
否则,请手动安装hcatalog并设置主路径。如果您已安装了hive,则hcatalog将随它一起安装。现在在
.bashrc
中设置HCAT\u HOME
,如下所示
cd ~
gedit .bashrc
export HCAT_HOME=${HIVE_HOME}/hcatalog/
export PATH=$HCAT_HOME/bin:$PATH
source .bashrc //to refresh the .bashrc file
否则,安装hcatalog saperately并设置home path。您能用
sqoop list tables列出表吗?连接jdbc:mysql://localhost/hadoopguide
?您能用sqoop列表表列出表吗?连接jdbc:mysql://localhost/hadoopguide
?