Nutch:org.apache.hadoop.mapreduce.lib.input.InvalidInputException:输入路径不存在
当我执行nutch命令创建crawldb文件夹和内容时:Nutch:org.apache.hadoop.mapreduce.lib.input.InvalidInputException:输入路径不存在,hadoop,nutch,Hadoop,Nutch,当我执行nutch命令创建crawldb文件夹和内容时: soporte@CNEOSYLAP /usr/local/apache-nutch-2.2.1/runtime/local $ bin/nutch crawl urls -dir crawl -depth 3 -topN 5 我得到这个错误: InjectorJob: Using class org.apache.gora.hbase.store.HBaseStore as the Gora storage class. Excepti
soporte@CNEOSYLAP /usr/local/apache-nutch-2.2.1/runtime/local
$ bin/nutch crawl urls -dir crawl -depth 3 -topN 5
我得到这个错误:
InjectorJob: Using class org.apache.gora.hbase.store.HBaseStore as the Gora storage class.
Exception in thread "main" org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: file:/C:/cygwin/usr/local/apache-nutch-2.2.1/runtime/local/crawl
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:224)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:241)
at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:50)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:233)
at org.apache.nutch.crawl.Crawler.runTool(Crawler.java:68)
at org.apache.nutch.crawl.Crawler.run(Crawler.java:136)
at org.apache.nutch.crawl.Crawler.run(Crawler.java:250)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.crawl.Crawler.main(Crawler.java:257)
我正在使用apache-nutch-2.2.1、hadoop-0.20.2-core.jar、hbase-0.90.4.jar和CygWin setup 2.774
我没有安装hadoop,只有nutch安装中的hadoop库,因此不是分布式的,而是本地的nutch安装
有什么想法吗?
提前谢谢
编辑:
手动创建目录时,我遇到另一个错误:
soporte@CNEOSYLAP /usr/local/apache-nutch-2.2.1/runtime/local
$ mkdir crawl
soporte@CNEOSYLAP /usr/local/apache-nutch-2.2.1/runtime/local
$ chmod 777 crawl
soporte@CNEOSYLAP /usr/local/apache-nutch-2.2.1/runtime/local
$ bin/nutch crawl urls -dir crawl -depth 3 -topN 5
cygpath: can't convert empty path
InjectorJob: Using class org.apache.gora.hbase.store.HBaseStore as the Gora storage class.
Exception in thread "main" java.lang.RuntimeException: job failed: name=inject crawl, jobid=null
at org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:54)
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:233)
at org.apache.nutch.crawl.Crawler.runTool(Crawler.java:68)
at org.apache.nutch.crawl.Crawler.run(Crawler.java:136)
at org.apache.nutch.crawl.Crawler.run(Crawler.java:250)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.crawl.Crawler.main(Crawler.java:257)
如果您想使用
-dir crawl
,首先需要创建文件夹文件:/C:/cygwin/usr/local/apache-nutch-2.2.1/runtime/local/crawl
。我以前尝试过这个,但我有另一个错误:java.lang.RuntimeException:job failed:name=inject crawl,jobid=Null你能粘贴堆栈跟踪吗?你有没有发现这个问题?这个版本似乎不推荐使用“nutch crawl”命令。改为使用“爬网”脚本,例如:“bin/crawl URL爬网1”。