MongoDb Hadoop连接器使用Pig支持连接问题
MongoDb Hadoop连接器使用Pig支持连接问题,mongodb,hadoop,apache-pig,Mongodb,Hadoop,Apache Pig,错误2118:无法连接到集合。无法连接到集合。 我的猪代码: REGISTER /home/auto/ykale/jars/mongo/mongo-hadoop-pig_cdh3u3-1.1.0.jar REGISTER /home/auto/ykale/jars/mongo/mongo-hadoop-core_cdh3u3-1.1.0.jar REGISTER /home/auto/ykale/jars/mongo/mongo-hadoop-streaming_cdh3u3-1.1.0.jar
错误2118:无法连接到集合。无法连接到集合。
我的猪代码:
REGISTER /home/auto/ykale/jars/mongo/mongo-hadoop-pig_cdh3u3-1.1.0.jar
REGISTER /home/auto/ykale/jars/mongo/mongo-hadoop-core_cdh3u3-1.1.0.jar
REGISTER /home/auto/ykale/jars/mongo/mongo-hadoop-streaming_cdh3u3-1.1.0.jar
REGISTER /home/auto/ykale/jars/mongo/com.mongodb_2.6.5.1.jar
--name1 = load 'mongodb://hfdvmprmongodb1.vm.itg.corp.us.shldcorp.com:27017/member_pricing.testData' USING com.mongodb.hadoop.pig.MongoLoader;
name1 = load 'mongodb://ykale:newpassword4@hfdvmprmongodb1.vm.itg.corp.us.shldcorp.com:27017/member_pricing.testData' USING com.mongodb.hadoop.pig.MongoLoader;
STORE name1 into '/user/ykale/mongo_dump/file1';
当我使用上面代码中注释掉的另一个load命令时,我得到如下输出,即分配0个map reduce作业
2013-12-11 05:16:24,769 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete
2013-12-11 05:16:24,769 [main] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 0 map reduce job(s) failed!
2013-12-11 05:16:24,770 [main] INFO org.apache.pig.tools.pigstats.PigStats - Script Statistics:
HadoopVersion PigVersion UserId StartedAt FinishedAt Features
0.20.2-cdh3u3 0.8.1-cdh3u3 ykale 2013-12-11 05:16:22 2013-12-11 05:16:24 UNKNOWN
Failed!
Failed Jobs:
JobId Alias Feature Message Outputs
Input(s):
Output(s):
Counters:
Total records written : 0
Total bytes written : 0
Spillable Memory Manager spill count : 0
Total bags proactively spilled: 0
Total records proactively spilled: 0
Job DAG:
null
2013-12-11 05:16:24,770 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Success!
我正在使用Dmongo.input.split.create\u input\u splits=false执行Pig脚本
非常感谢您的帮助。您的意思是错误无法连接到集合。
是在运行此pig脚本时出现的吗?您能提供更多与错误相关的日志吗?mongo的日志也会有帮助。当我使用pig connector并运行上面的脚本时,mongo日志将在哪里生成?mongo日志位于运行mongodb的机器上,这是在启动mongodb进程时指定的。如果您能够找到堆栈中出现错误的位置,那就太好了。