Java Hadoop LZO奇怪的本机LZO库不可用错误

Java Hadoop LZO奇怪的本机LZO库不可用错误,java,hadoop,lzo,Java,Hadoop,Lzo,我已经安装了Cloudera Hadoop LZO包,并在我的客户端环境安全阀中添加了以下设置: HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/* JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/native 但是,我得到了最奇怪的本地lzo库不可用错误:

我已经安装了Cloudera Hadoop LZO包,并在我的客户端环境安全阀中添加了以下设置:

HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/*
JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/native
但是,我得到了最奇怪的本地lzo库不可用错误:

13/08/05 23:59:06 INFO lzo.GPLNativeCodeLoader: Loaded native gpl library
13/08/05 23:59:06 INFO lzo.LzoCodec: Successfully loaded & initialized native-lzo library [hadoop-lzo rev 6298911ef75545c61859c08add6a74a83e0183ad]
13/08/05 23:59:07 INFO mapred.JobClient: Running job: job_201308052350_0003
13/08/05 23:59:08 INFO mapred.JobClient:  map 0% reduce 0%
13/08/05 23:59:18 INFO mapred.JobClient: Task Id : attempt_201308052350_0003_m_000000_0, Status : FAILED
java.lang.RuntimeException: native-lzo library not available
        at com.hadoop.compression.lzo.LzopCodec.getDecompressorType(LzopCodec.java:96)
        at org.apache.hadoop.io.compress.CodecPool.getDecompressor(CodecPool.java:131)
        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:86)
        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:478)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:671)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
        at org.apache.hadoop.mapred.Child.main(Child.java:262)

为什么它会说本机lzo库加载成功,然后抱怨本机lzo库不可用?这些异常是否来自DataNodes?

问题是我们没有在DataNodes上安装lzop!经过以下指示,一切正常:

sudo apt-get install lzop

希望有帮助

加载的日志表示本机lzo已加载到JobClient中。但投诉来自地图绘制者。这意味着TaskTracker无法找到本地lzo。