如何在MapReduce的配置对象中将整数数组作为属性值传递?
我们可以将整数作为配置属性传递,如下所示:如何在MapReduce的配置对象中将整数数组作为属性值传递?,mapreduce,Mapreduce,我们可以将整数作为配置属性传递,如下所示: Configuration conf = new Configuration(); conf.set("size", 4); 有没有办法将整数数组作为属性值发送 conf.set("list.of.nums", {2, 4, 6, 8}); // one way is to pass them as a String but it doesn't look good 您可以将数组对象序列化为一个文件,然后将该文件移动到HDFS Distribute
Configuration conf = new Configuration();
conf.set("size", 4);
有没有办法将整数数组作为属性值发送
conf.set("list.of.nums", {2, 4, 6, 8}); // one way is to pass them as a String but it doesn't look good
您可以将数组对象序列化为一个文件,然后将该文件移动到HDFS
DistributedCache.addCacheFile(new URI(dfsMetaPath + "#"
+ Constants.OBJSYMLINK0), conf);
DistributedCache.createSymlink(conf);
序列化可以按如下方式进行:-
public static <T> void serializeMetadata(T voObj,
String filePath) throws IOException,NullPointerException {
if(null==voObj){
throw new NullPointerException("NULL object found");
}
ObjectOutputStream oout = null;
FileOutputStream fsout = null;
try {
fsout = new FileOutputStream(filePath);
oout = new ObjectOutputStream(fsout);
oout.writeObject(voObj);
oout.close();
} finally {
if (null != fsout) {
fsout.close();
}
if (null != oout) {
oout.close();
}
}
}
对于反序列化,可以使用以下代码:-
public static <T> T deserializeMetadata(T voObj,
String filePath) throws IOException,NullPointerException, ClassNotFoundException {
FileInputStream fsin = null;
ObjectInputStream oin = null;
try {
fsin = new FileInputStream(filePath);
oin = new ObjectInputStream(fsin);
voObj = (T) oin.readObject();
return voObj;
} finally {
if (null != fsin) {
fsin.close();
}
if (null != oin) {
oin.close();
}
}
}
公共静态T反序列化元数据(T voObj,
字符串文件路径)引发IOException、NullPointerException、ClassNotFoundException{
FileInputStream fsin=null;
ObjectInputStream oin=null;
试一试{
fsin=新文件输入流(文件路径);
oin=新对象输入流(fsin);
voObj=(T)oin.readObject();
返回voObj;
}最后{
如果(空!=fsin){
fsin.close();
}
if(null!=oin){
oin.close();
}
}
}
public static <T> T deserializeMetadata(T voObj,
String filePath) throws IOException,NullPointerException, ClassNotFoundException {
FileInputStream fsin = null;
ObjectInputStream oin = null;
try {
fsin = new FileInputStream(filePath);
oin = new ObjectInputStream(fsin);
voObj = (T) oin.readObject();
return voObj;
} finally {
if (null != fsin) {
fsin.close();
}
if (null != oin) {
oin.close();
}
}
}