Hadoop 在配置单元中执行自定义UDF
我的蜂箱里有下表Hadoop 在配置单元中执行自定义UDF,hadoop,mapreduce,hive,Hadoop,Mapreduce,Hive,我的蜂箱里有下表 >describe weblogs; OK originatingip string clientidentity string userid string time
>describe weblogs;
OK
originatingip string
clientidentity string
userid string
time string
requesttype string
requestpage string
httpprotocolversion string
responsecode int
responsesize int
referrer string
useragent string
Time taken: 1.065 seconds, Fetched: 11 row(s)
package com.prithvi.hive.logprocessing.udf.ipgeo;
public class IpgeoHive extends UDF {
Text result = new Text();
String ipCountry, ipCity;
public Text evaluate(Text input) throws IOException {
if(input==null)return null;
URL database_path = getClass().getResource("/GeoLiteCity.dat");
File file;
try {
file = new File(database_path.toURI());
} catch(URISyntaxException e) {
file = new File(database_path.getPath());
}
LookupService cl = new LookupService(file);
Location location = cl.getLocation(input.toString());
if (location != null) {
ipCountry = location.countryName;
ipCity = location.city;
} else {
ipCountry = "Unknown";
ipCity = "Unknown";
}
result.set(ipCountry+"/"+ipCity);
return result;
}
}
ADD JAR MapReduce_Examples-0.0.1-SNAPSHOT-jar-with-dependencies.jar;
CREATE TEMPORARY FUNCTION IP2GEO AS 'com.prithvi.hive.logprocessing.udf.ipgeo.IpgeoHive';
SELECT originatingip, IP2GEO(originatingip) from weblogs limit 10;
错误表明hive不知道java字符串/文本 您必须将java字符串/文本转换为配置单元字符串 使用下面的代码
private JavaStringObjectInspector stringInspector;
stringInspector = PrimitiveObjectInspectorFactory.javaStringObjectInspector;
String ip = stringInspector.getPrimitiveJavaObject(input);
请将weblogs表中的示例记录添加到您的问题中。谢谢您使用的是哪一个geoip库？我仍然收到相同的错误：失败，异常为java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException:无法执行方法公共建议最受欢迎。。及@Omkar@Prithvi514你用其他方法解决了吗？？
private JavaStringObjectInspector stringInspector;
stringInspector = PrimitiveObjectInspectorFactory.javaStringObjectInspector;
String ip = stringInspector.getPrimitiveJavaObject(input);