Hadoop 在配置单元上使用RegexSerde时无法执行选择计数(*)
我正在从固定长度的平面文件读取数据,并应用了以下脚本:Hadoop 在配置单元上使用RegexSerde时无法执行选择计数(*),hadoop,hive,dbeaver,fixed-length-file,regexserde,Hadoop,Hive,Dbeaver,Fixed Length File,Regexserde,我正在从固定长度的平面文件读取数据,并应用了以下脚本: CREATE EXTERNAL TABLE `test_table`.`test_data` (test_column1 STRING, test_column2 STRING ) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe' WITH SERDEPROPERTIES ("input.regex" = "(.{10})
CREATE EXTERNAL TABLE `test_table`.`test_data`
(test_column1 STRING,
test_column2 STRING
)
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
WITH SERDEPROPERTIES ("input.regex" = "(.{10})(.{10})" )
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION '<specified location>';
请告诉我,为LLAP IO创建SerDe时出现的
错误究竟意味着什么,以及我如何着手解决问题?问题已解决。以下答案引自
'RegexSerDe'是'hive contrib'库的一部分。从Hive 0.10开始,serde是“Hive serde-.jar”的一部分
使用最新的serde“org.apache.hadoop.hive.serde2.RegexSerDe”而不是“org.apache.hadoop.hive.contrib.serde2.RegexSerDe”创建表
您还可以运行ALTERTABLE来修改serde,如下所示
ALTER TABLE SET SERDE'org.apache.hadoop.hive.serde2.RegexSerDe';
SQL Error [2] [08S01]: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1620900284233_0004_195_00, diagnostics=[Task failed, taskId=task_1620900284233_0004_195_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1620900284233_0004_195_00_000000_0:java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException: Error creating SerDe for LLAP IO
'RegexSerDe' is part of 'hive-contrib' library. From Hive 0.10 onwards the serde is part of 'hive-serde-<version>.jar'
Create the table with latest serde 'org.apache.hadoop.hive.serde2.RegexSerDe' instead of 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
You can also run alter table to modify the serde as below
ALTER TABLE <TABLENAME> SET SERDE 'org.apache.hadoop.hive.serde2.RegexSerDe';