跳过SparkyR中配置单元外部表的第一行
我有以下资料:跳过SparkyR中配置单元外部表的第一行,r,apache-spark,hive,sparklyr,R,Apache Spark,Hive,Sparklyr,我有以下资料: "ElemUID ElemName Kind Number DaySecFrom(UTC) DaySecTo(UTC)" "399126817 A648/13FKO-66 DEZ 2017-07-01 23:58:00.000 2017-07-01 23:59:00.000" "483492732 A661/18FRS-97 DEZ 120.00 2017-07-01 23:58:00.000 2017-07-01 23:59:0
"ElemUID ElemName Kind Number DaySecFrom(UTC) DaySecTo(UTC)"
"399126817 A648/13FKO-66 DEZ 2017-07-01 23:58:00.000 2017-07-01 23:59:00.000"
"483492732 A661/18FRS-97 DEZ 120.00 2017-07-01 23:58:00.000 2017-07-01 23:59:00.000"
"399126819 A648/12FKO-2 DEZ 60.00 2017-07-01 23:58:00.000 2017-07-01 23:59:00.000"
"399126818 A648/12FKO-1 DEZ 180.00 2017-07-01 23:58:00.000 2017-07-01 23:59:00.000"
"399126816 A648/13FKO-65 DEZ 2017-07-01 23:58:00.000 2017-07-01 23:59:00.000"
"398331142 A661/31OFN-1 DEZ 120.00 2017-07-01 23:58:00.000 2017-07-01 23:59:00.000"
"398331143 A661/31OFN-2 DEZ 2017-07-01 23:58:00.000 2017-07-01 23:59:00.000"
"483492739 A5/28FKN-65 DEZ 2017-07-01 23:58:00.000 2017-07-01 23:59:00.000"
"483492735 A661/23FRS-97 DEZ 60.00 2017-07-01 23:58:00.000 2017-07-01 23:59:00.000"
"483492740 B44/104FSN-33 DEZ 180.00 2017-07-01 23:58:00.000 2017-07-01 23:59:00.000"
我把它加载到HDFS。然后我在配置单元中定义了一个外部表:
CREATE EXTERNAL TABLE IF NOT EXISTS deg
(
ElemUID int,
ElemName string,
Kind string,
Number float,
timefromdeg string,
timetodeg string
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\n'
TBLPROPERTIES ("skip.header.line.count"="1");
然后,我使用LOAD DATA INPATH..
现在我想让它加载tbl()
到sparklyr。每次执行此操作时,我总是将标题作为第一行中的数据获取:
scape()
的输出:
变量:6
$elemuid NA,399126817,483492732,399126819,399126818,399126816,39。。。
$elemname“elemname”、“A648/13FKO-66”、“A661/18FRS-97”、“A648/12FKO-2”,。。。
$kind“kind”,“DEZ”,“DEZ”,“DEZ”,“DEZ”,“DEZ”,“DEZ”,“DEZ”,“DEZ”,“DEZ”,“DEZ…”。。。
$number楠,楠,120,60,180,楠,120,楠,楠,楠,60,180,楠,楠。。。
$timefrom NA,2017-07-01 23:58:00,2017-07-01 23:58:00,2017-07-01 23:00。。。
$timeto NA,2017-07-01 23:59:00,2017-07-01 23:59:00,2017-07-01 23:00。。。
我觉得这干扰了我以后的分析。在创建外部table()
时,我已经使用了tblproperty(“skip.header.line.count”=“1”)
有可能跳过第一行吗
谢谢!除elemenname和kind外,第一行大部分为空。请显示一个小的可复制示例和预期输出。我认为ElemnName和kind不是NA,因为这些列具有可复制格式(char)。请包括示例数据和表定义。不要认为这是个问题,请参阅
Variables: 6
$ elemuid <int> NA, 399126817, 483492732, 399126819, 399126818, 399126816, 39...
$ elemname <chr> "ElemName", "A648/13FKO-66", "A661/18FRS-97", "A648/12FKO-2",...
$ kind <chr> "Kind", "DEZ", "DEZ", "DEZ", "DEZ", "DEZ", "DEZ", "DEZ", "DEZ...
$ number <dbl> NaN, NaN, 120, 60, 180, NaN, 120, NaN, NaN, 60, 180, NaN, NaN...
$ timefrom <dttm> NA, 2017-07-01 23:58:00, 2017-07-01 23:58:00, 2017-07-01 23:...
$ timeto <dttm> NA, 2017-07-01 23:59:00, 2017-07-01 23:59:00, 2017-07-01 23:...