Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark 从ORC到配置单元列的自定义列映射_Apache Spark_Hive_Bigdata_Orc_Hiveddl - Fatal编程技术网

Apache spark 从ORC到配置单元列的自定义列映射

Apache spark 从ORC到配置单元列的自定义列映射,apache-spark,hive,bigdata,orc,hiveddl,Apache Spark,Hive,Bigdata,Orc,Hiveddl,我相信在配置单元表属性中使用orc.force.positional.evolution,我们可以强制位置演化,默认情况下是模式演化。我有一个奇怪的需要,我需要给一个自定义映射。例如,我的orc模式如下 root |-- action_date: timestamp (nullable = true) |-- action_type: string (nullable = true) |-- customer_id: string (nullable = true) |-- name

我相信在配置单元表属性中使用
orc.force.positional.evolution
,我们可以强制位置演化,默认情况下是模式演化。我有一个奇怪的需要,我需要给一个自定义映射。例如,我的orc模式如下

  root
 |-- action_date: timestamp (nullable = true)
 |-- action_type: string (nullable = true)
 |-- customer_id: string (nullable = true)
 |-- name: string (nullable = true)
  date timestamp,
  customer_id string,
  customer_name string,
  type string
我的蜂巢列名如下

  root
 |-- action_date: timestamp (nullable = true)
 |-- action_type: string (nullable = true)
 |-- customer_id: string (nullable = true)
 |-- name: string (nullable = true)
  date timestamp,
  customer_id string,
  customer_name string,
  type string
有没有一种方法可以提供像这样的自定义映射-

date -> action_date
customer_id -> customer_id
customer_name -> name
type -> action_type
请注意,在这种情况下,位置演化和模式演化都不起作用。更改ORC架构或配置单元列名不是我的选项。 提前谢谢