Warning: file_get_contents(/data/phpspider/zhask/data//catemap/3/apache-spark/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Apache spark spark sql时间戳数据类型是否实际存储时区?_Apache Spark_Apache Spark Sql_Databricks - Fatal编程技术网

Apache spark spark sql时间戳数据类型是否实际存储时区?

Apache spark spark sql时间戳数据类型是否实际存储时区?,apache-spark,apache-spark-sql,databricks,Apache Spark,Apache Spark Sql,Databricks,spark sqltimestamp数据类型是否实际存储时区 我使用的是databricks 6.5(ApacheSpark 2.4.5,Scala 2.11) 给出了这个结果 C1 C2 C3 ===================================================================== 2020-07-02T07:06:57.716+

spark sql
timestamp
数据类型是否实际存储时区

我使用的是databricks 6.5(ApacheSpark 2.4.5,Scala 2.11)

给出了这个结果

C1                              C2                              C3
=====================================================================
2020-07-02T07:06:57.716+0000    2020-07-02T16:36:57.716+0000    +0000
任何地方都没有时区指示器的迹象

我能找到的所有datetime函数都要求您指定时区。在我看来,时区部分不应该显示,因为它实际上不在数据中

这个问题是:不,它没有存储,但有人能确认吗

编辑
还有人认为+0000在这里有误导性吗?对我来说,这意味着日期有一个UTC时区,可以简明扼要地存储一个不同的时区。我来自SQL Server世界,那里不存储时区的datetime没有时区指示符。

在spark中,所有datetime操作/函数都是时区感知的,但spark内部从不存储时区,它们将时间存储在
int
long

来自spark doc-

* Helper functions for converting between internal and external date and time representations.
 * Dates are exposed externally as java.sql.Date and are represented internally as the number of
 * dates since the Unix epoch (1970-01-01). Timestamps are exposed externally as java.sql.Timestamp
 * and are stored internally as longs, which are capable of storing timestamps with microsecond
 * precision.

ref-

在spark中,所有datetime操作/函数都是时区感知的,但spark内部从不存储时区,它们将时间存储在
int
long

来自spark doc-

* Helper functions for converting between internal and external date and time representations.
 * Dates are exposed externally as java.sql.Date and are represented internally as the number of
 * dates since the Unix epoch (1970-01-01). Timestamps are exposed externally as java.sql.Timestamp
 * and are stored internally as longs, which are capable of storing timestamps with microsecond
 * precision.
参考-