Apache spark spark sql时间戳数据类型是否实际存储时区?
spark sqlApache spark spark sql时间戳数据类型是否实际存储时区?,apache-spark,apache-spark-sql,databricks,Apache Spark,Apache Spark Sql,Databricks,spark sqltimestamp数据类型是否实际存储时区 我使用的是databricks 6.5(ApacheSpark 2.4.5,Scala 2.11) 给出了这个结果 C1 C2 C3 ===================================================================== 2020-07-02T07:06:57.716+
timestamp
数据类型是否实际存储时区
我使用的是databricks 6.5(ApacheSpark 2.4.5,Scala 2.11)
给出了这个结果
C1 C2 C3
=====================================================================
2020-07-02T07:06:57.716+0000 2020-07-02T16:36:57.716+0000 +0000
任何地方都没有时区指示器的迹象
我能找到的所有datetime函数都要求您指定时区。在我看来,时区部分不应该显示,因为它实际上不在数据中
这个问题是:不,它没有存储,但有人能确认吗
编辑
还有人认为+0000在这里有误导性吗?对我来说,这意味着日期有一个UTC时区,可以简明扼要地存储一个不同的时区。我来自SQL Server世界,那里不存储时区的datetime没有时区指示符。在spark中,所有datetime操作/函数都是时区感知的,但spark内部从不存储时区,它们将时间存储在
int
和long
来自spark doc-
* Helper functions for converting between internal and external date and time representations.
* Dates are exposed externally as java.sql.Date and are represented internally as the number of
* dates since the Unix epoch (1970-01-01). Timestamps are exposed externally as java.sql.Timestamp
* and are stored internally as longs, which are capable of storing timestamps with microsecond
* precision.
ref-在spark中,所有datetime操作/函数都是时区感知的,但spark内部从不存储时区,它们将时间存储在
int
和long
来自spark doc-
* Helper functions for converting between internal and external date and time representations.
* Dates are exposed externally as java.sql.Date and are represented internally as the number of
* dates since the Unix epoch (1970-01-01). Timestamps are exposed externally as java.sql.Timestamp
* and are stored internally as longs, which are capable of storing timestamps with microsecond
* precision.
参考-