Fatal编程技术网
  • mysql/
  • Mysql 在时间序列数据表中添加缺失的日期记录

Mysql 在时间序列数据表中添加缺失的日期记录

mysql sql

Mysql 在时间序列数据表中添加缺失的日期记录,mysql,sql,Mysql,Sql,我正在使用一个包含工作日数据的表。这些数据基本上是关于每天的期末余额。数据如下所示： ID Name Some Val Other Val Date 10 Somebody 33001.93 33001.93 2018-10-01 10 Somebody 33481.93 33481.93 2018-10-02 10 Somebody 33001.93 33001.93 2018-10-03 10 Some

我正在使用一个包含工作日数据的表。这些数据基本上是关于每天的期末余额。数据如下所示：

ID  Name        Some Val    Other Val   Date

10  Somebody    33001.93    33001.93    2018-10-01
10  Somebody    33481.93    33481.93    2018-10-02
10  Somebody    33001.93    33001.93    2018-10-03
10  Somebody    33582.76    33582.76    2018-10-04
10  Somebody    33582.73    33582.79    2018-10-05
------- Missing row for 2018-10-06 ---------------
------- Missing row for 2018-10-07 ---------------
10  Somebody    33582.76    33582.76    2018-10-08
------- Missing row for 2018-10-09 ---------------
10  Somebody    33462.76    33462.76    2018-10-10
我的任务是计算平均每日余额每天结束时的总余额/总天数。为了进行计算，我需要确保我有所有天的数据。为此，最后一行需要替换丢失的数据

我的需要是：

ID  Name        Some Val    Other Val   Date

10  Somebody    33001.93    33001.93    2018-10-01
10  Somebody    33481.93    33481.93    2018-10-02
10  Somebody    33001.93    33001.93    2018-10-03
10  Somebody    33582.76    33582.76    2018-10-04
10  Somebody    33582.73    33582.79    2018-10-05    
10  Somebody    33582.73    33582.79    2018-10-06
10  Somebody    33582.73    33582.79    2018-10-07    
10  Somebody    33582.76    33582.76    2018-10-08
10  Somebody    33382.76    33582.76    2018-10-09
10  Somebody    33462.76    33462.76    2018-10-10
基本上，第5行写入缺失的第6行和第7行，第8行写入第9行

我得到了部分解决方案，如果只是缺少周末记录的话

select ID, Name, val1, val2, date from t
union all
select id, name, val1, val2, date + interval 1 day from t where dayofweek(date) = 6
union all
select id, name, val1, val2, date + interval 2 day from t where dayofweek(date) = 6
;
此部分解决方案的工作原理是假设只有周末记录丢失。通过复制周五到周六和周日的数据，创建了两个新表。最后，将所有三个数据集连接在一起

如果数据在工作周内丢失（例如公共假日），则解决方案将失败，因此仅填写第6行和第7行。第9行仍然是空的

我如何找到丢失的记录，用最后的记录信息填充它们，从而完成时间序列？我不熟悉SQL，但不熟悉编程。有了正确的指针，我就能想出一个解决方案。有人建议我如何解决这个问题

我使用的MySQL版本是：

mysql  Ver 14.14 Distrib 5.7.24, for Linux (x86_64) using  EditLine wrapper
如果您的mysql支持cte递归，您可以尝试使用它来创建日历表

然后使用case进行外部联接和子查询

模式MySQL v8.0

问题1

问题1

你的MySQL版本是什么？你的MySQL版本支持CTE吗？@ D-SHIH，用细节更新了原始帖子。你会考虑让你的MySQL支持CTE吗？因为你需要做一个日历表。只要解决方案有效，使用CTE将是一件好事，我不介意。你能提供一个我可以建立的东西吗？我为cte recursive@Sushant VasishtaThanks@D-Shih编写了一个版本，看起来是你用MySQL 8版写的。我的版本是5.7。我在尝试遵循您提供的解决方案时出错。更重要的是，你能简单地阐述一下方法吗？我是SQL新手，您的解释对于巩固我对利用率的理解/增强我修改解决方案的能力至关重要。关键是您需要制作一个日历表来填写日期，您缺少的。或者您可以为外部联接创建一个日历表。这里是一个示例@sushantvasishtat该表需要插入日期，您需要使用该日期，然后在此基础上进行外部联接。太棒了。我只是好奇这一部分：从tt中选择ID，其中tt.DateCREATE TABLE T( ID int, Name varchar(50), SomeVal float, OtherVal float, `Date` date ); insert into T values (10,'Somebody',33001.93,33001.93,'2018-10-01'); insert into T values (10,'Somebody',33481.93,33481.93,'2018-10-02'); insert into T values (10,'Somebody',33001.93,33001.93,'2018-10-03'); insert into T values (10,'Somebody',33582.76,33582.76,'2018-10-04'); insert into T values (10,'Somebody',33582.73,33582.79,'2018-10-05'); insert into T values (10,'Somebody',33582.76,33582.76,'2018-10-08'); insert into T values (10,'Somebody',33462.76,33462.76,'2018-10-10'); 
WITH recursive CTE as(
  SELECT MIN(Date) minDt,MAX(Date) maxDt
  FROM T
  UNION ALL
  SELECT date_add(minDt,INTERVAL 1 DAY),maxDt
  FROM CTE
  WHERE minDt < maxDt
)

SELECT  
    CASE WHEN ID IS NULL THEN (SELECT ID 
                            FROM T tt 
                            WHERE tt.Date < t1.minDt
                            ORDER BY tt.Date DESC
                            LIMIT 1)  
    ELSE ID END ID,
    CASE WHEN Name IS NULL THEN (SELECT Name 
                            FROM T tt 
                            WHERE tt.Date < t1.minDt
                            ORDER BY tt.Date DESC
                            LIMIT 1) 
    ELSE Name END Name,
    CASE WHEN SomeVal IS NULL THEN (SELECT SomeVal 
                            FROM T tt 
                            WHERE tt.Date < t1.minDt
                            ORDER BY tt.Date DESC
                            LIMIT 1) 
    ELSE SomeVal END SomeVal,
    CASE WHEN OtherVal IS NULL THEN (SELECT OtherVal 
                            FROM T tt 
                            WHERE tt.Date < t1.minDt
                            ORDER BY tt.Date DESC
                            LIMIT 1) 
    ELSE OtherVal END OtherVal,
    minDt
FROM CTE t1 
LEFT JOIN T t2 ON t1.minDt = t2.Date
ORDER BY t1.minDT;

| ID  | Name     | SomeVal        | OtherVal       | minDt      |
| --- | -------- | -------------- | -------------- | ---------- |
| 10  | Somebody | 33001.9296875  | 33001.9296875  | 2018-10-01 |
| 10  | Somebody | 33481.9296875  | 33481.9296875  | 2018-10-02 |
| 10  | Somebody | 33001.9296875  | 33001.9296875  | 2018-10-03 |
| 10  | Somebody | 33582.76171875 | 33582.76171875 | 2018-10-04 |
| 10  | Somebody | 33582.73046875 | 33582.7890625  | 2018-10-05 |
| 10  | Somebody | 33582.73046875 | 33582.7890625  | 2018-10-06 |
| 10  | Somebody | 33582.73046875 | 33582.7890625  | 2018-10-07 |
| 10  | Somebody | 33582.76171875 | 33582.76171875 | 2018-10-08 |
| 10  | Somebody | 33582.76171875 | 33582.76171875 | 2018-10-09 |
| 10  | Somebody | 33462.76171875 | 33462.76171875 | 2018-10-10 |

CREATE TABLE T(
   ID int,
   Name varchar(50),
   SomeVal float,   
   OtherVal float,   
   `Date` date
);




insert into T values (10,'Somebody',33001.93,33001.93,'2018-10-01');
insert into T values (10,'Somebody',33481.93,33481.93,'2018-10-02');
insert into T values (10,'Somebody',33001.93,33001.93,'2018-10-03');
insert into T values (10,'Somebody',33582.76,33582.76,'2018-10-04');
insert into T values (10,'Somebody',33582.73,33582.79,'2018-10-05');
insert into T values (10,'Somebody',33582.76,33582.76,'2018-10-08');
insert into T values (10,'Somebody',33462.76,33462.76,'2018-10-10');


CREATE Table calendar(
   minDt Date
);

INSERT INTO calendar values ('2018-10-01');
INSERT INTO calendar values ('2018-10-02');
INSERT INTO calendar values ('2018-10-03');
INSERT INTO calendar values ('2018-10-04');
INSERT INTO calendar values ('2018-10-05');
INSERT INTO calendar values ('2018-10-06');
INSERT INTO calendar values ('2018-10-07');
INSERT INTO calendar values ('2018-10-08');
INSERT INTO calendar values ('2018-10-09');
INSERT INTO calendar values ('2018-10-10');

SELECT  
    CASE WHEN ID IS NULL THEN (SELECT ID 
                            FROM T tt 
                            WHERE tt.Date < t1.minDt
                            ORDER BY tt.Date DESC
                            LIMIT 1)  
    ELSE ID END ID,
    CASE WHEN Name IS NULL THEN (SELECT Name 
                            FROM T tt 
                            WHERE tt.Date < t1.minDt
                            ORDER BY tt.Date DESC
                            LIMIT 1) 
    ELSE Name END Name,
    CASE WHEN SomeVal IS NULL THEN (SELECT SomeVal 
                            FROM T tt 
                            WHERE tt.Date < t1.minDt
                            ORDER BY tt.Date DESC
                            LIMIT 1) 
    ELSE SomeVal END SomeVal,
    CASE WHEN OtherVal IS NULL THEN (SELECT OtherVal 
                            FROM T tt 
                            WHERE tt.Date < t1.minDt
                            ORDER BY tt.Date DESC
                            LIMIT 1) 
    ELSE OtherVal END OtherVal,
    minDt
FROM calendar t1 
LEFT JOIN T t2 ON t1.minDt = t2.Date
ORDER BY t1.minDT;

| ID  | Name     | SomeVal        | OtherVal       | minDt      |
| --- | -------- | -------------- | -------------- | ---------- |
| 10  | Somebody | 33001.9296875  | 33001.9296875  | 2018-10-01 |
| 10  | Somebody | 33481.9296875  | 33481.9296875  | 2018-10-02 |
| 10  | Somebody | 33001.9296875  | 33001.9296875  | 2018-10-03 |
| 10  | Somebody | 33582.76171875 | 33582.76171875 | 2018-10-04 |
| 10  | Somebody | 33582.73046875 | 33582.7890625  | 2018-10-05 |
| 10  | Somebody | 33582.73046875 | 33582.7890625  | 2018-10-06 |
| 10  | Somebody | 33582.73046875 | 33582.7890625  | 2018-10-07 |
| 10  | Somebody | 33582.76171875 | 33582.76171875 | 2018-10-08 |
| 10  | Somebody | 33582.76171875 | 33582.76171875 | 2018-10-09 |
| 10  | Somebody | 33462.76171875 | 33462.76171875 | 2018-10-10 |