Python 如何连接两个单独的文本数据文件，并在不同的时间间隔和平均值上对齐数据

我有两个不同仪器的文本数据文件。一种是电梯，每隔7分钟和8分钟升降一次。我需要将一台仪器的数据与上下位置的数据匹配对齐，时间为7或8分钟。以下是仪表Picarro和电梯AEM的数据：

一个问题是：皮卡罗时间是以UTC时间记录的，所以实际上是下午6点，而不是午夜，而AEM是在午夜开始的

Loc_Strt值表示位置上部364或下部233

皮卡罗仪器

升降器

---

我希望能够合并这两个单独的文件并输出到一个新的列表中。从这个新列表中，我想对数据进行统计分析、平均值、标准偏差等。但首先，我必须在这些时间范围内调整数据。AEM的间隔模式似乎是7,8,8,7分钟，然后重复，因此需要为此创建一些循环，但这远远超出了我的Python技能。我想沿着这个模式创建间隔来证实数据

以下是您可以使用的一种方法：

import re
from datetime import datetime, timedelta
# Custom classes to hold your data.
class ElevatorInterval(object):
    def __init__(self, timestamp, record, loc_strt, loc_cut):
        timestamp = datetime.strptime(timestamp, "%Y-%m-%d %H:%M:%S") 
        self.timestop = self.timestart = timestamp + timedelta(hours=6) #UTC+6
        self.measures = []
        self.location = 'bottom' if float(loc_strt) < 300 else 'top'
class NH3Measure(object):
    def __init__(self, date, tim, nh3_raw):
        self.timestamp = datetime.strptime(date + tim, "%Y-%m-%d%H:%M:%S")
        self.nh3 = nh3_raw
    def __repr__(self):
        return str(self.nh3)
# Read data from file and assign them to elevator measures, and NH3 measures.
ele_intervals, nh3_measures = [], []
with open('aem.txt', 'r') as f:
    for line in f:
        linematch = re.match(r'^"([0-9-]+\s[0-9:]+)(?:\.[0-9])?",([0-9]+),([0-9.]+),([0-9.]+)', line)
        if linematch:
            ele_intervals.append(ElevatorInterval(*linematch.groups()))
            if len(ele_intervals) > 1: # Set timestop for the last elevator interval.
                ele_intervals[-2].timestop = ele_intervals[-1].timestart - timedelta(seconds=22)
del ele_intervals[-1] # Remove last interval as it has no stop time.
with open('pic.txt', 'r') as f:
    for line in f:
        linematch = re.match(r'^([0-9-]+)\s+([0-9:]+)[0-9.]*\s+([0-9E.+]+)', line)
        if linematch: nh3_measures.append(NH3Measure(*linematch.groups()))
# Assign NH3 measures to their proper interval, and output the intervals.
for ele in ele_intervals:
    ele.measures = filter(lambda x: ele.timestart < x.timestamp < ele.timestop, nh3_measures)
    print ele.location, ele.measures

和pic.txt：

打印结果：

bottom [3.3844673297E+000, 3.1585870007E+000, 3.2442662514E+000]
top [4.2442662514E+000, 4.4242662514E+000, 4.1064987772E+000]
bottom [3.2442662514E+000, 3.2442662514E+000, 3.1064987772E+000]

---

如何将日期与它们之间的6天对齐？是否要根据日期对值进行排序？文件将来自同一日期。我打错日期了。我现在已经编辑了。那么，你想按时间点对它们进行排序吗？是的。我想对电梯AEM在时间间隔内的所有Picarro数据取平均值。我正在测量上下两个高度处的气体浓度，233=下，378=上。例如，Picarro连续测量并记录约2/秒的数据点。我需要平均Picarro数据在每个高度（上部和下部）花费的时间。低7分钟，高8分钟，低8分钟，高7分钟，这是每30分钟一次的循环，然后循环重复。21-22秒是电梯移动到相反高度的行程时间。我想你可以把时间四舍五入到整分钟。在你使用'with open'pic.txt'，'r'作为f:'的地方，我尝试使用pd.read_csvfile，header=0，index_col=0作为f:，但不断得到一个exit属性错误。你知道为什么吗？Picarro文件是固定宽度的fwf文件，AEM是csv文件。我想使用pandas，因为它们有简单的模块read_fwf和read_csv来处理每种文件类型。对不起，我对那些模块了解不多。也许你应该再问一个问题。但是，您可以尝试，这是惊人的。我提供的AEM电梯示例文件比我在文本中提供的三列多得多。当尝试将其导入脚本时，我得到以下错误：init正好接受5个参数，只有4个给出了回溯最后一次调用：文件C:\Users\Jason\Google Drive\Python Programs\AEM\u Picarro\u Slice，第23行，在ele_Interval.appendElevatorInterval*linematch.groups中类型错误：uu init_uu恰好接受5个参数4 givenTry打印linematch.groups。在我的代码中，它应该返回一个包含4个元素的元组。你以某种方式改变了它，所以它只给出三个。

import re
from datetime import datetime, timedelta
# Custom classes to hold your data.
class ElevatorInterval(object):
    def __init__(self, timestamp, record, loc_strt, loc_cut):
        timestamp = datetime.strptime(timestamp, "%Y-%m-%d %H:%M:%S") 
        self.timestop = self.timestart = timestamp + timedelta(hours=6) #UTC+6
        self.measures = []
        self.location = 'bottom' if float(loc_strt) < 300 else 'top'
class NH3Measure(object):
    def __init__(self, date, tim, nh3_raw):
        self.timestamp = datetime.strptime(date + tim, "%Y-%m-%d%H:%M:%S")
        self.nh3 = nh3_raw
    def __repr__(self):
        return str(self.nh3)
# Read data from file and assign them to elevator measures, and NH3 measures.
ele_intervals, nh3_measures = [], []
with open('aem.txt', 'r') as f:
    for line in f:
        linematch = re.match(r'^"([0-9-]+\s[0-9:]+)(?:\.[0-9])?",([0-9]+),([0-9.]+),([0-9.]+)', line)
        if linematch:
            ele_intervals.append(ElevatorInterval(*linematch.groups()))
            if len(ele_intervals) > 1: # Set timestop for the last elevator interval.
                ele_intervals[-2].timestop = ele_intervals[-1].timestart - timedelta(seconds=22)
del ele_intervals[-1] # Remove last interval as it has no stop time.
with open('pic.txt', 'r') as f:
    for line in f:
        linematch = re.match(r'^([0-9-]+)\s+([0-9:]+)[0-9.]*\s+([0-9E.+]+)', line)
        if linematch: nh3_measures.append(NH3Measure(*linematch.groups()))
# Assign NH3 measures to their proper interval, and output the intervals.
for ele in ele_intervals:
    ele.measures = filter(lambda x: ele.timestart < x.timestamp < ele.timestop, nh3_measures)
    print ele.location, ele.measures

TIMESTAMP, RECORD, Loc_Strt, Loc_Cut
"2014-06-23 18:15:22.6",798,233.8,215
"2014-06-23 18:23:22",799,364,378.8
"2014-06-23 18:30:22.5",800,233.7,215.4
"2014-06-23 18:37:22.5",801,364,378.8

Date            Time            NH3_Raw              
2014-06-24      00:16:39.134    3.3844673297E+000  
2014-06-24      00:16:41.210    3.1585870007E+000 
2014-06-24      00:16:43.293    3.2442662514E+000
2014-06-24      00:24:45.293    4.2442662514E+000
2014-06-24      00:24:47.812    4.4242662514E+000
2014-06-24      00:24:49.335    4.1064987772E+000
2014-06-24      00:31:45.293    3.2442662514E+000
2014-06-24      00:31:47.812    3.2442662514E+000
2014-06-24      00:31:49.335    3.1064987772E+000

bottom [3.3844673297E+000, 3.1585870007E+000, 3.2442662514E+000]
top [4.2442662514E+000, 4.4242662514E+000, 4.1064987772E+000]
bottom [3.2442662514E+000, 3.2442662514E+000, 3.1064987772E+000]