Python 在时间戳列表中查找最大间隔(首选Perl)
我接受任何一种解释语言Perl、Python、Bash等。。但我更喜欢Perl,因为它正是我想要学习的。我有一个时间戳列表,如:Python 在时间戳列表中查找最大间隔(首选Perl),python,regex,perl,parsing,scripting,Python,Regex,Perl,Parsing,Scripting,我接受任何一种解释语言Perl、Python、Bash等。。但我更喜欢Perl,因为它正是我想要学习的。我有一个时间戳列表,如: 17:31:16 17:31:16 17:31:18 17:31:29 我想找到任意两个连续行之间的所有最大间隔(前5),并返回时间戳和行号。基本上,这是一个软件构建的日志文件,我试图确定哪些步骤花费的时间最长。我给出的示例实际上已经过过滤,行实际上看起来像: [15:57:42]:CC-net/sunrpc/xprtsock.o 如果你能给我一个解析这种格式的程序
17:31:16
17:31:16
17:31:18
17:31:29
我想找到任意两个连续行之间的所有最大间隔(前5),并返回时间戳和行号。基本上,这是一个软件构建的日志文件,我试图确定哪些步骤花费的时间最长。我给出的示例实际上已经过过滤,行实际上看起来像:
[15:57:42]:CC-net/sunrpc/xprtsock.o
如果你能给我一个解析这种格式的程序,它会更容易一些,并返回在时间上出现最大差异的行号
这就是我用来从日志中隔离时间戳的方法
perl -lane 'print $1 if $_ =~ /^\[(\d+:\d+:\d+)\]:*/'
我希望实现的输出类型如下:
line 574 20:04:54
line 575 20:24:55
Difference 00:20:01
如果你不想解决这个问题,我很乐意看到一些伪代码或得到任何建议。我已经花了很多时间,没有有用的代码来显示它。只是得到了最大的不同
perl -l -n -e 'BEGIN {$m=0;$last=0;$am=$.;} /(\d+):(\d+):(\d+)/; $v=($1*3600)+($2*60)+$3; if ($last && $v-$last > $m) { $am=$.; $m=$v-$last;} $last=$v; END { print "max diff ",$m, " at line $am\n" }' d.txt
前五名
我会将您的时间匹配正则表达式稍微升级一点,以单独捕获时间的组成部分。我们是否需要担心构建在午夜之前开始并一直运行到第二天凌晨
#!/usr/bin/env perl
use strict;
use warnings;
my $oldtime = ""; # hh:mm:ss for end of long interval
my $oldlineno = 0; # line number in the file of second line
my $oldoffset = 0; # offset in seconds from midnight of second command
my $olddiff = 0; # time taken for longest command
sub hhmmss
{
my($time) = @_;
my(@tm) = (int($time/3600), int($time/60)%60, $time%60);
return @tm;
}
while (<>)
{
chomp;
next unless m/^((\d\d):(\d\d):(\d\d))\s+/;
my $newoffset = (($2 * 60) + $3) * 60 + $4;
if ($oldoffset == 0)
{
$oldtime = $1;
$olddiff = 0;
$oldoffset = $newoffset;
$oldlineno = $.;
}
elsif (($newoffset - $oldoffset) > $olddiff)
{
$oldtime = $1;
$olddiff = $newoffset - $oldoffset;
$oldoffset = $newoffset;
$oldlineno = $.;
}
}
if ($oldoffset != 0)
{
my $prvlineno = $oldlineno - 1;
my $newoffset = $oldoffset - $olddiff;
my(@tm) = hhmmss($newoffset);
printf "line $prvlineno: %.2d:%.2d:%.2d\n", $tm[0], $tm[1], $tm[2];
print "line $oldlineno: $oldtime\n";
@tm = hhmmss($olddiff);
printf "diff: %.2d:%.2d:%.2d\n", $tm[0], $tm[1], $tm[2];
}
下面的scriptlet生成所示的输出:
$ for i in $(seq 1 9); do sed ${i}q data | perl dt.pl; done | so
line 0: 17:31:16
line 1: 17:31:16
diff: 00:00:00
line 1: 17:31:16
line 2: 17:31:18
diff: 00:00:02
line 2: 17:31:18
line 3: 17:31:29
diff: 00:00:11
line 3: 17:31:29
line 4: 17:33:59
diff: 00:02:30
line 4: 17:33:59
line 5: 18:00:21
diff: 00:26:22
line 4: 17:33:59
line 5: 18:00:21
diff: 00:26:22
line 6: 18:00:21
line 7: 18:41:25
diff: 00:41:04
line 7: 18:41:25
line 8: 19:51:54
diff: 01:10:29
line 7: 18:41:25
line 8: 19:51:54
diff: 01:10:29
$
我很想听听您在编写任何代码之前是如何考虑这个问题的
这显然是一个问题,需要保留前一行信息(相关部分)的记录,以计算其与当前行之间的差异。您还需要保持当前的最大差异,直到您读取第二条匹配行后才能正式确定该差异。这推动了设计。通过无条件地分配3个值和有条件地分配第四个值($olddiff
),可以将代码中的大重复减少为零。在那之后,它主要是一个力学和战术的问题
像这样跨多行匹配是一个麻烦的过程;你必须处理保持适当状态的问题。部分原因是经验问题;这种事情你做了几十次之后,下次就不用花那么长时间了。非常令人印象深刻,而且非常有用。非常感谢。我要研究一下这个问题。你能不能帮我把你的方法和思维过程分解一下?这是一份很棒的工作,谢谢!我很想听听您在编写任何代码之前是如何考虑这个问题的。我认为我不能像程序员一样处理这个问题对我来说是一个很大的障碍。思考的过程是什么?有时间间隔过午夜吗?是否有任何时间间隔与夏令时调整重叠?@pilcrow否,此特定数据集不存在类似问题。
#!/usr/bin/env perl
use strict;
use warnings;
my $oldtime = ""; # hh:mm:ss for end of long interval
my $oldlineno = 0; # line number in the file of second line
my $oldoffset = 0; # offset in seconds from midnight of second command
my $olddiff = 0; # time taken for longest command
sub hhmmss
{
my($time) = @_;
my(@tm) = (int($time/3600), int($time/60)%60, $time%60);
return @tm;
}
while (<>)
{
chomp;
next unless m/^((\d\d):(\d\d):(\d\d))\s+/;
my $newoffset = (($2 * 60) + $3) * 60 + $4;
if ($oldoffset == 0)
{
$oldtime = $1;
$olddiff = 0;
$oldoffset = $newoffset;
$oldlineno = $.;
}
elsif (($newoffset - $oldoffset) > $olddiff)
{
$oldtime = $1;
$olddiff = $newoffset - $oldoffset;
$oldoffset = $newoffset;
$oldlineno = $.;
}
}
if ($oldoffset != 0)
{
my $prvlineno = $oldlineno - 1;
my $newoffset = $oldoffset - $olddiff;
my(@tm) = hhmmss($newoffset);
printf "line $prvlineno: %.2d:%.2d:%.2d\n", $tm[0], $tm[1], $tm[2];
print "line $oldlineno: $oldtime\n";
@tm = hhmmss($olddiff);
printf "diff: %.2d:%.2d:%.2d\n", $tm[0], $tm[1], $tm[2];
}
17:31:16 line1
17:31:18 line2
17:31:29 line3
17:33:59 line4
18:00:21 line5
18:21:03 line6
18:41:25 line7
19:51:54 line8
19:52:34 line9
$ for i in $(seq 1 9); do sed ${i}q data | perl dt.pl; done | so
line 0: 17:31:16
line 1: 17:31:16
diff: 00:00:00
line 1: 17:31:16
line 2: 17:31:18
diff: 00:00:02
line 2: 17:31:18
line 3: 17:31:29
diff: 00:00:11
line 3: 17:31:29
line 4: 17:33:59
diff: 00:02:30
line 4: 17:33:59
line 5: 18:00:21
diff: 00:26:22
line 4: 17:33:59
line 5: 18:00:21
diff: 00:26:22
line 6: 18:00:21
line 7: 18:41:25
diff: 00:41:04
line 7: 18:41:25
line 8: 19:51:54
diff: 01:10:29
line 7: 18:41:25
line 8: 19:51:54
diff: 01:10:29
$