File shell脚本中的CSV文件日期处理
我有一个csv日志文件,有两列,每列都有请求(第1列)和响应(第2列)的时间戳 以下是一些示例数据:File shell脚本中的CSV文件日期处理,file,shell,csv,awk,log-analysis,File,Shell,Csv,Awk,Log Analysis,我有一个csv日志文件,有两列,每列都有请求(第1列)和响应(第2列)的时间戳 以下是一些示例数据: 2013-07-11 08:39:08.748,2013-07-11 08:39:08.748 2013-07-11 08:39:08.826,2013-07-11 08:39:08.826 2013-07-11 08:39:08.860,2013-07-11 08:39:08.860 2013-07-11 08:39:08.919,2013-07-11 08:39:08.919 2013-07
2013-07-11 08:39:08.748,2013-07-11 08:39:08.748
2013-07-11 08:39:08.826,2013-07-11 08:39:08.826
2013-07-11 08:39:08.860,2013-07-11 08:39:08.860
2013-07-11 08:39:08.919,2013-07-11 08:39:08.919
2013-07-11 08:39:08.941,2013-07-11 08:39:08.941
2013-07-11 08:39:09.390,2013-07-11 08:39:09.390
2013-07-11 08:39:09.594,2013-07-11 08:39:09.594
2013-07-11 08:39:09.619,2013-07-11 08:39:09.619
2013-07-11 08:39:09.787,2013-07-11 08:39:09.787
2013-07-11 08:39:10.006,2013-07-11 08:39:10.006
2013-07-11 08:39:10.017,2013-07-11 08:39:10.017
2013-07-11 08:39:10.088,2013-07-11 08:39:10.088
2013-07-11 08:39:10.214,2013-07-11 08:39:10.214
我想计算完整文件的两列(响应-请求)差异的平均值。这个文件一天可以包含一百万个条目
我正在寻找一种使用shell脚本的方法。
请帮忙
谢谢Fedorqui,我试过你的剧本:
while read line;
do
d1=$(echo $line | cut -d, -f1);
d2=$(echo $line | cut -d, -f2);
ds1=$(date -d"$d1" "+%s");
ds2=$(date -d"$d2" "+%s");
echo "$ds2 - $ds1 = $(( $ds1 - $ds2))";
done < requestResponse.csv
我需要以毫秒为单位的差异。我在基本shell脚本中以纳秒为单位完成了这项工作:
while read line
do
d1=$(echo $line | cut -d, -f1)
d2=$(echo $line | cut -d, -f2)
ds1=$(date -d"$d1" "+%s.%N")
ds2=$(date -d"$d2" "+%s.%N")
diff=$(echo "$ds2 - $ds1" | bc)
printf "$ds2 - $ds1 = %.3f\n" $diff
done < file
读取行时
做
d1=$(回声$线|切割-d,-f1)
d2=$(回声$行|切割-d,-f2)
ds1=$(日期-d“$d1”“+%s.%N”)
ds2=$(日期-d“$d2”“+%s.%N”)
差异=$(回声“$ds2-$ds1”| bc)
printf“$ds2-$ds1=%.3f\n”$diff
完成<文件
试验
$whilereadline;do d1=$(回声$线|切-d,-f1);d2=$(回声$行|切割-d,-f2);ds1=$(日期-d“$d1”“+%s.%N”);ds2=$(日期-d“$d2”“+%s.%N”);差异=$(回声“$ds2-$ds1”| bc);printf“$ds2-$ds1=%.3f\n”$diff;完成<文件
1373524748.748000000 - 1373524748.748000000 = 0.000
1373524748.826000000 - 1373524748.826000000 = 0.000
1373524748.860000000 - 1373524748.860000000 = 0.000
1373524748.919000000 - 1373524748.919000000 = 0.000
1373524748.941000000 - 1373524748.941000000 = 0.000
1373524749.390000000 - 1373524749.390000000 = 0.000
1373524749.594000000 - 1373524749.594000000 = 0.000
1373524749.619000000 - 1373524749.619000000 = 0.000
1373524749.787000000 - 1373524749.787000000 = 0.000
1373524750.006000000 - 1373524750.006000000 = 0.000
1373524750.017000000 - 1373524750.017000000 = 0.000
1373524750.088000000 - 1373524750.088000000 = 0.000
1373524750.214000000 - 1373524750.214000000 = 0.000
awk可以通过mktime()或外部date
来完成。但我可以在不编写任何代码的情况下立即告诉您示例的输出。:),它们都是“零”。:事实上是零。这就是为什么把它贴在这里的原因(读取行时;do d1=$(echo$line | cut-d,-f1);d2=$(echo$line | cut-d,-f2);ds1=$(date-d“$d1”“+%s”);ds2=$(date-d“$d2”“+%s”);echo“$ds2-$ds1=$($ds1-$ds2));完成
将为您提供“date1-date2=diff”转换为UNIX stamp后。使用此输出,您可以轻松计算平均值。@fedorqui我认为您应该将其放入答案中。此外,我猜(不确定)OP可能希望得到ms级别的结果。(仅从示例数据中猜测)@LearnMore当前数据始终为0。但是,我更新了答案,以毫秒级显示。非常适合我。非常感谢。
while read line
do
d1=$(echo $line | cut -d, -f1)
d2=$(echo $line | cut -d, -f2)
ds1=$(date -d"$d1" "+%s.%N")
ds2=$(date -d"$d2" "+%s.%N")
diff=$(echo "$ds2 - $ds1" | bc)
printf "$ds2 - $ds1 = %.3f\n" $diff
done < file
$ while read line; do d1=$(echo $line | cut -d, -f1); d2=$(echo $line | cut -d, -f2); ds1=$(date -d"$d1" "+%s.%N"); ds2=$(date -d"$d2" "+%s.%N"); diff=$(echo "$ds2 - $ds1" | bc); printf "$ds2 - $ds1 = %.3f\n" $diff; done < file
1373524748.748000000 - 1373524748.748000000 = 0.000
1373524748.826000000 - 1373524748.826000000 = 0.000
1373524748.860000000 - 1373524748.860000000 = 0.000
1373524748.919000000 - 1373524748.919000000 = 0.000
1373524748.941000000 - 1373524748.941000000 = 0.000
1373524749.390000000 - 1373524749.390000000 = 0.000
1373524749.594000000 - 1373524749.594000000 = 0.000
1373524749.619000000 - 1373524749.619000000 = 0.000
1373524749.787000000 - 1373524749.787000000 = 0.000
1373524750.006000000 - 1373524750.006000000 = 0.000
1373524750.017000000 - 1373524750.017000000 = 0.000
1373524750.088000000 - 1373524750.088000000 = 0.000
1373524750.214000000 - 1373524750.214000000 = 0.000