Python 3.x python基于主机名从复合日志文件中分离行_Python 3.x_Pandas

Python 3.x python基于主机名从复合日志文件中分离行

python-3.x pandas

Python 3.x python基于主机名从复合日志文件中分离行,python-3.x,pandas,Python 3.x,Pandas,我期待着一种方法来读取日志文件，读取基于第5列的主机名，并在所有迭代中保留主机名，直到新名称出现，或者说在找到新名称时只标记一个双空格，但需要打印整行正在读取文件： $ cat test.py with open("file", "r") as f: for line in f: fr = f.read() print(fr) $ ./test.py Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002

我期待着一种方法来读取日志文件，读取基于第5列的主机名，并在所有迭代中保留主机名，直到新名称出现，或者说在找到新名称时只标记一个双空格，但需要打印整行

正在读取文件：

$ cat test.py
with open("file", "r") as f:
    for line in f:
        fr = f.read()
        print(fr)

$ ./test.py
Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002   not responding, still trying
Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:13:20 hw070 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:17:26 hw070 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:16:21 hw071 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:17:24 hw071 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:18:27 hw071 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:43:07 hw073 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 15:45:59 hw074 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 17:17:12 hw074 kernel: nfs: server hpstor002   not responding, still trying
Jul 19 01:20:20 hw074 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 16:16:01 hw079 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:17:56 hw079 kernel: nfs: server hpstor001   not responding, still trying
Jul 17 04:34:08 anujv kernel: nfs: server hpstor003  not responding, still trying
Jul 17 21:18:42 sanujv kernel: nfs: server hpstor003  not responding, still trying
Jul 18 01:36:00 sanujv kernel: nfs: server hpstor003  not responding, still trying
Jul 18 04:16:10 sanujv kernel: nfs: server hpstor003  not responding, still trying

Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001   not responding, still trying


Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002   not responding, still trying
Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001   not responding, still trying


Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001   not responding, still trying

文件内容输出：

$ cat test.py
with open("file", "r") as f:
    for line in f:
        fr = f.read()
        print(fr)

$ ./test.py
Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002   not responding, still trying
Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:13:20 hw070 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:17:26 hw070 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:16:21 hw071 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:17:24 hw071 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:18:27 hw071 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:43:07 hw073 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 15:45:59 hw074 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 17:17:12 hw074 kernel: nfs: server hpstor002   not responding, still trying
Jul 19 01:20:20 hw074 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 16:16:01 hw079 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:17:56 hw079 kernel: nfs: server hpstor001   not responding, still trying
Jul 17 04:34:08 anujv kernel: nfs: server hpstor003  not responding, still trying
Jul 17 21:18:42 sanujv kernel: nfs: server hpstor003  not responding, still trying
Jul 18 01:36:00 sanujv kernel: nfs: server hpstor003  not responding, still trying
Jul 18 04:16:10 sanujv kernel: nfs: server hpstor003  not responding, still trying

Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001   not responding, still trying


Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002   not responding, still trying
Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001   not responding, still trying


Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001   not responding, still trying

所需输出：

$ cat test.py
with open("file", "r") as f:
    for line in f:
        fr = f.read()
        print(fr)

$ ./test.py
Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002   not responding, still trying
Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:13:20 hw070 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:17:26 hw070 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:16:21 hw071 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:17:24 hw071 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:18:27 hw071 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 20:43:07 hw073 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 15:45:59 hw074 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 17:17:12 hw074 kernel: nfs: server hpstor002   not responding, still trying
Jul 19 01:20:20 hw074 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 16:16:01 hw079 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:17:56 hw079 kernel: nfs: server hpstor001   not responding, still trying
Jul 17 04:34:08 anujv kernel: nfs: server hpstor003  not responding, still trying
Jul 17 21:18:42 sanujv kernel: nfs: server hpstor003  not responding, still trying
Jul 18 01:36:00 sanujv kernel: nfs: server hpstor003  not responding, still trying
Jul 18 04:16:10 sanujv kernel: nfs: server hpstor003  not responding, still trying

Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001   not responding, still trying


Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002   not responding, still trying
Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001   not responding, still trying


Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001   not responding, still trying

或者用多一点化妆品：

hw067 
Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002   not responding, still trying
Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001   not responding, still trying

hw068 
Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002   not responding, still trying
Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001   not responding, still trying

hw069
Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001   not responding, still trying
Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001   not responding, still trying

除了简单的python方法之外，我们还可以用python实现这一点吗。

-谢谢

你可以通过itertools和operator function学习，我是一个很好的学习者，我从我的一篇文章的ans中借用了代码，但是我收集了关于这两个函数的一些细节，我希望它们会有用

itertools.groupby（iterable，key=None或some func）获取一个iterable列表，并根据指定的键对其进行分组。该键指定要应用于每个单独iterable的操作，其结果将用作每个分组项的标题；最终具有相同“键”值的项将最终位于同一组中。返回值是一个类似于字典的iterable，其形式为{key:value}

操作符。itemgetter（n）构造一个可调用对象，该对象假定一个iterable对象（例如list、tuple、set）作为输入，并从中提取第n个元素

操作符。itemgetter（n）将为您提供：一个从类似列表的对象中获取项的函数。 itemgetter（3）从列表中获取第三项

更确切地说，这些是可调用的，而不是函数，但这是一个经常被忽略的区别

以下代码应适用于您：

根据上述代码的结果，我将您的文件用于测试

anujv
    Jul 17 04:34:08 anujv kernel: nfs: server hpstor003 not responding, still trying

hw060
    Jul 18 15:46:02 hw060 kernel: nfs: server hpstor001 not responding, still trying

hw067
    Jul 18 14:30:02 hw067 kernel: nfs: server hpstor002 not responding, still trying
    Jul 18 23:47:45 hw067 kernel: nfs: server hpstor001 not responding, still trying
    Jul 18 23:48:48 hw067 kernel: nfs: server hpstor001 not responding, still trying
    Jul 18 23:49:51 hw067 kernel: nfs: server hpstor001 not responding, still trying

hw068
    Jul 18 17:17:10 hw068 kernel: nfs: server hpstor002 not responding, still trying
    Jul 19 00:46:43 hw068 kernel: nfs: server hpstor001 not responding, still trying
    Jul 19 00:48:49 hw068 kernel: nfs: server hpstor001 not responding, still trying

hw069
    Jul 19 00:50:01 hw069 kernel: nfs: server hpstor001 not responding, still trying
    Jul 19 00:51:16 hw069 kernel: nfs: server hpstor001 not responding, still trying
    Jul 19 00:52:52 hw069 kernel: nfs: server hpstor001 not responding, still trying
    Jul 18 22:13:20 hw069 kernel: nfs: server hpstor001 not responding, still trying

hw070
    Jul 18 22:13:20 hw070 kernel: nfs: server hpstor001 not responding, still trying
    Jul 18 22:17:26 hw070 kernel: nfs: server hpstor002 not responding, still trying

hw071
    Jul 18 20:16:21 hw071 kernel: nfs: server hpstor002 not responding, still trying
    Jul 18 20:17:24 hw071 kernel: nfs: server hpstor002 not responding, still trying
    Jul 18 20:18:27 hw071 kernel: nfs: server hpstor002 not responding, still trying

hw073
    Jul 18 20:43:07 hw073 kernel: nfs: server hpstor002 not responding, still trying

hw074
    Jul 18 15:45:59 hw074 kernel: nfs: server hpstor001 not responding, still trying
    Jul 18 17:17:12 hw074 kernel: nfs: server hpstor002 not responding, still trying
    Jul 19 01:20:20 hw074 kernel: nfs: server hpstor002 not responding, still trying

hw079
    Jul 18 16:16:01 hw079 kernel: nfs: server hpstor001 not responding, still trying
    Jul 19 00:17:56 hw079 kernel: nfs: server hpstor001 not responding, still trying

sanujv
    Jul 17 21:18:42 sanujv kernel: nfs: server hpstor003 not responding, still trying
    Jul 18 01:36:00 sanujv kernel: nfs: server hpstor003 not responding, still trying
    Jul 18 04:16:10 sanujv kernel: nfs: server hpstor003 not responding, still trying

你为什么给熊猫贴标签？你想要纯文本输出还是数据帧？@xyzjayne，我对任何想法都没意见。即使使用熊猫数据帧，纯文本输出也很不错。