Linux 文件操作：删除字符串的每次出现，并在某个不同模式后执行第一次出现

我希望删除文件中多次出现的行，但希望保留某些行。我该怎么做

这是我文件的一部分，我想更改：

§M: 1, K: 2
    name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,     task, scale,  IPC,  CPU,  GHz 
Fastor2D, 0.00, 88378.00, 23272.00,  646.00,  1525.00,  241.00, 17447.00,     1, 0.26, 4.71, 5.07 
    name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,   CPU,  GHz 
Fastor2D, 0.00, 32083.00, 17690.00,  148.00,    28.00,  116.00, 6437.00,     1, 0.55, 25.96, 4.98 
    name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,   CPU,  GHz 
Fastor2D, 0.00, 31342.00, 17690.00,  161.00,    23.00,  100.00, 6125.00,     1, 0.56, 36.46, 5.12 

   name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,   CPU,  GHz 
Blitz2D, 0.00, 33418.00, 17933.00,  168.00,    83.00,  109.00, 6473.00,     1, 0.54, 12.50, 5.16 

     name, time,     cycle,    instr, L1-miss, LLC-miss, br-miss,     task, scale,  IPC,  CPU,  GHz 
XTensor2D, 0.00, 176180.00, 60830.00, 1586.00,  3040.00,  546.00, 36306.00,     1, 0.35, 1.21, 4.85 
     name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,  CPU,  GHz 
XTensor2D, 0.00, 39599.00, 21641.00,  258.00,    41.00,  169.00, 7962.00,     1, 0.55, 4.21, 4.97 
     name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,  CPU,  GHz 
XTensor2D, 0.00, 36483.00, 21641.00,  261.00,     5.00,  110.00, 7323.00,     1, 0.59, 5.84, 4.98 

   name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,  CPU,  GHz 
Eigen2D, 0.00, 47271.00, 24427.00,  296.00,   298.00,  143.00, 9354.00,     1, 0.52, 2.81, 5.05 
   name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,   CPU,  GHz 
Eigen2D, 0.00, 31979.00, 18101.00,  166.00,    19.00,  100.00, 6313.00,     1, 0.57, 16.23, 5.07 
   name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,   CPU,  GHz 
Eigen2D, 0.00, 32383.00, 18101.00,  175.00,    38.00,   96.00, 6513.00,     1, 0.56, 18.09, 4.97 




§M: 2, K: 2
    name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,   CPU,  GHz 
Fastor2D, 0.00, 31548.00, 17689.00,  149.00,    20.00,  101.00, 6112.00,     1, 0.56, 33.77, 5.16 
    name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,   CPU,  GHz 
Fastor2D, 0.00, 31404.00, 17689.00,  161.00,     0.00,   95.00, 6105.00,     1, 0.56, 41.82, 5.14 
    name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,   CPU,  GHz 
Fastor2D, 0.00, 31060.00, 17689.00,  167.00,     8.00,   93.00, 6145.00,     1, 0.57, 38.41, 5.05 

   name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,   CPU,  GHz 
Blitz2D, 0.00, 32130.00, 18122.00,  155.00,    47.00,  113.00, 6359.00,     1, 0.56, 16.87, 5.05 

     name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,  CPU,  GHz 
XTensor2D, 0.00, 42370.00, 22410.00,  245.00,   105.00,  193.00, 8393.00,     1, 0.53, 3.53, 5.05 
     name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,  CPU,  GHz 
XTensor2D, 0.00, 37526.00, 21853.00,  264.00,    26.00,  119.00, 7474.00,     1, 0.58, 5.25, 5.02 
     name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,  CPU,  GHz 
XTensor2D, 0.00, 37091.00, 21853.00,  247.00,    22.00,  107.00, 7377.00,     1, 0.59, 5.74, 5.03 

   name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,     task, scale,  IPC,  CPU,  GHz 
Eigen2D, 0.00, 55863.00, 25323.00,  321.00,   434.00,  166.00, 11266.00,     1, 0.45, 2.17, 4.96 
   name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,  CPU,  GHz 
Eigen2D, 0.00, 34626.00, 18984.00,  198.00,    47.00,  111.00, 6661.00,     1, 0.55, 9.93, 5.20 
   name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,   CPU,  GHz 
Eigen2D, 0.00, 32862.00, 18984.00,  191.00,    17.00,   97.00, 6572.00,     1, 0.58, 11.99, 5.00 

这种模式还在继续。我想要的是，

name，…

行在每个

§M:*，K:*

之后保留，但在这之后它被删除，直到模式2再次出现

最后，它应该看起来像这样：

§M: 1, K: 2
    name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,     task, scale,  IPC,  CPU,  GHz 
Fastor2D, 0.00, 88378.00, 23272.00,  646.00,  1525.00,  241.00, 17447.00,     1, 0.26, 4.71, 5.07 
Fastor2D, 0.00, 32083.00, 17690.00,  148.00,    28.00,  116.00, 6437.00,     1, 0.55, 25.96, 4.98 
Fastor2D, 0.00, 31342.00, 17690.00,  161.00,    23.00,  100.00, 6125.00,     1, 0.56, 36.46, 5.12 

Blitz2D, 0.00, 33418.00, 17933.00,  168.00,    83.00,  109.00, 6473.00,     1, 0.54, 12.50, 5.16 

XTensor2D, 0.00, 176180.00, 60830.00, 1586.00,  3040.00,  546.00, 36306.00,     1, 0.35, 1.21, 4.85 
XTensor2D, 0.00, 39599.00, 21641.00,  258.00,    41.00,  169.00, 7962.00,     1, 0.55, 4.21, 4.97 
XTensor2D, 0.00, 36483.00, 21641.00,  261.00,     5.00,  110.00, 7323.00,     1, 0.59, 5.84, 4.98 

Eigen2D, 0.00, 47271.00, 24427.00,  296.00,   298.00,  143.00, 9354.00,     1, 0.52, 2.81, 5.05 
Eigen2D, 0.00, 31979.00, 18101.00,  166.00,    19.00,  100.00, 6313.00,     1, 0.57, 16.23, 5.07 
Eigen2D, 0.00, 32383.00, 18101.00,  175.00,    38.00,   96.00, 6513.00,     1, 0.56, 18.09, 4.97 




§M: 2, K: 2
    name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,   CPU,  GHz 
Fastor2D, 0.00, 31548.00, 17689.00,  149.00,    20.00,  101.00, 6112.00,     1, 0.56, 33.77, 5.16 
Fastor2D, 0.00, 31404.00, 17689.00,  161.00,     0.00,   95.00, 6105.00,     1, 0.56, 41.82, 5.14 
Fastor2D, 0.00, 31060.00, 17689.00,  167.00,     8.00,   93.00, 6145.00,     1, 0.57, 38.41, 5.05 

Blitz2D, 0.00, 32130.00, 18122.00,  155.00,    47.00,  113.00, 6359.00,     1, 0.56, 16.87, 5.05 

XTensor2D, 0.00, 42370.00, 22410.00,  245.00,   105.00,  193.00, 8393.00,     1, 0.53, 3.53, 5.05 
XTensor2D, 0.00, 37526.00, 21853.00,  264.00,    26.00,  119.00, 7474.00,     1, 0.58, 5.25, 5.02 
XTensor2D, 0.00, 37091.00, 21853.00,  247.00,    22.00,  107.00, 7377.00,     1, 0.59, 5.74, 5.03 

Eigen2D, 0.00, 55863.00, 25323.00,  321.00,   434.00,  166.00, 11266.00,     1, 0.45, 2.17, 4.96 
Eigen2D, 0.00, 34626.00, 18984.00,  198.00,    47.00,  111.00, 6661.00,     1, 0.55, 9.93, 5.20 
Eigen2D, 0.00, 32862.00, 18984.00,  191.00,    17.00,   97.00, 6572.00,     1, 0.58, 11.99, 5.00 

---

这是sed公式还是awk中的sed公式？请解释一下你的台词是做什么的，我想了解一下。提前感谢

删除重复项的典型习惯用法是：

awk'！d[$0]+'

您可以向其添加任何谓词，该谓词可以基于上一行：

awk '
  /M:.+, K:/ { last_header_line = NR }
  last_header_line == NR-1 || !d[$0]++ { print }
'

使用awk：

# awk is a line oriented program.
# for each line in the input file, following procedures are run.
/^§/ { f = 1 } # if input line begins with a `§', flag is up
/^ *name,/ {   # if input line begins with zero or more spaces followed by `name,';
  if (f) {     # and if flag is up;
    print      # print this line
    f = 0      # put flag down
  }
  next         # break the cycle irrespective of flag's status, read a new line and start over
}
1 # when this is reached, print input line

用法：

awk '/^§/{f=1} /^ *name,/{if(f){print;f=0} next} 1' file

---

将输入视为一系列记录，其第一行从

§

开始。现在只需打印每条记录的第二行（即每条记录中的第一行

名称

），以及不以

名称

开头的每一行：

$ awk '/^§/{lineNr=0} (++lineNr==2) || (!/^ *name/)' file
§M: 1, K: 2
    name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,     task, scale,  IPC,  CPU,  GHz
Fastor2D, 0.00, 88378.00, 23272.00,  646.00,  1525.00,  241.00, 17447.00,     1, 0.26, 4.71, 5.07
Fastor2D, 0.00, 32083.00, 17690.00,  148.00,    28.00,  116.00, 6437.00,     1, 0.55, 25.96, 4.98
Fastor2D, 0.00, 31342.00, 17690.00,  161.00,    23.00,  100.00, 6125.00,     1, 0.56, 36.46, 5.12

Blitz2D, 0.00, 33418.00, 17933.00,  168.00,    83.00,  109.00, 6473.00,     1, 0.54, 12.50, 5.16

XTensor2D, 0.00, 176180.00, 60830.00, 1586.00,  3040.00,  546.00, 36306.00,     1, 0.35, 1.21, 4.85
XTensor2D, 0.00, 39599.00, 21641.00,  258.00,    41.00,  169.00, 7962.00,     1, 0.55, 4.21, 4.97
XTensor2D, 0.00, 36483.00, 21641.00,  261.00,     5.00,  110.00, 7323.00,     1, 0.59, 5.84, 4.98

Eigen2D, 0.00, 47271.00, 24427.00,  296.00,   298.00,  143.00, 9354.00,     1, 0.52, 2.81, 5.05
Eigen2D, 0.00, 31979.00, 18101.00,  166.00,    19.00,  100.00, 6313.00,     1, 0.57, 16.23, 5.07
Eigen2D, 0.00, 32383.00, 18101.00,  175.00,    38.00,   96.00, 6513.00,     1, 0.56, 18.09, 4.97




§M: 2, K: 2
    name, time,    cycle,    instr, L1-miss, LLC-miss, br-miss,    task, scale,  IPC,   CPU,  GHz
Fastor2D, 0.00, 31548.00, 17689.00,  149.00,    20.00,  101.00, 6112.00,     1, 0.56, 33.77, 5.16
Fastor2D, 0.00, 31404.00, 17689.00,  161.00,     0.00,   95.00, 6105.00,     1, 0.56, 41.82, 5.14
Fastor2D, 0.00, 31060.00, 17689.00,  167.00,     8.00,   93.00, 6145.00,     1, 0.57, 38.41, 5.05

Blitz2D, 0.00, 32130.00, 18122.00,  155.00,    47.00,  113.00, 6359.00,     1, 0.56, 16.87, 5.05

XTensor2D, 0.00, 42370.00, 22410.00,  245.00,   105.00,  193.00, 8393.00,     1, 0.53, 3.53, 5.05
XTensor2D, 0.00, 37526.00, 21853.00,  264.00,    26.00,  119.00, 7474.00,     1, 0.58, 5.25, 5.02
XTensor2D, 0.00, 37091.00, 21853.00,  247.00,    22.00,  107.00, 7377.00,     1, 0.59, 5.74, 5.03

Eigen2D, 0.00, 55863.00, 25323.00,  321.00,   434.00,  166.00, 11266.00,     1, 0.45, 2.17, 4.96
Eigen2D, 0.00, 34626.00, 18984.00,  198.00,    47.00,  111.00, 6661.00,     1, 0.55, 9.93, 5.20
Eigen2D, 0.00, 32862.00, 18984.00,  191.00,    17.00,   97.00, 6572.00,     1, 0.58, 11.99, 5.00

---

到目前为止，您尝试过什么？由于我在sed或awk方面没有任何实际经验，我没有尝试过任何东西，因为我只会做错事。请您解释一下此命令的作用，以及我如何将其应用于我的情况？我还想保留大部分副本，只是不是第一次出现在一个文件中之后的副本谢谢，这起作用了，你能解释一下它的作用吗，我得到了带有段落字符的部分，以及为什么在这之后第一次出现的部分没有被删除，但我不明白接下来会发生什么does@CleboSevic更新了我的答案，添加了更多的评论。请再读一遍，你不理解的具体部分是什么？@CleboSevic nope。如果awk达到

1

，它将打印输入行。在上面的脚本中，如果输入行以

name开始，

并且标志为down，我们将使用

next

过程阻止awk到达

1

。删除它并运行脚本以查看差异。@CleboSevic欢迎，我的英语有点差劲，所以我无法像我想的那样清楚地解释它，很抱歉，我对我的感觉也一样，所以我不确定我是否理解错了。这是非常好的，可以理解