Python 将多行连接到pfam输出文件中的单行
我有以下多行数据:Python 将多行连接到pfam输出文件中的单行,python,perl,awk,sed,Python,Perl,Awk,Sed,我有以下多行数据: TRINITY_GG_428_c0_g1_i1_或F1 PF13499.1 EF_hand_5 TRINITY_GG_428_c0_g1_i1_或F1 PF00036.27 efhand TRINITY_GG_428_c0_g1_i1_或F1 PF13405.1 EF_手4 TRINITY_GG_428_c0_g1_i1_或F1 PF13833.1 EF_hand_6 TRINITY_GG_428_c0_g1_i1_或F1 PF13202.1 EF_hand_3 TRINIT
TRINITY_GG_428_c0_g1_i1_或F1 PF13499.1 EF_hand_5
TRINITY_GG_428_c0_g1_i1_或F1 PF00036.27 efhand
TRINITY_GG_428_c0_g1_i1_或F1 PF13405.1 EF_手4
TRINITY_GG_428_c0_g1_i1_或F1 PF13833.1 EF_hand_6
TRINITY_GG_428_c0_g1_i1_或F1 PF13202.1 EF_hand_3
TRINITY_GG_429_c0_g1_i1_或F1 PF00156.22 Pribosyltran
TRINITY_GG_431_c5_g1_i1_或F1 PF00475.13 IGPD
TRINITY_GG_461_c0_g1_i1_或F1 PF01208.12 URO-D
TRINITY_GG_461_c0_g1_i1_或F1 PF12876.2类纤维素酶
我要做的是将它们转换为一行:
TRINITY_GGU 428_c0_g1_i1_或F1 PF13499.1 EF_hand_5 | PF00036.27 EF|PF13405.1 EF_hand_4 | PF13833.1 EF_hand_6 | PF13202.1 EF_hand_3
TRINITY_GG_429_c0_g1_i1_或F1 PF00156.22 Pribosyltran
TRINITY_GG_431_c5_g1_i1_或F1 PF00475.13 IGPD
TRINITY_GG_461_c0_g1_i1_或F1 PF01208.12 URO-D | PF12876.2类纤维素酶
匹配的线总是相邻的
如何在sed/awk/Perl/Python中解决这个问题?您可以使用Python正则表达式来执行类似的操作
import re
out_lines = []
with open('file.txt', 'r') as f:
key = None
key_lines = []
for line in f:
m = re.match(r'^(\S+)\s(.+)$', line)
k, v = m.group(1), m.group(2)
if k != key:
if key:
out_lines.append('{0} {1}'.format(key, ' | '.join(key_lines)))
key = k
key_lines = [v]
else:
key_lines.append(v)
else:
if key:
out_lines.append('{0} {1}'.format(key, ' | '.join(key_lines)))
with open('out.txt', 'w') as f:
f.write('\n'.join(out_lines))
使用GNU时:
$ sed -r ':a;N;s/^([^ ]*)( .*)\n\1(.*)$/\1\2 |\3/;ta;P;D' infile
TRINITY_GG_428_c0_g1_i1_orf1 PF13499.1 EF_hand_5 | PF00036.27 efhand | PF13405.1 EF_hand_4 | PF13833.1 EF_hand_6 | PF13202.1 EF_hand_3
TRINITY_GG_429_c0_g1_i1_orf1 PF00156.22 Pribosyltran
TRINITY_GG_431_c5_g1_i1_orf1 PF00475.13 IGPD
TRINITY_GG_461_c0_g1_i1_orf1 PF01208.12 URO-D | PF12876.2 Cellulase-like
主要部分是替换:它检查两行是否以相同的字符串开头(直到第一个空格),如果是,则连接行,从第二行删除字符串,并用管道替换换行符
拆分:
:label # Label to branch to
N # Append next line to pattern space
s/^([^ ]*)( .*)\n\1(.*)$/\1\2 |\3/ # Substitution
t label # Branch to label if the substitution took place
P # Strings weren't identical: print up to newline
D # Delete up to newline, start new cycle (second line become first line)
为了使BSD sed能够工作,我们必须在标签周围拆分命令,并使用-E
标志,而不是-r
:
sed -E -e ':a' -e 'N;s/^([^ ]*)( .*)\n\1(.*)$/\1\2 |\3/;ta' -e 'P;D' infile
为了更好地衡量,仔细看看替代:
s/ # Start substitution
^ # Anchor at start of pattern space
([^ ]*) # Match and capture non-space characters (group #1)
( .*) # Capture up to end of line (group #2)
\n # Match newline
\1 # Start of second line: match first capture group
(.*) # Capture rest of second line (group #3)
$ # Anchor at end of pattern space
/ # Delimiter for substitution
\1\2 |\3 # Substitute: captures groups 1 and 2, space, pipe, capture group 3
/ # End of substitution
这是一种非常常见的编程模式。您需要使用Perl哈希来累积属于每个不同初始字段(键)的所有数据。然后只需按所需的顺序和格式打印哈希 这个程序演示了。我假设您希望按键的词法顺序打印键。如果您需要任何不同的内容,例如它们在源数据中首次出现的顺序,那么请这样说—一个小的更改是必要的 该程序希望输入文件的路径作为命令行上的参数,并将其输出发送到STDOUT,STDOUT可以以正常方式重定向
use strict;
use warnings 'all';
my %data;
while ( <> ) {
chomp;
my ($key, $val) = split ' ', $_, 2;
push @{ $data{$key} }, $val;
}
print $_, ' ', join(' | ', @{ $data{$_} }), "\n" for sort keys %data;
只要在当前行的第一个字段与前一行相同时建立所有行的连接记录,然后在第一个字段的值发生变化时打印:
$ awk '
$1==prev { rec = rec " | " $2 " " $3 }
$1!=prev { if (NR>1) print rec; rec=$0 }
{ prev=$1 }
END { print rec }
' file
TRINITY_GG_428_c0_g1_i1_orf1 PF13499.1 EF_hand_5 | PF00036.27 efhand | PF13405.1 EF_hand_4 | PF13833.1 EF_hand_6 | PF13202.1 EF_hand_3
TRINITY_GG_429_c0_g1_i1_orf1 PF00156.22 Pribosyltran
TRINITY_GG_431_c5_g1_i1_orf1 PF00475.13 IGPD
TRINITY_GG_461_c0_g1_i1_orf1 PF01208.12 URO-D | PF12876.2 Cellulase-like
或者,如果您的输入行键不是连续的,并且您不关心输出顺序是否与输入顺序相同,并且您的输入文件足够小,可以将其全部保存在内存中,那么您可以使用散列方法,在不同的答案中建议:
$ awk '{a[$1]=($1 in a ? a[$1]" | "$2" "$3 : $0)} END{for (k in a) print a[k]}' file
TRINITY_GG_429_c0_g1_i1_orf1 PF00156.22 Pribosyltran
TRINITY_GG_461_c0_g1_i1_orf1 PF01208.12 URO-D | PF12876.2 Cellulase-like
TRINITY_GG_431_c5_g1_i1_orf1 PF00475.13 IGPD
TRINITY_GG_428_c0_g1_i1_orf1 PF13499.1 EF_hand_5 | PF00036.27 efhand | PF13405.1 EF_hand_4 | PF13833.1 EF_hand_6 | PF13202.1 EF_hand_3
匹配的行总是在连续的行上吗?是的,我想将重复的行连接成一条单行。当我将a[$1]替换为$0时,这样可以吗:
awk'{a[$1]=($1在a?$0“|“$2”“$3:$0)}END{for(k在a)打印[k]}文件
?我不确定我理解这个问题,但a[$1]=($1在a?$0“|“$2”“$3:$0)
最好写为a[$1]=$0($1在a?”|“$2”“$3:”)
因此您只需声明公共部分$0一次,并明确表示它无条件发生。