Linux 格式化Masscan输出

我从Masscan的-oG选项中获得以下输出：

如何使用awk、cut、grep、sed等操作此输出以获得以下格式：

192.168.1.1 80,8000
192.168.1.2 443,3478
192.168.1.3 80,8000
192.168.1.4 80,443

代码注释：

#!/bin/bash

# create the input file:
cat <<EOF >file
Host: 192.168.1.1 ()  Ports: 8000/open/tcp//unknown//
Host: 192.168.1.2 ()  Ports: 3478/open/tcp//unknown//
Host: 192.168.1.3 ()   Ports: 8000/open/tcp//unknown//
Host: 192.168.1.1 ()        Ports: 80/open/tcp//http//
Host: 192.168.1.2 ()        Ports: 443/open/tcp//https//
Host: 192.168.1.4 () Ports: 443/open/tcp//https//
Host: 192.168.1.3 () Ports: 80/open/tcp//http//
Host: 192.168.1.4 () Ports: 80/open/tcp//http//
EOF

# extract fields 2 and 5
<file awk '{print $2,$5}' |
# remove all that /open/tcp//https... part
sed 's@/.*@@' |
# Now merging is the worst part...
# script from https://stackoverflow.com/questions/19823941/join-lines-with-the-same-value-in-the-first-column
# This outputs `field1 , field2, field3, field4`
awk -F' ' -v OFS=' ' '{x=$1;$1="";a[x]=a[x]","$0}END{for(x in a) print x,a[x]}' |
# subsitute `, ` for `,` and remove the only remaining first ` ,`
sed 's/, /,/g' | sed 's/ ,/ /'

有没有办法以递增的方式对端口进行排序

当然。在awk之前，使用第二列或先使用第一列再使用第二列进行数字排序。awk将维持秩序

# extract fields 2 and 5
<file awk '{print $2,$5}' |
# remove all that /open/tcp//https... part
sed 's@/.*@@' |
# numeric sort using the second column (ie. port)
sort -t' ' -n -k2 |
# Now merging is the worst part...
# script from https://stackoverflow.com/questions/19823941/join-lines-with-the-same-value-in-the-first-column
# This outputs `field1 , field2, field3, field4`
awk -F' ' -v OFS=' ' '{x=$1;$1="";a[x]=a[x]","$0}END{for(x in a) print x,a[x]}' |
# subsitute `, ` for `,` and remove the only remaining first ` ,`
sed 's/, /,/g' | sed 's/ ,/ /'

试试这个：

#!/bin/bash

# define testcontent
content=$(cat << EOT
# Masscan 1.0.6 scan initiated Mon May  6 08:45:19 2019
# Ports scanned: TCP(13107;1-13107) UDP(0;) SCTP(0;) PROTOCOLS(0;)
Host: 192.168.1.1 ()  Ports: 8000/open/tcp//unknown//
Host: 192.168.1.2 ()  Ports: 3478/open/tcp//unknown//
Host: 192.168.1.3 ()   Ports: 8000/open/tcp//unknown//
Host: 192.168.1.1 ()        Ports: 80/open/tcp//http//
Host: 192.168.1.2 ()        Ports: 443/open/tcp//https//
Host: 192.168.1.4 () Ports: 443/open/tcp//https//
Host: 192.168.1.3 () Ports: 80/open/tcp//http//
Host: 192.168.1.4 () Ports: 80/open/tcp//http//
EOT
)

# declare associative array
declare -A dict 

# loop over all ip lines
while read -r ip port; do
   # save ports
   dict[$ip]+="$port "
         # ignore lines start with #, grep ip an port from content 
done < <(sed '/^#/d;s/Host: \([^ ]*\).*Ports: \([0-9]*\).*/\1 \2/' <<< "$content") 

# loop over assocative array
for key in  "${!dict[@]}"; do

   # sort ports in string
   sorted=$(echo "${dict[$key]}" | tr " " "\n" | sort -n | tr "\n" ,)

   # extract leading ,
   ports="${sorted#*,}"

   # print key an ports without tailing ,
   printf "%s %s\n" "$key" "${ports%,*}"
done | sort  

---

缺少转换逻辑的问题。撰稿人不应该猜测问题的意图。谢谢你的回答。我更喜欢这个，因为它可以做为一个班轮。有没有办法以递增的方式对端口进行排序？

# extract fields 2 and 5
<file awk '{print $2,$5}' |
# remove all that /open/tcp//https... part
sed 's@/.*@@' |
# numeric sort using the second column (ie. port)
sort -t' ' -n -k2 |
# Now merging is the worst part...
# script from https://stackoverflow.com/questions/19823941/join-lines-with-the-same-value-in-the-first-column
# This outputs `field1 , field2, field3, field4`
awk -F' ' -v OFS=' ' '{x=$1;$1="";a[x]=a[x]","$0}END{for(x in a) print x,a[x]}' |
# subsitute `, ` for `,` and remove the only remaining first ` ,`
sed 's/, /,/g' | sed 's/ ,/ /'

192.168.1.1 80,8000
192.168.1.2 443,3478
192.168.1.3 80,8000
192.168.1.4 80,443

#!/bin/bash

# define testcontent
content=$(cat << EOT
# Masscan 1.0.6 scan initiated Mon May  6 08:45:19 2019
# Ports scanned: TCP(13107;1-13107) UDP(0;) SCTP(0;) PROTOCOLS(0;)
Host: 192.168.1.1 ()  Ports: 8000/open/tcp//unknown//
Host: 192.168.1.2 ()  Ports: 3478/open/tcp//unknown//
Host: 192.168.1.3 ()   Ports: 8000/open/tcp//unknown//
Host: 192.168.1.1 ()        Ports: 80/open/tcp//http//
Host: 192.168.1.2 ()        Ports: 443/open/tcp//https//
Host: 192.168.1.4 () Ports: 443/open/tcp//https//
Host: 192.168.1.3 () Ports: 80/open/tcp//http//
Host: 192.168.1.4 () Ports: 80/open/tcp//http//
EOT
)

# declare associative array
declare -A dict 

# loop over all ip lines
while read -r ip port; do
   # save ports
   dict[$ip]+="$port "
         # ignore lines start with #, grep ip an port from content 
done < <(sed '/^#/d;s/Host: \([^ ]*\).*Ports: \([0-9]*\).*/\1 \2/' <<< "$content") 

# loop over assocative array
for key in  "${!dict[@]}"; do

   # sort ports in string
   sorted=$(echo "${dict[$key]}" | tr " " "\n" | sort -n | tr "\n" ,)

   # extract leading ,
   ports="${sorted#*,}"

   # print key an ports without tailing ,
   printf "%s %s\n" "$key" "${ports%,*}"
done | sort  

192.168.1.1 80,8000
192.168.1.2 443,3478
192.168.1.3 80,8000
192.168.1.4 80,443