Python csv切割柱的部分
我遇到了这个奇怪的问题 我还应该提到这在过去是有效的,所以我也在想可能是.csv或特定行本身出了问题 快速故障。我有一个脚本,可以从CVE(漏洞)数据的.csv文件中提取数据。然后,它使用cvss模块重新审视发现,我们使用输出作为衡量修补优先级和紧迫性的方法 (在我们实现新工具之前,此脚本是临时修复) 这就是问题的症结所在。下面是我的摄取文件输出现在的样子Python csv切割柱的部分,python,csv,Python,Csv,我遇到了这个奇怪的问题 我还应该提到这在过去是有效的,所以我也在想可能是.csv或特定行本身出了问题 快速故障。我有一个脚本,可以从CVE(漏洞)数据的.csv文件中提取数据。然后,它使用cvss模块重新审视发现,我们使用输出作为衡量修补优先级和紧迫性的方法 (在我们实现新工具之前,此脚本是临时修复) 这就是问题的症结所在。下面是我的摄取文件输出现在的样子 Vulnerability Title,Plugin ID,Original CVSS Score,Default Vector,Origi
Vulnerability Title,Plugin ID,Original CVSS Score,Default Vector,Original Severity,AWS Score,AWS Vector,AWS Severity,Hosts,Host Type,Percentage Impacted
Cisco IOS IKEv1 Packet Handling Remote Information Disclosure (cisco-sa-20160916-ikev1) (BENIGNCERTAIN),NES-93736,4.6,CVSS2#AV:N/AC:L/Au:N/C:P/I:N/A:N,,,AV:N/AC:L/Au:N/C:P/I:N/A:N,,26,26,
Cisco IOS Software TCP Memory Leak DoS (cisco-sa-20150325-tcpleak),NES-82568,4.9,CVSS2#AV:N/AC:L/Au:N/C:N/I:N/A:C,,,AV:N/AC:L/Au:N/C:N/I:N/A:C,,30,26,
RHEL 5 / 6 / 7 : nss and nss-util (RHSA-2016:2779),NES-94912,4.2,CVSS2#AV:N/AC:M/Au:N/C:C/I:C/A:C/E:F/RL:OF/RC:ND,,,AV:N/AC:M/Au:N/C:C/I:C/A:C/E:F/RL:OF/RC:ND,,5112,23,
这是我的脚本之后的输出(附在下面)
为了进一步解释,第1行以“ium”开头,这是单词Medium的一个截断点,它来自我脚本第128行的底部部分(表示#原始分数的部分)。应该说是中等。所以基本上,如果你看我输入的2个字,和输出比较,它删掉了整行,只添加了脚本试图添加的单词的一半。我想可能是因为所有的支架什么的,但我不确定
Cisco IOS IKEv1 Packet Handling Remote Information Disclosure (cisco-sa-20160916-ikev1) (BENIGNCERTAIN),NES-93736,4.6,CVSS2#AV:N/AC:L/Au:N/C:P/I:N/A:N,
下面是执行此功能的脚本。这有点难看,我知道,改进建议是受欢迎的,但找出它为什么会弄乱我的文件是我现在的首要任务。我曾考虑过换成熊猫,但这需要一点时间,因为我从来没有使用过它,所以我还不知道如何做到这一点
def rescore_function():
#headers
print 'Starting Rescore'
csv_in = open('/tmp/rescore_test.csv', 'rb')
csv_out = open('/tmp/rescored_vulnerabilities.csv', 'wb')
writer = csv.writer(csv_out)
reader = csv.reader(csv_in)
headers = next(reader, None)
if headers:
writer.writerow(headers)
print 'Creating Target Distrobution'
for row in csv.reader(csv_in):
#This is a terrible way of setting up the percentage of hosts impacted for target distrobution. Its ugly and horrible. Host count defines the host impacted, host_type identifies what kind of host it is. Such as Alinux, Rhel5, or Cisco IOS
host_count = float(row[8])
host_type = float(row[9])
alinux_impact = host_count / ALINUX_HOST
cisco_impact = host_count / CISCO_COUNT
juniper_impact = host_count / JUNIPER_COUNT
citrix_impact = host_count / CITRIX_COUNT
all_linux= host_count / LINUX_TOTAL
print 'math set'
#The reason for vul_id is 3 lists combined is simple. alinux_impact NEEDS to be 24, cisco NEEDs to be 26, juniper NEEDS to match 27, because vul_id is the softwares 'vulnerability ID type
#range falls into all_linux. So fillvalue=vul_os[-1] means if its not 24,26,27, it is "all_linux" which means it compares it to the All linux number.
vul_id = [24, 26, 27, 25] + range(24) + range(28,101)
vul_os = [alinux_impact, cisco_impact, juniper_impact, all_linux]
append_file = open('/tmp/rescored_vulnerabilities.csv', 'ab')
append_write = csv.writer(append_file)
#Does the for loop with the fillvalue as mentioned above. Basically Y is the host type (linux, Cisco IOS, etc) and X is the vulnerability type. So it runs through and figures out the TD and rescore methods.
#X equals the percetange of impacted, so the Metric will be based on amount/percentage of X impacted and does a regex search and replace based on that using the CVSS calculations.
print vul_id
print vul_os
for x,y in izip_longest(vul_os, vul_id, fillvalue=vul_os[-1]):
print x,y
print host_type
#VECTOR REGEXP, host_type is which OS/Device type. 23 = RHEL5, 24 = Alinux, 26 = Cisco, 27 = Juniper
if host_type == y:
row[10] = x
if x <= 0.25:
AC_Metric = 'A:C/CDP:L/TD:L/CR:H/IR:H/AR:H'
AP_Metric = 'A:P/CDP:L/TD:L/CR:H/IR:H/AR:H'
AN_Metric = 'A:N/CDP:L/TD:L/CR:H/IR:H/AR:H'
RCUC_Metric = 'RC:UC/CDP:L/TD:L/CR:H/IR:H/AR:H'
RCUR_Metric = 'RC:UR/CDP:L/TD:L/CR:H/IR:H/AR:H'
RCC_Metric = 'RC:C/CDP:L/TD:L/CR:H/IR:H/AR:H'
RCND_Metric = 'RC:ND/CDP:L/TD:L/CR:H/IR:H/AR:H'
elif 0.26 <= x <= 0.75:
AC_Metric = 'A:C/CDP:L/TD:M/CR:H/IR:H/AR:H'
AP_Metric = 'A:P/CDP:L/TD:M/CR:H/IR:H/AR:H'
AN_Metric = 'A:N/CDP:L/TD:M/CR:H/IR:H/AR:H'
RCUC_Metric = 'RC:UC/CDP:L/TD:M/CR:H/IR:H/AR:H'
RCUR_Metric = 'RC:UR/CDP:L/TD:M/CR:H/IR:H/AR:H'
RCC_Metric = 'RC:C/CDP:L/TD:M/CR:H/IR:H/AR:H'
RCND_Metric = 'RC:ND/CDP:L/TD:M/CR:H/IR:H/AR:H'
else:
AC_Metric = 'A:C/CDP:L/TD:H/CR:H/IR:H/AR:H'
AP_Metric = 'A:P/CDP:L/TD:H/CR:H/IR:H/AR:H'
AN_Metric = 'A:N/CDP:L/TD:H/CR:H/IR:H/AR:H'
RCUC_Metric = 'RC:UC/CDP:L/TD:H/CR:H/IR:H/AR:H'
RCUR_Metric = 'RC:UR/CDP:L/TD:H/CR:H/IR:H/AR:H'
RCC_Metric = 'RC:C/CDP:L/TD:H/CR:H/IR:H/AR:H'
RCND_Metric = 'RC:ND/CDP:L/TD:H/CR:H/IR:H/AR:H'
text = row[6]
text = re.sub(r'AV:N','AV:A',text)
text = re.sub(r'AC:L','AC:H',text)
text = re.sub(r'AC:M','AC:H',text)
text = re.sub(r'Au:N','Au:M',text)
text = re.sub(r'Au:S','Au:M',text)
text = re.sub(r'A:C$',AC_Metric,text)
text = re.sub(r'A:P$',AP_Metric,text)
text = re.sub(r'A:N$',AP_Metric,text)
text = re.sub(r'RC:UC',RCUC_Metric,text)
text = re.sub(r'RC:UR',RCUR_Metric,text)
text = re.sub(r'RC:C',RCC_Metric,text)
text = re.sub(r'RC:ND',RCND_Metric,text)
row[6] = text
#NEW SCORE, uses CVSS module to take the previous vector and find out the the numbered score. It then uses that number to define the severity word.
try:
vector = row[6]
c = CVSS2(vector)
row[5] = c.scores()[2]
vul_score = row[5]
if 0 <= vul_score <= 3.9:
vuln_word = 'Low'
elif 4.0 <= vul_score <=6.9:
vuln_word = 'Medium'
elif 7.0 <= vul_score <= 9.9:
vuln_word = 'High'
else:
vuln_word = 'Critical'
row[7] = vuln_word
except CVSS2MalformedError:
rescored_success = False
pass
#ORIGINAL SCORE, does the same as above for the original vector since NESSUS does not provide the Severity "word". This only finds the word, not the number value.
default_score = float(row[2])
if 0 <= default_score <= 3.9:
default_severity = 'Low'
elif 4.0 <= default_score <=6.9:
default_severity = 'Medium'
elif 7.0 <= default_score <= 9.9:
default_severity = 'High'
else:
default_severity = 'Critical'
row[4] = default_severity
append_write.writerow(row)
def rescore_函数():
#标题
打印“开始重新存储”
csv_in=open('/tmp/rescore_test.csv','rb')
csv_out=open('/tmp/rescored_漏洞.csv',wb')
writer=csv.writer(csv\u out)
读卡器=csv。读卡器(csv_in)
headers=next(读卡器,无)
如果标题为:
writer.writerow(标题)
打印“正在创建目标分发”
对于csv.reader中的行(csv\u in):
#这是设置受目标分发影响的主机百分比的糟糕方法。它丑陋可怕。主机计数定义受影响的主机,主机类型标识主机的类型。例如Alinux、Rhel5或Cisco IOS
主机计数=浮动(第[8]行)
主机类型=浮动(第[9]行)
alinux\u影响=主机计数/alinux\u主机
cisco\u影响=主机\u计数/cisco\u计数
juniper\u impact=主机计数/juniper\u计数
citrix_impact=主机_计数/citrix_计数
所有linux=主机计数/linux总数
打印“数学集”
#vul_id为3个列表的原因很简单。alinux_影响需要是24,cisco需要是26,juniper需要匹配27,因为vul_id是软件的漏洞id类型
#范围包括所有linux。所以fillvalue=vul_os[-1]意味着如果不是24,26,27,那么它就是“all_linux”,这意味着它将它与all-linux编号进行比较。
vul_id=[24,26,27,25]+范围(24)+范围(28101)
vul_os=[alinux_影响、cisco_影响、juniper_影响、all_linux]
append_file=open('/tmp/rescorred_漏洞.csv','ab')
append\u write=csv.writer(append\u文件)
#使用上面提到的fillvalue执行for循环。基本上Y是主机类型(linux、Cisco IOS等),X是漏洞类型。因此,它贯穿并计算出TD和rescore方法。
#X等于受影响的百分比,因此该度量将基于受影响X的数量/百分比,并使用CVSS计算基于该数量/百分比进行正则表达式搜索和替换。
打印vul_id
打印vul_os
对于izip_最长的x,y(vul_os,vul_id,fillvalue=vul_os[-1]):
打印x,y
打印主机类型
#VECTOR REGEXP,主机类型是哪个操作系统/设备类型。23=RHEL5,24=Alinux,26=Cisco,27=Juniper
如果主机类型==y:
行[10]=x
如果x您的代码非常大,很难复制,但我怀疑写文件句柄和所有在写模式下进行的缓冲/并发缓冲文件访问有问题。一团糟
首先,您使用csv\u out=open('/tmp/rescored\u-volabilities.csv',wb')打开/截断文件。
你写标题
对于每个迭代,尽管前面提到的句柄没有关闭,但可以在追加模式下打开文件:
append_file=open('/tmp/rescored_-volabilities.csv','ab')
您也不能关闭append\u文件
李>
我建议:
- 第一个是可以的
- 删除
append_file=open('/tmp/rescorred_-volabilities.csv','ab')
- 用
write
替换append\u write
(它会工作,write
指向同一个文件,并且仍然打开)
- 不要忘记在最后关闭
csv\u out
(或者将所有代码放在中,打开(…)作为csv\u out:
块
请注意,此问题仅适用于Un*x。在Windows文件系统上,它会立即抛出异常,因为文件不能在写入模式下打开两次(有时也是如此)。实际上,在python 2中r或rb是可以的。大多数python 2版本(但最新版本)在写入时需要“wb”
或插入空行(在windows上).这是一个错误,检查这里:啊,是的,是那个附加文件。我删除了它,并将其切换为只使用原始编写器,从而修复了所有问题。非常感谢您的帮助!我确实会在脚本完成后关闭文件写入文件,但我也会修复所有问题。太好了!我没有看到还有什么其他问题。
def rescore_function():
#headers
print 'Starting Rescore'
csv_in = open('/tmp/rescore_test.csv', 'rb')
csv_out = open('/tmp/rescored_vulnerabilities.csv', 'wb')
writer = csv.writer(csv_out)
reader = csv.reader(csv_in)
headers = next(reader, None)
if headers:
writer.writerow(headers)
print 'Creating Target Distrobution'
for row in csv.reader(csv_in):
#This is a terrible way of setting up the percentage of hosts impacted for target distrobution. Its ugly and horrible. Host count defines the host impacted, host_type identifies what kind of host it is. Such as Alinux, Rhel5, or Cisco IOS
host_count = float(row[8])
host_type = float(row[9])
alinux_impact = host_count / ALINUX_HOST
cisco_impact = host_count / CISCO_COUNT
juniper_impact = host_count / JUNIPER_COUNT
citrix_impact = host_count / CITRIX_COUNT
all_linux= host_count / LINUX_TOTAL
print 'math set'
#The reason for vul_id is 3 lists combined is simple. alinux_impact NEEDS to be 24, cisco NEEDs to be 26, juniper NEEDS to match 27, because vul_id is the softwares 'vulnerability ID type
#range falls into all_linux. So fillvalue=vul_os[-1] means if its not 24,26,27, it is "all_linux" which means it compares it to the All linux number.
vul_id = [24, 26, 27, 25] + range(24) + range(28,101)
vul_os = [alinux_impact, cisco_impact, juniper_impact, all_linux]
append_file = open('/tmp/rescored_vulnerabilities.csv', 'ab')
append_write = csv.writer(append_file)
#Does the for loop with the fillvalue as mentioned above. Basically Y is the host type (linux, Cisco IOS, etc) and X is the vulnerability type. So it runs through and figures out the TD and rescore methods.
#X equals the percetange of impacted, so the Metric will be based on amount/percentage of X impacted and does a regex search and replace based on that using the CVSS calculations.
print vul_id
print vul_os
for x,y in izip_longest(vul_os, vul_id, fillvalue=vul_os[-1]):
print x,y
print host_type
#VECTOR REGEXP, host_type is which OS/Device type. 23 = RHEL5, 24 = Alinux, 26 = Cisco, 27 = Juniper
if host_type == y:
row[10] = x
if x <= 0.25:
AC_Metric = 'A:C/CDP:L/TD:L/CR:H/IR:H/AR:H'
AP_Metric = 'A:P/CDP:L/TD:L/CR:H/IR:H/AR:H'
AN_Metric = 'A:N/CDP:L/TD:L/CR:H/IR:H/AR:H'
RCUC_Metric = 'RC:UC/CDP:L/TD:L/CR:H/IR:H/AR:H'
RCUR_Metric = 'RC:UR/CDP:L/TD:L/CR:H/IR:H/AR:H'
RCC_Metric = 'RC:C/CDP:L/TD:L/CR:H/IR:H/AR:H'
RCND_Metric = 'RC:ND/CDP:L/TD:L/CR:H/IR:H/AR:H'
elif 0.26 <= x <= 0.75:
AC_Metric = 'A:C/CDP:L/TD:M/CR:H/IR:H/AR:H'
AP_Metric = 'A:P/CDP:L/TD:M/CR:H/IR:H/AR:H'
AN_Metric = 'A:N/CDP:L/TD:M/CR:H/IR:H/AR:H'
RCUC_Metric = 'RC:UC/CDP:L/TD:M/CR:H/IR:H/AR:H'
RCUR_Metric = 'RC:UR/CDP:L/TD:M/CR:H/IR:H/AR:H'
RCC_Metric = 'RC:C/CDP:L/TD:M/CR:H/IR:H/AR:H'
RCND_Metric = 'RC:ND/CDP:L/TD:M/CR:H/IR:H/AR:H'
else:
AC_Metric = 'A:C/CDP:L/TD:H/CR:H/IR:H/AR:H'
AP_Metric = 'A:P/CDP:L/TD:H/CR:H/IR:H/AR:H'
AN_Metric = 'A:N/CDP:L/TD:H/CR:H/IR:H/AR:H'
RCUC_Metric = 'RC:UC/CDP:L/TD:H/CR:H/IR:H/AR:H'
RCUR_Metric = 'RC:UR/CDP:L/TD:H/CR:H/IR:H/AR:H'
RCC_Metric = 'RC:C/CDP:L/TD:H/CR:H/IR:H/AR:H'
RCND_Metric = 'RC:ND/CDP:L/TD:H/CR:H/IR:H/AR:H'
text = row[6]
text = re.sub(r'AV:N','AV:A',text)
text = re.sub(r'AC:L','AC:H',text)
text = re.sub(r'AC:M','AC:H',text)
text = re.sub(r'Au:N','Au:M',text)
text = re.sub(r'Au:S','Au:M',text)
text = re.sub(r'A:C$',AC_Metric,text)
text = re.sub(r'A:P$',AP_Metric,text)
text = re.sub(r'A:N$',AP_Metric,text)
text = re.sub(r'RC:UC',RCUC_Metric,text)
text = re.sub(r'RC:UR',RCUR_Metric,text)
text = re.sub(r'RC:C',RCC_Metric,text)
text = re.sub(r'RC:ND',RCND_Metric,text)
row[6] = text
#NEW SCORE, uses CVSS module to take the previous vector and find out the the numbered score. It then uses that number to define the severity word.
try:
vector = row[6]
c = CVSS2(vector)
row[5] = c.scores()[2]
vul_score = row[5]
if 0 <= vul_score <= 3.9:
vuln_word = 'Low'
elif 4.0 <= vul_score <=6.9:
vuln_word = 'Medium'
elif 7.0 <= vul_score <= 9.9:
vuln_word = 'High'
else:
vuln_word = 'Critical'
row[7] = vuln_word
except CVSS2MalformedError:
rescored_success = False
pass
#ORIGINAL SCORE, does the same as above for the original vector since NESSUS does not provide the Severity "word". This only finds the word, not the number value.
default_score = float(row[2])
if 0 <= default_score <= 3.9:
default_severity = 'Low'
elif 4.0 <= default_score <=6.9:
default_severity = 'Medium'
elif 7.0 <= default_score <= 9.9:
default_severity = 'High'
else:
default_severity = 'Critical'
row[4] = default_severity
append_write.writerow(row)