Python 3.x Pandas不会写入所有结果,它会覆盖并只给出最后一个结果
我正在进行网页抓取,我正在逐行从文本文件中提取姓名,在谷歌上搜索,并从搜索结果中抓取地址。我想在各自的名字前面加上这个结果。这是我的文本文件a.txt:Python 3.x Pandas不会写入所有结果,它会覆盖并只给出最后一个结果,python-3.x,pandas,beautifulsoup,Python 3.x,Pandas,Beautifulsoup,我正在进行网页抓取,我正在逐行从文本文件中提取姓名,在谷歌上搜索,并从搜索结果中抓取地址。我想在各自的名字前面加上这个结果。这是我的文本文件a.txt: 0.5BN FINHEALTH PRIVATE LIMITED 01 SYNERGY CO. 1 BY 0 SOLUTIONS 这是我的代码: import requests from bs4 import BeautifulSoup import pandas as pd USER_AGENT = "Mozilla/5.0 (
0.5BN FINHEALTH PRIVATE LIMITED
01 SYNERGY CO.
1 BY 0 SOLUTIONS
这是我的代码:
import requests
from bs4 import BeautifulSoup
import pandas as pd
USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:65.0) Gecko/20100101 Firefox/65.0"
out_fl = open('a.txt','r')
for line in out_fl:
query = line
query = query.replace(' ', '+')
# print(line)
URL = f"https://google.com/search?q={query}"
# print(URL)
headers = {"user-agent": USER_AGENT}
resp = requests.get(URL, headers=headers)
if resp.status_code == 200:
soup = BeautifulSoup(resp.content, "html.parser")
results = []
for g in soup.find_all('div', class_="i4J0ge"):
address = soup.find('span', class_="LrzXr")
if address:
address = (address.text)
else:
print("Not found")
phone = soup.find('span',class_="LrzXr zdqRlf kno-fv")
if phone:
phone = (phone.text)
else:
print("None")
company = line
item = {"company": line.replace('\n',''),"Address" : address,"Phone" : phone}
# print(item)
results.append(item)
print(results)
df = pd.DataFrame(results, columns=["company", "Address", "Phone"])
df.to_excel('filename.xlsx', sheet_name='sheet name', index=False)
out_fl.close()
我不知道它在哪里被覆盖,请帮助我。谢谢。试试这个,但你必须根据自己的喜好修改结果
import requests
from bs4 import BeautifulSoup
import pandas as pd
USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:65.0) Gecko/20100101 Firefox/65.0"
df = pd.DataFrame(results, columns=["company","result"])
out_fl = open('a.txt','r')
for line in out_fl:
query = line
query = query.replace(' ', '+')
# print(line)
URL = f"https://google.com/search?q={query}"
# print(URL)
headers = {"user-agent": USER_AGENT}
resp = requests.get(URL, headers=headers)
if resp.status_code == 200:
soup = BeautifulSoup(resp.content, "html.parser")
results = []
for g in soup.find_all('div', class_="i4J0ge"):
address = soup.find('span', class_="LrzXr")
if address:
address = (address.text)
else:
print("Not found")
phone = soup.find('span',class_="LrzXr zdqRlf kno-fv")
if phone:
phone = (phone.text)
else:
print("None")
company = line
item = {"company": line.replace('\n',''),"Address" : address,"Phone" : phone}
# print(item)
results.append(item)
print(results)
df.loc[query]=[query,results]
df.to_excel("results.xlsx",sheet_name="result", index=False)
尝试此操作,但您必须根据您的需要修改结果
import requests
from bs4 import BeautifulSoup
import pandas as pd
USER_AGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:65.0) Gecko/20100101 Firefox/65.0"
df = pd.DataFrame(results, columns=["company","result"])
out_fl = open('a.txt','r')
for line in out_fl:
query = line
query = query.replace(' ', '+')
# print(line)
URL = f"https://google.com/search?q={query}"
# print(URL)
headers = {"user-agent": USER_AGENT}
resp = requests.get(URL, headers=headers)
if resp.status_code == 200:
soup = BeautifulSoup(resp.content, "html.parser")
results = []
for g in soup.find_all('div', class_="i4J0ge"):
address = soup.find('span', class_="LrzXr")
if address:
address = (address.text)
else:
print("Not found")
phone = soup.find('span',class_="LrzXr zdqRlf kno-fv")
if phone:
phone = (phone.text)
else:
print("None")
company = line
item = {"company": line.replace('\n',''),"Address" : address,"Phone" : phone}
# print(item)
results.append(item)
print(results)
df.loc[query]=[query,results]
df.to_excel("results.xlsx",sheet_name="result", index=False)
此脚本将从您的输入文件a.txt生成公司/电话的CSV:
0.5BN FINHEALTH PRIVATE LIMITED
01 SYNERGY CO.
1 BY 0 SOLUTIONS
从LibreOffice生成data.csv屏幕截图:
此脚本将从您的输入文件a.txt生成公司/电话的CSV:
0.5BN FINHEALTH PRIVATE LIMITED
01 SYNERGY CO.
1 BY 0 SOLUTIONS
从LibreOffice生成data.csv屏幕截图:
给出您的输出和所需的输出。它只写最后一行。。表示它写得太多。公司地址电话0.5亿FINHEALTH PRIVATE LIMITED乳制品公司GNS Towers 18 BHIVE Forum二楼。。。None给出您的输出和所需的输出。它只写最后一行。。表示它写得太多。公司地址电话0.5亿FINHEALTH PRIVATE LIMITED乳制品公司GNS Towers 18 BHIVE Forum二楼。。。没有一个