Warning: file_get_contents(/data/phpspider/zhask/data//catemap/8/python-3.x/18.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python Scraper仅将数据从最后一个URL输出到CSV_Python_Python 3.x_Beautifulsoup - Fatal编程技术网

Python Scraper仅将数据从最后一个URL输出到CSV

Python Scraper仅将数据从最后一个URL输出到CSV,python,python-3.x,beautifulsoup,Python,Python 3.x,Beautifulsoup,我对Python非常陌生,并试图通过做一些小项目来学习。我目前正在尝试从各种网页收集一些信息,但是,每当它将抓取的数据输出到CSV时,它似乎只输出来自上一个URL的数据 理想情况下,我希望它能够写入CSV,而不是追加,因为我只希望CSV只包含最新刮取的最新数据 我在StackOverflow上查看了一些其他类似的查询,但我要么不理解它们,要么它们不适合我。可能是前者 任何帮助都将不胜感激 import csv import requests from bs4 import BeautifulSo

我对Python非常陌生,并试图通过做一些小项目来学习。我目前正在尝试从各种网页收集一些信息,但是,每当它将抓取的数据输出到CSV时,它似乎只输出来自上一个URL的数据

理想情况下,我希望它能够写入CSV,而不是追加,因为我只希望CSV只包含最新刮取的最新数据

我在StackOverflow上查看了一些其他类似的查询,但我要么不理解它们,要么它们不适合我。可能是前者

任何帮助都将不胜感激

import csv
import requests
from bs4 import BeautifulSoup
import pandas as pd

URL = ['URL1','URL2']

for URL in URL:
    response = requests.get(URL)
    soup = BeautifulSoup(response.content, 'html.parser')

    nameElement = soup.find('p', attrs={'class':'name'}).a
    nameText = nameElement.text.strip()

    priceElement = soup.find('span', attrs={'class':'price'})
    priceText = priceElement.text.strip()



columns = [['Name','Price'], [nameText, priceText]]


with open('index.csv', 'w', newline='') as csv_file:
    writer = csv.writer(csv_file)
    writer.writerows(columns)

您必须在for循环之前打开文件,并在for循环中写入每一行

或者,您必须在for循环之前创建列表,并将数据附加到此列表中

URL = ['URL1','URL2']

columns = [ ['Name','Price'] ]

for URL in URL:
    response = requests.get(URL)
    soup = BeautifulSoup(response.content, 'html.parser')

    nameElement = soup.find('p', attrs={'class':'name'}).a
    nameText = nameElement.text.strip()

    priceElement = soup.find('span', attrs={'class':'price'})
    priceText = priceElement.text.strip()

    columns.append( [nameText, priceText] )

with open('index.csv', 'w', newline='') as csv_file:
    writer = csv.writer(csv_file)
    writer.writerows(columns)

在for循环之前打开文件,并在for循环内部写入行。或者在for循环为所有结果创建列表之前,在for循环中附加到此列表,并使用此结果写入文件。非常感谢你的帮助!
URL = ['URL1','URL2']

columns = [ ['Name','Price'] ]

for URL in URL:
    response = requests.get(URL)
    soup = BeautifulSoup(response.content, 'html.parser')

    nameElement = soup.find('p', attrs={'class':'name'}).a
    nameText = nameElement.text.strip()

    priceElement = soup.find('span', attrs={'class':'price'})
    priceText = priceElement.text.strip()

    columns.append( [nameText, priceText] )

with open('index.csv', 'w', newline='') as csv_file:
    writer = csv.writer(csv_file)
    writer.writerows(columns)