Python 3.x 将webscraping的列表输出列表转换为数据帧

Python 3.x 将webscraping的列表输出列表转换为数据帧,python-3.x,web-scraping,Python 3.x,Web Scraping,我已经创建了以下代码来从一个网站上抓取地址,看起来效果不错。然而,输出是一个列表列表,我无法将其转换为数据帧 我尝试使用pd.DataFrame(地址),但这不会产生预期的输出。我还尝试了pd.DataFrame(list(zip(addresses)),但也没有得到预期的输出 from selenium import webdriver from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver

我已经创建了以下代码来从一个网站上抓取地址,看起来效果不错。然而,输出是一个列表列表,我无法将其转换为数据帧

我尝试使用pd.DataFrame(地址),但这不会产生预期的输出。我还尝试了pd.DataFrame(list(zip(addresses)),但也没有得到预期的输出

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException 
from bs4 import BeautifulSoup
import time
import pandas as pd
import re
base_url = 'https://www.thechristhospital.com/locations-search-results?Type=AdvancedSearch'
browser = webdriver.Chrome()
browser.get(base_url)
soup = BeautifulSoup(browser.page_source,'html.parser')

addresses = []
time.sleep(5)
button = browser.find_element_by_css_selector('#ctl00_ctl35_g_5f6e70e2_119c_48b6_a627_dbce7ca77728_cntrlPaging_btnPageFwd')
time.sleep(2)
count = 0
while True:
    try:

        WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#ctl00_ctl35_g_5f6e70e2_119c_48b6_a627_dbce7ca77728_cntrlPaging_btnPageFwd"))).click()
        count += 1
        time.sleep(2)
        soup = BeautifulSoup( browser.page_source,'html.parser')
        add= [add.text.strip() for add in soup.find_all('div',{'class':'address'})]
        addresses.append(add)
        time.sleep(2)
    except TimeoutException:
            break
for add in add:

browser.quit()
我期望的输出是一个数据帧,它列出了每个位置的地址。最好在不同的字段中按名称/地址进行拆分,但如果在一个字段中进行拆分,也可以


非常感谢您的帮助。

下面的方法有些不同-有点笨拙,可能很脆弱,但它确实起到了作用,您应该能够轻松地对其进行修补,以使其达到您想要的效果

我只在第一页上尝试了,所以为了捕获其他页面,您也必须修改它

data = pd.read_html(base_url)
info = data[0].iloc[:,0] #this is where the relevant info is located

#remove irrelevant parts and split into lists
places = []
for place in info:
    place_list = place.replace('Get Directions ','').replace('Hours','').replace('Providers  ','').replace('Services','NA').split('  ')[:-1]
    if len(place_list)== 6: #some entries don't have a second address line, some do
        place_list.insert(3,'NA')
    places.append(place_list)


#create the dataframe
columns = ['Hospital','Division','Street Address','Address 2','Address 3','Phone','Providers']

new_df = pd.DataFrame(places, columns=columns) 
new_df.head(3)
输出:

                   Hospital                      Division              Street Address   Address 2   Address 3             Phone     Providers
0   The Christ Hospital Interventional Radiology    The Christ Hospital     2139 Auburn Ave.    Level C - Interventional Radiology  Cincinnati, OH 45219    (513) 585-3072  Charity N. DeArmond, CNPVickie M. Dietrich, CNP
1   The Christ Hospital Inpatient Orthopedics   The Christ Hospital     2139 Auburn Ave.    NA  Cincinnati, OH 45219    (513) 585-2493  Stephanie L. Ellis, CNP
2   The Christ Hospital Inpatient Transplant    The Christ Hospital     2139 Auburn Ave.    NA  Cincinnati, OH 45219    (513) 585-2493  Rebecca K. Parks, CNP

作品完美,我会修改,以获得额外的页面