Python 3.x 从XPATH返回空值的Web抓取

Python 3.x 从XPATH返回空值的Web抓取,python-3.x,beautifulsoup,Python 3.x,Beautifulsoup,我正在努力为stubhub上的一场比赛削价。我没有成功连接到API,所以我想我应该改用这个方法,但是遇到了一些问题 https://www.stubhub.com/chicago-cubs-chicago-tickets-5-3-2021/event/104736306/ 为什么这个XPATH会产生一个空值 price = WebDriverWait(driver, 0.01).until(EC.presence_of_element_located((By.XPATH, '//*[@id=&

我正在努力为stubhub上的一场比赛削价。我没有成功连接到API,所以我想我应该改用这个方法,但是遇到了一些问题

https://www.stubhub.com/chicago-cubs-chicago-tickets-5-3-2021/event/104736306/
为什么这个XPATH会产生一个空值

price = WebDriverWait(driver, 0.01).until(EC.presence_of_element_located((By.XPATH, '//*[@id="main"]/div/div/div[1]/div[2]/section[1]/div[1]/ul/div/li[34]/div[2]/div[1]/div[1]/div'))).text
较大的代码段:

driver.get('https://www.stubhub.com/find/s/?q=chicago%20cubs')
matchups = [i.text for i in WebDriverWait(driver, 1).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, '.EventItem__TitleLink')))]
games = [i.get_attribute('href') for i in WebDriverWait(driver, 1).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, '.EventItem__TitleLink')))]

game_details = pd.DataFrame()
game_details['Matchup'] = matchups
game_details['Link'] = games
print(game_details)

#gather ticket data
urls = []
location = []
prices = []
details = []

for g in games:
    try:
        driver.get(g + '?priceWithFees=true')
        price = [i.text for i in WebDriverWait(driver, 100).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, '.AdvisoryPriceDisplay__content')))]
        price = price[:-2]
        loc = [i.text for i in WebDriverWait(driver, 100).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, '.RoyalTicketListPanel__SectionName')))]
        detail = [i.text for i in WebDriverWait(driver, 100).until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, '.RoyalTicketListPanel__SecondaryInfo')))]
        url = [str(g)] * len(price)
        urls.extend(url)
        prices.extend(price)
        location.extend(loc)
        details.extend(detail)
        print(str(g) + ": " + len(price) + " ")
    except:
        print('Failed: ' + str(g))
        pass

ticket_prices = pd.DataFrame()
ticket_prices['Price'] = prices
ticket_prices['Location'] = location
ticket_prices['Detail'] = details
ticket_prices['Link'] = urls

print(ticket_prices.shape)
print(ticket_prices)

writer = pd.ExcelWriter(final, engine='xlsxwriter')
game_details.to_excel(writer, sheet_name='Games')
ticket_prices.to_excel(writer, sheet_name='Tickets')
writer.save()


你打算通过滚动页面获得所有价格吗?我很想获得所有价格,但是绕过延迟加载对我来说是第二步。。。。到目前为止,我只希望第一个显示
prices=[I.text for I in WebDriverWait(driver,0.01)。直到(EC.presence_of_all_element_located((By.CSS_SELECTOR,'.AdvisoryPriceDisplay_content'))]
这仍然是失败的一半时间。。。知道是什么导致它有时起作用还是不起作用吗?不知道我该如何排除故障,也许是监视值的变化而不是状态?首先通过添加waits before来验证这是否是一个计时问题。如果是,探索最可靠的等待条件。