Python 3.x 漂亮的汤没有通过ID找到元素

我正试图从forexfactory.com上抓取即将到来的头条新闻

from bs4 import *
import requests

source = requests.get("https://www.forexfactory.com/news").text

soup = BeautifulSoup(source, 'lxml')

list = soup.find(id="ui-outer")

print(list)

当它应该返回包含所有标题的Div时，它将返回None。你知道会出什么问题吗？我尝试过通过div、Ul-id、li-id和其他一些方式进行搜索。它总是不返回任何值。

---

谢谢。

对于动态创建的元素，请尝试一下

from selenium import webdriver
driver = webdriver.Chrome(executable_path=r'C:\Program Files\ChromeDriver\chromedriver.exe')
URL = 'https://www.forexfactory.com/news'

driver.get(URL)

等待几秒钟，直到动态加载内容

driver.implicitly_wait(5) # wait for seconds

获取您的元素

uiOuter = driver.find_element_by_id('ui-outer')

    ['EU\'s Barnier says "fundamental divergences" persist in UK trade talks',
 'With end of crisis programs, Fed faces tricky post-pandemic transition',
 'Markets Look Past Near-Term Challenges',
 'Time is short, Divergences remain, but we continue to work hard for a deal',
 'EURUSD consolidates around 1.1900; indicators are flat',
 'Rush for emerging market company bonds as investors look beyond COVID-19',
 'Europe’s Virus Lockdowns Push Economy Into Another Contraction',
 'Interactive Brokers enhances Client Portal',
 'BoE’s Haldane: Risk That Anxiety Leads To Gloom Loop',
 'Sharpest fall in UK private sector output since May. Manufacturing growth offset by renewed...',
 'Remote Working Shift Offers Silver Lining for Finance Gender Gap',
 'EU Flash PMI signals steep downturn in November amid COVID-19 lockdowns',
 'German PMI drops to five-month low in November due to tightening of COVID-19 restrictions, but...',
 'Sharp decline in French business activity amid fresh COVID-19 lockdown',
 'Rishi Sunak says Spending Review will not spell austerity',
 'Remote Working Shift Offers Silver Lining for Finance Gender Gap',
 'Japan’s Labor Thanksgiving Day goes viral',
 'Ranking Asset Classes by Historical Returns (1985-2020)',
 'Time is short, Divergences remain, but we continue to work hard for a deal',
 'EURUSD consolidates around 1.1900; indicators are flat',
 'US Dollar stuck near support, NZ$ strikes two-year high',
 'US Dollar stuck near support, NZ$ strikes two-year high',
 'Georgia confirms results in latest setback for Trump bid to overturn Biden win',
 'Canada to roll over terms of EU trade deal with UK',
 'Time is short, Divergences remain, but we continue to work hard for a deal',
 'German PMI drops to five-month low in November due to tightening of COVID-19 restrictions, but...',
 "COVID-19: 'No return to austerity', says chancellor as he hints at public sector pay freeze",
 'EURUSD consolidates around 1.1900; indicators are flat',
 'New Zealand Dollar May Rise as RBNZ Holds Fire on Negative Rates',
 'Interactive Brokers enhances Client Portal']

所有链接的示例（故事平铺）

输出

uiOuter = driver.find_element_by_id('ui-outer')

    ['EU\'s Barnier says "fundamental divergences" persist in UK trade talks',
 'With end of crisis programs, Fed faces tricky post-pandemic transition',
 'Markets Look Past Near-Term Challenges',
 'Time is short, Divergences remain, but we continue to work hard for a deal',
 'EURUSD consolidates around 1.1900; indicators are flat',
 'Rush for emerging market company bonds as investors look beyond COVID-19',
 'Europe’s Virus Lockdowns Push Economy Into Another Contraction',
 'Interactive Brokers enhances Client Portal',
 'BoE’s Haldane: Risk That Anxiety Leads To Gloom Loop',
 'Sharpest fall in UK private sector output since May. Manufacturing growth offset by renewed...',
 'Remote Working Shift Offers Silver Lining for Finance Gender Gap',
 'EU Flash PMI signals steep downturn in November amid COVID-19 lockdowns',
 'German PMI drops to five-month low in November due to tightening of COVID-19 restrictions, but...',
 'Sharp decline in French business activity amid fresh COVID-19 lockdown',
 'Rishi Sunak says Spending Review will not spell austerity',
 'Remote Working Shift Offers Silver Lining for Finance Gender Gap',
 'Japan’s Labor Thanksgiving Day goes viral',
 'Ranking Asset Classes by Historical Returns (1985-2020)',
 'Time is short, Divergences remain, but we continue to work hard for a deal',
 'EURUSD consolidates around 1.1900; indicators are flat',
 'US Dollar stuck near support, NZ$ strikes two-year high',
 'US Dollar stuck near support, NZ$ strikes two-year high',
 'Georgia confirms results in latest setback for Trump bid to overturn Biden win',
 'Canada to roll over terms of EU trade deal with UK',
 'Time is short, Divergences remain, but we continue to work hard for a deal',
 'German PMI drops to five-month low in November due to tightening of COVID-19 restrictions, but...',
 "COVID-19: 'No return to austerity', says chancellor as he hints at public sector pay freeze",
 'EURUSD consolidates around 1.1900; indicators are flat',
 'New Zealand Dollar May Rise as RBNZ Holds Fire on Negative Rates',
 'Interactive Brokers enhances Client Portal']

---

我查了那个网站。虽然网站确实拥有一个

div

和

id=ui-outer

-当您从代码中打印

soup

的完整变量时，它没有该div。。。。。这可能是一个安全问题，因为该网站不允许抓取其数据。该网站使用Cloudflare保护脚本和机器人程序。为了获得数据，你需要绕过它。是的，谢谢你，这工作做得很好。你知道我如何在每一篇文章中加入链接吗？再次感谢。@RehanMahmood-好的，如果您指的是

href

中的URL，则相同的过程仅适用于

x.get\u属性（“href”）