Python 3.x 刮取角度列表数据
我想打印公司名单,但它什么也不打印Python 3.x 刮取角度列表数据,python-3.x,web-scraping,Python 3.x,Web Scraping,我想打印公司名单,但它什么也不打印 import requests from bs4 import BeautifulSoup url= requests.get('https://angel.co/companies').text soup= BeautifulSoup(url, 'lxml') for div in soup.find_all("div", class_="name"): print(div.text) 它工作正常因为该网站的内容是动态加载的,请求将无法捕获,除
import requests
from bs4 import BeautifulSoup
url= requests.get('https://angel.co/companies').text
soup= BeautifulSoup(url, 'lxml')
for div in soup.find_all("div", class_="name"):
print(div.text)
它工作正常因为该网站的内容是动态加载的,请求将无法捕获,除非您使用api执行
post
请求。您必须选择任何浏览器模拟器(如selenium)来获取该网页的内容。
import requests
from selenium import webdriver
from bs4 import BeautifulSoup
import time
driver=webdriver.Chrome()
driver.get('https://angel.co/companies?locations[]=1637-Dubai,+AE')
time.sleep(10)
soup= BeautifulSoup(driver.page_source, 'lxml')
name_data = []
for item in soup.select('.name'):
name=item.text.strip()
name_data.append(name)
print(name)
driver.quit()