Scape span，在DIV中使用Python

尝试使用Python从以下网站收集Amazon Prime可用电影的列表：

我试图找到电影的标题，但返回0。这必须是从网页加载JavaScript的第一，我已经在网络上寻找，但不能计算出我在寻找什么

我尝试了以下代码：

from requests import get
url = 'https://www.amazon.co.uk/s?i=instant-video&bbn=3010085031&rh=n%3A3010085031%2Cp_85%3A3282143031%2Cp_72%3A3289786031&dc&adult-product=0&field-genre=-family&field-review_count=3-&field-ways_to_watch=7448662031&p_n_entity_type=9739952031&qid=1557596014&qs-av_request_type=4&qs-is-prime-customer=0&rnid=3289782031&ref=sr_nr_p_72_4'
response = get(url)
from bs4 import BeautifulSoup
html_soup = BeautifulSoup(response.text, 'lxml')
type(html_soup)
movie_containers = html_soup.find_all('span', class_ = 'a-size-medium     a-color-base a-text-normal')
print(type(movie_containers))
print(len(movie_containers))

然后我试着循环：

for n in soup.find_all('span', {'class':'a-size-medium     a-color-base a-text-normal'}):
    title.append (n.text)

任何帮助都会很好。多谢各位

---

结果应提取每部电影的标题和链接。

您必须包含get请求的标题

import requests
from bs4 import BeautifulSoup

header = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36"}
a = requests.get("https://www.amazon.co.uk/s?i=instant-video&bbn=3010085031&rh=n%3A3010085031%2Cp_85%3A3282143031%2Cp_72%3A3289786031&dc&adult-product=0&field-genre=-family&field-review_count=3-&field-ways_to_watch=7448662031&p_n_entity_type=9739952031&qid=1557596014&qs-av_request_type=4&qs-is-prime-customer=0&rnid=3289782031&ref=sr_nr_p_72_4", headers=header)
b = BeautifulSoup(a.text, "html.parser")
for c in b.find_all("span", class_="a-size-medium"):
    print(c.text)

---

您必须包含get请求的标头

import requests
from bs4 import BeautifulSoup

header = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36"}
a = requests.get("https://www.amazon.co.uk/s?i=instant-video&bbn=3010085031&rh=n%3A3010085031%2Cp_85%3A3282143031%2Cp_72%3A3289786031&dc&adult-product=0&field-genre=-family&field-review_count=3-&field-ways_to_watch=7448662031&p_n_entity_type=9739952031&qid=1557596014&qs-av_request_type=4&qs-is-prime-customer=0&rnid=3289782031&ref=sr_nr_p_72_4", headers=header)
b = BeautifulSoup(a.text, "html.parser")
for c in b.find_all("span", class_="a-size-medium"):
    print(c.text)

---

谢谢你的回复。仍然无法使用上面的代码提取标题？谢谢！是否有办法进入第2页、第3页等？我建议分析每个URL的区别；第2页或第3页。当我更改页面引用时，输出是相同的。这很奇怪。你可以找到每个url参数的差异！谢谢你的回复。仍然无法使用上面的代码提取标题？谢谢！是否有办法进入第2页、第3页等？我建议分析每个URL的区别；第2页或第3页。当我更改页面引用时，输出是相同的。这很奇怪。你可以找到每个url参数的差异！