使用python之前：：之后的Web抓取内容返回[]

我对python非常陌生，甚至是作为一个整体进行编码。我已经成功地用这段代码抓取了大约10个网站，但是所有的悲伤都表明它不适用于这个网站。我想提取每个产品类的所有div。但是这些女主角不会出现在这本书里。我读过关于：：before和：：after的文章，但我找不到有效的解决方案。我的代码中可能有多个问题，但我找不到它们。我有两个星期的不眠之夜。请帮忙

from urllib.request import  urlopen as uReq
from bs4 import BeautifulSoup as soup

url = "https://www.woolworths.co.za/cat?No=60&Nrpp=60&Ns=p_pl30|0"

uClient=uReq(url)
page_html=uClient.read()
uClient.close()

page_soup=soup(page_html,"html.parser")
containers=page_soup.findAll("div",{"class":"product-list__item"})
print(containers)
        
quit()

我得到的结果 []

---

如果我必须提供更多信息，请告知我。

通过发送

GET

请求并将正确的

标题添加到以下位置，可从外部加载数据：

https://www.woolworths.co.za/server/searchCategory?pageURL=%2Fcat&No=60&Nrpp=60&Ns=p_pl30%7C0

您只需使用
请求
模块即可获取数据，无需使用
美化组
：

import requests


headers = {
    "Referer": "https://www.woolworths.co.za/cat?No=60&Nrpp=60&Ns=p_pl30|0",
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36",
}

URL = "https://www.woolworths.co.za/server/searchCategory?pageURL=%2Fcat&No=60&Nrpp=60&Ns=p_pl30%7C0"
response = requests.get(URL, headers=headers).json()
fmt_string = "{:<70} {:<15} {}"

print(fmt_string.format("Brand", "Price", "Image"))
print("-" * 200)

for d in response["contents"][0]["mainContent"][0]["contents"]:
    for dd in d["records"]:
        print(
            fmt_string.format(
                dd["attributes"]["p_displayName"],
                dd["startingPrice"]["p_pl30"],
                "https://images.woolworthsstatic.co.za/"
                + dd["attributes"]["p_externalImageReference"],
            )
        )

你的问题与
：：在
/
：：在
之前-你没有得到结果，因为页面正在动态加载内容，而你的div在获取内容时不在页面上。我不知道该怎么办，我尝试使用time（）等待页面加载，知道如何获取divz吗。这太好了，谢谢。我也设法得到了这个品牌，但是我如何得到价格和图片呢？谢谢，这很有效。你是我的朋友best@Vally很乐意帮忙。如果这个或任何答案解决了你的问题，考虑把它标记为
Brand                                                                  Price           Image
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Soda Water Sparkling Drink 200 ml                                      9.49            https://images.woolworthsstatic.co.za/Soda-Water-Sparkling-Drink-200-ml-6001009034250.jpg?V=50$J&o=eucyUmAbqcqMSs0IjPaS4WA$mzoj&
Salt & Vinegar Flavoured Potato Crisps 36 g                            9.49            https://images.woolworthsstatic.co.za/Salt-Vinegar-Flavoured-Potato-Crisps-36-g-6009175413541.jpg?V=6Pfl&o=Tyz@wbWHKvnW@Kc69RTJYM7WBUQj&
Salted Farmer's Crisps 36 g                                            9.49            https://images.woolworthsstatic.co.za/Salted-Farmer-s-Crisps-36-g-6009217630752.jpg?V=TM2n&o=eR0n3eqV0@15TKcRSRz1RzzVPW8j&
Lemonade Sugar Free Sparkling Flavoured Drink 200 ml                   9.49            https://images.woolworthsstatic.co.za/Lemonade-Sugar-Free-Sparkling-Flavoured-Drink-200-ml-6001009014238.jpg?V=9Skb&o=hqcUN6THi9J8YRixCQEica2ftcMj&
Ginger Ale Sugar Free Sparkling Flavoured Drink 200 ml                 9.49            https://images.woolworthsstatic.co.za/Ginger-Ale-Sugar-Free-Sparkling-Flavoured-Drink-200-ml-6001009014245.jpg?V=x9Pr&o=C7oKpoX27D3z2vf11X7bKRmUJEsj&
Cheddar Flavoured Crisps 36 g                                          9.49            https://images.woolworthsstatic.co.za/Cheddar-Flavoured-Crisps-36-g-6009217630776.jpg?V=zCPn&o=Ufn3jhhUzUGckf72QHLRaa64g20j&
...