用Python 3实现沃尔玛的降价

我对这个概念非常陌生，但我正在尝试学习如何使用python来操作HTML数据

我编写了一个python（3.4.1版）脚本，它获取URL并返回一些信息，我使用BeautifulSoup（4版）解析这些信息

在本例中，我试图获取Xbox One的价格。我选择这个div是因为它在网页上向用户显示价格。我知道有一个

$399.99

可以刮。从这一点来看，这是相当直截了当的，我更好奇的是，为什么价格不会出现在它应该出现的div中

---

我假设这与HTML标题有关，或者可能与使用标准web浏览器浏览时自动发送的某种POST/GET数据有关。有人能解释一下为什么价格没有显示出来，以及我需要做些什么才能让它们按预期显示出来吗？

价格是使用javascript加载的。请求如下：

Request URL:http://www.walmart.ca/ws/online/products
Request Method:POST

Form Data:
products:[{"productid":"6000187109066","skus":[[{"skuid":"6000187109066","status":"10"}]]}]
csrfToken:b08bfe580f3d9a0d893435fb

因为它包含一个csrfToken，所以在发出post请求之前，您需要弄清楚它是如何生成或提供的。他们可能也依赖于会话cookie

---

沃尔玛似乎有一个API：

价格是用javascript加载的。请求如下：

Request URL:http://www.walmart.ca/ws/online/products
Request Method:POST

Form Data:
products:[{"productid":"6000187109066","skus":[[{"skuid":"6000187109066","status":"10"}]]}]
csrfToken:b08bfe580f3d9a0d893435fb

因为它包含一个csrfToken，所以在发出post请求之前，您需要弄清楚它是如何生成或提供的。他们可能也依赖于会话cookie

---

沃尔玛似乎有一个API：

价格是用javascript加载的。请求如下：

Request URL:http://www.walmart.ca/ws/online/products
Request Method:POST

Form Data:
products:[{"productid":"6000187109066","skus":[[{"skuid":"6000187109066","status":"10"}]]}]
csrfToken:b08bfe580f3d9a0d893435fb

因为它包含一个csrfToken，所以在发出post请求之前，您需要弄清楚它是如何生成或提供的。他们可能也依赖于会话cookie

---

沃尔玛似乎有一个API：

价格是用javascript加载的。请求如下：

Request URL:http://www.walmart.ca/ws/online/products
Request Method:POST

Form Data:
products:[{"productid":"6000187109066","skus":[[{"skuid":"6000187109066","status":"10"}]]}]
csrfToken:b08bfe580f3d9a0d893435fb

因为它包含一个csrfToken，所以在发出post请求之前，您需要弄清楚它是如何生成或提供的。他们可能也依赖于会话cookie

沃尔玛似乎有一个API：免责声明：我在SerpApi工作

你可以通过SerpApi的包装来获取沃尔玛的产品价格。这是一个免费试用的付费API

导入操作系统
从serpapi导入谷歌搜索
params=dict(
engine=“沃尔玛”，
query=“xbox one”，
api_key=os.getenv（“api_key”），
)
搜索=谷歌搜索（参数）
结果=search.get_dict（）
对于有机结果[“有机结果”]：
产品标识=有机结果[“美国商品标识”]或有机结果[“产品标识”]
search.params_dict=dict(
engine=“沃尔玛产品”，
产品标识=产品标识，
api_key=os.getenv（“api_key”），
)
product\u result=search.get\u dict（）
印刷品(
f“价格{product_result['product_result']['title']}:{product_result['product_result']['Price_map']['Price']}{product_result['product_result']['Price_map']['currency']}”
)

输出

Price of 'Microsoft Xbox One S 1TB Console (Certified Refurbished)': '429 $'
Price of 'Star Wars Jedi: Fallen Order, Electronic Arts, Xbox One': '39.82 $'
Price of 'Microsoft - Xbox One S 1TB All-Digital Edition Console with Xbox One Wireless Controller (Renewed)': '499.99 $'
Price of 'Need for Speed: Heat, Electronic Arts, Xbox One, 014633373233': '25 $'
Price of 'Call of Duty: Black Ops Cold War, Activision, Xbox One': '49.94 $'
Price of 'NBA 2K21, 2K, Xbox One, 710425596858': '29.96 $'
Price of 'Refurbished Microsoft Factory Xbox One X 1TB, 4K Ultra HD Gaming Console in Black, FMQ-00042, 889842246971': '499.99 $'
Price of 'Madden NFL 21, Electronic Arts, Xbox One': '29.66 $'
Price of 'Need for Speed Payback, Electronic Arts, Xbox One, 014633370058': '14.96 $'
Price of 'Nickelodeon Kart Racers, Gamemill, Xbox One, 856131008060': '14.96 $'
Price of 'Sonic Mania (Sega), Xbox One, 010086640939': '14.96 $'
Price of 'Minecraft Master Collection, Microsoft, Xbox One, 889842394979': '42.88 $'
Price of 'Mortal Kombat 11, Warner Bros., Xbox One, 883929668977': '15 $'
Price of 'UFC 4, Electronic Arts, Xbox One': '29.96 $'
Price of 'Grand Theft Auto V, Rockstar Games, Xbox One': '22.8 $'
Price of 'Man Eater, Deep Silver, Xbox One, 816819017517': '24.96 $'
Price of 'Crash N. Sane Trilogy, Activision, Xbox One, 047875881969': '39.99 $'
Price of 'Spyro Reignited Trilogy, Activision, Xbox One, 047875882423': '29.83 $'
Price of 'Call of Duty: Modern Warfare, Activision, Xbox One, 0047875884366': '47.9 $'
Price of 'Cyberpunk 2077, Warner Bros, Xbox One': '39.74 $'
Price of 'Mass Effect Andromeda, Electronic Arts, Xbox One, 014633734096': '6.91 $'
Price of 'THQ theHunter: Call Of the Wild (Xbox One)': '19.93 $'
Price of 'Wreckfest, THQ-Nordic, Xbox One, 0811994021649': '29.83 $'
Price of 'Assassin’s Creed Valhalla Xbox Series X|S, Xbox One Standard Edition': '36.51 $'
Price of 'NBA 2K20, 2K, Xbox One, 710425595264': '21.99 $'
Price of 'LEGO Worlds, Warner Bros, Xbox One': '15 $'
Price of 'Call of Duty: Black Ops 3 Zombie Chronicles Edition, Activision, Xbox One, 047875881228': '37.88 $'
Price of 'Jumanji The Video Game, Bandai Namco, Xbox One, 819338020792': '19.93 $'
Price of 'Red Dead Redemption 2, Rockstar Games, Xbox One': '34.99 $'
Price of 'HALO 5, Microsoft, Xbox One, 885370928518': '19 $'
Price of 'LEGO Jurassic World, Warner, Xbox One, 883929472727': '19.8 $'
Price of 'Plants vs. Zombies: Battle for Neighborville, Electronic Arts, Xbox One, 014633736007': '14.96 $'
Price of 'The LEGO Marvel Collection, Warner Bros., Xbox One, 00883929670499': '24.98 $'
Price of 'LEGO The Incredibles, Warner Bros, Xbox One, 883929633005': '18 $'
Price of 'Assassin's Creed: Origins, Ubisoft, Xbox One, 887256028459': '19.93 $'
Price of 'Spongebob Squarepants: Battle for Bikini Bottom Rehydrated, THQ-Nordic, Xbox One, 811994022165': '29.77 $'
Price of 'The SIMS 4, Electronic Arts, Xbox One': '33.84 $'
Price of 'Star Wars Battlefront 2, Electronic Arts, Xbox One, 014633735321': '14.96 $'
Price of 'Pac-Man Championship Edition 2 + Arcade Game Series, Bandai/Namco, Xbox One, 722674220705': '14.49 $'
Price of 'Microsoft Xbox One S 1TB All Digital Edition 3 Game Bundle (Disc-free Gaming), White, NJP-00050': '198.95 $'

免责声明：我在SerpApi工作

你可以通过SerpApi的包装来获取沃尔玛的产品价格。这是一个免费试用的付费API

导入操作系统
从serpapi导入谷歌搜索
params=dict(
engine=“沃尔玛”，
query=“xbox one”，
api_key=os.getenv（“api_key”），
)
搜索=谷歌搜索（参数）
结果=search.get_dict（）
对于有机结果[“有机结果”]：
产品标识=有机结果[“美国商品标识”]或有机结果[“产品标识”]
search.params_dict=dict(
engine=“沃尔玛产品”，
产品标识=产品标识，
api_key=os.getenv（“api_key”），
)
product\u result=search.get\u dict（）
印刷品(
f“价格{product_result['product_result']['title']}:{product_result['product_result']['Price_map']['Price']}{product_result['product_result']['Price_map']['currency']}”
)

输出

Price of 'Microsoft Xbox One S 1TB Console (Certified Refurbished)': '429 $'
Price of 'Star Wars Jedi: Fallen Order, Electronic Arts, Xbox One': '39.82 $'
Price of 'Microsoft - Xbox One S 1TB All-Digital Edition Console with Xbox One Wireless Controller (Renewed)': '499.99 $'
Price of 'Need for Speed: Heat, Electronic Arts, Xbox One, 014633373233': '25 $'
Price of 'Call of Duty: Black Ops Cold War, Activision, Xbox One': '49.94 $'
Price of 'NBA 2K21, 2K, Xbox One, 710425596858': '29.96 $'
Price of 'Refurbished Microsoft Factory Xbox One X 1TB, 4K Ultra HD Gaming Console in Black, FMQ-00042, 889842246971': '499.99 $'
Price of 'Madden NFL 21, Electronic Arts, Xbox One': '29.66 $'
Price of 'Need for Speed Payback, Electronic Arts, Xbox One, 014633370058': '14.96 $'
Price of 'Nickelodeon Kart Racers, Gamemill, Xbox One, 856131008060': '14.96 $'
Price of 'Sonic Mania (Sega), Xbox One, 010086640939': '14.96 $'
Price of 'Minecraft Master Collection, Microsoft, Xbox One, 889842394979': '42.88 $'
Price of 'Mortal Kombat 11, Warner Bros., Xbox One, 883929668977': '15 $'
Price of 'UFC 4, Electronic Arts, Xbox One': '29.96 $'
Price of 'Grand Theft Auto V, Rockstar Games, Xbox One': '22.8 $'
Price of 'Man Eater, Deep Silver, Xbox One, 816819017517': '24.96 $'
Price of 'Crash N. Sane Trilogy, Activision, Xbox One, 047875881969': '39.99 $'
Price of 'Spyro Reignited Trilogy, Activision, Xbox One, 047875882423': '29.83 $'
Price of 'Call of Duty: Modern Warfare, Activision, Xbox One, 0047875884366': '47.9 $'
Price of 'Cyberpunk 2077, Warner Bros, Xbox One': '39.74 $'
Price of 'Mass Effect Andromeda, Electronic Arts, Xbox One, 014633734096': '6.91 $'
Price of 'THQ theHunter: Call Of the Wild (Xbox One)': '19.93 $'
Price of 'Wreckfest, THQ-Nordic, Xbox One, 0811994021649': '29.83 $'
Price of 'Assassin’s Creed Valhalla Xbox Series X|S, Xbox One Standard Edition': '36.51 $'
Price of 'NBA 2K20, 2K, Xbox One, 710425595264': '21.99 $'
Price of 'LEGO Worlds, Warner Bros, Xbox One': '15 $'
Price of 'Call of Duty: Black Ops 3 Zombie Chronicles Edition, Activision, Xbox One, 047875881228': '37.88 $'
Price of 'Jumanji The Video Game, Bandai Namco, Xbox One, 819338020792': '19.93 $'
Price of 'Red Dead Redemption 2, Rockstar Games, Xbox One': '34.99 $'
Price of 'HALO 5, Microsoft, Xbox One, 885370928518': '19 $'
Price of 'LEGO Jurassic World, Warner, Xbox One, 883929472727': '19.8 $'
Price of 'Plants vs. Zombies: Battle for Neighborville, Electronic Arts, Xbox One, 014633736007': '14.96 $'
Price of 'The LEGO Marvel Collection, Warner Bros., Xbox One, 00883929670499': '24.98 $'
Price of 'LEGO The Incredibles, Warner Bros, Xbox One, 883929633005': '18 $'
Price of 'Assassin's Creed: Origins, Ubisoft, Xbox One, 887256028459': '19.93 $'
Price of 'Spongebob Squarepants: Battle for Bikini Bottom Rehydrated, THQ-Nordic, Xbox One, 811994022165': '29.77 $'
Price of 'The SIMS 4, Electronic Arts, Xbox One': '33.84 $'
Price of 'Star Wars Battlefront 2, Electronic Arts, Xbox One, 014633735321': '14.96 $'
Price of 'Pac-Man Championship Edition 2 + Arcade Game Series, Bandai/Namco, Xbox One, 722674220705': '14.49 $'
Price of 'Microsoft Xbox One S 1TB All Digital Edition 3 Game Bundle (Disc-free Gaming), White, NJP-00050': '198.95 $'

免责声明：我在SerpApi工作

你可以通过SerpApi的包装来获取沃尔玛的产品价格。这是一个免费试用的付费API

导入操作系统
从serpapi导入谷歌搜索
params=dict(
engine=“沃尔玛”，
query=“xbox one”，
api_key=os.getenv（“api_key”），
)
搜索=谷歌搜索（参数）
结果=search.get_dict（）
对于有机结果[“有机结果”]：
产品标识=有机结果[“美国商品标识”]或有机结果[“产品标识”]
search.params_dict=dict(
engine=“沃尔玛产品”，
产品标识=产品标识，
api_key=os.getenv（“api_key”），
)
product\u result=search.get\u dict（）
印刷品(
f“价格{product_result['product_result']['title']}:{product_result['product_result']['Price_map']['Price']}{product_result['product_result']['Price_map']['currency']}”
)

输出

Price of 'Microsoft Xbox One S 1TB Console (Certified Refurbished)': '429 $'
Price of 'Star Wars Jedi: Fallen Order, Electronic Arts, Xbox One': '39.82 $'
Price of 'Microsoft - Xbox One S 1TB All-Digital Edition Console with Xbox One Wireless Controller (Renewed)': '499.99 $'
Price of 'Need for Speed: Heat, Electronic Arts, Xbox One, 014633373233': '25 $'
Price of 'Call of Duty: Black Ops Cold War, Activision, Xbox One': '49.94 $'
Price of 'NBA 2K21, 2K, Xbox One, 710425596858': '29.96 $'
Price of 'Refurbished Microsoft Factory Xbox One X 1TB, 4K Ultra HD Gaming Console in Black, FMQ-00042, 889842246971': '499.99 $'
Price of 'Madden NFL 21, Electronic Arts, Xbox One': '29.66 $'
Price of 'Need for Speed Payback, Electronic Arts, Xbox One, 014633370058': '14.96 $'
Price of 'Nickelodeon Kart Racers, Gamemill, Xbox One, 856131008060': '14.96 $'
Price of 'Sonic Mania (Sega), Xbox One, 010086640939': '14.96 $'
Price of 'Minecraft Master Collection, Microsoft, Xbox One, 889842394979': '42.88 $'
Price of 'Mortal Kombat 11, Warner Bros., Xbox One, 883929668977': '15 $'
Price of 'UFC 4, Electronic Arts, Xbox One': '29.96 $'
Price of 'Grand Theft Auto V, Rockstar Games, Xbox One': '22.8 $'
Price of 'Man Eater, Deep Silver, Xbox One, 816819017517': '24.96 $'
Price of 'Crash N. Sane Trilogy, Activision, Xbox One, 047875881969': '39.99 $'
Price of 'Spyro Reignited Trilogy, Activision, Xbox One, 047875882423': '29.83 $'
Price of 'Call of Duty: Modern Warfare, Activision, Xbox One, 0047875884366': '47.9 $'
Price of 'Cyberpunk 2077, Warner Bros, Xbox One': '39.74 $'
Price of 'Mass Effect Andromeda, Electronic Arts, Xbox One, 014633734096': '6.91 $'
Price of 'THQ theHunter: Call Of the Wild (Xbox One)': '19.93 $'
Price of 'Wreckfest, THQ-Nordic, Xbox One, 0811994021649': '29.83 $'
Price of 'Assassin’s Creed Valhalla Xbox Series X|S, Xbox One Standard Edition': '36.51 $'
Price of 'NBA 2K20, 2K, Xbox One, 710425595264': '21.99 $'
Price of 'LEGO Worlds, Warner Bros, Xbox One': '15 $'
Price of 'Call of Duty: Black Ops 3 Zombie Chronicles Edition, Activision, Xbox One, 047875881228': '37.88 $'
Price of 'Jumanji The Video Game, Bandai Namco, Xbox One, 819338020792': '19.93 $'
Price of 'Red Dead Redemption 2, Rockstar Games, Xbox One': '34.99 $'
Price of 'HALO 5, Microsoft, Xbox One, 885370928518': '19 $'
Price of 'LEGO Jurassic World, Warner, Xbox One, 883929472727': '19.8 $'
Price of 'Plants vs. Zombies: Battle for Neighborville, Electronic Arts, Xbox One, 014633736007': '14.96 $'
Price of 'The LEGO Marvel Collection, Warner Bros., Xbox One, 00883929670499': '24.98 $'
Price of 'LEGO The Incredibles, Warner Bros, Xbox One, 883929633005': '18 $'
Price of 'Assassin's Creed: Origins, Ubisoft, Xbox One, 887256028459': '19.93 $'
Price of 'Spongebob Squarepants: Battle for Bikini Bottom Rehydrated, THQ-Nordic, Xbox One, 811994022165': '29.77 $'
Price of 'The SIMS 4, Electronic Arts, Xbox One': '33.84 $'
Price of 'Star Wars Battlefront 2, Electronic Arts, Xbox One, 014633735321': '14.96 $'
Price of 'Pac-Man Championship Edition 2 + Arcade Game Series, Bandai/Namco, Xbox One, 722674220705': '14.49 $'
Price of 'Microsoft Xbox One S 1TB All Digital Edition 3 Game Bundle (Disc-free Gaming), White, NJP-00050': '198.95 $'

免责声明：我在SerpApi工作

你可以通过SerpApi的包装来获取沃尔玛的产品价格。这是一个免费试用的付费API

导入操作系统
从serpapi导入谷歌搜索
params=dict(
engine=“沃尔玛”，
query=“xbox one”，
api_key=os.getenv（“api_key”），
)
搜索=谷歌搜索（参数）
结果=search.get_dict（）
对于有机结果[“有机结果”]：
产品标识=有机结果[“美国商品标识”]或有机结果[“产品标识”]
search.params_dict=dict(
engine=“沃尔玛产品”，
产品标识=产品标识，
api_key=os.getenv（“api_key”），
)
product\u result=search.get\u dict（）
印刷品(
f“价格{product_result['product_result']['title']}:{product_result['product_result']['Price_map']['Price']}{product_result['product_result']['Price_map']['currency']}”
)

输出

Price of 'Microsoft Xbox One S 1TB Console (Certified Refurbished)': '429 $'
Price of 'Star Wars Jedi: Fallen Order, Electronic Arts, Xbox One': '39.82 $'
Price of 'Microsoft - Xbox One S 1TB All-Digital Edition Console with Xbox One Wireless Controller (Renewed)': '499.99 $'
Price of 'Need for Speed: Heat, Electronic Arts, Xbox One, 014633373233': '25 $'
Price of 'Call of Duty: Black Ops Cold War, Activision, Xbox One': '49.94 $'
Price of 'NBA 2K21, 2K, Xbox One, 710425596858': '29.96 $'
Price of 'Refurbished Microsoft Factory Xbox One X 1TB, 4K Ultra HD Gaming Console in Black, FMQ-00042, 889842246971': '499.99 $'
Price of 'Madden NFL 21, Electronic Arts, Xbox One': '29.66 $'
Price of 'Need for Speed Payback, Electronic Arts, Xbox One, 014633370058': '14.96 $'
Price of 'Nickelodeon Kart Racers, Gamemill, Xbox One, 856131008060': '14.96 $'
Price of 'Sonic Mania (Sega), Xbox One, 010086640939': '14.96 $'
Price of 'Minecraft Master Collection, Microsoft, Xbox One, 889842394979': '42.88 $'
Price of 'Mortal Kombat 11, Warner Bros., Xbox One, 883929668977': '15 $'
Price of 'UFC 4, Electronic Arts, Xbox One': '29.96 $'
Price of 'Grand Theft Auto V, Rockstar Games, Xbox One': '22.8 $'
Price of 'Man Eater, Deep Silver, Xbox One, 816819017517': '24.96 $'
Price of 'Crash N. Sane Trilogy, Activision, Xbox One, 047875881969': '39.99 $'
Price of 'Spyro Reignited Trilogy, Activision, Xbox One, 047875882423': '29.83 $'
Price of 'Call of Duty: Modern Warfare, Activision, Xbox One, 0047875884366': '47.9 $'
Price of 'Cyberpunk 2077, Warner Bros, Xbox One': '39.74 $'
Price of 'Mass Effect Andromeda, Electronic Arts, Xbox One, 014633734096': '6.91 $'
Price of 'THQ theHunter: Call Of the Wild (Xbox One)': '19.93 $'
Price of 'Wreckfest, THQ-Nordic, Xbox One, 0811994021649': '29.83 $'
Price of 'Assassin’s Creed Valhalla Xbox Series X|S, Xbox One Standard Edition': '36.51 $'
Price of 'NBA 2K20, 2K, Xbox One, 710425595264': '21.99 $'
Price of 'LEGO Worlds, Warner Bros, Xbox One': '15 $'
Price of 'Call of Duty: Black Ops 3 Zombie Chronicles Edition, Activision, Xbox One, 047875881228': '37.88 $'
Price of 'Jumanji The Video Game, Bandai Namco, Xbox One, 819338020792': '19.93 $'
Price of 'Red Dead Redemption 2, Rockstar Games, Xbox One': '34.99 $'
Price of 'HALO 5, Microsoft, Xbox One, 885370928518': '19 $'
Price of 'LEGO Jurassic World, Warner, Xbox One, 883929472727': '19.8 $'
Price of 'Plants vs. Zombies: Battle for Neighborville, Electronic Arts, Xbox One, 014633736007': '14.96 $'
Price of 'The LEGO Marvel Collection, Warner Bros., Xbox One, 00883929670499': '24.98 $'
Price of 'LEGO The Incredibles, Warner Bros, Xbox One, 883929633005': '18 $'
Price of 'Assassin's Creed: Origins, Ubisoft, Xbox One, 887256028459': '19.93 $'
Price of 'Spongebob Squarepants: Battle for Bikini Bottom Rehydrated, THQ-Nordic, Xbox One, 811994022165': '29.77 $'
Price of 'The SIMS 4, Electronic Arts, Xbox One': '33.84 $'
Price of 'Star Wars Battlefront 2, Electronic Arts, Xbox One, 014633735321': '14.96 $'
Price of 'Pac-Man Championship Edition 2 + Arcade Game Series, Bandai/Namco, Xbox One, 722674220705': '14.49 $'
Price of 'Microsoft Xbox One S 1TB All Digital Edition 3 Game Bundle (Disc-free Gaming), White, NJP-00050': '198.95 $'

---

对我来说，

是空的（尽管它看起来像是通过脚本修改的div）。为什么不从

$399.99

处购买呢？一个原因可能是，如果您计划对价格进行刮削，并且显示给用户的价格会发生变化，但是跨度e