BS4 Python：试图从Google获取页面链接，但我得到的URL都是一样的
我用BS4编写了一个python脚本，试图从google搜索中获得结果 问题：我只能从谷歌的第1页获取数据 尝试解决：我尝试提取谷歌底部的每页href，从1,2,3…10中获取每页的href，并重复我对第1页所做的过程 尝试问题：当我尝试提取页面1..10 URL时，这些页面链接与google.com上inspect元素中的链接不同。 编辑： 上述代码生成：BS4 Python：试图从Google获取页面链接，但我得到的URL都是一样的,python,html,google-chrome,web-scraping,beautifulsoup,Python,Html,Google Chrome,Web Scraping,Beautifulsoup,我用BS4编写了一个python脚本，试图从google搜索中获得结果 问题：我只能从谷歌的第1页获取数据 尝试解决：我尝试提取谷歌底部的每页href，从1,2,3…10中获取每页的href，并重复我对第1页所做的过程 尝试问题：当我尝试提取页面1..10 URL时，这些页面链接与google.com上inspect元素中的链接不同。 编辑： 上述代码生成： Title: Manga - Wikipedia Url: https://en.wikipedia.org/wiki/Manga De
Title: Manga - Wikipedia
Url: https://en.wikipedia.org/wiki/Manga
Description: Manga are comics or graphic novels originating from Japan. Most manga conform to a style developed in Japan in the late 19th century, though the art form has ...
###############
Title: Read the Best Manga - VIZ
Url: https://www.viz.com/read
Description: Action, adventure, fantasy, mystery, romance and more—thousands of manga volumes for every fan!
###############
Title: Manga Toon - Free manga, comic and novel reader online
Url: https://mangatoon.mobi/
Description: MangaToon is a Global APP for Reading Comic Manga and Novel. Different comics in Action, Romance, Boys' love, Comedy, Horror and more are updated ...
###############
Title: Read Popular Manga Online - Crunchyroll
Url: https://www.crunchyroll.com/comics/manga
Description: Read your favorite Japanese manga online on Crunchyroll including Attack on Titan, Fairy Tail, The Seven Deadly Sins, Fuuka, Knight's & Magic, and more.
###############
Title: Manga Books - Goodreads
Url: https://www.goodreads.com/genres/manga
Description: Manga. Japanese or Japanese-influenced comics and graphic novels. Usually printed in black-and-white. There are many genres inside manga, the most distinct being shojo (for girls) and shonen (for boys).
###############
Title: Manga and Anime Books | Barnes & Noble®
Url: https://www.barnesandnoble.com/b/books/graphic-novels-comics/manga/_/N-29Z8q8Zucc
Description: Discover an extensive collection of manga and anime books at Barnes & Noble. Shop a wide variety of Manga series, boxed sets, bestsellers, and more.
###############
Title: 50 Best Manga You Must Read Right Now: Classics And New ...
Url: https://bookriot.com/2020/05/26/best-manga/
Description: May 26, 2020 - New to reading manga and don't know where to start? Want to find a new series to dive into? Here's a list of the 50 best manga to add to your ...
###############
Title: MANGA Plus
Url: https://mangaplus.shueisha.co.jp/updates
Description: "MANGA Plus by SHUEISHA" is the official manga reader from Shueisha Inc., and is available globally. We publish the greatest manga in the world such as ...
###############
<td class="b d6cvqb"><span class="SJajHc" style="background:url(/images/nav_logo299.png) no-repeat;background-position:-24px 0;width:28px"></span></td>
<td class="YyVfkd"><span class="SJajHc" style="background:url(/images/nav_logo299.png) no-repeat;background-position:-53px 0;width:20px"></span>1</td>
<td><a aria-label="Page 2" class="fl" href="/search?q=manga&ei=E5r7XviHIMbn-QbH4b0Y&start=10&sa=N&ved=2ahUKEwi43ZGeqqrqAhXGc94KHcdwDwMQ8tMDegQIExAs"><span class="SJajHc NVbCr" style="background:url(/images/nav_logo299.png) no-repeat;background-position:-74px 0;width:20px"></span>2</a></td>
<td><a aria-label="Page 3" class="fl" href="/search?q=manga&ei=E5r7XviHIMbn-QbH4b0Y&start=20&sa=N&ved=2ahUKEwi43ZGeqqrqAhXGc94KHcdwDwMQ8tMDegQIExAu"><span class="SJajHc NVbCr" style="background:url(/images/nav_logo299.png) no-repeat;background-position:-74px 0;width:20px"></span>3</a></td>
<td><a aria-label="Page 4" class="fl" href="/search?q=manga&ei=E5r7XviHIMbn-QbH4b0Y&start=30&sa=N&ved=2ahUKEwi43ZGeqqrqAhXGc94KHcdwDwMQ8tMDegQIExAw"><span class="SJajHc NVbCr" style="background:url(/images/nav_logo299.png) no-repeat;background-position:-74px 0;width:20px"></span>4</a></td>
<td><a aria-label="Page 5" class="fl" href="/search?q=manga&ei=E5r7XviHIMbn-QbH4b0Y&start=40&sa=N&ved=2ahUKEwi43ZGeqqrqAhXGc94KHcdwDwMQ8tMDegQIExAy"><span class="SJajHc NVbCr" style="background:url(/images/nav_logo299.png) no-repeat;background-position:-74px 0;width:20px"></span>5</a></td>
<td><a aria-label="Page 6" class="fl" href="/search?q=manga&ei=E5r7XviHIMbn-QbH4b0Y&start=50&sa=N&ved=2ahUKEwi43ZGeqqrqAhXGc94KHcdwDwMQ8tMDegQIExA0"><span class="SJajHc NVbCr" style="background:url(/images/nav_logo299.png) no-repeat;background-position:-74px 0;width:20px"></span>6</a></td>
<td><a aria-label="Page 7" class="fl" href="/search?q=manga&ei=E5r7XviHIMbn-QbH4b0Y&start=60&sa=N&ved=2ahUKEwi43ZGeqqrqAhXGc94KHcdwDwMQ8tMDegQIExA2"><span class="SJajHc NVbCr" style="background:url(/images/nav_logo299.png) no-repeat;background-position:-74px 0;width:20px"></span>7</a></td>
<td><a aria-label="Page 8" class="fl" href="/search?q=manga&ei=E5r7XviHIMbn-QbH4b0Y&start=70&sa=N&ved=2ahUKEwi43ZGeqqrqAhXGc94KHcdwDwMQ8tMDegQIExA4"><span class="SJajHc NVbCr" style="background:url(/images/nav_logo299.png) no-repeat;background-position:-74px 0;width:20px"></span>8</a></td>
<td><a aria-label="Page 9" class="fl" href="/search?q=manga&ei=E5r7XviHIMbn-QbH4b0Y&start=80&sa=N&ved=2ahUKEwi43ZGeqqrqAhXGc94KHcdwDwMQ8tMDegQIExA6"><span class="SJajHc NVbCr" style="background:url(/images/nav_logo299.png) no-repeat;background-position:-74px 0;width:20px"></span>9</a></td>
<td><a aria-label="Page 10" class="fl" href="/search?q=manga&ei=E5r7XviHIMbn-QbH4b0Y&start=90&sa=N&ved=2ahUKEwi43ZGeqqrqAhXGc94KHcdwDwMQ8tMDegQIExA8"><span class="SJajHc NVbCr" style="background:url(/images/nav_logo299.png) no-repeat;background-position:-74px 0;width:20px"></span>10</a></td>
<td aria-level="3" class="b d6cvqb" role="heading"><a class="G0iuSb" href="/search?q=manga&ei=E5r7XviHIMbn-QbH4b0Y&start=10&sa=N&ved=2ahUKEwi43ZGeqqrqAhXGc94KHcdwDwMQ8NMDegQIExA-" id="pnnext" style="text-align:left"><span class="SJajHc NVbCr" style="background:url(/images/nav_logo299.png) no-repeat;background-position:-96px 0;width:71px"></span><span style="display:block;margin-left:53px">Next</span></a></td>
Process finished with exit code 0
标题：漫画-维基百科
网址：https://en.wikipedia.org/wiki/Manga
漫画是源自日本的漫画或图画小说。大多数漫画都符合19世纪末在日本发展起来的风格，尽管这种艺术形式已经。。。
###############
标题：阅读最佳漫画-VIZ
网址：https://www.viz.com/read
描述：动作，冒险，幻想，神秘，浪漫和更多的漫画卷为每个球迷！
###############
标题：漫画卡通-免费漫画，漫画和小说在线读者
网址：https://mangatoon.mobi/
描述：MangaToon是一款阅读漫画和小说的全球应用程序。不同的漫画在行动，浪漫，男孩的爱，喜剧，恐怖和更多的更新。。。
###############
标题：阅读流行漫画在线-Crunchyroll
网址：https://www.crunchyroll.com/comics/manga
描述：在Crunchyroll上在线阅读你最喜欢的日本漫画，包括《泰坦之战》、《仙女尾巴》、《七宗罪》、《福卡》、《骑士与魔法》等。
###############
标题：漫画书-古德雷兹
网址：https://www.goodreads.com/genres/manga
描述：漫画。日本或受日本影响的漫画和漫画小说。通常用黑白印刷。漫画中有很多类型，最明显的是shojo（女孩）和shonen（男孩）。
###############
标题：漫画和动漫书籍| Barnes&Noble®
网址：https://www.barnesandnoble.com/b/books/graphic-novels-comics/manga/_/N-29Z8q8Zucc
描述：在巴诺书店发现大量漫画和动漫书籍。商店有各种各样的漫画系列、盒装套装、畅销书等等。
###############
标题：你们现在必须阅读的50部最佳漫画：经典与新。。。
网址：https://bookriot.com/2020/05/26/best-manga/
描述：2020年5月26日-阅读漫画新手，不知道从哪里开始？想找一个新的系列来深入研究吗？这里有一个50个最好的漫画列表，添加到您的。。。
###############
标题：漫画加
网址：https://mangaplus.shueisha.co.jp/updates
描述：“SHUEISHA提供的漫画Plus”是SHUEISHA Inc.的官方漫画阅读器，可在全球范围内获得。我们出版了世界上最伟大的漫画，如。。。
###############
1.
进程已完成，退出代码为0
我如何测试链接是否正确：我进入chrome上的搜索栏，输入“google.com”，查看页码是否更改。我尝试了每个链接，我总是在第一页。此脚本打印google页面的所有标题，然后尝试查找下一个
import requests
from bs4 import BeautifulSoup
url = 'https://google.com/search?q=manga&hl=en'
headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:77.0) Gecko/20100101 Firefox/77.0'}
page = 1
while True:
print()
print('Page {}...'.format(page))
print('-' * 80)
soup = BeautifulSoup(requests.get(url, headers=headers).content, 'html.parser')
for h in soup.select('h3'):
print(h.get_text(strip=True))
next_link = soup.select_one('a:contains("Next")')
if not next_link:
break
url = 'https://google.com' + next_link['href']
page += 1
Page 1...
--------------------------------------------------------------------------------
Manga - Wikipedia
Read the Best Manga - VIZ
Manga Toon - Free manga, comic and novel reader online
Read Popular Manga Online - Crunchyroll
Manga Books - Goodreads
MANGA Plus
Videos
MANGA Plus by SHUEISHA - Apps on Google Play
MANGA.CLUB｜Read Free Official Manga Online!
50 Best Manga You Must Read Right Now: Classics And New ...
Images
Page 2...
--------------------------------------------------------------------------------
Top 10 Best Manga Series - YouTube
What are Manga and Anime - MIT
manga - Wiktionary
15 Best Manga Of All Time (According To MyAnimeList) | CBR
Images for manga
Guided Search Filters
Manga Storm on the App Store
A Beginner's Guide to Manga | The New York Public Library
Manga and Anime Books | Barnes & Noble®
Manga | Definition of Manga by Merriam-Webster
Manga Rock - Online Manga Reader
Page 3...
--------------------------------------------------------------------------------
manga - Urban Dictionary
...and so on.
源代码的内容并不是你想象的那样。
Page 1...
--------------------------------------------------------------------------------
Manga - Wikipedia
Read the Best Manga - VIZ
Manga Toon - Free manga, comic and novel reader online
Read Popular Manga Online - Crunchyroll
Manga Books - Goodreads
MANGA Plus
Videos
MANGA Plus by SHUEISHA - Apps on Google Play
MANGA.CLUB｜Read Free Official Manga Online!
50 Best Manga You Must Read Right Now: Classics And New ...
Images
Page 2...
--------------------------------------------------------------------------------
Top 10 Best Manga Series - YouTube
What are Manga and Anime - MIT
manga - Wiktionary
15 Best Manga Of All Time (According To MyAnimeList) | CBR
Images for manga
Guided Search Filters
Manga Storm on the App Store
A Beginner's Guide to Manga | The New York Public Library
Manga and Anime Books | Barnes & Noble®
Manga | Definition of Manga by Merriam-Webster
Manga Rock - Online Manga Reader
Page 3...
--------------------------------------------------------------------------------
manga - Urban Dictionary
...and so on.