Python 如何对站点URL的所有页面进行分页？

我试图从一个网站的url上抓取代理的姓名、电话和电子邮件。我已经完成了第一页的刮削工作。但我无法翻阅其余的几页。因为所有的页面似乎都有相同的URL。URL中没有作为查询字符串的页码

我正在使用

请求

和

美化组

顺便说一句

帮我把所有的书页都翻一遍。谢谢

网址是

第一页的代码：

import requests
from bs4 import BeautifulSoup as bs

count = 0

response = requests.get('https://propertycentral.co.za/property/all_property?adtype=3&areaname=Johannesburg&minprice=&maxprice=&bedrooms=&bathrooms=')
content = bs(response.text , 'lxml')

detail_page = content.find_all('div',class_='info')

for properte in detail_page:
    count += 1
    href = properte.find('a').get('href')
    property_name = properte.find('a').text.strip()
    properte_link = 'https://propertycentral.co.za{}'.format(href)

    print('\n********* Property#{} *********\n'.format(count))
    print('Property: {}'.format(property_name))
    print('Property Link: {}\n'.format(properte_link))

导入请求
从bs4导入BeautifulSoup
从itertools导入islice
参数={
“adtype”：“3”，
“区域名称”：“约翰内斯堡”，
“minprice”：“，
“maxprice”：“，
“卧室”：“，
“浴室”：”
}
数据={
“_EVENTTARGET”：”，
“事件参数”：“，
“ctl00$MainContent$hdn_区域ID”：”，
“ctl00$MainContent$hdn_onpagesrch_标志”：”，
“ctl00$MainContent$ddl_adtype”：“3”，
“ctl00$MainContent$ddl_省”：”，
“ctl00$MainContent$ddl_province_CascadingDropDown_ClientState:：：：：：”，
“ctl00$MainContent$ddl_Area_CascadingDropDown_ClientState:：：：：：”，
“ctl00$MainContent$ddl\u郊区\u CascadingDropDown\u客户状态”：：，
“ctl00$MainContent$ddl_卧室”：“，
“ctl00$MainContent$ddl_浴室”：“，
“ctl00$MainContent$ddl_最小价格”：”，
“ctl00$MainContent$ddl_最大价格”：”，
hiddenInputToUpdateATBuffer_CommonToolKitScript:“0”
}
def主（url）：
将requests.Session（）作为请求：
r=req.get（url，params=params）
soup=BeautifulSoup（r.content'html.parser'）
数据[''视图状态']=soup.find（“输入”，id=“\uu视图状态”）.get（“值”）
数据['''\u VIEWSTATEGENERATOR']=soup.find(
“输入”，id=“\uu VIEWSTATEGENERATOR”）.get（“值”）
目标=汤。选择（“div.info”）
打印（f“{'*'*80}提取页面#1{'*'*80}”）
对于目标中的焦油：
打印（“名称：{：”
对于范围（1,10）内的页面：
打印（f“{'*'*80}提取页面{Page+1}{'*'*80}”）
r=req.post（url，params=params，data=data）
soup=BeautifulSoup（r.content'html.parser'）
目标=汤。选择（“div.info”）
数据[''视图状态']=soup.find(
“输入”，id=“\uu VIEWSTATE”）.get（“值”）
数据['''\u VIEWSTATEGENERATOR']=soup.find(
“输入”，id=“\uu VIEWSTATEGENERATOR”）.get（“值”）
对于目标中的焦油：
打印（“名称：{：如果你在
=
（等号）之后添加链接末尾的数字（1,2,3，…），我想，内容会改变。你可以试试看！艾哈迈德日安-很高兴你带着你的想法来到这里。你有（至少有一个片段）吗我们可以查看的代码的列表…：这太棒了！@HumayunAhmadRajib No.。内容会发生变化，因为它会过滤到“bathrooms={anynumber}”“…它不会转到任何其他页面。请看这个，我认为它会对您有所帮助。@零我已经提供了第一页的代码。@AhmadFaraz您必须增加循环，仅此而已！请随意接受我的答案，如果它对您有帮助的话。我尝试增加循环，但没有帮助，它在前4页起作用，然后开始重复，没有帮助。”他一次又一次地看了同样的4页。我真的很努力，但我无法让它在所有的页面上都正常工作。请帮助…@AhmadFaraz让我检查一下，先生……你真是太棒了……非常感谢你
********************************************************************************Extracting Page# 1********************************************************************************
Name: 3 bedroom house for sale                          , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/3-Bedroom-Property/30347   
Name: 2 bedroom house for sale                          , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/2-Bedroom-Property/30344   
Name: 1 Bedroom Apartment For Sale in Auckland...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Auckland Park/1-Bedroom-Property/30343  
Name: 2 bedroom townhouse for sale                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/2-Bedroom-Property/30338   
Name: 2 bedroom townhouse for sale                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/2-Bedroom-Property/30337   
Name: 2 bedroom townhouse for sale                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/2-Bedroom-Property/30333   
Name: 3 bedroom house for sale                          , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/3-Bedroom-Property/30332   
Name: 2 bedroom townhouse for sale                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/2-Bedroom-Property/30330   
Name: 3 bedroom townhouse for sale                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/3-Bedroom-Property/30329   
Name: 3 bedroom house for sale                          , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/3-Bedroom-Property/30327   
********************************************************************************Extracting Page# 2********************************************************************************
Name: 3 Bedroom House For Sale in Lenasia               , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/3-Bedroom-Property/30294
Name: 2 Bedroom Freestanding For Sale in Devla...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/2-Bedroom-Property/30256   
Name: Property for sale in Anchorville                  , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/0-Bedroom-Property/29843        
Name: 1486 m² Land available in Fordsburg               , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/0-Bedroom-Property/29842   
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/0-Bedroom-Property/29836        
Name: Property for sale in Anchorville                  , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/0-Bedroom-Property/29835        
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/0-Bedroom-Property/29832        
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/0-Bedroom-Property/29831        
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/0-Bedroom-Property/29828        
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/0-Bedroom-Property/29827        
********************************************************************************Extracting Page# 3********************************************************************************
Name: Property for sale in Lenasia. POA                 , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/2-Bedroom-Property/29817
Name: Property for sale in Lenasia. POA                 , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/5-Bedroom-Property/29815        
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/4-Bedroom-Property/29811        
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/5-Bedroom-Property/29805        
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/3-Bedroom-Property/29802        
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/2-Bedroom-Property/29800        
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/6-Bedroom-Property/29797        
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/7-Bedroom-Property/29795        
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/2-Bedroom-Property/29788        
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/3-Bedroom-Property/29783        
********************************************************************************Extracting Page# 4********************************************************************************
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/3-Bedroom-Property/29780
Name: Property for sale in Lenasia South                , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia South/3-Bedroom-Property/29778
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/2-Bedroom-Property/29776      
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/3-Bedroom-Property/29766      
Name: Property for sale in Lenasia South                , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia South/2-Bedroom-Property/29763
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/0-Bedroom-Property/29762      
Name: Property for sale in Lenasia South                , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia South/3-Bedroom-Property/29760
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/0-Bedroom-Property/29754
Name: Property for sale in Ennerdale                    , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Ennerdale/3-Bedroom-Property/29745
Name: Property for sale in Lenasia South                , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia South/3-Bedroom-Property/29738
********************************************************************************Extracting Page# 5********************************************************************************
Name: Property for sale in Lenasia South                , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia South/3-Bedroom-Property/29736
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/0-Bedroom-Property/29734
Name: Property for sale in Lenasia South                , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia South/24-Bedroom-Property/29732
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/20-Bedroom-Property/29730
Name: Property for sale in Parkwood                     , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Parkwood/4-Bedroom-Property/29728
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/6-Bedroom-Property/29726
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/4-Bedroom-Property/29724
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/5-Bedroom-Property/29722
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/4-Bedroom-Property/29720
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/4-Bedroom-Property/29718
********************************************************************************Extracting Page# 6********************************************************************************
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/8-Bedroom-Property/29716
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/4-Bedroom-Property/29714
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/4-Bedroom-Property/29706
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/4-Bedroom-Property/29704
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/4-Bedroom-Property/29702
Name: Property for sale in Randburg Central             , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/3-Bedroom-Property/29699
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/6-Bedroom-Property/29692
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/5-Bedroom-Property/29688
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/6-Bedroom-Property/29686
Name: Property for sale in Lenasia South                , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/6-Bedroom-Property/29685
********************************************************************************Extracting Page# 7********************************************************************************
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/4-Bedroom-Property/29671
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/5-Bedroom-Property/29668
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/4-Bedroom-Property/29666
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/3-Bedroom-Property/29665
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/3-Bedroom-Property/29632
Name: Property for sale in Mayfair                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Mayfair West/4-Bedroom-Property/29631
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/4-Bedroom-Property/29630
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/3-Bedroom-Property/29628
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/4-Bedroom-Property/29627
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/4-Bedroom-Property/29626
********************************************************************************Extracting Page# 8********************************************************************************
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/3-Bedroom-Property/29624
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/3-Bedroom-Property/29623
Name: Property for sale in Lenasia South                , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia South/3-Bedroom-Property/29621
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/3-Bedroom-Property/29620
Name: Property for sale in Lenasia South                , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia South/3-Bedroom-Property/29617
Name: Property for sale in Lenasia South                , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia South/5-Bedroom-Property/29614
Name: Property for sale in Lenasia                      , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/3-Bedroom-Property/29611
Name: Situated in Lilliba Section                       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/2-Bedroom-Property/29483
Name: 3 Bedroom Freestanding For Sale in Lenas...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia/3-Bedroom-Property/29376
Name: 3 Bedroom Freehold For Sale in Lenasia S...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia South/3-Bedroom-Property/29364
********************************************************************************Extracting Page# 9********************************************************************************
Name: 2 Bedroom Apartment / Flat for sale in N...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Northcliff/2-Bedroom-Property/29354
Name: 2 Bedroom House For Sale in Ennerdale             , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Ennerdale/2-Bedroom-Property/29339
Name: 2 Bedroom House For Sale in Eldorado Par...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Eldorado Park/2-Bedroom-Property/29328
Name: 3 Bedroom House For Sale in Lenasia Sout...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Lenasia South/3-Bedroom-Property/29325
Name: 2 Bedroom Apartment / Flat for sale in J...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/JHB CBD/Johannesburg Central and CBD/2-Bedroom-Property/29093    
Name: 2 Bedroom Apartment / Flat for sale in J...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/JHB CBD/Johannesburg Central and CBD/2-Bedroom-Property/29005    
Name: 6 Bedroom House for sale in Ennerdale             , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Ennerdale/6-Bedroom-Property/28949
Name: Commercial Property for sale in Bertrams          , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Bertrams/0-Bedroom-Property/28650
Name: 1 Bedroom Apartment / Flat for sale in B...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Braamfontein Werf/1-Bedroom-Property/28622
Name: Apartment / Flat for sale in Joubert Par...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/0-Bedroom-Property/28397
********************************************************************************Extracting Page# 10********************************************************************************
Name: R5 750 000 | 2 Bedroom Apartment / Flat ...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Melrose/2-Bedroom-Property/26929
Name: Apartment / Flat For Sale in Johannesbur...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/0-Bedroom-Property/26788
Name: Apartment / Flat For Sale in Johannesbur...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/0-Bedroom-Property/26771
Name: Apartment / Flat For Sale in Johannesbur...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/0-Bedroom-Property/26762
Name: Apartment / Flat For Sale in Johannesbur...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/0-Bedroom-Property/26746
Name: Apartment / Flat For Sale in Maboneng, J...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/0-Bedroom-Property/26737
Name: Apartment / Flat For Sale in Johannesbur...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/1-Bedroom-Property/26736
Name: Apartment / Flat For Sale in Johannesbur...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/2-Bedroom-Property/26725
Name: Apartment / Flat For Sale in Johannesbur...       , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Johannesburg/0-Bedroom-Property/26712
Name: 3 Bedroom House For Sale in Parkhurst             , Url: https://propertycentral.co.za/property/House-For-Sale/Gauteng/Johannesburg/Parkhurst/3-Bedroom-Property/26578