为什么这个url不是用python打开的,而是可以直接从浏览器打开的?

为什么这个url不是用python打开的,而是可以直接从浏览器打开的?,python,web-scraping,beautifulsoup,urllib,Python,Web Scraping,Beautifulsoup,Urllib,这是我用python编写的打开url的代码 from urllib.request import urlopen from urllib.error import HTTPError from bs4 import BeautifulSoup import time import requests from random import randint import urllib.parse class AmazonReviews(): def __init__(self):

这是我用python编写的打开url的代码

from urllib.request import urlopen
from urllib.error import HTTPError
from bs4 import BeautifulSoup
import time
import requests
from random import randint
import urllib.parse

class AmazonReviews():
    def __init__(self):
           self.headers = {'User-Agent' : 'Mozilla/5.0'}

    def open_url(self,url):
        values = {}
        data = urllib.parse.urlencode(values).encode('utf-8')
        req = urllib.request.Request(url, data, self.headers)
        response = urllib.request.urlopen(req)
        html = response.read()
        return html

   def fetch_reviews(self,all_reviews_link):
        try:
            url = "https://www.amazon.in" + all_reviews_link
            print(url)
            html = self.open_url(url)
        except HTTPError as e:
            print(e)

review = AmazonReviews()
review.fetch_reviews('/gp/profile/amzn1.account.AFBWOEM2CWLC7ZRQ7WK6FQYXH6AA/ref=cm_cr_arp_d_gw_btm?ie=UTF8')
我之所以这样传递url,是因为在主项目中,此url是使用提供相对路径的href属性刮取的。 如果有任何方法获得绝对网址,请建议

输出-

https://www.amazon.in/gp/profile/amzn1.account.AFBWOEM2CWLC7ZRQ7WK6FQYXH6AA/ref=cm_cr_arp_d_gw_btm?ie=UTF8
HTTP Error 404: NotFound
代码链接

改用:

输出:


有趣的是,
requests.get(url,headers=self.headers)。状态代码为403。感谢您的解决方案。我使用了类似的代码来获取这个url,它工作了/gp/profile/amzn1.account.AFBWOEM2CWLC7ZRQ7WK6FQYXH6AA/ref=cm\u cr\u arp\u d\u gw\u btm?ie=UTF8'为什么它在这种情况下不工作@扎拉基·肯帕奇
from selenium import webdriver
import os

browser = webdriver.Chrome(executable_path=os.path.abspath(os.getcwd()) + "/chromedriver")
link = "https://www.amazon.in/gp/profile/amzn1.account.AFBWOEM2CWLC7ZRQ7WK6FQYXH6AA/ref=cm_cr_arp_d_gw_btm?ie=UTF8"
browser.get(link)
name = browser.find_element_by_xpath('//*[@id="customer-profile-name-header"]/div[2]/span').text
Dheeraj Malhotra