Web scraping 如何获取链接中的数据?为什么请求没有显示页面的全部内容?

Web scraping 如何获取链接中的数据?为什么请求没有显示页面的全部内容?,web-scraping,beautifulsoup,python-requests,Web Scraping,Beautifulsoup,Python Requests,我试图使用python从在线数据库中提取数据,但我必须浏览条款和条件页面才能访问。我使用request登录,但输出没有显示页面的全部内容。如何传递协议页面并访问页面源中显示但在我创建的soup对象中未显示的数据 import requests from bs4 import BeautifulSoup url = 'https://apps.usp.org/app/worldwide/medQualityDatabase/reportResults.html?country=Ethiopia

我试图使用python从在线数据库中提取数据,但我必须浏览条款和条件页面才能访问。我使用request登录,但输出没有显示页面的全部内容。如何传递协议页面并访问页面源中显示但在我创建的soup对象中未显示的数据

import requests
from bs4 import BeautifulSoup


url = 'https://apps.usp.org/app/worldwide/medQualityDatabase/reportResults.html?country=Ethiopia%2BGhana%2BKenya%2BMozambique%2BNigeria%2BCambodia%2BLao+PDR%2BPhilippines%2BThailand%2BViet+Nam%2BBolivia%2BColombia%2BEcuador%2BGuatemala%2BGuyana%2BPeru&period=2017%2B2016%2B2015%2B2014%2B2013%2B2012%2B2011%2B2010%2B2009%2B2008%2B2007%2B2006%2B2005%2B2004%2B2003&conclusion=Both&testType=Both&counterfeit=Both'

payload = {'username':'user', 
           'password': 'pass', 
           'agree': '', 
           'edit-submit':'Submit'}
s = requests.session()
s.post(url,data = payload)

r = s.get(url)
data = BeautifulSoup(r.content, 'lxml')
print(data)