Warning: file_get_contents(/data/phpspider/zhask/data//catemap/8/http/4.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181

Warning: file_get_contents(/data/phpspider/zhask/data//catemap/1/ms-access/4.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 带有MechanicalGroup的表单请求未显示预期结果_Python_Http_Web Scraping_Python Requests_Mechanicalsoup - Fatal编程技术网

Python 带有MechanicalGroup的表单请求未显示预期结果

Python 带有MechanicalGroup的表单请求未显示预期结果,python,http,web-scraping,python-requests,mechanicalsoup,Python,Http,Web Scraping,Python Requests,Mechanicalsoup,我是网络抓取和网络事物的新手(但对Python非常熟悉),我想了解如何将网站搜索集成到生物信息学研究工具中 目标:在上检索表单的输出 在输出中,我得到了完全相同的页面()。我尝试使用标准请求,但遇到另一种错误: from requests import get, Session url="http://www.lovd.nl/3.0/search" formurl = "http://www.lovd.nl/3.0/ajax/search_variant.php" client = Sessi

我是网络抓取和网络事物的新手(但对Python非常熟悉),我想了解如何将网站搜索集成到生物信息学研究工具中

目标:在上检索表单的输出

在输出中,我得到了完全相同的页面()。我尝试使用标准请求,但遇到另一种错误:

from requests import get, Session

url="http://www.lovd.nl/3.0/search"
formurl = "http://www.lovd.nl/3.0/ajax/search_variant.php"
client = Session()

#get the csrf
soup = BeautifulSoup(client.get(url).text, "html.parser")
csrf = soup.select('form input[name="csrf_token"]')[0]['value']

form_data = {
    "search": "",
    "csrf_token": csrf,
    "build": "hg19",
    "variant": "chr15:g.40699840C>T"
}

response = get(formurl, data=form_data)
html=response.content
return html
…这只返回一个

alert("Error while sending data.");
表单_数据字段取自XHR请求(来自developer->network选项卡)

我可以看到数据是通过ajax异步发送的,但我不理解这些信息的实际含义


需要一些指导

MechanicalSoup不做JavaScript。您尝试浏览的网站有:

<form id="websitevariantsearch"
      action=""
      onsubmit="if ...">


没有传统HTML表单意义上的操作,但是在提交时会执行一段JavaScript。机械组在这里帮不了忙。硒可能会起作用:

谢谢你,我按照你的建议用硒做了。
<form id="websitevariantsearch"
      action=""
      onsubmit="if ...">