Python 2.7 试图用Scrapy FormRequest填充表单,意外结果
我正在尝试填写www.wetseal.com/Stores上的表单,该表单允许选择显示门店的州Python 2.7 试图用Scrapy FormRequest填充表单,意外结果,python-2.7,scrapy,scrapy-spider,Python 2.7,Scrapy,Scrapy Spider,我正在尝试填写www.wetseal.com/Stores上的表单,该表单允许选择显示门店的州 <form action="http://www.wetseal.com/Stores?dwcont=C73689620" method="post" id="dwfrm_storelocator_state"> <fieldset> <div class="form-row r
<form action="http://www.wetseal.com/Stores?dwcont=C73689620" method="post" id="dwfrm_storelocator_state">
<fieldset>
<div class="form-row required ">
<label for="dwfrm_storelocator_address_states_stateUSCA">
<span>State</span>
<span class="required-indicator">*</span>
</label>
<select id="dwfrm_storelocator_address_states_stateUSCA" class="input-select required" name="dwfrm_storelocator_address_states_stateUSCA">
<option value="">Select...</option>
<option value="AK">Alaska</option>
<option value="AZ">Arizona</option>
<option value="AR">Arkansas</option>
<option value="CA">California</option>
<option value="CO">Colorado</option>
<option value="CT">Connecticut</option>
<option value="DE">Delaware</option>
<option value="FL">Florida</option>
<option value="GA">Georgia</option>
<option value="HI">Hawaii</option>
<option value="ID">Idaho</option>
<option value="IL">Illinois</option>
<option value="IN">Indiana</option>
<option value="KS">Kansas</option>
<option value="KY">Kentucky</option>
<option value="MD">Maryland</option>
<option value="MA">Massachusetts</option>
<option value="MI">Michigan</option>
<option value="MN">Minnesota</option>
<option value="MS">Mississippi</option>
<option value="MO">Missouri</option>
<option value="NE">Nebraska</option>
<option value="NV">Nevada</option>
<option value="NH">New Hampshire</option>
<option value="NJ">New Jersey</option>
<option value="NM">New Mexico</option>
<option value="NY">New York</option>
<option value="NC">North Carolina</option>
<option value="ND">North Dakota</option>
<option value="OH">Ohio</option>
<option value="OK">Oklahoma</option>
<option value="OR">Oregon</option>
<option value="PA">Pennsylvania</option>
<option value="PR">Puerto Rico</option>
<option value="RI">Rhode Island</option>
<option value="SC">South Carolina</option>
<option value="SD">South Dakota</option>
<option value="TN">Tennessee</option>
<option value="TX">Texas</option>
<option value="VA">Virginia</option>
<option value="WA">Washington</option>
<option value="WV">West Virginia</option>
<option value="WI">Wisconsin</option>
</select>
</div>
<button type="submit" name="dwfrm_storelocator_findbystate" value="Search">
Search
</button>
</fieldset>
</form>
当它开始发出FormRequest时,查看响应,一切似乎都正常:
但在回调方法中,我在响应中看到:
看起来像是在最后发出了GET请求,而url完全错误:
'http://www.wetseal.com/Search?q=&dwfrm_storelocator_findbystate=Search&dwfrm_storelocator_address_states_stateUSCA=AZ'
知道我做错了什么吗
谢谢 您正在使用
formname
但表单没有名称
尝试改用formxpath='id(“dwfrm\u storelocator\u state”)
。试试这个
states = response.xpath(
".//select[@id='dwfrm_storelocator_address_states_stateUSCA']//option[@value!='']/@value").extract()
url = self.get_text_from_node(response.xpath("//form[@id='dwfrm_storelocator_state']/@action"))
for state in states:
form_data = {'dwfrm_storelocator_address_states_stateUSCA': state,
"dwfrm_storelocator_findbystate": "Search"}
yield FormRequest(url,
formdata=form_data,
callback=self.your_Callback)
成功了!正如我所怀疑的,名字不能被使用,我认为Scrapy会聪明到可以同时查看“id”属性,但它不是:D,谢谢@Artur Gaspar
states = response.xpath(
".//select[@id='dwfrm_storelocator_address_states_stateUSCA']//option[@value!='']/@value").extract()
url = self.get_text_from_node(response.xpath("//form[@id='dwfrm_storelocator_state']/@action"))
for state in states:
form_data = {'dwfrm_storelocator_address_states_stateUSCA': state,
"dwfrm_storelocator_findbystate": "Search"}
yield FormRequest(url,
formdata=form_data,
callback=self.your_Callback)