Xpath 为什么我会得到一个空的瘙痒反应？

我开始

scrapy shell -s USER_AGENT='Mozilla/5.0' https://www.gumtree.com/p/property-to-rent/brand-new-modern-studio-flat-%C2%A31056pcm-all-bills-included-in-willesden-green-area/1303463798

下一步

In [5]: response                                                                                                                                                                                            
Out[5]: <405 https://www.gumtree.com/p/property-to-rent/brand-new-modern-studio-flat-%C2%A31056pcm-all-bills-included-in-willesden-green-area/1303463798>

复制外部HTML

<h1 itemprop="name" id="ad-title">Brand New Modern Studio Flat £1056pcm | All Bills Included | In Willesden Green area</h1>

全新现代工作室公寓，1056pcm英镑|包括所有账单|在Willesden绿地

图像视图响应

---

为什么？

尝试将用户代理设置为更现实的设置，例如：

Mozilla/5.0（X11；Ubuntu；Linux x86；rv:63.0）Gecko/20100101 Firefox/63.0

一些网站在用户代理上做一些基本的验证，如果他们检测到一些奇怪的东西，会将你重定向到一些特殊的页面

scrapy shell -s USER_AGENT='Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:63.0) Gecko/20100101 Firefox/63.0' https://www.gumtree.com/p/property-to-rent/brand-new-modern-studio-flat-%C2%A31056pcm-all-bills-included-in-willesden-green-area/1303463798
>>> response.xpath('//*[@id="ad-title"]').extract()
['<h1 itemprop="name" id="ad-title">Brand New Modern Studio Flat £1056pcm | All Bills Included | In Willesden Green area</h1>']
>>>

scrapy shell-s USER_AGENT='Mozilla/5.0（X11；Ubuntu；Linux x86_64；rv:63.0）Gecko/20100101 Firefox/63.0'https://www.gumtree.com/p/property-to-rent/brand-new-modern-studio-flat-%C2%A31056pcm-包括在willesden green area/1303463798中的所有账单
>>>response.xpath（'/*[@id=“ad title”]'）。extract（）
[“全新的现代工作室公寓，1056pcm英镑|包括所有账单|在Willesden绿地”]
>>>

如果仔细看，回答是

405

。这是一个错误代码。如何修复？

scrapy shell -s USER_AGENT='Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:63.0) Gecko/20100101 Firefox/63.0' https://www.gumtree.com/p/property-to-rent/brand-new-modern-studio-flat-%C2%A31056pcm-all-bills-included-in-willesden-green-area/1303463798
>>> response.xpath('//*[@id="ad-title"]').extract()
['<h1 itemprop="name" id="ad-title">Brand New Modern Studio Flat £1056pcm | All Bills Included | In Willesden Green area</h1>']
>>>