Python Scrapy https教程
各位! 我不熟悉Scrapy框架。我需要解析wisemapping.com。 起初,我阅读了官方的Scrapy教程,并试图访问其中一个“wisemap”,但出现了一个错误:Python Scrapy https教程,python,python-3.x,web-scraping,scrapy,Python,Python 3.x,Web Scraping,Scrapy,各位! 我不熟悉Scrapy框架。我需要解析wisemapping.com。 起初,我阅读了官方的Scrapy教程,并试图访问其中一个“wisemap”,但出现了一个错误: [scrapy.core.engine] DEBUG: Crawled (404) <GET https://app.wisemapping.com/robots.txt> (referer: None) [scrapy.downloadermiddlewares.retry] DEBUG: Gave up r
[scrapy.core.engine] DEBUG: Crawled (404) <GET https://app.wisemapping.com/robots.txt> (referer: None)
[scrapy.downloadermiddlewares.retry] DEBUG: Gave up retrying
<GET https://app.wisemapping.com/c/maps/576786/public> (failed 3 times): 500 Internal Server Error
[scrapy.core.engine] DEBUG: Crawled (500) <GET https://app.wisemapping.com/c/maps/576786/public> (referer: None)
[scrapy.spidermiddlewares.httperror] INFO: Ignoring response <500 https://app.wisemapping.com/c/maps/576786/public>: HTTP status code is not handled or not allowed
导航到会导致错误
“走开!!。这张地图已经不可用了。
您没有足够的权限查看此地图。此地图已更改为私有或已删除。“
这张地图存在吗?如果是这样,试着把它公开
如果您知道您尝试访问的地图确实存在,请验证您尝试访问的URL是否正确。导航到会出现错误
“走开!!。这张地图已经不可用了。
您没有足够的权限查看此地图。此地图已更改为私有或已删除。“
这张地图存在吗?如果是这样,试着把它公开
如果您知道您尝试访问的地图确实存在,请验证您尝试访问的URL是否正确
import scrapy
class QuotesSpider(scrapy.Spider):
name = "quotes"
def start_requests(self):
urls = [
'https://app.wisemapping.com/c/maps/576786/public',
]
for url in urls:
yield scrapy.Request(url=url, callback=self.parse)
def parse(self, response):
page = response.url.split("/")[-2]
filename = 'wisemape.html'
with open(filename, 'wb') as f:
f.write(response.body)
self.log('Saved file %s' % filename)