Warning: file_get_contents(/data/phpspider/zhask/data//catemap/5/sql/67.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Web scraping 在scrapy中进行刮取时没有数据_Web Scraping_Scrapy - Fatal编程技术网

Web scraping 在scrapy中进行刮取时没有数据

Web scraping 在scrapy中进行刮取时没有数据,web-scraping,scrapy,Web Scraping,Scrapy,下面是我试图从中获取数据的站点的代码和url,但程序运行时不向csv输出文件写入任何内容。另外,我从chrome inspector控制台获得了xpath,所以必须是正确的 代码如下: from scrapy.spiders import CrawlSpider, Rule from scrapy.selector import HtmlXPathSelector from insta.items import Insta class instagram(CrawlSpider): n

下面是我试图从中获取数据的站点的代码和url,但程序运行时不向csv输出文件写入任何内容。另外,我从chrome inspector控制台获得了xpath,所以必须是正确的

代码如下:

from scrapy.spiders import CrawlSpider, Rule
from scrapy.selector import HtmlXPathSelector
from insta.items import Insta

class instagram(CrawlSpider):
    name = "instagram"
    allowed_domains = ["zymanga.com"]
    start_urls = ['http://zymanga.com/millionplus/%sf' % page for page in range(1,163)]

    def parse_items(self, response):
        hxs = HtmlXPathSelector(response)
        titles = hxs.xpath('//*[@id="username"]/')

        items = []
        for title in titles:
            item = Insta()
            item["username"] = title.select("a/text()").extract()
            items.append(item)
        return(items) 

尝试
yield
而不是
return
如果您已经尝试过,则不会产生任何结果…您是否在shell中尝试过XPath?如果您未登录到站点,则无法访问数据(请使用专用导航器进行尝试)。检查