Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/294.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 我如何解决这个问题&引用;类型错误:';维基百科项目';对象不支持项分配“;_Python_Web Scraping_Scrapy_Wikipedia - Fatal编程技术网

Python 我如何解决这个问题&引用;类型错误:';维基百科项目';对象不支持项分配“;

Python 我如何解决这个问题&引用;类型错误:';维基百科项目';对象不支持项分配“;,python,web-scraping,scrapy,wikipedia,Python,Web Scraping,Scrapy,Wikipedia,我对python和scrapy都很陌生。我想从维基百科上搜集数据,但事情没有成功。每次我在维基上抓痒,我总是得到;“TypeError:'WikipediaItem'对象不支持项目分配”。我该如何解决这个问题,以及如何成功地从维基百科中获取细节 无论如何,我的代码如下: from scrapy.spider import BaseSpider from scrapy.selector import HtmlXPathSelector from wikipedia.items import Wik

我对python和scrapy都很陌生。我想从维基百科上搜集数据,但事情没有成功。每次我在维基上抓痒,我总是得到;“TypeError:'WikipediaItem'对象不支持项目分配”。我该如何解决这个问题,以及如何成功地从维基百科中获取细节

无论如何,我的代码如下:

from scrapy.spider import BaseSpider
from scrapy.selector import HtmlXPathSelector
from wikipedia.items import WikipediaItem

class WikipediaItem(BaseSpider):
    name = "wiki"
    allowed_domains = ["wikipedia.org"]
    start_urls = ["http://en.wikipedia.org/wiki/Main_Page"]

    def parse(self, response):
        hxs = HtmlXPathSelector(response)
        sites = hxs.select('//table[@id="mp-upper"]/tr')
        items = []
        for site in sites:
            item = WikipediaItem()
            item['title'] = site.select('.//a[@class="MainPageBG"]/text()').extract()
            item['link'] = site.select('.//a[@class="MainPageBG"]').extract()
            item['details'] = site.select('.//p/text()').extract()
            items.append(item)
        return items
下面是我得到的结果:

2013-04-18 23:56:54+0800 [scrapy] INFO: Scrapy 0.14.4 started (bot: wikipedia)
2013-04-18 23:56:54+0800 [scrapy] DEBUG: Enabled extensions: LogStats, TelnetConsole, CloseSpider, WebService, CoreStats, MemoryUsage, SpiderState
2013-04-18 23:56:54+0800 [scrapy] DEBUG: Enabled downloader middlewares: HttpAuthMiddleware, DownloadTimeoutMiddleware, UserAgentMiddleware, RetryMiddleware, DefaultHeadersMiddleware, RedirectMiddleware, CookiesMiddleware, HttpCompressionMiddleware, ChunkedTransferMiddleware, DownloaderStats
2013-04-18 23:56:54+0800 [scrapy] DEBUG: Enabled spider middlewares: HttpErrorMiddleware, OffsiteMiddleware, RefererMiddleware, UrlLengthMiddleware, DepthMiddleware    
2013-04-18 23:56:54+0800 [scrapy] DEBUG: Enabled item pipelines: 
2013-04-18 23:56:54+0800 [wiki] INFO: Spider opened
2013-04-18 23:56:54+0800 [wiki] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2013-04-18 23:56:54+0800 [scrapy] DEBUG: Telnet console listening on 0.0.0.0:6023
2013-04-18 23:56:54+0800 [scrapy] DEBUG: Web service listening on 0.0.0.0:6080
2013-04-18 23:56:56+0800 [wiki] DEBUG: Crawled (200) <GET http://en.wikipedia.org/wiki/Main_Page> (referer: None)
2013-04-18 23:56:56+0800 [wiki] ERROR: Spider error processing <GET http://en.wikipedia.org/wiki/Main_Page>
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 1178, in mainLoop
        self.runUntilCurrent()
      File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 800, in runUntilCurrent
        call.func(*call.args, **call.kw)
      File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 368, in callback
        self._startRunCallbacks(result)
      File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 464, in _startRunCallbacks
        self._runCallbacks()
    --- <exception caught here> ---
      File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 551, in _runCallbacks
        current.result = callback(current.result, *args, **kw)
      File "/home/jean/wiki/wikipedia/spiders/wikipedia_spider.py", line 17, in parse
        item['title'] = row.select('.//a[@class="MainPageBG"]/text()').extract()
    exceptions.TypeError: 'WikipediaItem' object does not support item assignment
2013-04-18 23:56:56+0800 [wiki] INFO: Closing spider (finished)
2013-04-18 23:56:56+0800 [wiki] INFO: Dumping spider stats:
    {'downloader/request_bytes': 215,
     'downloader/request_count': 1,    
     'downloader/request_method_count/GET': 1,
     'downloader/response_bytes': 17762,
     'downloader/response_count': 1,
     'downloader/response_status_count/200': 1,
     'finish_reason': 'finished',
     'finish_time': datetime.datetime(2013, 4, 18, 15, 56, 56, 244255),    
     'scheduler/memory_enqueued': 1,
     'spider_exceptions/TypeError': 1,
     'start_time': datetime.datetime(2013, 4, 18, 15, 56, 54, 592948)}
2013-04-18 23:56:56+0800 [wiki] INFO: Spider closed (finished)
2013-04-18 23:56:56+0800 [scrapy] INFO: Dumping global stats:
    {'memusage/max': 28065792, 'memusage/startup': 28065792}

您将刮刀命名为与导入的
WikipediaItem
相同的名称:

from wikipedia.items import WikipediaItem

class WikipediaItem(BaseSpider):
    # ...
因此,
parse
使用的是您的
BaseSpider
子类,而不是您在
wikipedia.items
中定义的任何子类。也许您想重命名该类:

class WikipediaSpider(BaseSpider):
    # ...

WikipediaItem
来自哪里?这是来自外部项目还是您自己编写的类?然后您需要向我们展示您的
items.py
class WikipediaSpider(BaseSpider):
    # ...