Parse platform Post Scrapy返回要分析的结果

Parse platform Post Scrapy返回要分析的结果,parse-platform,scrapy,scrapy-pipeline,Parse Platform,Scrapy,Scrapy Pipeline,我做了一个管道 PARSE='api.PARSE.com' 端口=443 但是,我找不到在Parse中发布数据的正确方法。因为每次它都会在我的Parse DB中创建未定义的对象 class Newscrawlbotv01Pipeline(object): def process_item(self, item, spider): for data in item: if not data: raise DropIt

我做了一个管道

PARSE='api.PARSE.com' 端口=443

但是,我找不到在Parse中发布数据的正确方法。因为每次它都会在我的Parse DB中创建未定义的对象

 class Newscrawlbotv01Pipeline(object):
    def process_item(self, item, spider):
        for data in item:
            if not data:
                raise DropItem("Missing data!")
        connection = httplib.HTTPSConnection(
            settings['PARSE'],
            settings['PORT']
        )
        connection.connect()
        connection.request('POST', '/1/classes/articlulos', json.dumps({item}), {
       "X-Parse-Application-Id": "XXXXXXXXXXXXXXXX",
       "X-Parse-REST-API-Key": "XXXXXXXXXXXXXXXXXXX",
       "Content-Type": "application/json"
     })
        log.msg("Question added to PARSE !", level=log.DEBUG, spider=spider)
        return item
        #self.collection.update({'url': item['url']}, dict(item), upsert=True)
错误示例:

    2016-03-16 20:13:19 [scrapy] ERROR: Error processing {'image': 'http://eedl.eodi.org/wp-content/uploads/sites/3/2016/01/Figaro.png',
 'language': 'FR',
 'publishedDate': u'2016-03-16T18:52:24+01:00',
 'publisher': 'Le Figaro',
 'theme': 'Actualites',
 'title': u'Interpellations Paris: \xable niveau de menace reste tr\xe8s \xe9lev\xe9\xbb selon Hollande',
 'url': u'http://www.lefigaro.fr/flash-actu/2016/03/16/97001-20160316FILWWW00315-interpellations-paris-la-menace-reste-tres-elevee-selon-hollande.php'}
Traceback (most recent call last):
  File "c:\python27\lib\site-packages\twisted\internet\defer.py", line 588, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "C:\Users\simon\Documents\NewsSwipe\PROTOTYPE\v0.1\NewsCrawlBotV0_1\NewsCrawlBotV0_1\pipelines.py", line 49, in process_item
    connection.request('POST', '/1/classes/articlulos', json.dumps({data}), {
  File "c:\python27\lib\json\__init__.py", line 243, in dumps
    return _default_encoder.encode(obj)
  File "c:\python27\lib\json\encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "c:\python27\lib\json\encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "c:\python27\lib\json\encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: set(['theme']) is not JSON serializable

您需要使用,它将在其
process\u item
方法上处理所有输出项,在那里您可以对该项执行任何操作。

Scrapy有一个内置的JSON文件提要导出器,您只需添加

-o example.json

到您的scrapy命令行。请参阅。

现在它是重复的,可能您应该接受创建管道的答案,然后我们可以继续您关于其他问题的管道。很抱歉,您如何接受答案?我是新来的。哦,等等,我明白了,我想你明白了,请记住把问题保持原样。