如何将从Scrapy到FTP的爬网数据存储为csv?

如何将从Scrapy到FTP的爬网数据存储为csv?,scrapy,ftp,scrapy-pipeline,Scrapy,Ftp,Scrapy Pipeline,My scrapy settings.py from datetime import datetime file_name = datetime.today().strftime('%Y-%m-%d_%H%M_') save_name = file_name + 'Mobile_Nshopping' FEED_URI = 'ftp://myusername:mypassword@ftp.mymail.com/uploads/%(save_name)s.csv' 当我运行我的蜘蛛抓取我的\u项目

My scrapy settings.py

from datetime import datetime
file_name = datetime.today().strftime('%Y-%m-%d_%H%M_')
save_name = file_name + 'Mobile_Nshopping'
FEED_URI = 'ftp://myusername:mypassword@ftp.mymail.com/uploads/%(save_name)s.csv'
当我运行我的蜘蛛抓取我的\u项目\u名称时出现错误。。。 我可以创建一个管道吗

\scrapy\extensions\feedexport.py:247: ScrapyDeprecationWarning: The `FEED_URI` and `FEED_FORMAT` settings have been deprecated in favor of the `FEEDS` setting. Please see the `FEEDS` setting docs for more details
 exporter = cls(crawler)
Traceback (most recent call last):
 File "c:\users\viren\appdata\local\programs\python\python38\lib\runpy.py", line 194, in _run_module_as_main
   return _run_code(code, main_globals, None,
 File "c:\users\viren\appdata\local\programs\python\python38\lib\runpy.py", line 87, in _run_code
   exec(code, run_globals)
 File "C:\Users\viren\AppData\Local\Programs\Python\Python38\Scripts\scrapy.exe\__main__.py", line 7, in <module>
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\cmdline.py", line 145, in execute
   _run_print_help(parser, _run_command, cmd, args, opts)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\cmdline.py", line 100, in _run_print_help
   func(*a, **kw)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\cmdline.py", line 153, in _run_command
   cmd.run(args, opts)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\commands\crawl.py", line 22, in run
   crawl_defer = self.crawler_process.crawl(spname, **opts.spargs)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\crawler.py", line 191, in crawl
   crawler = self.create_crawler(crawler_or_spidercls)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\crawler.py", line 224, in create_crawler
   return self._create_crawler(crawler_or_spidercls)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\crawler.py", line 229, in _create_crawler
   return Crawler(spidercls, self.settings)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\crawler.py", line 72, in __init__
   self.extensions = ExtensionManager.from_crawler(self)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\middleware.py", line 53, in from_crawler
   return cls.from_settings(crawler.settings, crawler)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\middleware.py", line 35, in from_settings
   mw = create_instance(mwcls, settings, crawler)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\utils\misc.py", line 167, in create_instance
   instance = objcls.from_crawler(crawler, *args, **kwargs)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 247, in from_crawler
   exporter = cls(crawler)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 282, in __init__
   if not self._storage_supported(uri, feed_options):
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 427, in _storage_supported
   self._get_storage(uri, feed_options)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 458, in _get_storage
   instance = build_instance(feedcls.from_crawler, crawler)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 455, in build_instance
   return build_storage(builder, uri, feed_options=feed_options, preargs=preargs)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 46, in build_storage
   return builder(*preargs, uri, *args, **kwargs)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 201, in from_crawler
   return build_storage(
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 46, in build_storage
   return builder(*preargs, uri, *args, **kwargs)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 192, in __init__
   self.port = int(u.port or '21')
 File "c:\users\viren\appdata\local\programs\python\python38\lib\urllib\parse.py", line 174, in port
raise ValueError(message) from None
ValueError: Port could not be cast to integer value as 'Edh=)9sd'
\scrapy\extensions\feedexport.py:247:scrapydeproduction警告:“提要URI”和“提要格式”设置已被弃用,取而代之的是“提要”设置。有关详细信息,请参阅“提要”设置文档
导出器=cls(爬虫程序)
回溯(最近一次呼叫最后一次):
文件“c:\users\viren\appdata\local\programs\python\38\lib\runpy.py”,第194行,位于\u run\u模块\u as\u main中
返回运行代码(代码、主全局、无、,
文件“c:\users\viren\appdata\local\programs\python\python38\lib\runpy.py”,第87行,在运行代码中
exec(代码、运行\全局)
文件“C:\Users\viren\AppData\Local\Programs\Python38\Scripts\scrapy.exe\ \uuuuuu main\uuuuuuu.py”,第7行,在
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\cmdline.py”,执行中的第145行
_运行\u打印\u帮助(解析器、\u运行\u命令、cmd、args、opts)
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\cmdline.py”,第100行,在\u run\u print\u帮助中
func(*a,**千瓦)
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\cmdline.py”,第153行,在\u run\u命令中
cmd.run(参数、选项)
文件“c:\users\viren\appdata\local\programs\python38\lib\site packages\scrapy\commands\crawl.py”,第22行,正在运行
crawl\u defer=self.crawler\u process.crawl(spname,**opts.spargs)
文件“c:\users\viren\appdata\local\programs\python38\lib\site packages\scrapy\crawler.py”,第191行,在爬网中
爬虫=自我。创建爬虫(爬虫或蜘蛛)
文件“c:\users\viren\appdata\local\programs\python38\lib\site packages\scrapy\crawler.py”,第224行,位于create\u crawler中
返回自我。创建爬虫程序(爬虫程序或蜘蛛程序)
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\crawler.py”,第229行,位于创建爬虫程序中
返回爬虫程序(spidercls、self.settings)
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\crawler.py”,第72行,在\uu init中__
self.extensions=ExtensionManager.from_crawler(self)
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\middleware.py”,第53行,来自爬虫程序
返回cls.from_设置(crawler.settings,crawler)
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\middleware.py”,第35行,来自\u设置
mw=创建_实例(mwcls、设置、爬虫程序)
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\utils\misc.py”,第167行,位于create\U实例中
instance=objcls.from_crawler(crawler,*args,**kwargs)
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\extensions\feedexport.py”,第247行,from_crawler
导出器=cls(爬虫程序)
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\extensions\feedexport.py”,第282行,在uu init中__
如果不支持自存储(uri、提要选项):
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\extensions\feedexport.py”,第427行,在支持的存储中
self.\u获取\u存储(uri、提要\u选项)
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\extensions\feedexport.py”,第458行,位于get\U存储器中
实例=构建实例(feedcls.from_crawler,crawler)
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\extensions\feedexport.py”,第455行,在build\u实例中
返回生成存储(生成器、uri、提要选项=提要选项、预args=预args)
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\extensions\feedexport.py”,第46行,在build\u存储器中
返回生成器(*preargs,uri,*args,**kwargs)
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\extensions\feedexport.py”,第201行,from\u crawler
返回生成存储(
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\extensions\feedexport.py”,第46行,在build\u存储器中
返回生成器(*preargs,uri,*args,**kwargs)
文件“c:\users\viren\appdata\local\programs\python\38\lib\site packages\scrapy\extensions\feedexport.py”,第192行,在uu init中__
self.port=int(u.port或'21')
文件“c:\users\viren\appdata\local\programs\python\python38\lib\urllib\parse.py”,第174行,在端口中
从无引发值错误(消息)
ValueError:端口无法转换为整数值为'Edh=)9sd'
我不知道如何将CSV存储到FTP中。 因为我的密码是int,所以出现错误? 有什么我忘了写的吗

我可以创建一个管道吗

\scrapy\extensions\feedexport.py:247: ScrapyDeprecationWarning: The `FEED_URI` and `FEED_FORMAT` settings have been deprecated in favor of the `FEEDS` setting. Please see the `FEEDS` setting docs for more details
 exporter = cls(crawler)
Traceback (most recent call last):
 File "c:\users\viren\appdata\local\programs\python\python38\lib\runpy.py", line 194, in _run_module_as_main
   return _run_code(code, main_globals, None,
 File "c:\users\viren\appdata\local\programs\python\python38\lib\runpy.py", line 87, in _run_code
   exec(code, run_globals)
 File "C:\Users\viren\AppData\Local\Programs\Python\Python38\Scripts\scrapy.exe\__main__.py", line 7, in <module>
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\cmdline.py", line 145, in execute
   _run_print_help(parser, _run_command, cmd, args, opts)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\cmdline.py", line 100, in _run_print_help
   func(*a, **kw)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\cmdline.py", line 153, in _run_command
   cmd.run(args, opts)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\commands\crawl.py", line 22, in run
   crawl_defer = self.crawler_process.crawl(spname, **opts.spargs)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\crawler.py", line 191, in crawl
   crawler = self.create_crawler(crawler_or_spidercls)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\crawler.py", line 224, in create_crawler
   return self._create_crawler(crawler_or_spidercls)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\crawler.py", line 229, in _create_crawler
   return Crawler(spidercls, self.settings)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\crawler.py", line 72, in __init__
   self.extensions = ExtensionManager.from_crawler(self)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\middleware.py", line 53, in from_crawler
   return cls.from_settings(crawler.settings, crawler)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\middleware.py", line 35, in from_settings
   mw = create_instance(mwcls, settings, crawler)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\utils\misc.py", line 167, in create_instance
   instance = objcls.from_crawler(crawler, *args, **kwargs)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 247, in from_crawler
   exporter = cls(crawler)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 282, in __init__
   if not self._storage_supported(uri, feed_options):
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 427, in _storage_supported
   self._get_storage(uri, feed_options)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 458, in _get_storage
   instance = build_instance(feedcls.from_crawler, crawler)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 455, in build_instance
   return build_storage(builder, uri, feed_options=feed_options, preargs=preargs)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 46, in build_storage
   return builder(*preargs, uri, *args, **kwargs)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 201, in from_crawler
   return build_storage(
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 46, in build_storage
   return builder(*preargs, uri, *args, **kwargs)
 File "c:\users\viren\appdata\local\programs\python\python38\lib\site-packages\scrapy\extensions\feedexport.py", line 192, in __init__
   self.port = int(u.port or '21')
 File "c:\users\viren\appdata\local\programs\python\python38\lib\urllib\parse.py", line 174, in port
raise ValueError(message) from None
ValueError: Port could not be cast to integer value as 'Edh=)9sd'
是的,您可能应该创建一个管道。如中所示,基本概念是:发送请求,响应返回并由爬行器处理,最后,管道对爬行器返回的项目进行处理。在您的情况下,您可以创建一个管道,将数据保存在CSV文件中,并将其上载到ftp服务器。有关更多信息,请参阅

我不知道如何将CSV存储到FTP中。因为我的密码是int,所以会出现错误?有什么我忘记写的吗

我认为这是由于下面的弃用错误(显示在您提供的错误顶部):
scrapydeproductionwarning:FEED\u URI和FEED\u格式设置已被弃用,取而代之的是FEED设置。有关详细信息,请参阅FEED设置文档


尝试将
FEED\u URI
替换为
FEED
;请参阅。

您还需要指定端口

可以在“设置”中指定此选项

另请参见scrapy文档中的类定义

类FTPFilesStore:

FTP_USERNAME = None
FTP_PASSWORD = None
USE_ACTIVE_MODE = None

def __init__(self, uri):
    if not uri.startswith("ftp://"):
        raise ValueError(f"Incorrect URI scheme in {uri}, expected 'ftp'")
    u = urlparse(uri)
    self.port = u.port
    self.host = u.hostname
    self.port = int(u.port or 21)
    self.username = u.username or self.FTP_USERNAME
    self.password = u.password or self.FTP_PASSWORD
    self.basedir = u.path.rstrip('/')
我不认为你的
提要URI中的
%(save_name)s
在做你想做的事情