Warning: file_get_contents(/data/phpspider/zhask/data//catemap/8/python-3.x/15.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 3.x Scrapy ImportError:没有名为';设置';_Python 3.x_Scrapy - Fatal编程技术网

Python 3.x Scrapy ImportError:没有名为';设置';

Python 3.x Scrapy ImportError:没有名为';设置';,python-3.x,scrapy,Python 3.x,Scrapy,我想从网站上抓取一些数据,并将其存储到postgree数据库中,但出现以下错误: ImportError:没有名为“设置”的模块 文件夹的树视图为: Coder ├── coder_app │   ├── __init__.py │   ├── items.py │   ├── models.py │   ├── pipelines.py │   ├── settings.py │   └── spiders │   ├── __init__.py │   └── livings

我想从网站上抓取一些数据,并将其存储到postgree数据库中,但出现以下错误:

ImportError:没有名为“设置”的模块

文件夹的树视图为:

Coder
├── coder_app
│   ├── __init__.py
│   ├── items.py
│   ├── models.py
│   ├── pipelines.py
│   ├── settings.py
│   └── spiders
│       ├── __init__.py
│       └── livingsocial.py
├── output.json
└── scrapy.cfg
Spider代码如下:

# -*- coding: utf-8 -*-
import scrapy
from coder_app.items import CoderItem
# from scrapy.loader import ItemLoader


class LivingsocialSpider(scrapy.Spider):
    name = "livingsocial"
    allowed_domains = ["livingsocial.com"]
    start_urls = (
        'http://www.livingsocial.com/cities/15-san-francisco',
    )

    def parse(self, response):
        # deals = response.xpath('//li')
        for deal in response.xpath('//li[a//h2/text()]'):
            item = CoderItem()
            item['title'] = deal.xpath('a//h2/text()').extract_first()
            item['link'] = deal.xpath('a/@href').extract_first()
            item['location'] = deal.xpath('a//p[@class="location"]/text()').extract_first()
            item['price'] = deal.xpath('a//div[@class="deal-price"]/text()').extract_first()
            item['end_date'] = deal.xpath('a//p[@class="dates"]/text()').extract_first()
            yield item
Pipeline.py是:

from sqlalchemy.orm import sessionmaker
from coder_app.models import Deals, db_connect, create_deals_table


class CoderPipeline(object):
    def process_item(self, item, spider):
        return item

class LivingSocialPipeline(object):
    """Livingsocial pipeline for storing scraped items in the database"""
    def __init__(self):
        """
        Initializes database connection and sessionmaker.
        Creates deals table.
        """
        engine = db_connect()
        create_deals_table(engine)
        self.Session = sessionmaker(bind=engine)

    def process_item(self, item, spider):
        """Save deals in the database.

        This method is called for every item pipeline component.

        """
        session = self.Session()
        deal = Deals(**item)

        try:
            session.add(deal)
            session.commit()
        except:
            session.rollback()
            raise
        finally:
            session.close()

        return item
Model.py代码为:

from sqlalchemy import create_engine, Column, Integer, String, DateTime
from sqlalchemy.engine.url import URL
from sqlalchemy.ext.declarative import declarative_base

import settings

DeclarativeBase = declarative_base()


def db_connect():
    """
    Performs database connection using database settings from settings.py.
    Returns sqlalchemy engine instance
    """
    return create_engine(URL(**settings.DATABASE))


def create_deals_table(engine):
    """
    Performs table creation
    """
    DeclarativeBase.metadata.create_all(engine)

class Deals(DeclarativeBase):
    """Sqlalchemy deals model"""
    __tablename__ = "deals"

    id = Column(Integer, primary_key=True)
    title = Column('title', String)
    link = Column('link', String, nullable=True)
    location = Column('location', String, nullable=True)
    #original_price = Column('original_price', String, nullable=True)
    price = Column('price', String, nullable=True)
    end_date = Column('end_date', DateTime, nullable=True)
和settings.py是:

BOT_NAME = 'coder_app'

SPIDER_MODULES = ['coder_app.spiders']
NEWSPIDER_MODULE = 'coder_app.spiders'

DATABASE = {
    'drivername': 'postgres',
    'host': 'localhost',
    'port': '5432',
    'username': 'mohib',
    'password': '100200',
    'database': 'scrape'
}
ITEM_PIPELINES = {
   'coder_app.pipelines.LivingSocialPipeline': 300,
}
为什么我会犯这个错误,我也试过了

code_app.import settings
在model.py中,但我得到了以下错误:

NameError: name 'settings' is not defined
我真的被困在这里了。有人能帮我吗?

代替

import settings

然后在函数或方法中:

settings = get_project_settings()

希望能有所帮助。

我遇到了这样一个问题,因为我有一个与我的项目同名的python模块


尝试用您的项目名称运行pip卸载。

您是否尝试过从代码应用程序导入设置中执行
?效果很好!谢谢你:)
settings = get_project_settings()