Python Scrapy:如何将数据插入SQLite
我正在尝试将字典写入sql数据库,但没有成功地给出:Python Scrapy:如何将数据插入SQLite,python,sqlite,scrapy,Python,Sqlite,Scrapy,我正在尝试将字典写入sql数据库,但没有成功地给出: cursor.execute('INSERT INTO posts(url, First_and_Last_Name) VALUES("%s", "%s")' % (url, First_and_Last_Name)) NameError: name 'url' is not defined 蜘蛛网: from scrapy.crawler import CrawlerProcess import scrapy import sqlite3
cursor.execute('INSERT INTO posts(url, First_and_Last_Name) VALUES("%s", "%s")' % (url, First_and_Last_Name))
NameError: name 'url' is not defined
蜘蛛网:
from scrapy.crawler import CrawlerProcess
import scrapy
import sqlite3
class Wellness(scrapy.Spider):
name = "wellness"
start_urls = ['https://www.wellness.com/dir/6022571/acupuncturist/ri/east-greenwich/hwasook-lee-phoenix-fertility-center-dac']
def parse(self, response):
item = {
'url' : response.request.url,
'First_and_Last_Name' : response.css('h1::text').get(),
}
directions_link = response.css('#directions_tab a::attr(href)').get()
yield scrapy.Request(response.urljoin(directions_link), callback=self.parse_directions, meta={'item': item})
def parse_directions(self, response):
item = response.meta['item']
item['Phone'] = response.css('.tel::text').get()
yield item
db = sqlite3.connect('posts.db')
db.execute('''CREATE TABLE IF NOT EXISTS posts(id INTEGER PRIMARY KEY AUTOINCREMENT,url TEXT, First_and_Last_Name TEXT);''')
db.commit()
cursor = db.cursor()
cursor.execute('INSERT INTO posts(url, First_and_Last_Name) VALUES("%s", "%s")' % (url, First_and_Last_Name))
db.commit()
db.close()
if __name__ == '__main__':
process = CrawlerProcess()
process.crawl(Wellness)
process.start()
我做错了什么。非常感谢您的帮助。谢谢 您想使用
项['url']
而不是url
,与首名和姓氏相同
:
cursor.execute('INSERT INTO posts(url、First_和Last_Name)值(“%s”、““%s”)”)%(项['url'],项['First_和Last_Name']))
另外,不要对SQL语句使用字符串格式,因为这会使代码容易受到SQL注入攻击。让该方法为您执行转义操作:
cursor.execute('INSERT INTO posts(url,First_和Last_Name)值(?,),(item['url'],item['First_和Last_Name']))
您使用的url
但是您在哪里定义了它??