Scrapy 我可以使用start_url来刮取url列表吗?
我有一个url列表,我想从中获取数据。它来自一个我想更新的数据库,但我不确定如何继续Scrapy 我可以使用start_url来刮取url列表吗?,scrapy,Scrapy,我有一个url列表,我想从中获取数据。它来自一个我想更新的数据库,但我不确定如何继续 import scrapy import sqlite3 from datetime import datetime, timedelta class A1hrlaterSpider(scrapy.Spider): name = 'onehrlater' allowed_domains = ['donedeal.ie'] timenow = datetime.now() del
import scrapy
import sqlite3
from datetime import datetime, timedelta
class A1hrlaterSpider(scrapy.Spider):
name = 'onehrlater'
allowed_domains = ['donedeal.ie']
timenow = datetime.now()
delta = timedelta(minutes=0)
delta2 = timedelta(minutes=1)
past_time = timenow - delta
past_time2 = timenow - delta2
conn = sqlite3.connect('ddother.db')
c = conn.cursor()
c.execute("SELECT adUrl FROM database WHERE timestamp BETWEEN ? AND ?", (past_time2, past_time))
all_urls = c.fetchall()
urllist = [item[0] for item in all_urls]
print(urllist)
conn.commit()
conn.close()
Urllist是我想要刮取的URL列表。但我不确定如何使用start\u URL
如果这确实是正确的方式,请遵循链接。我可以说start_url=urlist,还是这是错误的
任何帮助都将不胜感激。谢谢查看
start\u请求
。好的,我会的,谢谢Gallaecio查看start\u请求
。好的,我会的,谢谢Gallaecio