Python 自动抓取日期为未来9天的动态URL
每周五,我都想用crontab从opetable.com网站上搜刮1000磅。以后预订两个星期天的房间。页面的URL是提交的结果Python 自动抓取日期为未来9天的动态URL,python,datetime,url,dynamic,web-scraping,Python,Datetime,Url,Dynamic,Web Scraping,每周五,我都想用crontab从opetable.com网站上搜刮1000磅。以后预订两个星期天的房间。页面的URL是提交的结果 html = urlopen('http://www.opentable.com/s/PopRestaurantList?covers=2¤tview=list&datetime=2016-03-06+19%3A00&metroid=15&onlypoptimes=true&personalizer=true&
html = urlopen('http://www.opentable.com/s/PopRestaurantList?covers=2¤tview=list&datetime=2016-03-06+19%3A00&metroid=15&onlypoptimes=true&personalizer=true&ref=2213&showmap=false&size=100&sort=Popularity')
使用urlparse或其他模块;datetime如何更改URL,使其始终具有刮板运行后9天的日期
bsObj = BeautifulSoup(html.read().decode('utf-8'),"lxml")
table = bsObj.findAll("div", {"class":"content-section-list infinite-results-list"})
headlines = table[0].findAll("a", {"class":"rest-row-name rest-name"})
for headline in headlines:
restaurant = (headline.get_text().encode('utf-8'))
print(restaurant)
print '\n'
我已将日期时间增量设置为9天
date = datetime.datetime.today()
date += datetime.timedelta(days=9)
print(date)
您可以使用字符串格式和date.strftime来执行此操作:
date = datetime.datetime.today()
date += datetime.timedelta(days=9)
url = 'http://www.opentable.com/s/PopRestaurantList?covers=2¤tview=list&datetime={date}+19%3A00&metroid=15&onlypoptimes=true&personalizer=true&ref=2213&showmap=false&size=100&sort=Popularity'
html = urlopen(url.format(date=date.strftime('%Y-%m-%d')))
当我尝试在打印后出现语法错误时(date.strftime(“%Y-%m-%d”)^SyntaxError:invalid syntaxi如果在urlopen之后有print语句,则我的示例代码中最后一行的末尾缺少一个paren,这可能是导致错误的原因