Python 2.7 在youtube上抓取私人播放列表的数据?
我用python中的scrapy制作了爬虫。但它不起作用。它似乎成功地进行了身份验证。但在这之后,它必须移动到我的私人列表,并抓取私人列表中剪辑的每个标题。但它不起作用 我认为,在它成功地进行身份验证之后,它会转到与我预期不同的另一页Python 2.7 在youtube上抓取私人播放列表的数据?,python-2.7,scrapy,scrapy-spider,Python 2.7,Scrapy,Scrapy Spider,我用python中的scrapy制作了爬虫。但它不起作用。它似乎成功地进行了身份验证。但在这之后,它必须移动到我的私人列表,并抓取私人列表中剪辑的每个标题。但它不起作用 我认为,在它成功地进行身份验证之后,它会转到与我预期不同的另一页 import scrapy from scrapy.http import Request, FormRequest from scrapy.selector import HtmlXPathSelector class LoginSpider(scrapy.
import scrapy
from scrapy.http import Request, FormRequest
from scrapy.selector import HtmlXPathSelector
class LoginSpider(scrapy.Spider):
name = 'youtube'
start_urls = ['https://accounts.google.com/ServiceLoginAuth']
def parse(self, response):
return scrapy.FormRequest.from_response(
response,
#formdata={'Email': 'xxx@gmail.com', 'Passwd': 'xxxx'},
callback=self.after_login
)
def after_login(self, response):
print "---------------------------------------------------------------------------"
# print response.xpath('//div[contains(@class, "yt-alert-message")]').extract()[1]
print response.headers["x-frame-options"]
print "---------------------------------------------------------------------------"
# check login succeed before going on
if "error-msg" in response.body:
self.logger.error("Login failed")
return
# We've successfully authenticated, let's have some fun!
else:
return Request(url="https://www.youtube.com/playlist?list=xxxxxxxx",
callback=self.parse_tastypage)
def parse_tastypage(self, response):
print response.xpath("//div[@class='pl-video-title']/a/@href").extract()
它不是为我登录的