Javascript 如何使用ChereIO数据抓取来解析javascipt代码
我在网上看了视频,然后我想用不同的方式练习代码。 我想从网站上提取数据,然后如何解决JS代码的问题 从IMDB网站获取数据Javascript 如何使用ChereIO数据抓取来解析javascipt代码,javascript,jquery,node.js,cheerio,request-promise,Javascript,Jquery,Node.js,Cheerio,Request Promise,我在网上看了视频,然后我想用不同的方式练习代码。 我想从网站上提取数据,然后如何解决JS代码的问题 从IMDB网站获取数据 const option = { uri: "https://www.imdb.com/chart/moviemeter/?ref_=nv_mv_mpm", headers: { Accept: "text/html,application/xhtml+xml,applicatio
const option = {
uri: "https://www.imdb.com/chart/moviemeter/?ref_=nv_mv_mpm",
headers: {
Accept: "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "en-US,en;q=0.5",
},
json: true,
transform: (body) => cheerio.load(body),
};
await rp(option)
.then(($) => {
process.stdout.write("loading...\n");
let table = [];
$("tbody[class='lister-list'] tr").each((i, el) => {
//self call function
process.stdout.write(`.`);
let link =
"https://www.imdb.com" +
$(el).find("td[class='titleColumn'] a").attr("href");
goToNextPage(link, table);
});
console.table(table);
})
.catch((e) => console.log(e));
请求承诺代码
const option = {
uri: "https://www.imdb.com/chart/moviemeter/?ref_=nv_mv_mpm",
headers: {
Accept: "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "en-US,en;q=0.5",
},
json: true,
transform: (body) => cheerio.load(body),
};
await rp(option)
.then(($) => {
process.stdout.write("loading...\n");
let table = [];
$("tbody[class='lister-list'] tr").each((i, el) => {
//self call function
process.stdout.write(`.`);
let link =
"https://www.imdb.com" +
$(el).find("td[class='titleColumn'] a").attr("href");
goToNextPage(link, table);
});
console.table(table);
})
.catch((e) => console.log(e));
我使用参数调用了request promise JS代码中的函数goToNextPage
const goToNextPage = (link, table) => {
let $ = cheerio.load(link);
let title = $("div[class='title_wrapper'] > h1").text().trim(),
ratingValue = $("div[class='ratingValue'] > strong > span")
.text()
.trim(),
director = $("div[class='credit_summary_item'] > a")
.first()
.text()
.trim(),
filmType = $("div[class='subtext'] > a")
.not('a[title="See more release dates"]')
.text()
.replace(/([A-Z])/g, " $1")
.trim();
table.push({
title,
director,
filmType,
ratingValue,
});
};
您的代码生成带有以下内容的点:
process.stdout.write(`.`);
删除那部分代码,你就不会有点了