Javascript 木偶演员:将DefaultNavigationTimeout设置为0仍然超时
每次我运行这个脚本都超时了。Javascript 木偶演员:将DefaultNavigationTimeout设置为0仍然超时,javascript,puppeteer,Javascript,Puppeteer,每次我运行这个脚本都超时了。 setDefaultNavigationTimeout是否确实阻止超时 我浏览了大约26个URL,每个页面都有大量的图片。无法想象木偶演员仅仅因为沉重的图像而无法处理这些页面 const url = 'test.com'; const jsonReturn = []; async function runScraper() { const browser = await puppeteer.launch(prodConfig); const page
setDefaultNavigationTimeout
是否确实阻止超时
我浏览了大约26个URL,每个页面都有大量的图片。无法想象木偶演员仅仅因为沉重的图像而无法处理这些页面
const url = 'test.com';
const jsonReturn = [];
async function runScraper() {
const browser = await puppeteer.launch(prodConfig);
const page = await browser.newPage({
timeout: 0
});
page.setDefaultNavigationTimeout(0);
await page.goto(url, { waitUntil: 'domcontentloaded' });
await page.waitForSelector('.featured-shows-featured-show');
let featuredShowsURLs = await page.$$eval('.featured-shows-featured-show > a', (links) => {
return links.map(link => {
return link.href;
});
});
featuredShowsURLs = _.uniq(featuredShowsURLs)
for (const featuredShowsURL of featuredShowsURLs) {
const page = await browser.newPage({
timeout: 0
});
try {
await page.goto(featuredShowsURL);
await page.waitForSelector('.show-title');
} catch (e) {
featuredShowsURL;
debugger;
}
const showTitle = await findAndReturnSelectorText('.show-title', page);
const showDates = await findAndReturnSelectorText('.show-dates', page);
const showLocation = await findAndReturnSelectorText('.show-location', page);
const showGallery = await findAndReturnSelectorText('.entity-link', page);
const showDetail = await findAndReturnSelectorText('.show-press-release', page);
const newItem = {
showTitle,
showDates,
showLocation,
showGallery,
showDetail,
};
const id = hash(newItem);
jsonReturn.push({
...newItem,
id
});
}
await browser.close();
}
runScraper();
你从哪里得到超时时间?您可以共享此消息吗?如果您从
page.waitForSelector()
获得超时,您需要消除此问题。