Javascript 导航到URL,但未得到响应
我使用casperjs,我有一系列链接,我打开了每个页面。阵列的循环始终停止。我似乎没有得到任何回应。我孤立的情况下,我只是无法获得信息从该链接在所有 它只是停了下来: 。。。 [信息][幻影]跑步套件:2个步骤 [debug][phantom]正在打开url:,HTTP GET 请求的[调试][幻影]导航:url=,类型=其他, willNavigate=true,isMainFrame=true 请求的[调试][幻影]导航:url=,类型=Oth 呃,willNavigate=true,isMainFrame=true 以下是隔离代码:Javascript 导航到URL,但未得到响应,javascript,node.js,web-scraping,phantomjs,casperjs,Javascript,Node.js,Web Scraping,Phantomjs,Casperjs,我使用casperjs,我有一系列链接,我打开了每个页面。阵列的循环始终停止。我似乎没有得到任何回应。我孤立的情况下,我只是无法获得信息从该链接在所有 它只是停了下来: 。。。 [信息][幻影]跑步套件:2个步骤 [debug][phantom]正在打开url:,HTTP GET 请求的[调试][幻影]导航:url=,类型=其他, willNavigate=true,isMainFrame=true 请求的[调试][幻影]导航:url=,类型=Oth 呃,willNavigate=true,is
var casper = require("casper").create({
verbose : true,
logLevel : "debug",
pageSettings: {
userAgent: "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36",
customHeaders: {
"Server": "Apache/2.4.1 (Unix)"
}
},
clientScripts: ["jquery.js"] ,
viewportSize: {
width: 1920,
height: 1080
}
});
casper.start().thenOpen("http://finishline.com")
.then(function(){
console.log("title : ", this.getTitle())
})
casper.run();
为什么我不能从finishline获得回复
如果在没有响应的情况下无法获得响应以继续使用代码,该如何处理?我使用Slimerjs运行了您的代码:
casperjs --engine=slimerjs my_casper_script.js
它运行时没有问题:
[info] [phantom] Starting...
[info] [phantom] Running suite: 2 steps
[debug] [phantom] opening url: http://finishline.com/, HTTP GET
[debug] [phantom] Navigation requested: url=http://finishline.com/, type=Undefined, willNavigate=true, isMainFrame=true
[debug] [phantom] url changed to "http://www.finishline.com/"
[debug] [phantom] Navigation requested: url=https://a7474150781.cdn.optimizely.com/client_storage/a7474150781.html, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=https://a7474150781.cdn.optimizely.com/client_storage/a7474150781.html, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=https://cdns.us1.gigya.com/gs/webSdk/Api.aspx?apiKey=3_wvYmaqsebd4zKyXKtXex5iT6qVdAQNm8T5Vjh1LhGavP4EApJp4T5CcdKuXozmAe#origin=http://www.finishline.com/&hasGmid=false, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=https://cdns.us1.gigya.com/gs/webSdk/Api.aspx?apiKey=3_wvYmaqsebd4zKyXKtXex5iT6qVdAQNm8T5Vjh1LhGavP4EApJp4T5CcdKuXozmAe#origin=http://www.finishline.com/&hasGmid=false, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=https://4978775.fls.doubleclick.net/activityi;src=4978775;type=aa;cat=links00;dc_lat=;dc_rdid=;tag_for_child_directed_treatment=;&ord=0.17442537640363165, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=https://4978775.fls.doubleclick.net/activityi;src=4978775;type=aa;cat=links00;dc_lat=;dc_rdid=;tag_for_child_directed_treatment=;&ord=0.17442537640363165, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=http://login.dotomi.com/ucm/UCMController?dtm_com=28&dtm_fid=101&dtm_cid=61247&dtm_cmagic=963c2b&dtm_format=5&cli_promo_id=1&dtm_user_id=o867995714&dtmc_ref=&dtmc_loc=http%3A//www.finishline.com/&dtm_user_token=, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=http://login.dotomi.com/ucm/UCMController?dtm_com=28&dtm_fid=101&dtm_cid=61247&dtm_cmagic=963c2b&dtm_format=5&cli_promo_id=1&dtm_user_id=o867995714&dtmc_ref=&dtmc_loc=http%3A//www.finishline.com/&dtm_user_token=, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=https://20725988p.rfihub.com/ca.html?rfiidc=1048283195230333342&rfiaid=8c49df89b5394c92b2630ec07269f9c3&ver=9&rb=24949&ca=20725988&_o=24949&_t=20725988&pe=https%3A%2F%2F4978775.fls.doubleclick.net%2Factivityi%3Bdc_pre%3DCPyZuc2nuNQCFXah7Qod9IsPqg%3Bsrc%3D4978775%3Btype%3Daa%3Bcat%3Dlinks00%3Bdc_lat%3D%3Bdc_rdid%3D%3Btag_for_child_directed_treatment%3D%3B%26ord%3D0.17442537640363165&pf=http%3A%2F%2Fwww.finishline.com%2F&ra=82836050341159, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=https://20725988p.rfihub.com/ca.html?rfiidc=1048283195230333342&rfiaid=8c49df89b5394c92b2630ec07269f9c3&ver=9&rb=24949&ca=20725988&_o=24949&_t=20725988&pe=https%3A%2F%2F4978775.fls.doubleclick.net%2Factivityi%3Bdc_pre%3DCPyZuc2nuNQCFXah7Qod9IsPqg%3Bsrc%3D4978775%3Btype%3Daa%3Bcat%3Dlinks00%3Bdc_lat%3D%3Bdc_rdid%3D%3Btag_for_child_directed_treatment%3D%3B%26ord%3D0.17442537640363165&pf=http%3A%2F%2Fwww.finishline.com%2F&ra=82836050341159, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=http://dis.us.criteo.com/dis/dis.aspx?p=3616&cb=23181624377&ref=&sc_r=1280x1024&sc_d=24, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=http://dis.us.criteo.com/dis/dis.aspx?p=3616&cb=23181624377&ref=&sc_r=1280x1024&sc_d=24, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Automatically injected jquery.js client side
[debug] [phantom] Successfully injected Casper client-side utilities
[info] [phantom] Step anonymous 2/2 http://www.finishline.com/ (HTTP 301)
title : Finish Line: Shoes, Sneakers & Athletic Gear
[info] [phantom] Step anonymous 2/2: done in 7686ms.
[info] [phantom] Done 2 steps in 7694ms
当我使用phantomjs作为引擎运行您的代码时,我得到了与您相同的结果。
有关如何安装slimerjs的说明如下。
但如果您安装了npm,则可以使用此命令安装它
npm install slimerjs -g
希望这能有所帮助 我使用Slimerjs运行了您的代码:
casperjs --engine=slimerjs my_casper_script.js
它运行时没有问题:
[info] [phantom] Starting...
[info] [phantom] Running suite: 2 steps
[debug] [phantom] opening url: http://finishline.com/, HTTP GET
[debug] [phantom] Navigation requested: url=http://finishline.com/, type=Undefined, willNavigate=true, isMainFrame=true
[debug] [phantom] url changed to "http://www.finishline.com/"
[debug] [phantom] Navigation requested: url=https://a7474150781.cdn.optimizely.com/client_storage/a7474150781.html, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=https://a7474150781.cdn.optimizely.com/client_storage/a7474150781.html, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=https://cdns.us1.gigya.com/gs/webSdk/Api.aspx?apiKey=3_wvYmaqsebd4zKyXKtXex5iT6qVdAQNm8T5Vjh1LhGavP4EApJp4T5CcdKuXozmAe#origin=http://www.finishline.com/&hasGmid=false, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=https://cdns.us1.gigya.com/gs/webSdk/Api.aspx?apiKey=3_wvYmaqsebd4zKyXKtXex5iT6qVdAQNm8T5Vjh1LhGavP4EApJp4T5CcdKuXozmAe#origin=http://www.finishline.com/&hasGmid=false, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=https://4978775.fls.doubleclick.net/activityi;src=4978775;type=aa;cat=links00;dc_lat=;dc_rdid=;tag_for_child_directed_treatment=;&ord=0.17442537640363165, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=https://4978775.fls.doubleclick.net/activityi;src=4978775;type=aa;cat=links00;dc_lat=;dc_rdid=;tag_for_child_directed_treatment=;&ord=0.17442537640363165, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=http://login.dotomi.com/ucm/UCMController?dtm_com=28&dtm_fid=101&dtm_cid=61247&dtm_cmagic=963c2b&dtm_format=5&cli_promo_id=1&dtm_user_id=o867995714&dtmc_ref=&dtmc_loc=http%3A//www.finishline.com/&dtm_user_token=, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=http://login.dotomi.com/ucm/UCMController?dtm_com=28&dtm_fid=101&dtm_cid=61247&dtm_cmagic=963c2b&dtm_format=5&cli_promo_id=1&dtm_user_id=o867995714&dtmc_ref=&dtmc_loc=http%3A//www.finishline.com/&dtm_user_token=, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=https://20725988p.rfihub.com/ca.html?rfiidc=1048283195230333342&rfiaid=8c49df89b5394c92b2630ec07269f9c3&ver=9&rb=24949&ca=20725988&_o=24949&_t=20725988&pe=https%3A%2F%2F4978775.fls.doubleclick.net%2Factivityi%3Bdc_pre%3DCPyZuc2nuNQCFXah7Qod9IsPqg%3Bsrc%3D4978775%3Btype%3Daa%3Bcat%3Dlinks00%3Bdc_lat%3D%3Bdc_rdid%3D%3Btag_for_child_directed_treatment%3D%3B%26ord%3D0.17442537640363165&pf=http%3A%2F%2Fwww.finishline.com%2F&ra=82836050341159, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=https://20725988p.rfihub.com/ca.html?rfiidc=1048283195230333342&rfiaid=8c49df89b5394c92b2630ec07269f9c3&ver=9&rb=24949&ca=20725988&_o=24949&_t=20725988&pe=https%3A%2F%2F4978775.fls.doubleclick.net%2Factivityi%3Bdc_pre%3DCPyZuc2nuNQCFXah7Qod9IsPqg%3Bsrc%3D4978775%3Btype%3Daa%3Bcat%3Dlinks00%3Bdc_lat%3D%3Bdc_rdid%3D%3Btag_for_child_directed_treatment%3D%3B%26ord%3D0.17442537640363165&pf=http%3A%2F%2Fwww.finishline.com%2F&ra=82836050341159, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=http://dis.us.criteo.com/dis/dis.aspx?p=3616&cb=23181624377&ref=&sc_r=1280x1024&sc_d=24, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Navigation requested: url=http://dis.us.criteo.com/dis/dis.aspx?p=3616&cb=23181624377&ref=&sc_r=1280x1024&sc_d=24, type=Undefined, willNavigate=true, isMainFrame=false
[debug] [phantom] Automatically injected jquery.js client side
[debug] [phantom] Successfully injected Casper client-side utilities
[info] [phantom] Step anonymous 2/2 http://www.finishline.com/ (HTTP 301)
title : Finish Line: Shoes, Sneakers & Athletic Gear
[info] [phantom] Step anonymous 2/2: done in 7686ms.
[info] [phantom] Done 2 steps in 7694ms
当我使用phantomjs作为引擎运行您的代码时,我得到了与您相同的结果。
有关如何安装slimerjs的说明如下。
但如果您安装了npm,则可以使用此命令安装它
npm install slimerjs -g
希望这能有所帮助 出于好奇,为什么在请求时您会发送这些实际上应该是响应头的自定义头?出于好奇,为什么在请求时您会发送这些实际上应该是响应头的自定义头?我会在有机会时对此进行测试。谢谢你的输入。哦,如果你在使用Ubuntu,我写了一个bash脚本,可以自动安装casperjs、slimerjs、phantomjs和必要的图形引擎,使slimerjs完全无头。你可以得到它,我会在有机会的时候测试它。谢谢你的输入。哦,如果你使用Ubuntu,我写了一个bash脚本,可以自动安装casperjs、slimerjs、phantomjs和必要的图形引擎,使slimerjs完全无头,你可以得到它