Warning: file_get_contents(/data/phpspider/zhask/data//catemap/9/javascript/455.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Javascript 导航到URL,但未得到响应_Javascript_Node.js_Web Scraping_Phantomjs_Casperjs - Fatal编程技术网

Javascript 导航到URL,但未得到响应

Javascript 导航到URL,但未得到响应,javascript,node.js,web-scraping,phantomjs,casperjs,Javascript,Node.js,Web Scraping,Phantomjs,Casperjs,我使用casperjs,我有一系列链接,我打开了每个页面。阵列的循环始终停止。我似乎没有得到任何回应。我孤立的情况下,我只是无法获得信息从该链接在所有 它只是停了下来: 。。。 [信息][幻影]跑步套件:2个步骤 [debug][phantom]正在打开url:,HTTP GET 请求的[调试][幻影]导航:url=,类型=其他, willNavigate=true,isMainFrame=true 请求的[调试][幻影]导航:url=,类型=Oth 呃,willNavigate=true,is

我使用casperjs,我有一系列链接,我打开了每个页面。阵列的循环始终停止。我似乎没有得到任何回应。我孤立的情况下,我只是无法获得信息从该链接在所有

它只是停了下来:

。。。 [信息][幻影]跑步套件:2个步骤 [debug][phantom]正在打开url:,HTTP GET 请求的[调试][幻影]导航:url=,类型=其他, willNavigate=true,isMainFrame=true 请求的[调试][幻影]导航:url=,类型=Oth 呃,willNavigate=true,isMainFrame=true

以下是隔离代码:

var casper = require("casper").create({
    verbose : true,
    logLevel : "debug",
    pageSettings: {
        userAgent: "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36",
         customHeaders: {
            "Server": "Apache/2.4.1 (Unix)"
        }
     },
    clientScripts: ["jquery.js"] ,
    viewportSize: {
        width: 1920,
        height: 1080
    }
});

casper.start().thenOpen("http://finishline.com")
.then(function(){
    console.log("title : ", this.getTitle())
})
casper.run();
为什么我不能从finishline获得回复


如果在没有响应的情况下无法获得响应以继续使用代码,该如何处理?

我使用Slimerjs运行了您的代码:

casperjs --engine=slimerjs my_casper_script.js
它运行时没有问题:

    [info] [phantom] Starting... 
[info] [phantom] Running suite: 2 steps 
[debug] [phantom] opening url: http://finishline.com/, HTTP GET 
[debug] [phantom] Navigation requested: url=http://finishline.com/, type=Undefined, willNavigate=true, isMainFrame=true 
[debug] [phantom] url changed to "http://www.finishline.com/" 
[debug] [phantom] Navigation requested: url=https://a7474150781.cdn.optimizely.com/client_storage/a7474150781.html, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=https://a7474150781.cdn.optimizely.com/client_storage/a7474150781.html, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=https://cdns.us1.gigya.com/gs/webSdk/Api.aspx?apiKey=3_wvYmaqsebd4zKyXKtXex5iT6qVdAQNm8T5Vjh1LhGavP4EApJp4T5CcdKuXozmAe#origin=http://www.finishline.com/&hasGmid=false, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=https://cdns.us1.gigya.com/gs/webSdk/Api.aspx?apiKey=3_wvYmaqsebd4zKyXKtXex5iT6qVdAQNm8T5Vjh1LhGavP4EApJp4T5CcdKuXozmAe#origin=http://www.finishline.com/&hasGmid=false, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=https://4978775.fls.doubleclick.net/activityi;src=4978775;type=aa;cat=links00;dc_lat=;dc_rdid=;tag_for_child_directed_treatment=;&ord=0.17442537640363165, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=https://4978775.fls.doubleclick.net/activityi;src=4978775;type=aa;cat=links00;dc_lat=;dc_rdid=;tag_for_child_directed_treatment=;&ord=0.17442537640363165, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=http://login.dotomi.com/ucm/UCMController?dtm_com=28&dtm_fid=101&dtm_cid=61247&dtm_cmagic=963c2b&dtm_format=5&cli_promo_id=1&dtm_user_id=o867995714&dtmc_ref=&dtmc_loc=http%3A//www.finishline.com/&dtm_user_token=, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=http://login.dotomi.com/ucm/UCMController?dtm_com=28&dtm_fid=101&dtm_cid=61247&dtm_cmagic=963c2b&dtm_format=5&cli_promo_id=1&dtm_user_id=o867995714&dtmc_ref=&dtmc_loc=http%3A//www.finishline.com/&dtm_user_token=, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=https://20725988p.rfihub.com/ca.html?rfiidc=1048283195230333342&rfiaid=8c49df89b5394c92b2630ec07269f9c3&ver=9&rb=24949&ca=20725988&_o=24949&_t=20725988&pe=https%3A%2F%2F4978775.fls.doubleclick.net%2Factivityi%3Bdc_pre%3DCPyZuc2nuNQCFXah7Qod9IsPqg%3Bsrc%3D4978775%3Btype%3Daa%3Bcat%3Dlinks00%3Bdc_lat%3D%3Bdc_rdid%3D%3Btag_for_child_directed_treatment%3D%3B%26ord%3D0.17442537640363165&pf=http%3A%2F%2Fwww.finishline.com%2F&ra=82836050341159, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=https://20725988p.rfihub.com/ca.html?rfiidc=1048283195230333342&rfiaid=8c49df89b5394c92b2630ec07269f9c3&ver=9&rb=24949&ca=20725988&_o=24949&_t=20725988&pe=https%3A%2F%2F4978775.fls.doubleclick.net%2Factivityi%3Bdc_pre%3DCPyZuc2nuNQCFXah7Qod9IsPqg%3Bsrc%3D4978775%3Btype%3Daa%3Bcat%3Dlinks00%3Bdc_lat%3D%3Bdc_rdid%3D%3Btag_for_child_directed_treatment%3D%3B%26ord%3D0.17442537640363165&pf=http%3A%2F%2Fwww.finishline.com%2F&ra=82836050341159, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=http://dis.us.criteo.com/dis/dis.aspx?p=3616&cb=23181624377&ref=&sc_r=1280x1024&sc_d=24, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=http://dis.us.criteo.com/dis/dis.aspx?p=3616&cb=23181624377&ref=&sc_r=1280x1024&sc_d=24, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Automatically injected jquery.js client side 
[debug] [phantom] Successfully injected Casper client-side utilities 
[info] [phantom] Step anonymous 2/2 http://www.finishline.com/ (HTTP 301) 
title :  Finish Line: Shoes, Sneakers & Athletic Gear 
[info] [phantom] Step anonymous 2/2: done in 7686ms. 
[info] [phantom] Done 2 steps in 7694ms 
当我使用phantomjs作为引擎运行您的代码时,我得到了与您相同的结果。 有关如何安装slimerjs的说明如下。 但如果您安装了npm,则可以使用此命令安装它

npm install slimerjs -g

希望这能有所帮助

我使用Slimerjs运行了您的代码:

casperjs --engine=slimerjs my_casper_script.js
它运行时没有问题:

    [info] [phantom] Starting... 
[info] [phantom] Running suite: 2 steps 
[debug] [phantom] opening url: http://finishline.com/, HTTP GET 
[debug] [phantom] Navigation requested: url=http://finishline.com/, type=Undefined, willNavigate=true, isMainFrame=true 
[debug] [phantom] url changed to "http://www.finishline.com/" 
[debug] [phantom] Navigation requested: url=https://a7474150781.cdn.optimizely.com/client_storage/a7474150781.html, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=https://a7474150781.cdn.optimizely.com/client_storage/a7474150781.html, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=https://cdns.us1.gigya.com/gs/webSdk/Api.aspx?apiKey=3_wvYmaqsebd4zKyXKtXex5iT6qVdAQNm8T5Vjh1LhGavP4EApJp4T5CcdKuXozmAe#origin=http://www.finishline.com/&hasGmid=false, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=https://cdns.us1.gigya.com/gs/webSdk/Api.aspx?apiKey=3_wvYmaqsebd4zKyXKtXex5iT6qVdAQNm8T5Vjh1LhGavP4EApJp4T5CcdKuXozmAe#origin=http://www.finishline.com/&hasGmid=false, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=https://4978775.fls.doubleclick.net/activityi;src=4978775;type=aa;cat=links00;dc_lat=;dc_rdid=;tag_for_child_directed_treatment=;&ord=0.17442537640363165, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=https://4978775.fls.doubleclick.net/activityi;src=4978775;type=aa;cat=links00;dc_lat=;dc_rdid=;tag_for_child_directed_treatment=;&ord=0.17442537640363165, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=http://login.dotomi.com/ucm/UCMController?dtm_com=28&dtm_fid=101&dtm_cid=61247&dtm_cmagic=963c2b&dtm_format=5&cli_promo_id=1&dtm_user_id=o867995714&dtmc_ref=&dtmc_loc=http%3A//www.finishline.com/&dtm_user_token=, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=http://login.dotomi.com/ucm/UCMController?dtm_com=28&dtm_fid=101&dtm_cid=61247&dtm_cmagic=963c2b&dtm_format=5&cli_promo_id=1&dtm_user_id=o867995714&dtmc_ref=&dtmc_loc=http%3A//www.finishline.com/&dtm_user_token=, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=https://20725988p.rfihub.com/ca.html?rfiidc=1048283195230333342&rfiaid=8c49df89b5394c92b2630ec07269f9c3&ver=9&rb=24949&ca=20725988&_o=24949&_t=20725988&pe=https%3A%2F%2F4978775.fls.doubleclick.net%2Factivityi%3Bdc_pre%3DCPyZuc2nuNQCFXah7Qod9IsPqg%3Bsrc%3D4978775%3Btype%3Daa%3Bcat%3Dlinks00%3Bdc_lat%3D%3Bdc_rdid%3D%3Btag_for_child_directed_treatment%3D%3B%26ord%3D0.17442537640363165&pf=http%3A%2F%2Fwww.finishline.com%2F&ra=82836050341159, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=https://20725988p.rfihub.com/ca.html?rfiidc=1048283195230333342&rfiaid=8c49df89b5394c92b2630ec07269f9c3&ver=9&rb=24949&ca=20725988&_o=24949&_t=20725988&pe=https%3A%2F%2F4978775.fls.doubleclick.net%2Factivityi%3Bdc_pre%3DCPyZuc2nuNQCFXah7Qod9IsPqg%3Bsrc%3D4978775%3Btype%3Daa%3Bcat%3Dlinks00%3Bdc_lat%3D%3Bdc_rdid%3D%3Btag_for_child_directed_treatment%3D%3B%26ord%3D0.17442537640363165&pf=http%3A%2F%2Fwww.finishline.com%2F&ra=82836050341159, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=http://dis.us.criteo.com/dis/dis.aspx?p=3616&cb=23181624377&ref=&sc_r=1280x1024&sc_d=24, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Navigation requested: url=http://dis.us.criteo.com/dis/dis.aspx?p=3616&cb=23181624377&ref=&sc_r=1280x1024&sc_d=24, type=Undefined, willNavigate=true, isMainFrame=false 
[debug] [phantom] Automatically injected jquery.js client side 
[debug] [phantom] Successfully injected Casper client-side utilities 
[info] [phantom] Step anonymous 2/2 http://www.finishline.com/ (HTTP 301) 
title :  Finish Line: Shoes, Sneakers & Athletic Gear 
[info] [phantom] Step anonymous 2/2: done in 7686ms. 
[info] [phantom] Done 2 steps in 7694ms 
当我使用phantomjs作为引擎运行您的代码时,我得到了与您相同的结果。 有关如何安装slimerjs的说明如下。 但如果您安装了npm,则可以使用此命令安装它

npm install slimerjs -g

希望这能有所帮助

出于好奇,为什么在请求时您会发送这些实际上应该是响应头的自定义头?出于好奇,为什么在请求时您会发送这些实际上应该是响应头的自定义头?我会在有机会时对此进行测试。谢谢你的输入。哦,如果你在使用Ubuntu,我写了一个bash脚本,可以自动安装casperjs、slimerjs、phantomjs和必要的图形引擎,使slimerjs完全无头。你可以得到它,我会在有机会的时候测试它。谢谢你的输入。哦,如果你使用Ubuntu,我写了一个bash脚本,可以自动安装casperjs、slimerjs、phantomjs和必要的图形引擎,使slimerjs完全无头,你可以得到它