Warning: file_get_contents(/data/phpspider/zhask/data//catemap/6/rest/5.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Javascript 如何按顺序生成page.open()请求?_Javascript_Phantomjs_Httprequest - Fatal编程技术网

Javascript 如何按顺序生成page.open()请求?

Javascript 如何按顺序生成page.open()请求?,javascript,phantomjs,httprequest,Javascript,Phantomjs,Httprequest,我写了这个,但它仍然是非顺序的。我希望在函数中,它会等到实际请求完成后,再调用新的请求。。。。但这不起作用 问题1:页面.open()调用不是顺序调用,如您在此处所见: 6 protocol: https: type: Content 7 protocol: https: type: Content 8 protocol: https: type: Content 9 protocol: https: type: C

我写了这个,但它仍然是非顺序的。我希望在函数中,它会等到实际请求完成后,再调用新的请求。。。。但这不起作用

问题1:页面.open()调用不是顺序调用,如您在此处所见:

6 protocol: https: type: Content 7 protocol: https: type: Content 8 protocol: https: type: Content 9 protocol: https: type: Content LINE: https://www.roller.de/einrichten/ 10 protocol: https: type: Content 11 protocol: https: type: Content 12 protocol: https: type: Content LINE: https://www.roller.de/einrichten/anrichte/ LINE: https://www.roller.de/einrichten/arbeitsstuhl/ LINE: https://www.roller.de/einrichten/arbeitstisch/ LINE: https://www.roller.de/einrichten/armlehnstuehle/ LINE: https://www.roller.de/einrichten/badezimmermoebel LINE: https://www.roller.de/einrichten/bistrostuehle/ LINE: https://www.roller.de/einrichten/buecherregal/ 13 protocol: https: type: Content 14 protocol: https: type: Content 15 protocol: https: type: Content 16 protocol: https: type: Content LINE: https://www.roller.de/einrichten/buerocontainer/ LINE: https://www.roller.de/einrichten/bueroregale/ 17 protocol: https: type: Content 18 protocol: https: type: Content
现在它可以使用这段代码,iFrames似乎会触发onLoadFinished()两次,所以我用hasOnLoadFinished检查它,以防止函数中出现多个条目(在PhantomJS中一次使用多个page.open()是一个非常糟糕的主意)

请注意,由于未知原因(大多数情况下没有错误消息,很少出现“QThread::start:未能创建线程()”),2.0将因太多链接/太多URL(在我的示例中为120-180)而崩溃

为了防止这种情况,使用1.9.8版本而不是2.0版本,这似乎是一个bug,在Github上用转储文件填充了一份崩溃报告

/编辑在3836个链接和1.9.8个链接后崩溃,但未显示错误消息………..幻影

console.log('Hello, world!');
var fs = require('fs');
var stream = fs.open('linklist.de.txt', 'r');
var webPage = require('webpage');
var i = 1;
var hasFound = Array();
var hasonLoadFinished = Array();

function handle_page(link) {
var page = webPage.create();
page.settings.loadImages = false;
page.open(link, function() {});

page.onResourceRequested = function(requestData, request) {
    var match = requestData.url.match(/example.com\/searchmeI'maString/g)
    if (match != null) {
        hasFound[link] = true;
        var targetString = decodeURI(JSON.stringify(requestData.url));
        var klammerauf = targetString.indexOf("{");
        var jsonobjekt = targetString.substr(klammerauf,     (targetString.indexOf("}") - klammerauf) + 1);
        targetJSON = (decodeURIComponent(jsonobjekt));
        var t = JSON.parse(targetJSON);
        console.log(i + "   " + t + "       " + t['id']);
        //console.log(targetJSON);
        //console.log("");
        request.abort;
    } else {
        request.abort;
        return;
    }

};
page.onLoadFinished = function(status) {    
    if (!hasonLoadFinished[link]) {
        hasonLoadFinished[link] = true;
        //console.log(" " + status + "  " + link);
        //console.log("onLoadFinished()")   
        //setTimeout(function(){/* Look mah! No name! */},1000);
        if (!hasFound[link]) {
            console.log(i + " :NOT FOUND: " + link);
            console.log("");
        }
        i++;
        page.close();
        nextPage();
    }
}
};

function nextPage() {
var link = stream.readLine();
if (!link) {
    end = Date.now();
    console.log("");
    console.log(((end - start) / 1000) + " Sekunden");
    console.log("FILE ENDS HERE!!!");
    phantom.exit(0);
}
hasFound[link] = false;
hasonLoadFinished[link] = false;
handle_page(link);
}

start = Date.now();
nextPage();

现在它可以使用这段代码,iFrames似乎会触发onLoadFinished()两次,所以我用hasOnLoadFinished检查它,以防止函数中出现多个条目(在PhantomJS中一次使用多个page.open()是一个非常糟糕的主意)

请注意,由于未知原因(大多数情况下没有错误消息,很少出现“QThread::start:未能创建线程()”),2.0将因太多链接/太多URL(在我的示例中为120-180)而崩溃

为了防止这种情况,使用1.9.8版本而不是2.0版本,这似乎是一个bug,在Github上用转储文件填充了一份崩溃报告

/编辑在3836个链接和1.9.8个链接后崩溃,但未显示错误消息………..幻影

console.log('Hello, world!');
var fs = require('fs');
var stream = fs.open('linklist.de.txt', 'r');
var webPage = require('webpage');
var i = 1;
var hasFound = Array();
var hasonLoadFinished = Array();

function handle_page(link) {
var page = webPage.create();
page.settings.loadImages = false;
page.open(link, function() {});

page.onResourceRequested = function(requestData, request) {
    var match = requestData.url.match(/example.com\/searchmeI'maString/g)
    if (match != null) {
        hasFound[link] = true;
        var targetString = decodeURI(JSON.stringify(requestData.url));
        var klammerauf = targetString.indexOf("{");
        var jsonobjekt = targetString.substr(klammerauf,     (targetString.indexOf("}") - klammerauf) + 1);
        targetJSON = (decodeURIComponent(jsonobjekt));
        var t = JSON.parse(targetJSON);
        console.log(i + "   " + t + "       " + t['id']);
        //console.log(targetJSON);
        //console.log("");
        request.abort;
    } else {
        request.abort;
        return;
    }

};
page.onLoadFinished = function(status) {    
    if (!hasonLoadFinished[link]) {
        hasonLoadFinished[link] = true;
        //console.log(" " + status + "  " + link);
        //console.log("onLoadFinished()")   
        //setTimeout(function(){/* Look mah! No name! */},1000);
        if (!hasFound[link]) {
            console.log(i + " :NOT FOUND: " + link);
            console.log("");
        }
        i++;
        page.close();
        nextPage();
    }
}
};

function nextPage() {
var link = stream.readLine();
if (!link) {
    end = Date.now();
    console.log("");
    console.log(((end - start) / 1000) + " Sekunden");
    console.log("FILE ENDS HERE!!!");
    phantom.exit(0);
}
hasFound[link] = false;
hasonLoadFinished[link] = false;
handle_page(link);
}

start = Date.now();
nextPage();
在if(match!=null)中添加一个else,您可能会得到100张打印,然后在if(match!=null)中添加一个else,您可能会得到100张打印