Warning: file_get_contents(/data/phpspider/zhask/data//catemap/9/javascript/399.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Javascript 套接字挂起foreach中的多个get调用_Javascript_Arrays_Node.js_Get_Fetch - Fatal编程技术网

Javascript 套接字挂起foreach中的多个get调用

Javascript 套接字挂起foreach中的多个get调用,javascript,arrays,node.js,get,fetch,Javascript,Arrays,Node.js,Get,Fetch,我试图从不同的url获取多个图像,所以我同时使用forEach和fetch(url)。我还尝试通过管道将它们传输到流,以将它们保存到磁盘。问题是在大约900个请求之后,请求计数急剧下降,最终出现错误。这是我使用的代码 const fetch = require('node-fetch') // savedImageURLS is an array of many image urls savedImageURLS.forEach((url) => { fetc

我试图从不同的url获取多个图像,所以我同时使用forEach和fetch(url)。我还尝试通过管道将它们传输到流,以将它们保存到磁盘。问题是在大约900个请求之后,请求计数急剧下降,最终出现错误。这是我使用的代码

   const fetch = require('node-fetch')
   // savedImageURLS is an array of many image urls
   savedImageURLS.forEach((url) => {
      fetch(url).then((res) => {
        console.log(count)
        const dest = fs.createWriteStream(`images/image-${count++}.png`);
        res.body.pipe(dest)
      }).catch((err) => {
        console.log('something went wrong', err)
      })
   })
我得到这个错误:

something went wrong { FetchError: request to https://scontent-yyz1-1.cdninstagram.com/vp/0b261beb7775c9accecc15ef782c2d9e/5E51A24B/t51.2885-15/e35/66648821_222609615365883_2370330998081102527_n.jpg?_nc_ht=scontent-yyz1-1.cdninstagram.com&_nc_cat=103&se=7&ig_cache_key=MjA4NTI1NTY3NTY5Mjk0NDk3NA%3D%3D.2 failed, reason: socket hang up
    at ClientRequest.<anonymous> (/Documents/batcave/instagram-saved-downloader/node_modules/node-fetch/lib/index.js:1455:11)
    at ClientRequest.emit (events.js:160:13)
    at TLSSocket.socketErrorListener (_http_client.js:389:9)
    at TLSSocket.emit (events.js:160:13)
    at emitErrorNT (internal/streams/destroy.js:64:8)
    at process._tickCallback (internal/process/next_tick.js:152:19)
  message: 'request to https://scontent-yyz1-1.cdninstagram.com/vp/0b261beb7775c9accecc15ef782c2d9e/5E51A24B/t51.2885-15/e35/66648821_222609615365883_2370330998081102527_n.jpg?_nc_ht=scontent-yyz1-1.cdninstagram.com&_nc_cat=103&se=7&ig_cache_key=MjA4NTI1NTY3NTY5Mjk0NDk3NA%3D%3D.2 failed, reason: socket hang up'
出现问题{FetchError:请求https://scontent-yyz1-1.cdninstagram.com/vp/0b261beb7775c9accecc15ef782c2d9e/5E51A24B/t51.2885-15/e35/66648821_222609615365883_2370330998081102527_n.jpg?_nc_ht=scontent-yyz1-1.cdnistagram.com&_nc_cat=103&se=7&ig_cache_key=MjA4NTI1NTY3NTY5Mjk0NDk3NA%3D%3D.2失败,原因:套接字挂起
在ClientRequest上。(/Documents/batcave/instagram saved downloader/node_modules/node fetch/lib/index.js:1455:11)
在ClientRequest.emit(events.js:160:13)
在TLSSocket.socketErrorListener(_http_client.js:389:9)
在TLSSocket.emit(events.js:160:13)
发出错误时(内部/streams/destroy.js:64:8)
在进程中。_tick回调(内部/process/next_tick.js:152:19)
信息:'请求https://scontent-yyz1-1.cdninstagram.com/vp/0b261beb7775c9accecc15ef782c2d9e/5E51A24B/t51.2885-15/e35/66648821_222609615365883_2370330998081102527_n.jpg?_nc_ht=scontent-yyz1-1.cdnistagram.com&_nc_cat=103&se=7&ig_cache_key=MjA4NTI1NTY3NTY5Mjk0NDk3NA%3D%3D.2失败,原因:套接字挂起'
我不知道出了什么问题。图像URL似乎不是无效的。get请求返回的速度不够快吗?我的内存不足吗?如果有任何帮助,我们将不胜感激


旁注:我也用节点https.get(url)尝试了这个方法当您使用node fetch调用映像时,需要记住的一点是,它是基于承诺的,这意味着它本质上是异步的。如果您进行了N次fetch调用,则节点请求I/O操作,从而阻止了一个线程执行操作。如果分配的线程数如果节点进程超过,您将始终得到此错误

另外,使用forEach循环请求900个映像是一个坏主意。这必然会阻塞节点队列。您可以将请求打包成900个的X(X>5)映像,并以串行方式进行900/X调用,以并行方式进行X调用(仍然不推荐这样做)