节点:致命错误:调用和重试上次分配失败-JavaScript堆内存不足

节点:致命错误:调用和重试上次分配失败-JavaScript堆内存不足,javascript,json,node.js,Javascript,Json,Node.js,我正在生成一个大对象,并试图将它们安全地保存到一个.json文件中。但是,即使使用流式处理,对象似乎正在缓存堆,并且从未被释放 这就是我调用的函数: saveObjectAsJSON (o: any, filename: string) { const content = JSON.stringify(o); const wstream = fs.createWriteStream(filename); wstream.write(content); wstrea

我正在生成一个大对象,并试图将它们安全地保存到一个
.json
文件中。但是,即使使用流式处理,对象似乎正在缓存堆,并且从未被释放

这就是我调用的函数:

saveObjectAsJSON (o: any, filename: string) {
    const content = JSON.stringify(o);
    const wstream = fs.createWriteStream(filename);
    wstream.write(content);
    wstream.end();
}
它适用于较小的对象,但我现在正在生成一个相当大的对象(外部级别大约有100万个对象)。这是否达到节点的容量和能力?我应该使用Mongo或其他东西来存储/检索它们吗

错误跟踪:

<--- Last few GCs --->

[6583:0x102801e00]   474566 ms: Mark-sweep 2948.9 (4039.7) -> 2948.8 (4024.7) MB, 6035.6 / 0.0 ms  allocation failure GC in old space requested
[6583:0x102801e00]   480764 ms: Mark-sweep 2948.8 (4024.7) -> 2948.8 (3993.7) MB, 6198.0 / 0.0 ms  last resort GC in old space requested
[6583:0x102801e00]   487025 ms: Mark-sweep 2948.8 (3993.7) -> 2948.8 (3963.7) MB, 6260.8 / 0.0 ms  last resort GC in old space requested
<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x934a4ba5529 <JSObject>
    1: fromString(aka fromString) [buffer.js:336] [bytecode=0x9340cbd6b51 offset=161](this=0x934c6e822d1 <undefined>,string=0x9350da7d189 <Very long string[936824012]>,encoding=0x934a4bb6419 <String[4]: utf8>)
    2: from [buffer.js:189] [bytecode=0x9340cbd6611 offset=14](this=0x934616bd3b1 <JSFunction Buffer (sfi = 0x934e8b83821)>,value=0x9350da7d189 <Very long string[936824012]>,encodingOrOffse...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/usr/local/bin/node]
 2: node::FatalTryCatch::~FatalTryCatch() [/usr/local/bin/node]
 3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/local/bin/node]
 4: v8::internal::Factory::NewRawOneByteString(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
 5: v8::internal::String::SlowFlatten(v8::internal::Handle<v8::internal::ConsString>, v8::internal::PretenureFlag) [/usr/local/bin/node]
 6: v8::String::WriteUtf8(char*, int, int*, int) const [/usr/local/bin/node]
 7: node::StringBytes::Write(v8::Isolate*, char*, unsigned long, v8::Local<v8::Value>, node::encoding, int*) [/usr/local/bin/node]
 8: node::Buffer::New(v8::Isolate*, v8::Local<v8::String>, node::encoding) [/usr/local/bin/node]
 9: node::Buffer::(anonymous namespace)::CreateFromString(v8::FunctionCallbackInfo<v8::Value> const&) [/usr/local/bin/node]
10: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) [/usr/local/bin/node]
11: v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) [/usr/local/bin/node]
12: v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) [/usr/local/bin/node]
13: 0x39ffa47042fd
14: 0x39ffa47bd196

[6583:0x102801e00]474566毫秒:标记扫描2948.9(4039.7)->2948.8(4024.7)MB,6035.6/0.0毫秒请求的旧空间中的GC分配失败
[6583:0x102801e00]480764毫秒:标记扫描2948.8(4024.7)->2948.8(3993.7)MB,6198.0/0.0毫秒请求旧空间中的最后一次GC
[6583:0x102801e00]487025毫秒:标记扫描2948.8(3993.7)->2948.8(3963.7)MB,6260.8/0.0毫秒请求的旧空间中的最后一次GC
==JS堆栈跟踪=========================================
安全上下文:0x934a4ba5529
1:fromString(又称fromString)[buffer.js:336][bytecode=0x9340cbd6b51 offset=161](this=0x934c6e822d1,string=0x9350da7d189,encoding=0x934a4bb6419)
2:from[buffer.js:189][bytecode=0x9340cbd6611 offset=14](this=0x934616bd3b1,value=0x9350da7d189,encodingOrOffse。。。
致命错误:调用和重试上次分配失败-JavaScript堆内存不足
1:node::Abort()[/usr/local/bin/node]
2:node::FatalTryCatch::~FatalTryCatch()[/usr/local/bin/node]
3:v8::internal::v8::FatalProcessOutOfMemory(char const*,bool)[/usr/local/bin/node]
4:v8::internal::Factory::NewRawOneByteString(int,v8::internal::PreforeFlag)[/usr/local/bin/node]
5:v8::internal::String::SlowFlatten(v8::internal::Handle,v8::internal::PreforeFlag)[/usr/local/bin/node]
6:v8::String::WriteUtf8(char*,int,int*,int)常量[/usr/local/bin/node]
7:node::StringBytes::Write(v8::Isolate*,char*,无符号长,v8::Local,node::encoding,int*)[/usr/Local/bin/node]
8:node::Buffer::New(v8::Isolate*,v8::Local,node::encoding)[/usr/Local/bin/node]
9:node::Buffer::(匿名命名空间)::CreateFromString(v8::FunctionCallbackInfo const&)[/usr/local/bin/node]
10:v8::internal::FunctionCallbackArguments::Call(void(*)(v8::FunctionCallbackInfo const&)[/usr/local/bin/node]
11:v8::internal::MaybeHandle v8::internal::(匿名命名空间)::HandLeapIcalHelper(v8::internal::Isolate*,v8::internal::Handle,v8::internal::Handle,v8::internal::BuiltinArguments)[/usr/local/bin/node]
12:v8::internal::Builtin_Impl_HandleapCall(v8::internal::BuiltinaGuments,v8::internal::Isolate*)[/usr/local/bin/node]
13:0x39ffa47042fd
14:0x39ffa47bd196

外部级别大约有100万个对象
,需要数据库或键值store@JonasW.这只是为系统的另一部分生成一些测试数据;它只需要运行一次,仅此而已。如果可以对其进行优化以完成,但运行时间很长(甚至数小时),这是很好的.但也许Node作为一种语言在这方面做得不够好?也许你应该使用一个json序列化程序来输出一个流?听起来不错,我会查出来的。谢谢。