C# ASP.NET内核禁用响应缓冲
我正在尝试将动态构建的大型JSON文件流式传输到客户端(可能是500 MB+)。我尝试禁用响应缓冲有多种原因,尽管主要是为了内存效率 我曾尝试直接写入C# ASP.NET内核禁用响应缓冲,c#,asp.net,asp.net-core,asp.net-web-api,asp.net-core-webapi,C#,Asp.net,Asp.net Core,Asp.net Web Api,Asp.net Core Webapi,我正在尝试将动态构建的大型JSON文件流式传输到客户端(可能是500 MB+)。我尝试禁用响应缓冲有多种原因,尽管主要是为了内存效率 我曾尝试直接写入HttpContext.Response.BodyWriter,但在写入输出之前,响应似乎已缓冲在内存中。此方法的返回类型为Task HttpContext.Response.ContentType = "application/json"; HttpContext.Response.ContentLength = null;
HttpContext.Response.BodyWriter,但在写入输出之前,响应似乎已缓冲在内存中。此方法的返回类型为Task
HttpContext.Response.ContentType = "application/json";
HttpContext.Response.ContentLength = null;
await HttpContext.Response.StartAsync(cancellationToken);
var bodyStream = HttpContext.Response.BodyWriter.AsStream(true);
await bodyStream.WriteAsync(Encoding.UTF8.GetBytes("["), cancellationToken);
await foreach (var item in cursor.WithCancellation(cancellationToken)
.ConfigureAwait(false))
{
await bodyStream.WriteAsync(JsonSerializer.SerializeToUtf8Bytes(item, DefaultSettings.JsonSerializerOptions), cancellationToken);
await bodyStream.WriteAsync(Encoding.UTF8.GetBytes(","), cancellationToken);
await bodyStream.FlushAsync(cancellationToken);
await Task.Delay(100,cancellationToken);
}
await bodyStream.WriteAsync(Encoding.UTF8.GetBytes("]"), cancellationToken);
bodyStream.Close();
await HttpContext.Response.CompleteAsync().ConfigureAwait(false);
注意:我意识到这段代码非常粗糙,试图让它工作,然后清理它
我正在使用Task.Delay
验证本地测试时响应未被缓冲,因为我没有完整的生产数据。我也尝试了IAsyncEnumerable
和yield-return
,但是失败了,因为响应太大了,以至于Kestrel认为枚举是无限的
我试过了
设置为较小的数字,甚至为0李>
使用HttpContext.Response.WriteAsync写入
使用HttpContext.Response.BodyWriter.AsStream()编写
使用管道编写器模式和HttpContext.Response.BodyWriter
删除所有中间件
删除对IApplicationBuilder.UserResponseCompression的调用
更新
在设置ContentType
之前(在写入响应之前)尝试禁用响应缓冲,但没有效果
尝试禁用响应未来的缓冲:
HttpContext.Features.Get<IHttpResponseBodyFeature>().DisableBuffering()
//As mentioned in documentation, to take effect, call it before any writes
在我的例子中,它给出了结果:在第一次请求之后,与newcoreapp2.2相比,总内存分配增加了80%以上,但没有更多的内存泄漏 对于仍感兴趣的用户,使用curl时,此代码会立即发送数据:
public async Task Invoke(HttpContext context)
{
var g = context.Features.Get<IHttpResponseBodyFeature>();
g.DisableBuffering(); // doesn't seem to make a difference
context.Response.StatusCode = 200;
context.Response.ContentType = "text/plain; charset=utf-8";
//context.Response.ContentLength = null;
await g.StartAsync();
for (int i = 0; i < 10; ++i)
{
var line = $"this is line {i}\r\n";
var bytes = utf8.GetBytes(line);
// it seems context.Response.Body.WriteAsync() and
// context.Response.BodyWriter.WriteAsync() work exactly the same
await g.Writer.WriteAsync(new ReadOnlyMemory<byte>(bytes));
await g.Writer.FlushAsync();
await Task.Delay(1000);
}
await g.CompleteAsync();
}
公共异步任务调用(HttpContext上下文)
{
var g=context.Features.Get();
g、 DisableBuffering();//似乎没有什么不同
context.Response.StatusCode=200;
context.Response.ContentType=“text/plain;charset=utf-8”;
//context.Response.ContentLength=null;
等待g.StartAsync();
对于(int i=0;i<10;++i)
{
var line=$“这是第{i}行\r\n”;
var bytes=utf8.GetBytes(行);
//似乎是context.Response.Body.WriteAsync()和
//context.Response.BodyWriter.WriteAsync()的工作原理完全相同
等待g.Writer.WriteAsync(新的只读内存(字节));
等待g.Writer.FlushAsync();
等待任务。延迟(1000);
}
等待g.CompleteAsync();
}
我尝试过使用和不使用DisableBufering()
以及写入管道(ihttppresponsebodyfeature.Writer
vsHttpContext.Response.Body
)的变体,但似乎没有什么不同
在curl中,它会立即显示消息,但在Chrome和一些rest客户端中,它会等待整个流显示出来
因此,我建议使用不等待整个流呈现的客户端测试代码行为。另一个选项是,我仍在检查aspnet core是否会在客户端请求时自动选择压缩可能性,即使压缩未在管道中配置。
因此,我建议更新,不,我有同样的问题,任何大的JSON,是的,我的答案不是一个解决方案,只是解决办法,以避免“OutOfMemory”。
HttpContext.Features.Get<IHttpResponseBodyFeature>().DisableBuffering()
//As mentioned in documentation, to take effect, call it before any writes
var pipe = context.HttpContext.Response.BodyWriter;
await pipe.WriteAsync(startArray);
using (var writer = new Utf8JsonWriter(pipe,
new JsonWriterOptions
{
Indented = option.WriteIndented,
Encoder = option.Encoder,
SkipValidation = true
}))
{
var dotSet = false;
foreach (var item in enumerable)
{
if (dotSet)
await pipe.WriteAsync(dot);
JsonSerializer.Serialize(writer, item, itemType, option);
await pipe.FlushAsync();
writer.Reset();
dotSet = true;
}
}
await pipe.WriteAsync(endArray);
public async Task Invoke(HttpContext context)
{
var g = context.Features.Get<IHttpResponseBodyFeature>();
g.DisableBuffering(); // doesn't seem to make a difference
context.Response.StatusCode = 200;
context.Response.ContentType = "text/plain; charset=utf-8";
//context.Response.ContentLength = null;
await g.StartAsync();
for (int i = 0; i < 10; ++i)
{
var line = $"this is line {i}\r\n";
var bytes = utf8.GetBytes(line);
// it seems context.Response.Body.WriteAsync() and
// context.Response.BodyWriter.WriteAsync() work exactly the same
await g.Writer.WriteAsync(new ReadOnlyMemory<byte>(bytes));
await g.Writer.FlushAsync();
await Task.Delay(1000);
}
await g.CompleteAsync();
}