C# 更改超大流的编码
更改流的编码时,我得到一个C# 更改超大流的编码,c#,character-encoding,C#,Character Encoding,更改流的编码时,我得到一个OutOfMemoryException。到目前为止,数据流相对较小(小于50MB),但现在我遇到了一个不同的场景,它们大约是1.78GB的数据流。所以它们相当大 至于基础设施,它位于Azure云服务中,拥有7GB内存,并且正在将其最大化。(它作为x64进程运行)。我知道根本的问题是我一次在内存中有太多的reportStream(超过5或6个) 例外情况: System.OutOfMemoryException: Array dimensions exceeded su
OutOfMemoryException
。到目前为止,数据流相对较小(小于50MB),但现在我遇到了一个不同的场景,它们大约是1.78GB的数据流。所以它们相当大
至于基础设施,它位于Azure云服务中,拥有7GB内存,并且正在将其最大化。(它作为x64进程运行)。我知道根本的问题是我一次在内存中有太多的reportStream
(超过5或6个)
例外情况:
System.OutOfMemoryException: Array dimensions exceeded supported range.
at System.Text.Encoding.GetChars(System.Byte[] bytes, System.Int32 index, System.Int32 count) at offset 9
at System.Text.Encoding.Convert(System.Text.Encoding srcEncoding, System.Text.Encoding dstEncoding, System.Byte[] bytes, System.Int32 index, System.Int32 count) at offset 61
at System.Text.Encoding.Convert(System.Text.Encoding srcEncoding, System.Text.Encoding dstEncoding, System.Byte[] bytes) at offset 21
at MyNamespace.MyClass.ChangeEncoding(System.IO.Stream reportStream) at offset 72 in ...
代码:
为了回答这个问题,我如何更改此设置以将编码转换为块而不是一次转换整个流?@user1666620我没有使用
StreamReader
。答案在MSDN中--“如果要转换的数据仅在顺序块中可用(例如从流读取的数据)或者,如果数据量太大,需要将其划分为更小的块,则应用程序应分别使用派生类的GetDecoder方法或GetEncoder方法提供的解码器或编码器。“如果有人遇到这种情况,使用StreamReader是一个好主意:
private static Stream ChangeEncoding(Stream reportStream)
{
var utf8 = Encoding.UTF8;
// The reports aren't actually in ASCII encoding.
// There in a superset of ASCII that's specific to Windows called "Windows-1252".
// Windows-1252 contains some special characters, whereas ASCII doesn't have any special
// characters at all.
// https://en.wikipedia.org/wiki/Windows-1252
var win = Encoding.GetEncoding("Windows-1252");
var length = (int) reportStream.Length;
var buffer = new byte[length];
int count;
var sum = 0;
// Read until Read method returns 0 (end of the stream has been reached)
while ((count = reportStream.Read(buffer, sum, length - sum)) > 0)
{
sum += count;
}
var convertedBytes = Encoding.Convert(win, utf8, buffer);
var outputStream = new MemoryStream();
outputStream.Write(convertedBytes, 0, convertedBytes.Length);
// Reset the position otherwise it won't zip and subsequently upload the report.
outputStream.Position = 0;
return outputStream;
}