.net 读取小二进制文件,内存不足异常
我正在将一个小的二进制文件(大约32mb)逐字节读取到一个数据表中,并将前8个字节从十六进制转换为二进制 对于大文件,我遇到系统内存不足异常错误消息,我无法思考如何改进代码以使其运行更平稳或更快 这是我的密码:.net 读取小二进制文件,内存不足异常,.net,vb.net,binary,.net,Vb.net,Binary,我正在将一个小的二进制文件(大约32mb)逐字节读取到一个数据表中,并将前8个字节从十六进制转换为二进制 对于大文件,我遇到系统内存不足异常错误消息,我无法思考如何改进代码以使其运行更平稳或更快 这是我的密码: Public Function DecToBin(ByVal DeciValue As Long, Optional ByVal NoOfBits As Integer = 8) As String Dim i As Integer Do While DeciValue
Public Function DecToBin(ByVal DeciValue As Long, Optional ByVal NoOfBits As Integer = 8) As String
Dim i As Integer
Do While DeciValue > (2 ^ NoOfBits) - 1
NoOfBits = NoOfBits + 8
Loop
DecToBin = vbNullString
For i = 0 To (NoOfBits - 1)
DecToBin = CStr((DeciValue And 2 ^ i) / 2 ^ i) & DecToBin
Next i
End Function
Private Sub importData(ByVal openFile As String)
If fullFilePath.EndsWith("variable.dat") Then
Dim importedData As New DataTable
Cursor.Current = Cursors.WaitCursor
Using reader As New BinaryReader(File.Open(openFile, FileMode.Open))
Dim pos As Integer = 0
Dim length As Integer = reader.BaseStream.Length
loadingBar.loadingLabel.Text = "Lines of data: " & length
loadingBar.loadingProgress.Minimum = 0
loadingBar.loadingProgress.Maximum = length
loadingBar.loadingProgress.Step = 1
importedData.Columns.Add("Data0")
importedData.Columns.Add("Data1")
importedData.Columns.Add("Data2")
importedData.Columns.Add("Data3")
importedData.Columns.Add("Data4")
importedData.Columns.Add("Data5")
importedData.Columns.Add("Data6")
importedData.Columns.Add("Data7")
importedData.Columns.Add("Time")
importedData.Columns.Add("Date")
importedData.Columns.Add("Day")
importedData.Columns.Add("Month")
importedData.Columns.Add("Year")
importedData.Columns.Add("Blank")
While pos < length
For i = 0 To 15
Dim value As Decimal = reader.ReadByte
rawDataArray(i) = value.ToString()
pos += 1
Next
loadingBar.BringToFront()
loadingBar.Show()
Cursor.Current = Cursors.WaitCursor
loadingBar.loadingProgress.Value = pos
convertedDataArray(0) = DecToBin(rawDataArray(0))
convertedDataArray(1) = DecToBin(rawDataArray(1))
convertedDataArray(2) = DecToBin(rawDataArray(2))
convertedDataArray(3) = DecToBin(rawDataArray(3))
convertedDataArray(4) = DecToBin(rawDataArray(4))
convertedDataArray(5) = DecToBin(rawDataArray(5))
convertedDataArray(6) = DecToBin(rawDataArray(6))
convertedDataArray(7) = DecToBin(rawDataArray(7))
convertedDataArray(8) = Format((rawDataArray(12) + 0), "00") & ":" & Format((rawDataArray(11) + 0), "00") & ":" & Format((rawDataArray(10) + 0), "00") & "." & Format((rawDataArray(8) * 256) + rawDataArray(9), "000")
convertedDataArray(9) = Format((rawDataArray(13) + 0), "00") & "." & Format((rawDataArray(14) + 0), "00") & "." & "20" & Format((rawDataArray(15) + 0), "00")
convertedDataArray(10) = Format((rawDataArray(13) + 0), "00")
convertedDataArray(11) = Format((rawDataArray(14) + 0), "00")
convertedDataArray(12) = Format((rawDataArray(15) + 0), "00")
importedData.Rows.Add(convertedDataArray)
Application.DoEvents()
End While
loadingBar.Close()
importedData.DefaultView.Sort = "Year,Month,Day,Time"
dataGrid.DataSource = importedData.DefaultView
End Using
dataGrid.Columns(0).Visible = False ' Data 0
dataGrid.Columns(1).Visible = False ' Data 1
dataGrid.Columns(2).Visible = False ' Data 2
dataGrid.Columns(3).Visible = False ' Data 3
dataGrid.Columns(4).Visible = False ' Data 4
dataGrid.Columns(5).Visible = False ' Data 5
dataGrid.Columns(6).Visible = False ' Data 6
dataGrid.Columns(7).Visible = False ' Data 7
dataGrid.Columns(8).Visible = True ' Data Time
dataGrid.Columns(9).Visible = True ' Data Date
dataGrid.Columns(10).Visible = False ' Data Day
dataGrid.Columns(11).Visible = False ' Data Month
dataGrid.Columns(12).Visible = False ' Data Year
dataGrid.Columns(13).Visible = False ' Data Blank
Else
MsgBox("Wrong file selected")
End If
End Sub
公共函数DecToBin(ByVal DeciValue为Long,可选ByVal NoOfBits为Integer=8)作为字符串
作为整数的Dim i
Do While DeciValue>(2^NoOfBits)-1
NoOfBits=NoOfBits+8
环
DecToBin=vbNullString
对于i=0到(NoOfBits-1)
DecToBin=CStr((DeciValue和2^i)/2^i)和DecToBin
接下来我
端函数
私有子导入数据(ByVal openFile作为字符串)
如果fullFilePath.EndsWith(“variable.dat”),则
将导入的数据作为新数据表进行Dim
Cursor.Current=Cursors.WaitCursor
将读取器用作新的二进制读取器(File.Open(openFile,FileMode.Open))
Dim pos作为整数=0
Dim长度为整数=reader.BaseStream.length
loadingBar.loadingLabel.Text=“数据行:”&长度
loadingBar.loadingProgress.Minimum=0
loadingBar.loadingProgress.Maximum=长度
loadingBar.loadingProgress.Step=1
importedData.Columns.Add(“Data0”)
importedData.Columns.Add(“数据1”)
importedData.Columns.Add(“数据2”)
importedData.Columns.Add(“Data3”)
importedData.Columns.Add(“Data4”)
importedData.Columns.Add(“Data5”)
importedData.Columns.Add(“Data6”)
importedData.Columns.Add(“Data7”)
importedData.Columns.Add(“时间”)
importedData.Columns.Add(“日期”)
importedData.Columns.Add(“日”)
importedData.Columns.Add(“月”)
importedData.Columns.Add(“年”)
importedData.Columns.Add(“空白”)
而pos
正在读取的数据示例:
3F F3 45 C6 03 00 11 00 6F 1F 1D 16 12 07 0D
输出示例:
00111111110011 01000101 11000110000011 00000000 00010011 00000000 22:29:31.111 18.07.2013 18.07 13
获取
System.OutOfMemoryException
意味着不存在所需大小的单个连续未分配内存区域。这并不一定意味着你没有剩下的内存,而是说内存碎片太多了
解决此问题的一种方法可能是使用类,例如,该类(顾名思义)将进程的部分虚拟内存映射到硬盘上的数据
您可以尝试处理该问题的另一种方法是调用GC.Collect
,这将作为垃圾收集过程的一部分压缩内存。尽管如此,我还是强烈反对使用这种方法
您可以使用或windbg(作为的一部分提供)查看内存的使用位置
至于您的代码,您似乎没有任何可疑之处(除了Marc Gravell已经指出的DecToBin)
希望这有帮助。问题属于codereview@peer我不知道怎么做;它询问的是一个特定的问题-OOM在处理什么时,尽管OP的描述是一个小fil