C# .NET内核在Docker中的高内存使用率（AWS ECS Fargate）_C#_.net_.net Core_Amazon Ecs_Hangfire

C# .NET内核在Docker中的高内存使用率（AWS ECS Fargate）

c# .net .net-core

C# .NET内核在Docker中的高内存使用率（AWS ECS Fargate）,c#,.net,.net-core,amazon-ecs,hangfire,C#,.net,.net Core,Amazon Ecs,Hangfire,下面是应用程序的说明。我有一个运行Hangfire的控制台应用程序。有几个工作是经常性的。特别是一个作业，它解析多个大型XML文件。我将此控制台应用程序部署为AWS ECS Fargate容器每次循环作业运行时，容器内存使用量不断增加，直到应用程序无响应或AWS终止容器。这是ECS上的16GB容器（大），所以它应该能够处理我扔给它的任何东西。我可以断定，一旦内存使用量接近13GB，它就注定要失败更重要的是，我在本地运行这个应用程序，它的内存使用率从来不会很高。我将从我的工作站和下面的容器中发

下面是应用程序的说明。我有一个运行Hangfire的控制台应用程序。有几个工作是经常性的。特别是一个作业，它解析多个大型XML文件。我将此控制台应用程序部署为AWS ECS Fargate容器

每次循环作业运行时，容器内存使用量不断增加，直到应用程序无响应或AWS终止容器。这是ECS上的16GB容器（大），所以它应该能够处理我扔给它的任何东西。我可以断定，一旦内存使用量接近13GB，它就注定要失败

更重要的是，我在本地运行这个应用程序，它的内存使用率从来不会很高。我将从我的工作站和下面的容器中发布几个示例日志

谢谢你的洞察力

解析XML：

使用XmlSerializer放入JSON格式的对象
使用XDocument只需浏览文档即可
使用XSD文件生成类，然后使用XmlSerializer填充

这三个都在本地工作，但在服务器上使用了大量内存。这些是不变的XML文件，我在本地和容器中运行完全相同的文件

仅供参考：我已关闭容器上的服务器垃圾收集。那没用

编辑2019年3月5日
下面的日志来自以下代码。它没有充分的应用程序，这是相当复杂的呼吸。此代码位于注册到Hangfire的方法中，每天运行一次。它从FTP站点提取XML文件，然后运行下面的代码来处理这些文件。每个文件的大小最多为10 MB

XmlSerializer serializer = new XmlSerializer(Model); foreach (var file in files) { Console.WriteLine($"Processing file: {file.Name}"); Model model; using (XmlReader reader = XmlReader.Create(file.FullName, new XmlReaderSettings { CheckCharacters = false })) { model = (Model)serializer.Deserialize(reader); } SaveToDatabase(model); Console.WriteLine($"Application Memory Usage - Working: {Process.GetCurrentProcess()?.WorkingSet64 ?? 0}, Private: {Process.GetCurrentProcess()?.PrivateMemorySize64 ?? 0}"); }
工作站日志：

[13:52:58 INF] Processing file: price_feed_supplemental_002981.xml [13:52:58 INF] Application Memory Usage - Working: 1.4 GB, Private: 1.6 GB [13:52:58 INF] Processing file: price_feed_002982.xml [13:53:01 INF] Application Memory Usage - Working: 1.4 GB, Private: 1.6 GB [13:53:01 INF] Processing file: price_feed_supplemental_002982.xml [13:53:01 INF] Application Memory Usage - Working: 1.4 GB, Private: 1.6 GB [13:53:01 INF] Processing file: price_feed_002983.xml [13:53:01 INF] Application Memory Usage - Working: 1.4 GB, Private: 1.6 GB [13:53:01 INF] Processing file: price_feed_supplemental_002983.xml [13:53:02 INF] Application Memory Usage - Working: 1.4 GB, Private: 1.6 GB [13:53:02 INF] Processing file: price_feed_002984.xml
容器日志：

[19:05:35 INF] Application Memory Usage - Working: 1.9 GB, Private: 0.0 bytes [19:05:35 INF] Processing file: price_feed_supplemental_002987.xml [19:05:35 INF] Application Memory Usage - Working: 1.9 GB, Private: 0.0 bytes [19:05:35 INF] Processing file: price_feed_supplemental_002988.xml [19:05:35 INF] Application Memory Usage - Working: 1.9 GB, Private: 0.0 bytes [19:05:35 INF] Processing file: price_feed_002988.xml [19:05:40 INF] Application Memory Usage - Working: 1.9 GB, Private: 0.0 bytes [19:05:40 INF] Processing file: price_feed_002989.xml [19:05:43 INF] Application Memory Usage - Working: 2.1 GB, Private: 0.0 bytes [19:05:43 INF] Processing file: price_feed_supplemental_002989.xml [19:05:43 INF] Application Memory Usage - Working: 2.1 GB, Private: 0.0 bytes [19:05:43 INF] Processing file: price_feed_supplemental_002990.xml [19:05:43 INF] Application Memory Usage - Working: 2.1 GB, Private: 0.0 bytes [19:05:43 INF] Processing file: price_feed_002990.xml [19:05:51 INF] Application Memory Usage - Working: 2.6 GB, Private: 0.0 bytes [19:05:51 INF] Processing file: price_feed_002991.xml [19:05:57 INF] Application Memory Usage - Working: 2.2 GB, Private: 0.0 bytes [19:05:57 INF] Processing file: price_feed_supplemental_002991.xml [19:06:00 INF] Application Memory Usage - Working: 2.2 GB, Private: 0.0 bytes [19:06:00 INF] Processing file: price_feed_002992.xml [19:06:19 INF] Application Memory Usage - Working: 2.1 GB, Private: 0.0 bytes [19:06:19 INF] Processing file: price_feed_supplemental_002992.xml [19:06:21 INF] Application Memory Usage - Working: 2.3 GB, Private: 0.0 bytes [19:06:21 INF] Uncompressing file: usa_LTSF_20190228_002993_001_INC.zip [19:06:22 INF] Source file: price_feed_002993_1.xml [19:06:22 INF] Source file: price_feed_supplemental_002993_8.xml [19:06:23 INF] Source file: price_feed_supplemental_002993_11.xml [19:06:23 INF] Source file: price_feed_002993_2.xml [19:06:23 INF] Source file: price_feed_supplemental_002993_5.xml [19:06:23 INF] Source file: price_feed_supplemental_002993_4.xml [19:06:24 INF] Source file: price_feed_supplemental_002993_17.xml [19:06:24 INF] Source file: price_feed_supplemental_002993_15.xml [19:06:24 INF] Source file: price_feed_supplemental_002993_3.xml [19:06:25 INF] Source file: price_feed_supplemental_002993_1.xml [19:06:25 INF] Source file: price_feed_supplemental_002993_9.xml [19:06:25 INF] Source file: price_feed_supplemental_002993_7.xml [19:06:26 INF] Source file: price_feed_supplemental_002993_6.xml [19:06:26 INF] Source file: price_feed_supplemental_002993_14.xml [19:06:26 INF] Source file: price_feed_supplemental_002993_13.xml [19:06:27 INF] Source file: price_feed_supplemental_002993_16.xml [19:06:27 INF] Source file: price_feed_supplemental_002993_12.xml [19:06:27 INF] Source file: price_feed_supplemental_002993_2.xml [19:06:28 INF] Source file: price_feed_supplemental_002993_10.xml [19:06:28 INF] Processing file: price_feed_002984.xml [19:06:28 INF] Application Memory Usage - Working: 2.5 GB, Private: 0.0 bytes [19:06:28 INF] Processing file: price_feed_supplemental_002984.xml [19:06:28 INF] Application Memory Usage - Working: 2.5 GB, Private: 0.0 bytes [19:06:28 INF] Processing file: price_feed_supplemental_002985.xml [19:06:28 INF] Application Memory Usage - Working: 2.5 GB, Private: 0.0 bytes [19:06:28 INF] Processing file: price_feed_002985.xml [19:06:28 INF] Application Memory Usage - Working: 2.5 GB, Private: 0.0 bytes [19:06:28 INF] Processing file: price_feed_supplemental_002986.xml [19:06:28 INF] Application Memory Usage - Working: 2.5 GB, Private: 0.0 bytes [19:06:28 INF] Processing file: price_feed_002986.xml [19:06:28 INF] Application Memory Usage - Working: 2.5 GB, Private: 0.0 bytes [19:06:28 INF] Processing file: price_feed_002987.xml [19:06:30 INF] Application Memory Usage - Working: 2.6 GB, Private: 0.0 bytes [19:06:30 INF] Processing file: price_feed_supplemental_002987.xml [19:06:30 INF] Application Memory Usage - Working: 2.6 GB, Private: 0.0 bytes [19:06:30 INF] Processing file: price_feed_supplemental_002988.xml [19:06:30 INF] Application Memory Usage - Working: 2.6 GB, Private: 0.0 bytes [19:06:30 INF] Processing file: price_feed_002988.xml [19:06:35 INF] Application Memory Usage - Working: 2.9 GB, Private: 0.0 bytes [19:06:35 INF] Processing file: price_feed_002989.xml [19:06:38 INF] Application Memory Usage - Working: 3.1 GB, Private: 0.0 bytes [19:06:38 INF] Processing file: price_feed_supplemental_002989.xml [19:06:38 INF] Application Memory Usage - Working: 3.1 GB, Private: 0.0 bytes [19:06:38 INF] Processing file: price_feed_supplemental_002990.xml [19:06:38 INF] Application Memory Usage - Working: 3.1 GB, Private: 0.0 bytes [19:06:38 INF] Processing file: price_feed_002990.xml [19:06:47 INF] Application Memory Usage - Working: 3.7 GB, Private: 0.0 bytes [19:06:47 INF] Processing file: price_feed_002991.xml [19:06:53 INF] Application Memory Usage - Working: 4.1 GB, Private: 0.0 bytes [19:06:53 INF] Processing file: price_feed_supplemental_002991.xml [19:06:55 INF] Application Memory Usage - Working: 4.3 GB, Private: 0.0 bytes [19:06:55 INF] Processing file: price_feed_002992.xml [19:07:05 INF] Application Memory Usage - Working: 4.9 GB, Private: 0.0 bytes [19:07:05 INF] Processing file: price_feed_supplemental_002992.xml [19:07:07 INF] Application Memory Usage - Working: 5.0 GB, Private: 0.0 bytes [19:07:07 INF] Processing file: price_feed_002993_1.xml [19:09:40 INF] 1 servers were removed due to timeout [19:10:49 INF] Application Memory Usage - Working: 10.0 GB, Private: 0.0 bytes [19:10:49 INF] Processing file: price_feed_supplemental_002993_8.xml [19:12:10 INF] Application Memory Usage - Working: 12.5 GB, Private: 0.0 bytes [19:12:10 INF] Processing file: price_feed_supplemental_002993_11.xml [19:13:08 INF] Application Memory Usage - Working: 13.5 GB, Private: 0.0 bytes [19:13:08 INF] Processing file: price_feed_002993_2.xml [19:14:06 INF] Application Memory Usage - Working: 13.8 GB, Private: 0.0 bytes [19:14:06 INF] Processing file: price_feed_supplemental_002993_5.xml

如果处理本地设备上的所有日志文件，内存会发生什么变化？本地框的统计信息只显示到
002984.xml
，在此之前的内存保持不变。在你的服务器上，有几个文件的内存也保持不变。我没有包括所有的本地日志，因为它们是所有日志的确切内存使用情况。这是本地运行的，并通过Visual Studio的调试器运行，结果相同。我想知道这是否与Nick遇到的问题有关。运行容器的机器的高内核数可能导致堆在GC发生之前大量增长。您能否通过定期调用
GC.Collect
来检查GC是否未运行或内存泄漏？您还能否共享运行时的版本，等等@JohnD这是在.NET Core 2.2下运行的。