GTX 1080 ti、特斯拉k80、特斯拉v100上针对同一pytorch型号的不同内存分配_Pytorch_Gpu_Nvidia_Huggingface Transformers_Tesla

GTX 1080 ti、特斯拉k80、特斯拉v100上针对同一pytorch型号的不同内存分配

pytorch

GTX 1080 ti、特斯拉k80、特斯拉v100上针对同一pytorch型号的不同内存分配,pytorch,gpu,nvidia,huggingface-transformers,tesla,Pytorch,Gpu,Nvidia,Huggingface Transformers,Tesla,我曾尝试在pytorch中加载3个不同GPU上的distilbert模型（GeForce GTX 1080 ti、特斯拉k80、特斯拉v100）。根据pytorch cuda profiler，所有这些GPU（534MB）的内存消耗是相同的。但“nvidia smi”显示了它们各自不同的内存消耗（GTX 1080 ti-1181MB、特斯拉k80-898MB、特斯拉v100-1714MB）我选择了v100，希望能够容纳更多的进程，因为它有额外的内存。因此，与k80相比，我无法在v100中容纳更

我曾尝试在pytorch中加载3个不同GPU上的distilbert模型（GeForce GTX 1080 ti、特斯拉k80、特斯拉v100）。根据pytorch cuda profiler，所有这些GPU（534MB）的内存消耗是相同的。但“nvidia smi”显示了它们各自不同的内存消耗（GTX 1080 ti-1181MB、特斯拉k80-898MB、特斯拉v100-1714MB）

我选择了v100，希望能够容纳更多的进程，因为它有额外的内存。因此，与k80相比，我无法在v100中容纳更多的进程

版本：Python 3.6.11，transformers==2.3.0，火炬==1.6.0

任何帮助都将不胜感激

以下是GPU中的内存消耗

----------------GTX 1080ti---------------------

2020-10-19 02:11:04,147 - CE - INFO - torch.cuda.max_memory_allocated() : 514.33154296875
2020-10-19 02:11:04,147 - CE - INFO - torch.cuda.memory_allocated() : 514.33154296875
2020-10-19 02:11:04,147 - CE - INFO - torch.cuda.memory_reserved() : 534.0
2020-10-19 02:11:04,148 - CE - INFO - torch.cuda.max_memory_reserved() : 534.0

2020-10-19 12:15:37,030 - CE - INFO - torch.cuda.max_memory_allocated() : 514.33154296875
2020-10-19 12:15:37,031 - CE - INFO - torch.cuda.memory_allocated() : 514.33154296875
2020-10-19 12:15:37,031 - CE - INFO - torch.cuda.memory_reserved() : 534.0
2020-10-19 12:15:37,031 - CE - INFO - torch.cuda.max_memory_reserved() : 534.0

2020-10-20 08:18:42,952 - CE - INFO - torch.cuda.max_memory_allocated() : 514.33154296875
2020-10-20 08:18:42,952 - CE - INFO - torch.cuda.memory_allocated() : 514.33154296875
2020-10-20 08:18:42,953 - CE - INFO - torch.cuda.memory_reserved() : 534.0
2020-10-20 08:18:42,953 - CE - INFO - torch.cuda.max_memory_reserved() : 534.0

“nvidia smi”的输出：

----------------特斯拉k80---------------------

2020-10-19 02:11:04,147 - CE - INFO - torch.cuda.max_memory_allocated() : 514.33154296875
2020-10-19 02:11:04,147 - CE - INFO - torch.cuda.memory_allocated() : 514.33154296875
2020-10-19 02:11:04,147 - CE - INFO - torch.cuda.memory_reserved() : 534.0
2020-10-19 02:11:04,148 - CE - INFO - torch.cuda.max_memory_reserved() : 534.0

2020-10-19 12:15:37,030 - CE - INFO - torch.cuda.max_memory_allocated() : 514.33154296875
2020-10-19 12:15:37,031 - CE - INFO - torch.cuda.memory_allocated() : 514.33154296875
2020-10-19 12:15:37,031 - CE - INFO - torch.cuda.memory_reserved() : 534.0
2020-10-19 12:15:37,031 - CE - INFO - torch.cuda.max_memory_reserved() : 534.0

2020-10-20 08:18:42,952 - CE - INFO - torch.cuda.max_memory_allocated() : 514.33154296875
2020-10-20 08:18:42,952 - CE - INFO - torch.cuda.memory_allocated() : 514.33154296875
2020-10-20 08:18:42,953 - CE - INFO - torch.cuda.memory_reserved() : 534.0
2020-10-20 08:18:42,953 - CE - INFO - torch.cuda.max_memory_reserved() : 534.0

“nvidia smi”的输出：

----------------特斯拉v100---------------------

2020-10-19 02:11:04,147 - CE - INFO - torch.cuda.max_memory_allocated() : 514.33154296875
2020-10-19 02:11:04,147 - CE - INFO - torch.cuda.memory_allocated() : 514.33154296875
2020-10-19 02:11:04,147 - CE - INFO - torch.cuda.memory_reserved() : 534.0
2020-10-19 02:11:04,148 - CE - INFO - torch.cuda.max_memory_reserved() : 534.0

2020-10-19 12:15:37,030 - CE - INFO - torch.cuda.max_memory_allocated() : 514.33154296875
2020-10-19 12:15:37,031 - CE - INFO - torch.cuda.memory_allocated() : 514.33154296875
2020-10-19 12:15:37,031 - CE - INFO - torch.cuda.memory_reserved() : 534.0
2020-10-19 12:15:37,031 - CE - INFO - torch.cuda.max_memory_reserved() : 534.0

2020-10-20 08:18:42,952 - CE - INFO - torch.cuda.max_memory_allocated() : 514.33154296875
2020-10-20 08:18:42,952 - CE - INFO - torch.cuda.memory_allocated() : 514.33154296875
2020-10-20 08:18:42,953 - CE - INFO - torch.cuda.memory_reserved() : 534.0
2020-10-20 08:18:42,953 - CE - INFO - torch.cuda.max_memory_reserved() : 534.0

“nvidia smi”的输出：