运行多GPU CUDA示例(simpleP2P)时P2P内存访问失败
我正在尝试对运行包含在CUDA示例中的simpleP2P示例程序时发现的错误进行故障排除。错误如下:运行多GPU CUDA示例(simpleP2P)时P2P内存访问失败,cuda,nvidia,p2p,multi-gpu,Cuda,Nvidia,P2p,Multi Gpu,我正在尝试对运行包含在CUDA示例中的simpleP2P示例程序时发现的错误进行故障排除。错误如下: $ ./simpleP2P [./simpleP2P] - Starting... Checking for multiple GPUs... CUDA-capable device count: 2 > GPU0 = " Tesla K20c" IS capable of Peer-to-Peer (P2P) > GPU1 = " Tesla K20c" IS
$ ./simpleP2P
[./simpleP2P] - Starting...
Checking for multiple GPUs...
CUDA-capable device count: 2
> GPU0 = " Tesla K20c" IS capable of Peer-to-Peer (P2P)
> GPU1 = " Tesla K20c" IS capable of Peer-to-Peer (P2P)
Checking GPU(s) for support of peer to peer memory access...
> Peer-to-Peer (P2P) access from Tesla K20c (GPU0) -> Tesla K20c (GPU1) : No
> Peer-to-Peer (P2P) access from Tesla K20c (GPU1) -> Tesla K20c (GPU0) : No
Two or more GPUs with SM 2.0 or higher capability are required for ./simpleP2P.
Peer to Peer access is not available between GPU0 <-> GPU1, waiving test.
从nvidia smi获得的有关连接的其他信息:
$ nvidia-smi topo -m
GPU0 GPU1 CPU Affinity
GPU0 X SOC 0-5,12-17
GPU1 SOC X 6-11,18-23
Legend:
X = Self
SOC = Path traverses a socket-level link (e.g. QPI)
PHB = Path traverses a PCIe host bridge
PXB = Path traverses multiple PCIe internal switches
PIX = Path traverses a PCIe internal switch
最后,lspci工具提供了更详细的输出
03:00.0 3D controller: NVIDIA Corporation GK110GL [Tesla K20c] (rev a1)
Subsystem: NVIDIA Corporation Device 0982
Flags: bus master, fast devsel, latency 0, IRQ 11
Memory at f9000000 (32-bit, non-prefetchable)
Memory at d0000000 (64-bit, prefetchable)
Memory at ce000000 (64-bit, prefetchable)
Capabilities: <access denied>
Kernel driver in use: nvidia
Kernel modules: nvidia_346, nouveau, nvidiafb
...
83:00.0 3D controller: NVIDIA Corporation GK110GL [Tesla K20c] (rev a1)
Subsystem: NVIDIA Corporation Device 0982
Flags: bus master, fast devsel, latency 0, IRQ 11
Memory at cc000000 (32-bit, non-prefetchable)
Memory at b0000000 (64-bit, prefetchable)
Memory at ae000000 (64-bit, prefetchable)
Capabilities: <access denied>
Kernel driver in use: nvidia
Kernel modules: nvidia_346, nouveau, nvidiafb
03:00.0 3D控制器:英伟达公司GK110GL[Tesla K20c](a1版)
子系统:NVIDIA公司设备0982
标志:总线主控、快速devsel、延迟0、IRQ 11
f9000000处的内存(32位,不可预取)
d0000000处的内存(64位,可预取)
ce000000处的内存(64位,可预取)
能力:
正在使用的内核驱动程序:nvidia
核心模块:nvidia_346、nouveau、nvidiafb
...
83:00.0 3D控制器:英伟达公司GK110GL[特斯拉K20c](a1版)
子系统:NVIDIA公司设备0982
标志:总线主控、快速devsel、延迟0、IRQ 11
cc000000处的内存(32位,不可预取)
b0000000处的内存(64位,可预取)
ae000000处的内存(64位,可预取)
能力:
正在使用的内核驱动程序:nvidia
核心模块:nvidia_346、nouveau、nvidiafb
你们中有谁有一些信息可以帮助我排除故障,或者至少更好地了解问题出在哪里?像往常一样感谢你的阅读/帮助Omar当GPU通过套接字级链路互连时(QPI用于基于Intel的系统): 那么这两个GPU之间就不可能进行P2P交易 参与P2P的GPU有许多要求。其中之一是,它们通常必须位于同一PCIE根复合体上。通过套接字级链路(例如QPI)连接的GPU位于两个不同的“套接字”上,即2个不同的CPU,因此它们属于两个不同的PCIE根复合体 请注意,一般来说,P2P支持可能因GPU或GPU系列而异。在一种GPU类型或GPU系列上运行P2P的能力并不一定表示它将在另一种GPU类型或系列上工作,即使在相同的系统/设置中也是如此。GPU P2P支持的最终决定因素是提供的工具,这些工具通过
cudaDeviceCanAccessPeer
查询运行时。P2P支持也可能因系统和其他因素而异。这里所做的任何声明都不能保证在任何特定设置中对任何特定GPU的P2P支持
03:00.0 3D controller: NVIDIA Corporation GK110GL [Tesla K20c] (rev a1)
Subsystem: NVIDIA Corporation Device 0982
Flags: bus master, fast devsel, latency 0, IRQ 11
Memory at f9000000 (32-bit, non-prefetchable)
Memory at d0000000 (64-bit, prefetchable)
Memory at ce000000 (64-bit, prefetchable)
Capabilities: <access denied>
Kernel driver in use: nvidia
Kernel modules: nvidia_346, nouveau, nvidiafb
...
83:00.0 3D controller: NVIDIA Corporation GK110GL [Tesla K20c] (rev a1)
Subsystem: NVIDIA Corporation Device 0982
Flags: bus master, fast devsel, latency 0, IRQ 11
Memory at cc000000 (32-bit, non-prefetchable)
Memory at b0000000 (64-bit, prefetchable)
Memory at ae000000 (64-bit, prefetchable)
Capabilities: <access denied>
Kernel driver in use: nvidia
Kernel modules: nvidia_346, nouveau, nvidiafb
GPU0 X SOC 0-5,12-17
GPU1 SOC X 6-11,18-23
^^^