如何使Tensorflow2.2访问databricks GPU集群上的GPU

我的问题与我先前的问题有关

我创建了一个新帖子，因为我需要发布更多的代码

我通过运行shell脚本在databricks上安装了Tensorflow2.2：

我试图在databricks GPU集群（p2.xlarge）上运行一些Tensorflow（2.2）代码

python3代码位于：

https://www.tensorflow.org/guide/gpu 

我的代码从databricks笔记本运行：

import tensorflow as tf
print("Num GPUs Available: ", len(tf.config.experimental.list_physical_devices('GPU')))
# 0 

import tensorflow as tf
print("Tensorflow version : {}".format(tf.__version__))
print("Num XLA_GPUs Available: ", len(tf.config.experimental.list_physical_devices('XLA_GPU')))
print("Num GPUs Available: ", len(tf.config.experimental.list_physical_devices('GPU')))
print("phy devices Available: ", tf.config.experimental.list_physical_devices())

# Tensorflow version : 2.2.0
# Num XLA_GPUs Available:  1
# Num GPUs Available:  0
# phy devices Available:  [PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'), 
# PhysicalDevice(name='/physical_device:XLA_CPU:0', device_type='XLA_CPU'), PhysicalDevice(name='/physical_device:XLA_GPU:0', device_type='XLA_GPU')]

# use gpu https://www.tensorflow.org/guide/gpu
tf.debugging.set_log_device_placement(True)

 # Create some tensors
 a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
 b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
 c = tf.matmul(a, b)

 print(c)

上述代码的输出：

tf.Tensor([[22. 28.][49. 64.]], shape=(2, 2), dtype=float32)

但是，它只在CPU上运行，不在GPU上运行

gpus = tf.config.experimental.list_physical_devices('XLA_GPU')
 
phy_devices = tf.config.list_physical_devices()
print(phy_devices)

 visible_devices = tf.config.get_visible_devices()
 print('visible_devices : ' + str(visible_devices))

 if not gpus:
    print('no gpus')
 else:
    print(gpus)
    print(gpus[0])
    # Restrict TensorFlow to only use the first GPU
try:
    tf.config.experimental.set_visible_devices(gpus[0], 'XLA_GPU') # exception poped here !!!
    logical_gpus = tf.config.experimental.list_logical_devices('XLA_GPU')
    print('logical_gpus : ' + str(logical_gpus))
    print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPU")
 except RuntimeError as e:
   # Visible devices must be set before GPUs have been initialized
   print(e)

输出：

visible_devices : [PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')][PhysicalDevice(name='/physical_device:XLA_GPU:0', device_type='XLA_GPU')]
PhysicalDevice(name='/physical_device:XLA_GPU:0', device_type='XLA_GPU')
Visible devices cannot be modified after being initialized

---

如何让tensorflow访问GPU？

我也有同样的问题

visible_devices : [PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')][PhysicalDevice(name='/physical_device:XLA_GPU:0', device_type='XLA_GPU')]
PhysicalDevice(name='/physical_device:XLA_GPU:0', device_type='XLA_GPU')
Visible devices cannot be modified after being initialized