Tensorflow 通过云TPU和x27生成梯度摘要;s host_call_fn()?
我的理解是host_call和host_call_fn()将统计数据从TPU传输到主机。然而,关于如何为任何非标量的内容生成摘要的说明并不十分清楚 例如,我试图修改官方的mnist_tpu.py,以生成训练期间生成的渐变的摘要。模型_fn()是添加更改的位置:Tensorflow 通过云TPU和x27生成梯度摘要;s host_call_fn()?,tensorflow,tensorboard,google-cloud-tpu,Tensorflow,Tensorboard,Google Cloud Tpu,我的理解是host_call和host_call_fn()将统计数据从TPU传输到主机。然而,关于如何为任何非标量的内容生成摘要的说明并不十分清楚 例如,我试图修改官方的mnist_tpu.py,以生成训练期间生成的渐变的摘要。模型_fn()是添加更改的位置: ... optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate) if FLAGS.use_tpu: optimizer = tf.contr
...
optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)
if FLAGS.use_tpu:
optimizer = tf.contrib.tpu.CrossShardOptimizer(optimizer)
grads = optimizer.compute_gradients(loss)
train_op = optimizer.apply_gradients(grads, global_step)
if not FLAGS.skip_host_call:
def host_call_fn(gs, loss, lr, grads):
gs = gs[0]
with summary.create_file_write(FLAGS.model_dir).as_default():
summary.scalar('loss', loss[0], step=gs)
summary.scalar('learning_rate', lr[0], step=gs)
for index, grad in enumerate(grads):
summary.histogram('{}-grad'.format(grads[index][1].name),
grads[index])
return summary.all_summary_ops()
gs_t = tf.reshape(global_step, [1])
loss_t = tf.reshape(loss, [1])
lr_t = tf.reshape(learning_rate, [1])
grads_t = grads
host_call = (host_call_fn, [gs_t, loss_t, lr_t, grads_t])
return tf.contrib.tpu.TPUEstimatorSpec(
mode=mode,
loss=loss,
train_op=train_op
)
....
不幸的是,上面的添加似乎不像基于CPU的训练期间直方图生成那样有效。知道如何正确地生成非标量张量的直方图吗?主机调用的参数。问题是梯度是一对梯度张量和变量。在将变量名传递给host_call_fn之前,应该先提取出变量名,然后只传递梯度张量。实现这一点的一种方法是将host_call_fn的参数更改为**kwargs,其中关键字参数名是变量名,并将字典作为张量列表传递