Tensorflow 将tf.train.SyncReplicasOptimizer与多个优化器一起使用

我正在尝试在分布式设置中运行deeplabresnet（）。我选择了同步数据并行训练方法，类似于Inception的分布式训练示例中演示的方法。（）

在本例中，tf.train.syncReplicateSOptimizer用于聚合每个使用单个RMSPropOptimizer的工人的梯度。全局_步长变量也由函数更新。以下代码片段描述了此典型案例：

# Optimizer in every worker that performs gradient descent. 
opt=tf.train.RMSPropOptimizer(lr,RMSPROP_DECAY,
momentum=RMSPROP_MOMENTUM, 
epsilon=RMSPROP_EPSILON)

# A synchronous replica optimizer that wraps the RMSoptimizer
  opt = tf.train.SyncReplicasOptimizer(opt,
      replicas_to_aggregate=num_replicas_to_aggregate,
      total_num_replicas=num_workers,
      variable_averages=exp_moving_averager,
      variables_to_average=variables_to_average)

#Apply gradients and update global_step
apply_gradients_op =opt.apply_gradients(grads,global_step=global_step) 

在我的例子中，使用了三个具有不同学习率的优化器，每个优化器用于网络的特定部分

#Three optimizers declared with different learning rates
opt_conv = tf.train.MomentumOptimizer(learning_rate, args.momentum)
opt_fc_w = tf.train.MomentumOptimizer(learning_rate * 10.0, args.momentum)
opt_fc_b = tf.train.MomentumOptimizer(learning_rate * 20.0, args.momentum)

#Scope for every optimizer 
grads = tf.gradients(reduced_loss, conv_trainable + fc_w_trainable + fc_b_trainable)
grads_conv = grads[:len(conv_trainable)]
grads_fc_w = grads[len(conv_trainable) : (len(conv_trainable) + len(fc_w_trainable))]
grads_fc_b = grads[(len(conv_trainable) + len(fc_w_trainable)):]

#Gradients applied to various portions of the network
train_op_conv = opt_conv.apply_gradients(zip(grads_conv, conv_trainable))
train_op_fc_w = opt_fc_w.apply_gradients(zip(grads_fc_w, fc_w_trainable))
train_op_fc_b = opt_fc_b.apply_gradients(zip(grads_fc_b, fc_b_trainable)) 

#train_op groups all optimizer operations
train_op = tf.group(train_op_conv, train_op_fc_w, train_op_fc_b)

我不知道如何将tf.train.SyncReplicasOptimizer用于多个优化器。另外，我不知道如何更新global_step变量并在本例中使用chief_queue_runner。请帮我做这个