tensorflow中的优化器在非线性工程中不起作用

我是tensorflow的新手，曾尝试实现一个简单的单层线性网络，类似于

该计划按预期运行，我在这方面没有问题。然而，我尝试添加另一层，但只发现

W1、b1、W2

learned都是零矩阵，并且只有bias

b2

包含非零值。下面是我修改过的网络

  x = tf.placeholder(tf.float32, [None, IN_SIZE], name="input") 

  W1 = tf.Variable(tf.zeros([IN_SIZE, L1_SIZE]), name="Weight1") 
  b1 = tf.Variable(tf.zeros([L1_SIZE]), name="bias1") 
  y = tf.matmul(x, W1) + b1 

  W2 = tf.Variable(tf.zeros([L1_SIZE, OUT_SIZE]), name="Weight2") 
  b2 = tf.Variable(tf.zeros([OUT_SIZE]), name="bias2") 
  y = tf.nn.relu(y) 
  y = tf.matmul(y, W2) + b2 

  # Define loss and optimizer 
  y_ = tf.placeholder(tf.float32, [None, OUT_SIZE], name="target") 

  cross_entropy = tf.reduce_mean( 
      tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y)) 
  train_step = tf.train.AdamOptimizer(1e-3).minimize(cross_entropy) 

---

问题是，如果在带有零的relu之前初始化权重矩阵，梯度将始终为零，并且不会发生学习。您需要进行随机初始化

  x = tf.placeholder(tf.float32, [None, IN_SIZE], name="input") 

  W1 = tf.Variable(tf.zeros([IN_SIZE, L1_SIZE]), name="Weight1") 
  b1 = tf.Variable(tf.zeros([L1_SIZE]), name="bias1") 
  y = tf.matmul(x, W1) + b1 

  W2 = tf.Variable(tf.zeros([L1_SIZE, OUT_SIZE]), name="Weight2") 
  b2 = tf.Variable(tf.zeros([OUT_SIZE]), name="bias2") 
  y = tf.nn.relu(y) 
  y = tf.matmul(y, W2) + b2 

  # Define loss and optimizer 
  y_ = tf.placeholder(tf.float32, [None, OUT_SIZE], name="target") 

  cross_entropy = tf.reduce_mean( 
      tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y)) 
  train_step = tf.train.AdamOptimizer(1e-3).minimize(cross_entropy)