Python 3.x Tensorflow收敛，但在mnist数据集中使用alexnet的训练精度非常低

我写了一个tensoflow程序来尝试使用MNIST数据集的Alexnet，但奇怪的是我的网络收敛得非常快，损失几乎没有改变。而且每个批次的准确度都非常低，低于0.1，如下所示：

step 0   loss 2.29801 train_accuracy 0.14
step 100 loss 2.30258 train_accuracy 0.07
step 200 loss 2.30258 train_accuracy 0.15
step 300 loss 2.30258 train_accuracy 0.09
step 400 loss 2.30258 train_accuracy 0.08
step 500 loss 2.30258 train_accuracy 0.06
step 600 loss 2.30258 train_accuracy 0.15
step 700 loss 2.30258 train_accuracy 0.16
....

这是我的密码

import tensorflow as tf
from tensorflow.python import debug as tf_debug
import numpy as np
from tensorflow.examples.tutorials.mnist import input_data

data_dir='./minist'
mnist = input_data.read_data_sets(data_dir,one_hot=True)

def conv2d(name, x, ws, bs, strides=1):
    w = tf.Variable(tf.truncated_normal(ws,stddev=0.01))
    b = tf.Variable(tf.constant(0.,shape=bs))
    x = tf.nn.conv2d(x, w, strides=[1,strides,strides,1], padding='SAME')
    x = tf.nn.bias_add(x,b)
    return tf.nn.relu(x, name=name)

def maxpool2d(name, x, k=2):
    return tf.nn.max_pool(x, ksize=[1, k, k, 1], strides=[1, k, k, 1],
                      padding='SAME', name=name)
def fc_op(name, x, n_out):
    n_in = x.get_shape()[-1].value
    w = tf.Variable(tf.truncated_normal([n_in,n_out],stddev=0.01))
    b = tf.Variable(tf.constant(0.,shape=[n_out]))
    x = tf.matmul(x, w)
    x = tf.nn.bias_add(x, b)
    return tf.nn.relu(x, name=name)

def alex_net(x,num_classes):
    conv1 = conv2d('conv1', x, [11,11,1,96] , [96], strides=4)
    pool1 = maxpool2d('pool1', conv1)
    conv2 = conv2d('conv2', pool1, [5,5,96,256] , [256])
    pool2 = maxpool2d('pool2', conv2)
    conv3 = conv2d('conv3', pool2, [3,3,256,384] , [384])
    conv4 = conv2d('conv4', conv3, [3,3,384,384] , [384])
    conv5 = conv2d('conv5', conv4, [3,3,384,256] , [256])
    pool5 = maxpool2d('pool5', conv5)

    shp = pool5.get_shape()
    flattened_shape = shp[1].value*shp[2].value*shp[3].value
    resh = tf.reshape(pool5, shape=[-1,flattened_shape], name='resh')

    fc1 = fc_op('fc1', resh, 4096)
    fc2 = fc_op('fc2', fc1, 4096)
    fc3 = fc_op('fc3', fc2, num_classes)
    return fc3

# ############################ arguments setting
learning_rate = 0.01
train_steps= 8000
num_classes = 10  
x = tf.placeholder(shape=[None, 784],dtype=tf.float32)
x_image = tf.reshape(x, [-1, 28, 28, 1])
y=tf.placeholder(shape=[None,10],dtype=tf.float32)

output=alex_net(x_image,num_classes)  

# ################################### trian

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=output,labels=y))
train_op = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)

# ###################################  inference
correct_pred = tf.equal(tf.argmax(output,1),tf.argmax(y,1))
accuracy = tf.reduce_mean(tf.cast(correct_pred,tf.float32))

init = tf.global_variables_initializer()

sess = tf.InteractiveSession()
sess.run(init)

for i in range(train_steps):
    xs, ys = mnist.train.next_batch(100)
    sess.run(train_op,feed_dict={x:xs,y:ys})
    if i%100==0:
        loss,train_accuracy = sess.run([cost,accuracy],feed_dict={x:xs,y:ys})
        print('step',i,'loss',loss,'train_accuracy',train_accuracy)

---

实际上，我不仅尝试了MNIST，还尝试了cifar-10，但遇到了相同的问题

问题是您的网络有很多层，而且每层的深度（过滤器数量）非常高。此外，您正在从头开始训练网络。而且您的MNIST数据集（60000张图像）非常少。此外，每个MNIST图像的大小仅为28x28x1

两个备选方案我可以建议你重新培训一个预先培训过的模型，即do。看看这个重量档案。该体系结构与您在代码中所做的略有不同。这种实现方式的好处是，它将抵消您拥有的很少数据

另一个更好的选择是减少层的数量和每层中过滤器的数量。通过这种方式，您可以从头开始训练模型，而且速度非常快。（假设您没有GPU）。看一看

---

希望答案对您有所帮助。

谢谢您的回答！在我减少了卷积层并在网络中使用了更小的学习率后，我得到了更好的结果。欢迎您。请接受我的答案并投票，这样会鼓励我回答更多问题。你真的需要最后一个完全连接的层上的relu吗？