Python 验证准确性没有提高

Python 验证准确性没有提高,python,tensorflow,machine-learning,keras,neural-network,Python,Tensorflow,Machine Learning,Keras,Neural Network,无论我使用了多少个时代或改变了学习率,我的验证准确率只保持在50年代。我现在使用1个退出层,如果我使用2个退出层,我的最大训练精度为40%,验证精度为59%。目前有一个退出层,下面是我的结果: 2527/2527 [==============================] - 26s 10ms/step - loss: 1.2076 - accuracy: 0.7944 - val_loss: 3.0905 - val_accuracy: 0.5822 Epoch 10/20 2527/2

无论我使用了多少个时代或改变了学习率,我的验证准确率只保持在50年代。我现在使用1个退出层,如果我使用2个退出层,我的最大训练精度为40%,验证精度为59%。目前有一个退出层,下面是我的结果:

2527/2527 [==============================] - 26s 10ms/step - loss: 1.2076 - accuracy: 0.7944 - val_loss: 3.0905 - val_accuracy: 0.5822
Epoch 10/20
2527/2527 [==============================] - 26s 10ms/step - loss: 1.1592 - accuracy: 0.7991 - val_loss: 3.0318 - val_accuracy: 0.5864
Epoch 11/20
2527/2527 [==============================] - 26s 10ms/step - loss: 1.1143 - accuracy: 0.8034 - val_loss: 3.0511 - val_accuracy: 0.5866
Epoch 12/20
2527/2527 [==============================] - 26s 10ms/step - loss: 1.0686 - accuracy: 0.8079 - val_loss: 3.0169 - val_accuracy: 0.5872
Epoch 13/20
2527/2527 [==============================] - 31s 12ms/step - loss: 1.0251 - accuracy: 0.8126 - val_loss: 3.0173 - val_accuracy: 0.5895
Epoch 14/20
2527/2527 [==============================] - 26s 10ms/step - loss: 0.9824 - accuracy: 0.8165 - val_loss: 3.0013 - val_accuracy: 0.5917
Epoch 15/20
2527/2527 [==============================] - 26s 10ms/step - loss: 0.9417 - accuracy: 0.8216 - val_loss: 2.9909 - val_accuracy: 0.5938
Epoch 16/20
2527/2527 [==============================] - 26s 10ms/step - loss: 0.9000 - accuracy: 0.8264 - val_loss: 3.0269 - val_accuracy: 0.5943
Epoch 17/20
2527/2527 [==============================] - 26s 10ms/step - loss: 0.8584 - accuracy: 0.8332 - val_loss: 3.0011 - val_accuracy: 0.5934
Epoch 18/20
2527/2527 [==============================] - 26s 10ms/step - loss: 0.8172 - accuracy: 0.8378 - val_loss: 2.9918 - val_accuracy: 0.5949
Epoch 19/20
2527/2527 [==============================] - 26s 10ms/step - loss: 0.7796 - accuracy: 0.8445 - val_loss: 2.9974 - val_accuracy: 0.5929
Epoch 20/20
2527/2527 [==============================] - 25s 10ms/step - loss: 0.7407 - accuracy: 0.8502 - val_loss: 3.0005 - val_accuracy: 0.5907
同样,最大值可以达到59%。以下是获得的图表:

无论我做了多少更改,验证准确率最高可达59%。 这是我的密码:

BATCH_SIZE = 64
EPOCHS = 20
LSTM_NODES = 256
NUM_SENTENCES = 3000
MAX_SENTENCE_LENGTH = 50
MAX_NUM_WORDS = 5000
EMBEDDING_SIZE = 100

encoder_inputs_placeholder = Input(shape=(max_input_len,))
x = embedding_layer(encoder_inputs_placeholder)
encoder = LSTM(LSTM_NODES, return_state=True)

encoder_outputs, h, c = encoder(x)
encoder_states = [h, c]

decoder_inputs_placeholder = Input(shape=(max_out_len,))

decoder_embedding = Embedding(num_words_output, LSTM_NODES)
decoder_inputs_x = decoder_embedding(decoder_inputs_placeholder)

decoder_lstm = LSTM(LSTM_NODES, return_sequences=True, return_state=True)
decoder_outputs, _, _ = decoder_lstm(decoder_inputs_x, initial_state=encoder_states)

decoder_dropout1 = Dropout(0.2)
decoder_outputs = decoder_dropout1(decoder_outputs)

decoder_dense1 = Dense(num_words_output, activation='softmax')
decoder_outputs = decoder_dense1(decoder_outputs)

opt = tf.keras.optimizers.RMSprop()

model = Model([encoder_inputs_placeholder,
  decoder_inputs_placeholder],
  decoder_outputs)
model.compile(
    optimizer=opt,
    loss='categorical_crossentropy',
    metrics=['accuracy']
)

history = model.fit(
    [encoder_input_sequences, decoder_input_sequences],
    decoder_targets_one_hot,
    batch_size=BATCH_SIZE,
    epochs=EPOCHS,
    validation_split=0.1,
)
我很困惑为什么只有我的训练准确度在更新,而不是验证准确度

以下是模型摘要:

Model: "model_1"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            (None, 25)           0                                            
__________________________________________________________________________________________________
input_2 (InputLayer)            (None, 23)           0                                            
__________________________________________________________________________________________________
embedding_1 (Embedding)         (None, 25, 100)      299100      input_1[0][0]                    
__________________________________________________________________________________________________
embedding_2 (Embedding)         (None, 23, 256)      838144      input_2[0][0]                    
__________________________________________________________________________________________________
lstm_1 (LSTM)                   [(None, 256), (None, 365568      embedding_1[0][0]                
__________________________________________________________________________________________________
lstm_2 (LSTM)                   [(None, 23, 256), (N 525312      embedding_2[0][0]                
                                                                 lstm_1[0][1]                     
                                                                 lstm_1[0][2]                     
__________________________________________________________________________________________________
dropout_1 (Dropout)             (None, 23, 256)      0           lstm_2[0][0]                     
__________________________________________________________________________________________________
dense_1 (Dense)                 (None, 23, 3274)     841418      dropout_1[0][0]                  
==================================================================================================
Total params: 2,869,542
Trainable params: 2,869,542
Non-trainable params: 0
__________________________________________________________________________________________________
None

训练数据集的大小小于3K。而可训练参数的数量约为300万。你的问题的答案是经典的过度拟合-模型非常庞大,只需记住训练子集而不是泛化

如何改善现状:

  • 尝试生成或查找更多数据
  • 降低模型的复杂性:
    • 使用预先训练过的嵌入(,,等)
    • 减少LSTM节点的数量

培训数据集的数量是多少?模型(model.summary())的训练参数有多少?@DanyloBaibak我编辑了我的帖子,现在已经添加了模型summary@DanyloBaibak这是我的数据集: