Tensorflow 深度学习：从第一个时代开始，培训和验证损失之间的巨大差异_Tensorflow_Neural Network_Keras_Deep Learning_Convolution

Tensorflow 深度学习：从第一个时代开始，培训和验证损失之间的巨大差异

tensorflow neural-network keras deep-learning

Tensorflow 深度学习：从第一个时代开始，培训和验证损失之间的巨大差异,tensorflow,neural-network,keras,deep-learning,convolution,Tensorflow,Neural Network,Keras,Deep Learning,Convolution,我的神经网络是本文提出的模型的稍微修改版本：我的目标是将文本分为9个不同的类别。我使用了29个卷积层，并将任何文本的最大长度设置为256个字符培训数据有900k个样本，验证数据有35k个样本。数据是非常不平衡的，因此我做了一些数据扩充来平衡训练数据（显然没有触及验证数据），然后在训练中使用类权重 Layer (type) Output Shape Param # =================================

我的神经网络是本文提出的模型的稍微修改版本：

我的目标是将文本分为9个不同的类别。我使用了29个卷积层，并将任何文本的最大长度设置为256个字符

培训数据有900k个样本，验证数据有35k个样本。数据是非常不平衡的，因此我做了一些数据扩充来平衡训练数据（显然没有触及验证数据），然后在训练中使用类权重

Layer (type)                 Output Shape              Param #   
=================================================================
input (InputLayer)           (None, 256)               0         
_________________________________________________________________
embedding_1 (Embedding)      (None, 256, 16)           1152      
_________________________________________________________________
conv1d_1 (Conv1D)            (None, 256, 64)           3136      
_________________________________________________________________
sequential_1 (Sequential)    (None, 256, 64)           25216     
_________________________________________________________________
sequential_2 (Sequential)    (None, 256, 64)           25216     
_________________________________________________________________
sequential_3 (Sequential)    (None, 256, 64)           25216     
_________________________________________________________________
sequential_4 (Sequential)    (None, 256, 64)           25216     
_________________________________________________________________
sequential_5 (Sequential)    (None, 256, 64)           25216     
_________________________________________________________________
sequential_6 (Sequential)    (None, 256, 64)           25216     
_________________________________________________________________
sequential_7 (Sequential)    (None, 256, 64)           25216     
_________________________________________________________________
sequential_8 (Sequential)    (None, 256, 64)           25216     
_________________________________________________________________
sequential_9 (Sequential)    (None, 256, 64)           25216     
_________________________________________________________________
sequential_10 (Sequential)   (None, 256, 64)           25216     
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 128, 64)           0         
_________________________________________________________________
sequential_11 (Sequential)   (None, 128, 128)          75008     
_________________________________________________________________
sequential_12 (Sequential)   (None, 128, 128)          99584     
_________________________________________________________________
sequential_13 (Sequential)   (None, 128, 128)          99584     
_________________________________________________________________
sequential_14 (Sequential)   (None, 128, 128)          99584     
_________________________________________________________________
sequential_15 (Sequential)   (None, 128, 128)          99584     
_________________________________________________________________
sequential_16 (Sequential)   (None, 128, 128)          99584     
_________________________________________________________________
sequential_17 (Sequential)   (None, 128, 128)          99584     
_________________________________________________________________
sequential_18 (Sequential)   (None, 128, 128)          99584     
_________________________________________________________________
sequential_19 (Sequential)   (None, 128, 128)          99584     
_________________________________________________________________
sequential_20 (Sequential)   (None, 128, 128)          99584     
_________________________________________________________________
max_pooling1d_2 (MaxPooling1 (None, 64, 128)           0         
_________________________________________________________________
sequential_21 (Sequential)   (None, 64, 256)           297472    
_________________________________________________________________
sequential_22 (Sequential)   (None, 64, 256)           395776    
_________________________________________________________________
sequential_23 (Sequential)   (None, 64, 256)           395776    
_________________________________________________________________
sequential_24 (Sequential)   (None, 64, 256)           395776    
_________________________________________________________________
max_pooling1d_3 (MaxPooling1 (None, 32, 256)           0         
_________________________________________________________________
sequential_25 (Sequential)   (None, 32, 512)           1184768   
_________________________________________________________________
sequential_26 (Sequential)   (None, 32, 512)           1577984   
_________________________________________________________________
sequential_27 (Sequential)   (None, 32, 512)           1577984   
_________________________________________________________________
sequential_28 (Sequential)   (None, 32, 512)           1577984   
_________________________________________________________________
lambda_1 (Lambda)            (None, 4096)              0         
_________________________________________________________________
dense_1 (Dense)              (None, 2048)              8390656   
_________________________________________________________________
dense_2 (Dense)              (None, 2048)              4196352   
_________________________________________________________________
dense_3 (Dense)              (None, 9)                 18441     
=================================================================
Total params: 21,236,681
Trainable params: 21,216,713
Non-trainable params: 19,968

对于给定的模型，我收到以下结果：

对我来说，损失曲线看起来很奇怪，因为我无法发现曲线上典型的过度拟合效应，但训练和验证损失之间的差异仍然很大。此外，在第1个历元，训练损失远低于任何历元的验证损失

这是我应该担心的吗？我该如何改进我的模型

谢谢

为了缩小培训和验证错误之间的差距，我建议两件事：

如果可以找到，请尝试使用更多数据
在密集层之间使用衰减层进行正则化

仅供参考，此差距称为概括错误：）