Python 对模型进行良好的培训会导致糟糕的结果
我有一个属于6类的图像数据集。我将数据分为训练集和测试集,测试规模为10%。我使用Python 对模型进行良好的培训会导致糟糕的结果,python,tensorflow,machine-learning,keras,deep-learning,Python,Tensorflow,Machine Learning,Keras,Deep Learning,我有一个属于6类的图像数据集。我将数据分为训练集和测试集,测试规模为10%。我使用ImageDataGenerator加载和扩充数据。这里我创建了两个生成器。一个用于培训,一个用于验证,这是培训数据的子集 # Create train_generator train_generator = train_datagen.flow_from_directory( train_path, # Path to data target_size=(30, 30), # Average
ImageDataGenerator
加载和扩充数据。这里我创建了两个生成器。一个用于培训,一个用于验证,这是培训数据的子集
# Create train_generator
train_generator = train_datagen.flow_from_directory(
train_path, # Path to data
target_size=(30, 30), # Average target size (32 + 28)/2 = 30
batch_size=32, # Batch size
class_mode='categorical', # Categorical Class mode
classes=classes, # Classes
subset='training', # Training Subset
color_mode='grayscale'
)
现在我已经为100个时代训练了一个简单的自制模型
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 28, 28, 128) 1280
_________________________________________________________________
activation (Activation) (None, 28, 28, 128) 0
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 14, 14, 128) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 12, 12, 64) 73792
_________________________________________________________________
activation_1 (Activation) (None, 12, 12, 64) 0
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 6, 6, 64) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 4, 4, 32) 18464
_________________________________________________________________
activation_2 (Activation) (None, 4, 4, 32) 0
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 2, 2, 32) 0
_________________________________________________________________
flatten (Flatten) (None, 128) 0
_________________________________________________________________
dense (Dense) (None, 128) 16512
_________________________________________________________________
activation_3 (Activation) (None, 128) 0
_________________________________________________________________
dropout (Dropout) (None, 128) 0
_________________________________________________________________
dense_1 (Dense) (None, 6) 774
_________________________________________________________________
activation_4 (Activation) (None, 6) 0
=================================================================
Total params: 110,822
Trainable params: 110,822
Non-trainable params: 0
训练的历史是这样的。
之后,我创建了一个测试生成器,如
test_generator = ImageDataGenerator(rescale=1./255).flow_from_directory(
test_path, # Path to data
target_size=(30, 30), # Average target size (32 + 28)/2 = 30
batch_size=32, # Batch size
class_mode='categorical', # Categorical Class mode
classes=classes, # Classes
color_mode='grayscale'
)
我用这个生成器做预测
#Confution Matrix and Classification Report
Y_pred = model.predict_generator(test_generator, test_generator.n // test_generator.batch_size+1)
y_pred = np.argmax(Y_pred, axis=1)
现在,当我用这些预测制作混乱矩阵和分类报告时,我得到了非常糟糕的结果
Confusion Matrix:
[[166 31 1 140 135 152]
[ 33 4 0 20 25 27]
[ 17 1 0 10 11 10]
[130 25 1 111 115 142]
[126 17 2 107 81 124]
[153 24 1 141 129 159]]
如果我做错了什么,请您指导我或指导我解决问题。答案不完整,但您的测试集准确性/损失正在恶化,这一事实强烈表明模型拟合过度。根据架构和数据集的不同,100个纪元通常是多余的。我可以尝试提前停止。但今天早上早些时候,我只对模型进行了10个时代的训练,根本没有过度拟合。但是模型在数据上的表现仍然很差。我想新生成器中的类顺序可能会发生变化?或者您忘记了
列车生成器
中的某个预处理,但新生成器中没有?
Confusion Matrix:
[[166 31 1 140 135 152]
[ 33 4 0 20 25 27]
[ 17 1 0 10 11 10]
[130 25 1 111 115 142]
[126 17 2 107 81 124]
[153 24 1 141 129 159]]
Classification Report:
precision recall f1-score support
Bag 0.27 0.27 0.27 625
Sandal 0.04 0.04 0.04 109
automobile 0.00 0.00 0.00 49
bird 0.21 0.21 0.21 524
truck 0.16 0.18 0.17 457
Ankle boot 0.26 0.26 0.26 607
accuracy 0.22 2371
macro avg 0.16 0.16 0.16 2371
weighted avg 0.22 0.22 0.22 2371