Python 为什么fit_发生器的精度与Keras中评估_发生器的精度不同?
我的工作:Python 为什么fit_发生器的精度与Keras中评估_发生器的精度不同?,python,tensorflow,machine-learning,keras,conv-neural-network,Python,Tensorflow,Machine Learning,Keras,Conv Neural Network,我的工作: 我正在用Kerasfit\u generator()培训一名经过预先培训的CNN。这将在每个历元之后生成评估指标(损失、acc、val_损失、val_acc)。在对模型进行培训后,我使用evaluate\u generator()生成评估指标(loss,acc) 我的期望: 如果我对模型进行一个历元的训练,我希望通过fit\u generator()和evaluate\u generator()获得的度量是相同的。它们都应该基于整个数据集导出度量 我观察到的: loss和a
- 我正在用Keras
培训一名经过预先培训的CNN。这将在每个历元之后生成评估指标(fit\u generator()
)。在对模型进行培训后,我使用损失、acc、val_损失、val_acc
生成评估指标(evaluate\u generator()
)loss,acc
- 如果我对模型进行一个历元的训练,我希望通过
和fit\u generator()
获得的度量是相同的。它们都应该基于整个数据集导出度量evaluate\u generator()
和loss
都不同于acc
和fit\u generator()
:evaluate\u generator()
- 为什么
的精度不高 与fit\u generator()
evaluate\u generator()
def generate_data(path, imagesize, nBatches):
datagen = ImageDataGenerator(rescale=1./255)
generator = datagen.flow_from_directory\
(directory=path, # path to the target directory
target_size=(imagesize,imagesize), # dimensions to which all images found will be resize
color_mode='rgb', # whether the images will be converted to have 1, 3, or 4 channels
classes=None, # optional list of class subdirectories
class_mode='categorical', # type of label arrays that are returned
batch_size=nBatches, # size of the batches of data
shuffle=True) # whether to shuffle the data
return generator
[……]
[……]
[……]
更新 我发现一些网站对此问题进行了报道:
acc
和loss
仍然不同于fit_generator()
和evaluate_generator()
,即使使用同一台发电机生成的完全相同的数据进行培训和验证。以下是我尝试过的:
- 静态设置整个脚本的学习阶段,或在向预先训练的层添加新层之前
- 从预先训练的模型中解冻所有批处理规范化层
- 不将辍学或批量标准化添加为未经培训的图层
- 使用不同的预训练CNN作为基础模型(VGG19、InceptionV3、InceptionResNetV2、Exception)
请让我知道我是否缺少其他解决方案。在这种情况下,一个时代的培训可能不够有用。此外,您的训练和测试数据可能不完全相同,因为您没有为
flow\u from\u directory
方法设置随机种子。看一看
也许,您可以设置一个种子,删除增强(如果有的话),并保存经过训练的模型权重,以便稍后加载以进行检查。我现在设法使用相同的评估指标。我更改了以下内容:
- 我按照@Anakin的建议,在
中设置flow\u from\u directory()
seed
- 我在
中根据警告设置fit\u generator()
:use\u multiprocessing=False
use\u multiprocessing=True,多个工作人员可能会复制您的数据
- 我统一了python设置,如关于如何在开发过程中使用Keras获得可复制结果的文章中所建议的那样
- 我现在使用以下工具生成数据,而不是使用
重新缩放输入图像:datagen=ImageDataGenerator(rescale=1./255)
有了这个,我设法从
fit_generator()
和evaluate_generator()
获得了类似的精确度和损耗。此外,使用相同的数据进行培训和测试现在会产生类似的指标剩余差异的原因在。集合在fit\u生成器
级别使用多处理=False
解决了问题,但大大降低了培训速度。一个更好但仍不完善的解决方法是,将验证生成器设置为仅使用_multiprocessing=False,如下代码是从keras的fit_生成器函数修改而来的
...
try:
if do_validation:
if val_gen and workers > 0:
# Create an Enqueuer that can be reused
val_data = validation_data
if isinstance(val_data, Sequence):
val_enqueuer = OrderedEnqueuer(val_data,
**use_multiprocessing=False**)
validation_steps = len(val_data)
else:
val_enqueuer = GeneratorEnqueuer(val_data,
**use_multiprocessing=False**)
val_enqueuer.start(workers=workers,
max_queue_size=max_queue_size)
val_enqueuer_gen = val_enqueuer.get()
...
请参考此链接。培训损失是每批培训数据损失的平均值,当模型进行培训时,此损失将在每批中发生变化。尝试创建验证生成器的两个实例,将一个实例传递给model.fit,另一个实例传递给evaluation_generator,并查看是否生成相同的结果。由于在许多情况下,批次大小并不精确地划分样本数量,因此在确定步骤数量时,整数划分可能会跳过一个批次,然后由评估生成器使用该批次,从而产生稍微不同的度量。
def evaluate_model(model, generator):
score = model.evaluate_generator(generator=generator, # Generator yielding tuples
steps=
generator.samples//nBatches) # number of steps (batches of samples) to yield from generator before stopping
print("%s: Model evaluated:"
"\n\t\t\t\t\t\t Loss: %.3f"
"\n\t\t\t\t\t\t Accuracy: %.3f" %
(datetime.now().strftime('%Y-%m-%d_%H-%M-%S'),
score[0], score[1]))
def main():
# Create model
modelUntrained = create_model(imagesize, nBands, nClasses)
# Prepare training and validation data
trainGenerator = generate_data(imagePathTraining, imagesize, nBatches)
valGenerator = generate_data(imagePathValidation, imagesize, nBatches)
# Train and save model
history, modelTrained = train_model(modelUntrained, nBatches, nEpochs, trainGenerator, valGenerator, resultPath)
# Evaluate on validation data
print("%s: Model evaluation (valX, valY):" % datetime.now().strftime('%Y-%m-%d_%H-%M-%S'))
evaluate_model(modelTrained, valGenerator)
# Evaluate on training data
print("%s: Model evaluation (trainX, trainY):" % datetime.now().strftime('%Y-%m-%d_%H-%M-%S'))
evaluate_model(modelTrained, trainGenerator)
K.set_learning_phase(0) # testing
K.set_learning_phase(1) # training
for i in range(len(model.layers)):
if str.startswith(model.layers[i].name, 'bn'):
model.layers[i].trainable=True
# Create pre-trained base model
basemodel = ResNet50(include_top=False, # exclude final pooling and fully connected layer in the original model
weights='imagenet', # pre-training on ImageNet
input_tensor=None, # optional tensor to use as image input for the model
input_shape=(imagesize, # shape tuple
imagesize,
nBands),
pooling=None, # output of the model will be the 4D tensor output of the last convolutional layer
classes=nClasses) # number of classes to classify images into
# Create new untrained layers
x = basemodel.output
x = GlobalAveragePooling2D()(x) # global spatial average pooling layer
x = Dense(1024, activation='relu')(x) # fully-connected layer
y = Dense(nClasses, activation='softmax')(x) # logistic layer making sure that probabilities sum up to 1
# Create model combining pre-trained base model and new untrained layers
model = Model(inputs=basemodel.input,
outputs=y)
# Freeze weights on pre-trained layers
for layer in basemodel.layers:
layer.trainable = False
# Define learning optimizer
learningRate = 0.01
optimizerSGD = optimizers.SGD(lr=learningRate, # learning rate.
momentum=0.9, # parameter that accelerates SGD in the relevant direction and dampens oscillations
decay=learningRate/nEpochs, # learning rate decay over each update
nesterov=True) # whether to apply Nesterov momentum
# Compile model
model.compile(optimizer=optimizerSGD, # stochastic gradient descent optimizer
loss='categorical_crossentropy', # objective function
metrics=['accuracy'], # metrics to be evaluated by the model during training and testing
loss_weights=None, # scalar coefficients to weight the loss contributions of different model outputs
sample_weight_mode=None, # sample-wise weights
weighted_metrics=None, # metrics to be evaluated and weighted by sample_weight or class_weight during training and testing
target_tensors=None) # tensor model's target, which will be fed with the target data during training
from keras.applications.vgg19 import VGG19
basemodel = VGG19(include_top=False, # exclude final pooling and fully connected layer in the original model
weights='imagenet', # pre-training on ImageNet
input_tensor=None, # optional tensor to use as image input for the model
input_shape=(imagesize, # shape tuple
imagesize,
nBands),
pooling=None, # output of the model will be the 4D tensor output of the last convolutional layer
classes=nClasses) # number of classes to classify images into
def generate_data(path, imagesize, nBatches):
datagen = ImageDataGenerator(rescale=1./255)
generator = datagen.flow_from_directory(directory=path, # path to the target directory
target_size=(imagesize,imagesize), # dimensions to which all images found will be resize
color_mode='rgb', # whether the images will be converted to have 1, 3, or 4 channels
classes=None, # optional list of class subdirectories
class_mode='categorical', # type of label arrays that are returned
batch_size=nBatches, # size of the batches of data
shuffle=True, # whether to shuffle the data
seed=42) # random seed for shuffling and transformations
return generator
history = model.fit_generator(generator=trainGenerator,
steps_per_epoch=trainGenerator.samples//nBatches, # total number of steps (batches of samples)
epochs=nEpochs, # number of epochs to train the model
verbose=2, # verbosity mode. 0 = silent, 1 = progress bar, 2 = one line per epoch
callbacks=callback, # keras.callbacks.Callback instances to apply during training
validation_data=valGenerator, # generator or tuple on which to evaluate the loss and any model metrics at the end of each epoch
validation_steps=
valGenerator.samples//nBatches, # number of steps (batches of samples) to yield from validation_data generator before stopping at the end of every epoch
class_weight=None, # optional dictionary mapping class indices (integers) to a weight (float) value, used for weighting the loss function
max_queue_size=10, # maximum size for the generator queue
workers=1, # maximum number of processes to spin up when using process-based threading
use_multiprocessing=False, # whether to use process-based threading
shuffle=False, # whether to shuffle the order of the batches at the beginning of each epoch
initial_epoch=0) # epoch at which to start training
import tensorflow as tf
import random as rn
from keras import backend as K
np.random.seed(42)
rn.seed(12345)
session_conf = tf.ConfigProto(intra_op_parallelism_threads=1,
inter_op_parallelism_threads=1)
tf.set_random_seed(1234)
sess = tf.Session(graph=tf.get_default_graph(), config=session_conf)
K.set_session(sess)
from keras.applications.resnet50 import preprocess_input
datagen = ImageDataGenerator(preprocessing_function=preprocess_input)
...
try:
if do_validation:
if val_gen and workers > 0:
# Create an Enqueuer that can be reused
val_data = validation_data
if isinstance(val_data, Sequence):
val_enqueuer = OrderedEnqueuer(val_data,
**use_multiprocessing=False**)
validation_steps = len(val_data)
else:
val_enqueuer = GeneratorEnqueuer(val_data,
**use_multiprocessing=False**)
val_enqueuer.start(workers=workers,
max_queue_size=max_queue_size)
val_enqueuer_gen = val_enqueuer.get()
...