Python 每个历元都有不同的数据增强参数
我正在使用keras和一个简单的cnn模型。 我想在训练中给图像添加高斯噪声。我想根据一些函数在每个历元改变噪声参数(平均值和西格玛)。比如说,Python 每个历元都有不同的数据增强参数,python,keras,Python,Keras,我正在使用keras和一个简单的cnn模型。 我想在训练中给图像添加高斯噪声。我想根据一些函数在每个历元改变噪声参数(平均值和西格玛)。比如说, in epoch 1 i want to add noise with sigma=1 in epoch 2 i want to add noise with sigma=2 in epoch 3 i want to add noise with sigma=3 # note-mean is always zero 等等 低效的解决方法是使用for循
in epoch 1 i want to add noise with sigma=1
in epoch 2 i want to add noise with sigma=2
in epoch 3 i want to add noise with sigma=3
# note-mean is always zero
等等
低效的解决方法是使用for循环,在每个历元后保存和加载模式,并调用增强函数。
更有效的方法是使用自定义回调或生成器,这是我没有成功做到的
低效方式:
total_num_of_epochs=100
def sigma_function(current_epoch):
sigma_fun=current_epoch/total_num_of_epochs
return sigma_fun
for i in range(total_num_of_epochs):
x_train += np.random.normal(mean=0,sigma=sigma_fun(i),size=x_train shape) # augment x_train based on sigma_function and current epochs
model.compile(...)
model.fit(x_train ,y_train...initial_epoch=i,epochs=i+1) #load the model
# from previous loop
save model
load model for next loop
期望的结果(我尝试使用ImageDataGenerator,但可能回调可以做到):
编辑
根据Daniel Möller提出的解决方案,我尝试了这种方法,但仍然出错
sigmaParam = 1
def apply_sigma(x):
return x + np.random.normal(mean=0,scale=sigmaParam,size=(3,32,32))
imgGen = ImageDataGenerator( preprocesing_function=apply_sigma)
generator = imgGen.flow_from_directory('data/train') # folder that contains
# only x_train and y_train
from keras.utils import Sequence
class SigmaGenerator(Sequence):
def __init__(self, keras_generator):
self.keras_generator = keras_generator
def __len__(self):
return len(self.keras_generator)
def __getitem__(self,i):
return self.keras_generator[i]
def on_epoch_end(self):
sigmaParam += 1
self.keras_generator.on_epoch_end()
training_generator = SigmaGenerator(generator)
model.fit_generator(training_generator,validation_data=(x_test,y_test),
steps_per_epoch=x_train.shape[0]//batch_size,epochs=100)
我得到的错误是:
process finished with exit code -1073741819 (0xC0000005)
您可以尝试以下方法:
sigmaParam = 1
def applySigma(x):
return x + np.random.normal(mean=0,scale=sigmaParam,size=x.shape)
创建原始生成器:
imgGen = ImageDataGenerator(..., preprocesing_function=apply_sigma)
generator = imgGen.flow_from_directory(....)
创建一个自定义生成器来包装原始生成器,替换它的on\u epoch\u end
方法来更新sigmaParam
from keras.utils import Sequence
class SigmaGenerator(Sequence):
def __init__(self, keras_generator):
self.keras_generator = keras_generator
def __len__(self):
return len(self.keras_generator)
def __getitem__(self,i):
return self.keras_generator[i]
def on_epoch_end(self):
sigmaParam += 1
self.keras_generator.on_epoch_end()
training_generator = SigmaGenerator(generator)
谢谢你的回答,但我还是有一个错误。请参阅上面的“编辑”部分。这不是完整的错误消息,您应该有一个堆栈跟踪以查看发生了什么。
from keras.utils import Sequence
class SigmaGenerator(Sequence):
def __init__(self, keras_generator):
self.keras_generator = keras_generator
def __len__(self):
return len(self.keras_generator)
def __getitem__(self,i):
return self.keras_generator[i]
def on_epoch_end(self):
sigmaParam += 1
self.keras_generator.on_epoch_end()
training_generator = SigmaGenerator(generator)