Warning: file_get_contents(/data/phpspider/zhask/data//catemap/2/python/361.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python 每个历元都有不同的数据增强参数_Python_Keras - Fatal编程技术网

Python 每个历元都有不同的数据增强参数

Python 每个历元都有不同的数据增强参数,python,keras,Python,Keras,我正在使用keras和一个简单的cnn模型。 我想在训练中给图像添加高斯噪声。我想根据一些函数在每个历元改变噪声参数(平均值和西格玛)。比如说, in epoch 1 i want to add noise with sigma=1 in epoch 2 i want to add noise with sigma=2 in epoch 3 i want to add noise with sigma=3 # note-mean is always zero 等等 低效的解决方法是使用for循

我正在使用keras和一个简单的cnn模型。 我想在训练中给图像添加高斯噪声。我想根据一些函数在每个历元改变噪声参数(平均值和西格玛)。比如说,

in epoch 1 i want to add noise with sigma=1
in epoch 2 i want to add noise with sigma=2
in epoch 3 i want to add noise with sigma=3
# note-mean is always zero
等等

低效的解决方法是使用for循环,在每个历元后保存和加载模式,并调用增强函数。 更有效的方法是使用自定义回调或生成器,这是我没有成功做到的

低效方式:

total_num_of_epochs=100
def sigma_function(current_epoch):
     sigma_fun=current_epoch/total_num_of_epochs
     return sigma_fun

for i in range(total_num_of_epochs):
    x_train += np.random.normal(mean=0,sigma=sigma_fun(i),size=x_train shape) # augment x_train based on sigma_function and current epochs

    model.compile(...)
    model.fit(x_train ,y_train...initial_epoch=i,epochs=i+1) #load the model 
    # from previous loop
    save model
    load model for next loop
期望的结果(我尝试使用ImageDataGenerator,但可能回调可以做到):

编辑 根据Daniel Möller提出的解决方案,我尝试了这种方法,但仍然出错

sigmaParam = 1

def apply_sigma(x):
    return x + np.random.normal(mean=0,scale=sigmaParam,size=(3,32,32))

imgGen = ImageDataGenerator( preprocesing_function=apply_sigma)
generator = imgGen.flow_from_directory('data/train') # folder that contains 
# only x_train and y_train 


from keras.utils import Sequence

class SigmaGenerator(Sequence):

    def __init__(self, keras_generator):
        self.keras_generator = keras_generator

    def __len__(self):
        return len(self.keras_generator)

    def __getitem__(self,i):
        return self.keras_generator[i]

    def on_epoch_end(self):
        sigmaParam += 1
        self.keras_generator.on_epoch_end()

training_generator = SigmaGenerator(generator)

model.fit_generator(training_generator,validation_data=(x_test,y_test),
                steps_per_epoch=x_train.shape[0]//batch_size,epochs=100)
我得到的错误是:

process finished with exit code -1073741819 (0xC0000005)
您可以尝试以下方法:

sigmaParam = 1

def applySigma(x):
    return x + np.random.normal(mean=0,scale=sigmaParam,size=x.shape)
创建原始生成器:

imgGen = ImageDataGenerator(..., preprocesing_function=apply_sigma)
generator = imgGen.flow_from_directory(....)
创建一个自定义生成器来包装原始生成器,替换它的
on\u epoch\u end
方法来更新sigmaParam

from keras.utils import Sequence

class SigmaGenerator(Sequence):

    def __init__(self, keras_generator):
        self.keras_generator = keras_generator

    def __len__(self):
        return len(self.keras_generator)

    def __getitem__(self,i):
        return self.keras_generator[i]

    def on_epoch_end(self):
        sigmaParam += 1
        self.keras_generator.on_epoch_end()

training_generator = SigmaGenerator(generator)

谢谢你的回答,但我还是有一个错误。请参阅上面的“编辑”部分。这不是完整的错误消息,您应该有一个堆栈跟踪以查看发生了什么。
from keras.utils import Sequence

class SigmaGenerator(Sequence):

    def __init__(self, keras_generator):
        self.keras_generator = keras_generator

    def __len__(self):
        return len(self.keras_generator)

    def __getitem__(self,i):
        return self.keras_generator[i]

    def on_epoch_end(self):
        sigmaParam += 1
        self.keras_generator.on_epoch_end()

training_generator = SigmaGenerator(generator)