Python Keras:ResourceExhausterRor（回溯见上文）：当使用形状[26671,32,32,64]分配张量时，OOM

我正在tensorflow后端（Keras版本2.1）上使用Keras训练我的网络，我在互联网上尝试了许多可用的东西，但没有找到任何解决方案

My Training set and labels: 26721(each image have size (32, 32,1)) , (26721, 10) 
Validation set and labels:  6680(each image have size(32,32,1), (6680,10)

到目前为止，这是我的模型，我正在使用Python3

def CNN(input_, num_classes):

model = Sequential()

model.add(Convolution2D(16, kernel_size=(7, 7),  border_mode='same',
                 input_shape=input_))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2), strides=(1, 1) ,  border_mode='same' ))
model.add(Convolution2D(64, (3, 3),  padding ='same'))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(3, 3), strides=(1,1),  border_mode='same' ))
model.add(Flatten())
model.add(Dense(96))
model.add(Activation('relu'))

model.add(Dense(num_classes))
model.add(Activation('softmax'))
return model

model = CNN(image_size, num_classes)

model.compile(loss=keras.losses.categorical_crossentropy,
          optimizer=keras.optimizers.SGD(lr=0.01),
          metrics=['accuracy'])

print(model.summary())
csv_logger = CSVLogger('training.log')
early_stop = EarlyStopping('val_acc', patience=200, verbose=1)
model_checkpoint = ModelCheckpoint(model_save_path,
                                    'val_acc', verbose=0,
                                    save_best_only=True)

model_callbacks = [early_stop, model_checkpoint, csv_logger]
# print "len(train_dataset) ", len(train_dataset)
print("int(len(train_dataset)/batch_size) ", int(len(train_dataset)/batch_size))
K.get_session().run(tf.global_variables_initializer())
 model.fit_generator(train,
              steps_per_epoch=np.ceil(len(train_dataset)/batch_size),
              epochs=num_epochs,
              verbose=1,
              validation_data=valid,
              validation_steps=batch_size,
              callbacks=model_callbacks)

模型摘要：

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 32, 32, 16)        800       
_________________________________________________________________
batch_normalization_1 (Batch (None, 32, 32, 16)        64        
_________________________________________________________________
activation_1 (Activation)    (None, 32, 32, 16)        0         
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 32, 32, 16)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 32, 32, 64)        9280      
_________________________________________________________________
batch_normalization_2 (Batch (None, 32, 32, 64)        256       
_________________________________________________________________
activation_2 (Activation)    (None, 32, 32, 64)        0         
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 32, 32, 64)        0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 65536)             0         
_________________________________________________________________
dense_1 (Dense)              (None, 96)                6291552   
_________________________________________________________________
activation_3 (Activation)    (None, 96)                0         
_________________________________________________________________
dense_2 (Dense)              (None, 10)                970       
_________________________________________________________________
activation_4 (Activation)    (None, 10)                0         
=================================================================
Total params: 6,302,922
Trainable params: 6,302,762
Non-trainable params: 160

我根据批量大小发送图像。这是我的发电机功能：

# Generate images according to batch size


def gen(dataset, labels, batch_size):

images = []
digits = []
i = 0
while True:
    images.append(dataset[i])
    digits.append(labels[i]) 
    i+=1
    if i == batch_size:
        yield (np.array(images), np.array(digits))
        images = []
        digits = []
    # Generate remaining images also
    if i == len(dataset):
        yield (np.array(images), np.array(digits))
        images, digits = [], []
        i = 0

   train = gen(train_data, train_labels, batch_size)
   valid = gen(valid_data, valid_lables, batch_size)

终端上的错误日志：

请检查此链接是否存在完全错误：

谁能帮帮我，我做错了什么

---

提前感谢

您正在整个列车组上训练您的网络，该列车组太大，无法放入内存，而且对于您的gpu来说太大

机器学习的标准是创建小批量的数据并对其进行培训。批量大小通常为16、32、64或其他2的幂，但它可以是任何东西，通常必须通过交叉验证找到正确的批量大小。

从日志中可以看出，在分配edge_1094_丢失之前，内存已经满了。检查使用中的限值

---

这可能是因为旧型号占用了内存。解决这个问题的快速方法就是简单地终止这个进程。这将释放旧型号所消耗的所有内存，这些内存可能没有被垃圾收集。

我正在发送一批100幅图像，我在post中编辑了gen函数，我正在使用generator将图像发送到网络。您的generator函数出现故障，因为它在某个地方生成了一批26671幅图像。每次迭代都打印批处理的大小，以查看出错的地方我的配置与以前的配置完全相同：批处理大小=8，而现在，它不工作了。