Warning: file_get_contents(/data/phpspider/zhask/data//catemap/8/sorting/2.json): failed to open stream: No such file or directory in /data/phpspider/zhask/libs/function.php on line 167

Warning: Invalid argument supplied for foreach() in /data/phpspider/zhask/libs/tag.function.php on line 1116

Notice: Undefined index: in /data/phpspider/zhask/libs/function.php on line 180

Warning: array_chunk() expects parameter 1 to be array, null given in /data/phpspider/zhask/libs/function.php on line 181
Python K.eval()在keras中出现内存不足错误_Python_Tensorflow_Keras_Out Of Memory_Tensor - Fatal编程技术网

Python K.eval()在keras中出现内存不足错误

Python K.eval()在keras中出现内存不足错误,python,tensorflow,keras,out-of-memory,tensor,Python,Tensorflow,Keras,Out Of Memory,Tensor,我正在使用keras.backend.eval()将从图像分类网络生成的超柱模型的输出张量转换为numpy数组,并且对于一些张量(如120张量)它运行良好。我正在使用一个生成器将图像逐个加载到模型的输入中,但在运行了120个张量之后,它仍然给我内存不足错误。这背后的潜在问题是什么?还有其他方法可以将张量转换为numpy数组吗 代码如下: def gethypercols(model,image): model = hypercol_model(model) output = mo

我正在使用
keras.backend.eval()
将从图像分类网络生成的超柱模型的输出张量转换为numpy数组,并且对于一些张量(如120张量)它运行良好。我正在使用一个生成器将图像逐个加载到模型的输入中,但在运行了120个张量之后,它仍然给我
内存不足
错误。这背后的潜在问题是什么?还有其他方法可以将张量转换为numpy数组吗

代码如下:

def gethypercols(model,image):
    model = hypercol_model(model)
    output = model.predict(image)
    factor = 2
    for i in range(len(output)):
            output[i] = tf.convert_to_tensor(output[i],np.float32)
            output[i] = (ZeroPadding2D((1,1)))(output[i])
            output[i] = (Conv2D(100,(3,3)))(output[i])
            if i in (0,1,2):
                    output[i] = UpSampling2D(size = (factor**2,factor**2),interpolation='bilinear')(output[i])
            elif i in (3,4,5,6):
                    output[i] = UpSampling2D(size = (factor**3,factor**3),interpolation='bilinear')(output[i])
            elif i in (7,8,9,10,11,12):
                    output[i] = UpSampling2D(size = (factor**4,factor**4),interpolation='bilinear')(output[i])
            else:
                    output[i] = UpSampling2D(size = (factor**5,factor**5),interpolation='bilinear')(output[i])
    out = Add()([output[0],output[1],output[2],output[3],output[4],output[5],output[6],output[7],output[8],output[9],output[10],output[11],output[12],output[13],output[14],output[15]])
    out = (Activation('sigmoid'))(out)
    print(out.shape)
    out = (ZeroPadding2D((1,1)))(out)
    out = (Conv2D(20,(3,3)))(out)
    print('conversion of tensor to numpy')
    outarray = tf.squeeze(out)
    outarray = K.eval(outarray)
    ourarray = np.array(outarray)
    return outarray

def hypercol_model(model):
        hypercol_model = Model(inputs=[model.input],outputs=[model.get_layer('add_1').output,model.get_layer('add_2').output,model.get_layer('add_3').output,model.get_layer('add_4').output,model.get_layer('add_5').output,model.get_layer('add_6').output,model.get_layer('add_7').output,model.get_layer('add_8').output,model.get_layer('add_9').output,model.get_layer('add_10').output,model.get_layer('add_11').output,model.get_layer('add_12').output,model.get_layer('add_13').output,model.get_layer('add_14').output,model.get_layer('add_15').output,model.get_layer('add_16').output])
        return hypercol_model
这里的模型是这样加载的图像分类模型:

model = load_model('ResnetTL4.h5',custom_objects={'precision_m': precision_m,'recall_m':recall_m})
图像由下面给出的生成器加载:

def initgenerator(batch_size,target_size,train_images_list):
        mean_pixel = np.array([104.0, 117.0, 123.0])
        train_images_list = iter(train_images_list)
        target_size = target_size[:2]
        while True:
            img_out = np.zeros(((batch_size,)+target_size+(3,)))
            for i in range(batch_size):
                img_file = next(train_images_list)
                print(img_file)                                                                                                                                                             img_path = 'JPEGImages/'+img_file+'.jpg'                                                                                                                                    print(img_path)                                                                                                                                                             # Read and resize image and mask
                img = cv2.imread(img_path, cv2.IMREAD_COLOR) # BGR                                                                                                                          img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) #RGB                                                                                                                             print(img.shape)                                                                                                                                                            img = nd.zoom(img.astype('float32'),(224/float(img.shape[0]),224/float(img.shape[1]), 1.0),order=1)                                                                        
                img = img - mean_pixel                                                                                                                                                      
                img_out[i,:,:,:] = img.astype(np.float32)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       
            yield(img_out)         
编辑2:

    def gethypercols(model,image):
        model = hypercol_model(model)
        output = model.predict(image)                                                                                                                                               
        print(len(output))
        gen = gen_model(model)
        outarray = gen.predict(image)
        print(outarray.shape)
        print(type(outarray))
        outarray = np.squeeze(outarray)
        return outarray

    def gen_model(model):
            output = model.output
            print(type(output))
            factor = 2
            print(len(output))
            for i in range(len(output)):
                    output[i] = (ZeroPadding2D((1,1)))(output[i])
                    output[i] = (Conv2D(100,(3,3)))(output[i])
                    if i in (0,1,2):
                            output[i] = UpSampling2D(size = (factor**2,factor**2),interpolation='bilinear')(output[i])
                    elif i in (3,4,5,6):
                            output[i] = UpSampling2D(size = (factor**3,factor**3),interpolation='bilinear')(output[i])
                    elif i in (7,8,9,10,11,12):
                            output[i] = UpSampling2D(size = (factor**4,factor**4),interpolation='bilinear')(output[i])
                    else:
                            output[i] = UpSampling2D(size = (factor**5,factor**5),interpolation='bilinear')(output[i])
            out = Add(name='add100')([output[0],output[1],output[2],output[3],output[4],output[5],output[6],output[7],output[8],output[9],output[10],output[11],output[12],output[13],output[14],output[15]])
            out = (Activation('sigmoid',name='act2'))(out)
            out = (ZeroPadding2D((1,1)))(out)
            out = (Conv2D(20,(3,3)))(out)
            model = Model(inputs=[model.input],outputs=[out])
            return model

    def hypercol_model(model):
            hypercol_model = Model(inputs=[model.input],outputs=[model.get_layer('add_1').output,model.get_layer('add_2').output,model.get_layer('add_3').output,model.get_layer('add_4').output,model.get_layer('add_5').output,model.get_layer('add_6').output,model.get_layer('add_7').output,model.get_layer('add_8').output,model.get_layer('add_9').output,model.get_layer('add_10').output,model.get_layer('add_11').output,model.get_layer('add_12').output,model.get_layer('add_13').output,model.get_layer('add_14').output,model.get_layer('add_15').output,model.get_layer('add_16').output])
            return hypercol_model

请添加完整的代码,您可能做错了什么,实际使用了整个内存。@MatiasValdenegro,添加了代码。我试图在这里实现超列的概念。您的代码内存效率非常低,需要将输出模型转换为张量并将其提供给另一个模型吗?您应该定义所有模型,并使用正确的数据调用predict。@MatiasValdenegro,您的意思是不需要将numpy转换为张量?我正在构建一个弱监督模型,我想将此输出存储到pickle文件中,当我试图使用
pickle.dump
将张量直接存储在pickle中时,它给了我
TypeError:cannotpickle\u thread.lock objects
错误。我还想进一步访问张量的每个元素,除了将其转换为numpy数组之外,还有其他更好的方法吗?您需要输入numpy数组并获得numpy数组输出,构建模型并使用model.predict