Machine learning Keras运行时错误：GpuCorrMM无法分配576 x 802816的工作内存
我试图在Keras中运行一个深度学习代码，但始终收到以下错误消息。我到处找了，花了很多时间，但还是没能把它修好。我是一条鱼，任何帮助都将不胜感激Machine learning Keras运行时错误：GpuCorrMM无法分配576 x 802816的工作内存,machine-learning,neural-network,runtime-error,deep-learning,keras,Machine Learning,Neural Network,Runtime Error,Deep Learning,Keras,我试图在Keras中运行一个深度学习代码，但始终收到以下错误消息。我到处找了，花了很多时间，但还是没能把它修好。我是一条鱼，任何帮助都将不胜感激 runfile('E:/dilation-keras/predict.py', wdir='E:/dilation-keras') Using Theano backend. Using gpu device 0: GeForce GT 635M (CNMeM is enabled with initial size: 90.0% of memory,
runfile('E:/dilation-keras/predict.py', wdir='E:/dilation-keras')
Using Theano backend.
Using gpu device 0: GeForce GT 635M (CNMeM is enabled with initial size: 90.0% of memory, cuDNN not available)
____________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
====================================================================================================
input_3 (InputLayer) (None, 3, 900, 900) 0
____________________________________________________________________________________________________
conv1_1 (Convolution2D) (None, 64, 898, 898) 1792 input_3[0][0]
____________________________________________________________________________________________________
conv1_2 (Convolution2D) (None, 64, 896, 896) 36928 conv1_1[0][0]
____________________________________________________________________________________________________
pool1 (MaxPooling2D) (None, 64, 448, 448) 0 conv1_2[0][0]
____________________________________________________________________________________________________
conv2_1 (Convolution2D) (None, 128, 446, 446) 73856 pool1[0][0]
____________________________________________________________________________________________________
conv2_2 (Convolution2D) (None, 128, 444, 444) 147584 conv2_1[0][0]
____________________________________________________________________________________________________
pool2 (MaxPooling2D) (None, 128, 222, 222) 0 conv2_2[0][0]
____________________________________________________________________________________________________
conv3_1 (Convolution2D) (None, 256, 220, 220) 295168 pool2[0][0]
____________________________________________________________________________________________________
conv3_2 (Convolution2D) (None, 256, 218, 218) 590080 conv3_1[0][0]
____________________________________________________________________________________________________
conv3_3 (Convolution2D) (None, 256, 216, 216) 590080 conv3_2[0][0]
____________________________________________________________________________________________________
pool3 (MaxPooling2D) (None, 256, 108, 108) 0 conv3_3[0][0]
____________________________________________________________________________________________________
conv4_1 (Convolution2D) (None, 512, 106, 106) 1180160 pool3[0][0]
____________________________________________________________________________________________________
conv4_2 (Convolution2D) (None, 512, 104, 104) 2359808 conv4_1[0][0]
____________________________________________________________________________________________________
conv4_3 (Convolution2D) (None, 512, 102, 102) 2359808 conv4_2[0][0]
____________________________________________________________________________________________________
conv5_1 (AtrousConvolution2D) (None, 512, 98, 98) 2359808 conv4_3[0][0]
____________________________________________________________________________________________________
conv5_2 (AtrousConvolution2D) (None, 512, 94, 94) 2359808 conv5_1[0][0]
____________________________________________________________________________________________________
conv5_3 (AtrousConvolution2D) (None, 512, 90, 90) 2359808 conv5_2[0][0]
____________________________________________________________________________________________________
fc6 (AtrousConvolution2D) (None, 4096, 66, 66) 102764544 conv5_3[0][0]
____________________________________________________________________________________________________
drop6 (Dropout) (None, 4096, 66, 66) 0 fc6[0][0]
____________________________________________________________________________________________________
fc7 (Convolution2D) (None, 4096, 66, 66) 16781312 drop6[0][0]
____________________________________________________________________________________________________
drop7 (Dropout) (None, 4096, 66, 66) 0 fc7[0][0]
____________________________________________________________________________________________________
fc-final (Convolution2D) (None, 21, 66, 66) 86037 drop7[0][0]
____________________________________________________________________________________________________
zeropadding2d_3 (ZeroPadding2D) (None, 21, 132, 132) 0 fc-final[0][0]
____________________________________________________________________________________________________
ct_conv1_1 (Convolution2D) (None, 42, 130, 130) 7980 zeropadding2d_3[0][0]
____________________________________________________________________________________________________
ct_conv1_2 (Convolution2D) (None, 42, 128, 128) 15918 ct_conv1_1[0][0]
____________________________________________________________________________________________________
ct_conv2_1 (AtrousConvolution2D) (None, 84, 124, 124) 31836 ct_conv1_2[0][0]
____________________________________________________________________________________________________
ct_conv3_1 (AtrousConvolution2D) (None, 168, 116, 116) 127176 ct_conv2_1[0][0]
____________________________________________________________________________________________________
ct_conv4_1 (AtrousConvolution2D) (None, 336, 100, 100) 508368 ct_conv3_1[0][0]
____________________________________________________________________________________________________
ct_conv5_1 (AtrousConvolution2D) (None, 672, 68, 68) 2032800 ct_conv4_1[0][0]
____________________________________________________________________________________________________
ct_fc1 (Convolution2D) (None, 672, 66, 66) 4064928 ct_conv5_1[0][0]
____________________________________________________________________________________________________
ct_final (Convolution2D) (None, 21, 66, 66) 14133 ct_fc1[0][0]
____________________________________________________________________________________________________
permute_5 (Permute) (None, 66, 66, 21) 0 ct_final[0][0]
____________________________________________________________________________________________________
reshape_5 (Reshape) (None, 4356, 21) 0 permute_5[0][0]
____________________________________________________________________________________________________
activation_3 (Activation) (None, 4356, 21) 0 reshape_5[0][0]
____________________________________________________________________________________________________
reshape_6 (Reshape) (None, 66, 66, 21) 0 activation_3[0][0]
____________________________________________________________________________________________________
permute_6 (Permute) (None, 21, 66, 66) 0 reshape_6[0][0]
====================================================================================================
Total params: 141,149,720
Trainable params: 141,149,720
Non-trainable params: 0
____________________________________________________________________________________________________
batch_size is: 1
Traceback (most recent call last):
File "<ipython-input-3-641fac717a39>", line 1, in <module>
runfile('E:/dilation-keras/predict.py', wdir='E:/dilation-keras')
File "c:\users\lenovo\anaconda2\lib\site-packages\spyder\utils\site\sitecustomize.py", line 866, in runfile
execfile(filename, namespace)
File "c:\users\lenovo\anaconda2\lib\site-packages\spyder\utils\site\sitecustomize.py", line 87, in execfile
exec(compile(scripttext, filename, 'exec'), glob, loc)
File "E:/dilation-keras/predict.py", line 74, in <module>
y_img = predict(im, model, ds)
File "E:/dilation-keras/predict.py", line 46, in predict
prob = model.predict(model_in,batch_size=batch_size)[0]
File "c:\users\lenovo\anaconda2\lib\site-packages\keras\engine\training.py", line 1272, in predict
batch_size=batch_size, verbose=verbose)
File "c:\users\lenovo\anaconda2\lib\site-packages\keras\engine\training.py", line 945, in _predict_loop
batch_outs = f(ins_batch)
File "c:\users\lenovo\anaconda2\lib\site-packages\keras\backend\theano_backend.py", line 959, in __call__
return self.function(*inputs)
File "c:\users\lenovo\anaconda2\lib\site-packages\theano\compile\function_module.py", line 886, in __call__
storage_map=getattr(self.fn, 'storage_map', None))
File "c:\users\lenovo\anaconda2\lib\site-packages\theano\gof\link.py", line 325, in raise_with_op
reraise(exc_type, exc_value, exc_trace)
File "c:\users\lenovo\anaconda2\lib\site-packages\theano\compile\function_module.py", line 873, in __call__
self.fn() if output_subset is None else\
RuntimeError: GpuCorrMM failed to allocate working memory of 576 x 802816
Apply node that caused the error: GpuCorrMM{valid, (1, 1), (1, 1)}(GpuContiguous.0, GpuContiguous.0)
Toposort index: 95
Inputs types: [CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D)]
Inputs shapes: [(1, 64, 898, 898), (64, 64, 3, 3)]
Inputs strides: [(0, 806404, 898, 1), (576, 9, 3, 1)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[GpuElemwise{Composite{(i0 * ((i1 + i2) + Abs((i1 + i2))))}}[(0, 1)](CudaNdarrayConstant{[[[[ 0.5]]]]}, GpuCorrMM{valid, (1, 1), (1, 1)}.0, GpuDimShuffle{x,0,x,x}.0)]]
HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
runfile（'E:/diability keras/predict.py'，wdir='E:/diability keras'）
使用Theano后端。
使用gpu设备0:GeForce GT 635M（启用CNMeM时初始大小为：90.0%内存，cuDNN不可用）
____________________________________________________________________________________________________
层（类型）输出形状参数#连接到
====================================================================================================
输入_3（输入层）（无，3900900）0
____________________________________________________________________________________________________
conv1_1（卷积2d）（无，64898898）1792输入_3[0][0]
____________________________________________________________________________________________________
conv1_2（卷积2d）（无，64896896）36928 conv1_1[0][0]
____________________________________________________________________________________________________
pool1（MaxPoolg2D）（无，64448448）0 conv1_2[0][0]
____________________________________________________________________________________________________
conv2_1（卷积2D）（无，128446446）73856池1[0][0]
____________________________________________________________________________________________________
conv2_2（卷积2d）（无，1284444444）147584 conv2_1[0][0]
____________________________________________________________________________________________________
pool2（MaxPoolg2D）（无、128、222、222）0 conv2_2[0][0]
____________________________________________________________________________________________________
conv3_1(卷积2d)(无,256,220,220)295168 pool2[0][0]
____________________________________________________________________________________________________
conv3_2（卷积2d）（无，256，218，218）59008 conv3_1[0][0]
____________________________________________________________________________________________________
conv3_3（卷积2d）（无，256，216，216）59008 conv3_2[0][0]
____________________________________________________________________________________________________
pool3（MaxPoolg2D）（无、256、108、108）0 conv3_3[0][0]
____________________________________________________________________________________________________
conv4_1（卷积2D）（无、512、106、106）1180160池3[0][0]
____________________________________________________________________________________________________
conv4_2（卷积2d）（无、512、104、104）2359808 conv4_1[0][0]
____________________________________________________________________________________________________
conv4_3（卷积2d）（无、512、102、102）2359808 conv4_2[0][0]
____________________________________________________________________________________________________
conv5_1（atrusconvolution2d）（无、512、98、98）2359808 conv4_3[0][0]
____________________________________________________________________________________________________
conv5_2（atrusconvolution2d）（无、512、94、94）2359808 conv5_1[0][0]
____________________________________________________________________________________________________
conv5_3（atrusconvolution2d）（无、512、90、90）2359808 conv5_2[0][0]
____________________________________________________________________________________________________
fc6（AtrusConvolution2D）（无、4096、66、66）102764544 conv5_3[0][0]
____________________________________________________________________________________________________
辍学6（辍学）（无、4096、66、66）0 fc6[0][0]
____________________________________________________________________________________________________
fc7（卷积2D）（无、4096、66、66）16781312下降6[0][0]
____________________________________________________________________________________________________
辍学7（辍学）（无、4096、66、66）0 fc7[0][0]
____________________________________________________________________________________________________
fc最终（卷积2D）（无、21、66、66）86037下降7[0][0]
____________________________________________________________________________________________________
零填充2D_3（零填充2D）（无、21132132）0 fc最终[0][0]
____________________________________________________________________________________________________
ct_conv1_1（卷积2D）（无，42130130）7980零填充2D_3[0][0]
____________________________________________________________________________________________________
ct_conv1_2（卷积2D）（无，42，128，128）15918 ct_conv1_1[0][0]
____________________________________________________________________________________________________
ct_conv2_1（atrusconvolution2d）（无，84124124）31836 ct_conv1_2[0][0]
____________________________________________