Python 用Chainer编写的CNN模型的训练代码
我正在编写训练代码,它是一个双流卷积神经网络。该模型通过网络的两个流预测正在评估的补丁的质量分数。在下面的培训中,我使用了上面GitHub链接中提供的测试数据集 培训代码如下:Python 用Chainer编写的CNN模型的训练代码,python,conv-neural-network,training-data,chainer,Python,Conv Neural Network,Training Data,Chainer,我正在编写训练代码,它是一个双流卷积神经网络。该模型通过网络的两个流预测正在评估的补丁的质量分数。在下面的培训中,我使用了上面GitHub链接中提供的测试数据集 培训代码如下: ## prepare training data test_label_path = 'data_list/test.txt' test_img_path = 'data/live/' test_Graimg_path = 'data/live_grad/' save_model_path = '/models/nr_
## prepare training data
test_label_path = 'data_list/test.txt'
test_img_path = 'data/live/'
test_Graimg_path = 'data/live_grad/'
save_model_path = '/models/nr_sana_2stream.model'
patches_per_img = 256
patchSize = 32
print('-------------Load data-------------')
final_train_set = []
with open(test_label_path, 'rt') as f:
for l in f:
line, la = l.strip().split() # for debug
tic = time.time()
full_path = os.path.join(test_img_path, line)
Grafull_path = os.path.join(test_Graimg_path, line)
f = Image.open(full_path)
Graf = Image.open(Grafull_path)
img = np.asarray(f, dtype=np.float32)
Gra = np.asarray(Graf, dtype=np.float32)
img = img.transpose(2, 0, 1)
Gra = Gra.transpose(2, 0, 1)
img1 = np.zeros((1, 3, Gra.shape[1], Gra.shape[2]))
img1[0, :, :, :] = img
Gra1 = np.zeros((1, 3, Gra.shape[1], Gra.shape[2]))
Gra1[0, :, :, :] = Gra
patches = extract_patches(img, (3, patchSize, patchSize), patchSize)
Grapatches = extract_patches(Gra, (3, patchSize, patchSize), patchSize)
X = patches.reshape((-1, 3, patchSize, patchSize))
GraX = Grapatches.reshape((-1, 3, patchSize, patchSize))
temp_slice1 = [X[int(float(index))] for index in range(256)]
temp_slice2 = [GraX[int(float(index))] for index in range(256)]
##############################################
for j in range(len(temp_slice1)):
temp_slice1[j] = xp.array(temp_slice1[j].astype(np.float32))
temp_slice2[j] = xp.array(temp_slice2[j].astype(np.float32))
final_train_set.append((temp_slice1[j], temp_slice2[j], int(la)))
final_train_set = np.asarray(final_train_set)
##############################################
#
print('--------------Done!----------------')
print('--------------Iterator!----------------')
train_iter = iterators.SerialIterator(final_train_set, batch_size=4)
optimizer = optimizers.Adam()
optimizer.use_cleargrads()
optimizer.setup(model)
updater = training.StandardUpdater(train_iter, optimizer, device=0)
print('--------------Trainer!----------------')
trainer = training.Trainer(updater, (50, 'epoch'), out='result')
trainer.extend(extensions.LogReport())
trainer.extend(extensions.PrintReport(['epoch', 'iteration', 'main/loss', 'elapsed_time']))
print('--------------Running trainer!----------------')
trainer.run()
但代码产生了如下错误:
Exception in main training loop: Unsupported dtype object
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/chainer/training/trainer.py", line 307, in run
update()
File "/usr/local/lib/python2.7/dist-packages/chainer/training/updaters/standard_updater.py", line 165, in update
self.update_core()
File "/usr/local/lib/python2.7/dist-packages/chainer/training/updaters/standard_updater.py", line 171, in update_core
in_arrays = self.converter(batch, self.device)
File "/usr/local/lib/python2.7/dist-packages/chainer/dataset/convert.py", line 149, in concat_examples
return to_device(device, _concat_arrays(batch, padding))
File "/usr/local/lib/python2.7/dist-packages/chainer/dataset/convert.py", line 37, in to_device
return cuda.to_gpu(x, device)
File "/usr/local/lib/python2.7/dist-packages/chainer/backends/cuda.py", line 288, in to_gpu
return _array_to_gpu(array, device_, stream)
File "/usr/local/lib/python2.7/dist-packages/chainer/backends/cuda.py", line 336, in _array_to_gpu
return cupy.asarray(array)
File "/usr/local/lib/python2.7/dist-packages/cupy/creation/from_data.py", line 60, in asarray
return core.array(a, dtype, False)
File "cupy/core/core.pyx", line 2174, in cupy.core.core.array
File "cupy/core/core.pyx", line 2207, in cupy.core.core.array
Will finalize trainer extensions and updater before reraising the exception.
Traceback (most recent call last):
File "train.py", line 126, in <module>
trainer.run()
File "/usr/local/lib/python2.7/dist-packages/chainer/training/trainer.py", line 321, in run
six.reraise(*sys.exc_info())
File "/usr/local/lib/python2.7/dist-packages/chainer/training/trainer.py", line 307, in run
update()
File "/usr/local/lib/python2.7/dist-packages/chainer/training/updaters/standard_updater.py", line 165, in update
self.update_core()
File "/usr/local/lib/python2.7/dist-packages/chainer/training/updaters/standard_updater.py", line 171, in update_core
in_arrays = self.converter(batch, self.device)
File "/usr/local/lib/python2.7/dist-packages/chainer/dataset/convert.py", line 149, in concat_examples
return to_device(device, _concat_arrays(batch, padding))
File "/usr/local/lib/python2.7/dist-packages/chainer/dataset/convert.py", line 37, in to_device
return cuda.to_gpu(x, device)
File "/usr/local/lib/python2.7/dist-packages/chainer/backends/cuda.py", line 288, in to_gpu
return _array_to_gpu(array, device_, stream)
File "/usr/local/lib/python2.7/dist-packages/chainer/backends/cuda.py", line 336, in _array_to_gpu
return cupy.asarray(array)
File "/usr/local/lib/python2.7/dist-packages/cupy/creation/from_data.py", line 60, in asarray
return core.array(a, dtype, False)
File "cupy/core/core.pyx", line 2174, in cupy.core.core.array
File "cupy/core/core.pyx", line 2207, in cupy.core.core.array
ValueError: Unsupported dtype object
主训练循环中出现异常:不支持的dtype对象
回溯(最近一次呼叫最后一次):
文件“/usr/local/lib/python2.7/dist packages/chainer/training/trainer.py”,第307行,运行中
更新()
文件“/usr/local/lib/python2.7/dist packages/chainer/training/updaters/standard_updater.py”,第165行,在更新中
self.update_core()
文件“/usr/local/lib/python2.7/dist packages/chainer/training/updaters/standard_updater.py”,第171行,在update_core中
in_Array=self.converter(批处理,self.device)
concat_示例中的文件“/usr/local/lib/python2.7/dist packages/chainer/dataset/convert.py”,第149行
返回到_设备(设备,_concat_数组(批处理,填充))
文件“/usr/local/lib/python2.7/dist packages/chainer/dataset/convert.py”,第37行,在to_设备中
将cuda.返回到gpu(x,设备)
文件“/usr/local/lib/python2.7/dist packages/chainer/backends/cuda.py”,第288行,在to_gpu中
返回\u数组\u到\u gpu(数组、设备\u、流)
文件“/usr/local/lib/python2.7/dist packages/chainer/backends/cuda.py”,第336行,在_-array\u-to\u-gpu中
返回cupy.asarray(数组)
asarray中的文件“/usr/local/lib/python2.7/dist packages/cupy/creation/from_data.py”,第60行
返回core.array(a,数据类型,False)
文件“cupy/core/core.pyx”,第2174行,位于cupy.core.core.array中
文件“cupy/core/core.pyx”,第2207行,位于cupy.core.core.array中
将在重新引发异常之前完成培训师扩展和更新程序。
回溯(最近一次呼叫最后一次):
文件“train.py”,第126行,在
trainer.run()
文件“/usr/local/lib/python2.7/dist packages/chainer/training/trainer.py”,第321行,运行中
六、重放(*sys.exc_info())
文件“/usr/local/lib/python2.7/dist packages/chainer/training/trainer.py”,第307行,运行中
更新()
文件“/usr/local/lib/python2.7/dist packages/chainer/training/updaters/standard_updater.py”,第165行,在更新中
self.update_core()
文件“/usr/local/lib/python2.7/dist packages/chainer/training/updaters/standard_updater.py”,第171行,在update_core中
in_Array=self.converter(批处理,self.device)
concat_示例中的文件“/usr/local/lib/python2.7/dist packages/chainer/dataset/convert.py”,第149行
返回到_设备(设备,_concat_数组(批处理,填充))
文件“/usr/local/lib/python2.7/dist packages/chainer/dataset/convert.py”,第37行,在to_设备中
将cuda.返回到gpu(x,设备)
文件“/usr/local/lib/python2.7/dist packages/chainer/backends/cuda.py”,第288行,在to_gpu中
返回\u数组\u到\u gpu(数组、设备\u、流)
文件“/usr/local/lib/python2.7/dist packages/chainer/backends/cuda.py”,第336行,在_-array\u-to\u-gpu中
返回cupy.asarray(数组)
asarray中的文件“/usr/local/lib/python2.7/dist packages/cupy/creation/from_data.py”,第60行
返回core.array(a,数据类型,False)
文件“cupy/core/core.pyx”,第2174行,位于cupy.core.core.array中
文件“cupy/core/core.pyx”,第2207行,位于cupy.core.core.array中
ValueError:不支持的数据类型对象
我使用了上面提供的github链接中的数据集。
我是Chainer的新手,请帮帮我 错误说
ValueError:不支持的数据类型对象
Chainer支持numpy.float32
和cupy.float32
数组。试着按如下方式转换数据数组dtype
怎么样
final_train_set = np.asarray(final_train_set).astype(np.float32)
错误说
ValueError:不支持的数据类型对象
Chainer支持numpy.float32
和cupy.float32
数组。试着按如下方式转换数据数组dtype
怎么样
final_train_set = np.asarray(final_train_set).astype(np.float32)
这使得final\u train\u set
成为混合类型的元组列表(numpy.ndarray
和int
)。
因此,np.asarray(最终列车集)
会产生dtype=numpy.object
,Chainer不支持该结果
为了将它传递给SerialIterator
,我认为正确的方法是
# list of tuples of data and labels
final_train_set.append((
numpy.asarray((temp_slice1[j], temp_slice2[j])).astype(numpy.float32),
int(la)
))
循环结束后,什么也不做
这使得final\u train\u set
成为混合类型的元组列表(numpy.ndarray
和int
)。
因此,np.asarray(最终列车集)
会产生dtype=numpy.object
,Chainer不支持该结果
为了将它传递给SerialIterator
,我认为正确的方法是
# list of tuples of data and labels
final_train_set.append((
numpy.asarray((temp_slice1[j], temp_slice2[j])).astype(numpy.float32),
int(la)
))
循环后,什么也不做。应用更改后,培训模块引发了另一个错误。我已经修改了上面的代码。请提供帮助。我认为您应该回复该问题并发布另一个问题,以便其他面临相同错误(即ValueError)的人。应用更改后,培训模块会引发另一个错误。我已经修改了上面的代码。请提供帮助。我认为您应该回复问题并发布另一个问题,以便其他面临相同错误的人(即ValueError)。应用此行会引发另一个错误,即
ValueError:使用序列设置数组元素。
应用此行会引发另一个错误,这是ValueError:设置带有序列的数组元素。