Python 用Chainer编写的CNN模型的训练代码

Python 用Chainer编写的CNN模型的训练代码,python,conv-neural-network,training-data,chainer,Python,Conv Neural Network,Training Data,Chainer,我正在编写训练代码,它是一个双流卷积神经网络。该模型通过网络的两个流预测正在评估的补丁的质量分数。在下面的培训中,我使用了上面GitHub链接中提供的测试数据集 培训代码如下: ## prepare training data test_label_path = 'data_list/test.txt' test_img_path = 'data/live/' test_Graimg_path = 'data/live_grad/' save_model_path = '/models/nr_

我正在编写训练代码,它是一个双流卷积神经网络。该模型通过网络的两个流预测正在评估的补丁的质量分数。在下面的培训中,我使用了上面GitHub链接中提供的测试数据集

培训代码如下:

## prepare training data 
test_label_path = 'data_list/test.txt'
test_img_path = 'data/live/'
test_Graimg_path = 'data/live_grad/'
save_model_path = '/models/nr_sana_2stream.model'

patches_per_img = 256
patchSize = 32

print('-------------Load data-------------')
final_train_set = []
with open(test_label_path, 'rt') as f:
    for l in f:
        line, la = l.strip().split()  # for debug

        tic = time.time()
        full_path = os.path.join(test_img_path, line)
        Grafull_path = os.path.join(test_Graimg_path, line)

        f = Image.open(full_path)
        Graf = Image.open(Grafull_path)
        img = np.asarray(f, dtype=np.float32)
        Gra = np.asarray(Graf, dtype=np.float32)
        img = img.transpose(2, 0, 1)
        Gra = Gra.transpose(2, 0, 1)

        img1 = np.zeros((1, 3, Gra.shape[1], Gra.shape[2]))
        img1[0, :, :, :] = img
        Gra1 = np.zeros((1, 3, Gra.shape[1], Gra.shape[2]))
        Gra1[0, :, :, :] = Gra

        patches = extract_patches(img, (3, patchSize, patchSize), patchSize)
        Grapatches = extract_patches(Gra, (3, patchSize, patchSize), patchSize)

        X = patches.reshape((-1, 3, patchSize, patchSize))
        GraX = Grapatches.reshape((-1, 3, patchSize, patchSize))

        temp_slice1 = [X[int(float(index))] for index in range(256)]
        temp_slice2 = [GraX[int(float(index))] for index in range(256)]
        ##############################################  
        for j in range(len(temp_slice1)):
            temp_slice1[j] = xp.array(temp_slice1[j].astype(np.float32))
            temp_slice2[j] = xp.array(temp_slice2[j].astype(np.float32))

            final_train_set.append((temp_slice1[j], temp_slice2[j], int(la)))

    final_train_set = np.asarray(final_train_set)       
        ##############################################  

#
print('--------------Done!----------------')

print('--------------Iterator!----------------')    
train_iter = iterators.SerialIterator(final_train_set, batch_size=4)
optimizer = optimizers.Adam()
optimizer.use_cleargrads()
optimizer.setup(model)

updater = training.StandardUpdater(train_iter, optimizer, device=0)

print('--------------Trainer!----------------') 
trainer = training.Trainer(updater, (50, 'epoch'), out='result')

trainer.extend(extensions.LogReport())

trainer.extend(extensions.PrintReport(['epoch', 'iteration', 'main/loss', 'elapsed_time']))

print('--------------Running trainer!----------------') 
trainer.run()
但代码产生了如下错误:

Exception in main training loop: Unsupported dtype object
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/chainer/training/trainer.py", line 307, in run
    update()
  File "/usr/local/lib/python2.7/dist-packages/chainer/training/updaters/standard_updater.py", line 165, in update
    self.update_core()
  File "/usr/local/lib/python2.7/dist-packages/chainer/training/updaters/standard_updater.py", line 171, in update_core
    in_arrays = self.converter(batch, self.device)
  File "/usr/local/lib/python2.7/dist-packages/chainer/dataset/convert.py", line 149, in concat_examples
    return to_device(device, _concat_arrays(batch, padding))
  File "/usr/local/lib/python2.7/dist-packages/chainer/dataset/convert.py", line 37, in to_device
    return cuda.to_gpu(x, device)
  File "/usr/local/lib/python2.7/dist-packages/chainer/backends/cuda.py", line 288, in to_gpu
    return _array_to_gpu(array, device_, stream)
  File "/usr/local/lib/python2.7/dist-packages/chainer/backends/cuda.py", line 336, in _array_to_gpu
    return cupy.asarray(array)
  File "/usr/local/lib/python2.7/dist-packages/cupy/creation/from_data.py", line 60, in asarray
    return core.array(a, dtype, False)
  File "cupy/core/core.pyx", line 2174, in cupy.core.core.array
  File "cupy/core/core.pyx", line 2207, in cupy.core.core.array
Will finalize trainer extensions and updater before reraising the exception.
Traceback (most recent call last):
  File "train.py", line 126, in <module>
    trainer.run()
  File "/usr/local/lib/python2.7/dist-packages/chainer/training/trainer.py", line 321, in run
    six.reraise(*sys.exc_info())
  File "/usr/local/lib/python2.7/dist-packages/chainer/training/trainer.py", line 307, in run
    update()
  File "/usr/local/lib/python2.7/dist-packages/chainer/training/updaters/standard_updater.py", line 165, in update
    self.update_core()
  File "/usr/local/lib/python2.7/dist-packages/chainer/training/updaters/standard_updater.py", line 171, in update_core
    in_arrays = self.converter(batch, self.device)
  File "/usr/local/lib/python2.7/dist-packages/chainer/dataset/convert.py", line 149, in concat_examples
    return to_device(device, _concat_arrays(batch, padding))
  File "/usr/local/lib/python2.7/dist-packages/chainer/dataset/convert.py", line 37, in to_device
    return cuda.to_gpu(x, device)
  File "/usr/local/lib/python2.7/dist-packages/chainer/backends/cuda.py", line 288, in to_gpu
    return _array_to_gpu(array, device_, stream)
  File "/usr/local/lib/python2.7/dist-packages/chainer/backends/cuda.py", line 336, in _array_to_gpu
    return cupy.asarray(array)
  File "/usr/local/lib/python2.7/dist-packages/cupy/creation/from_data.py", line 60, in asarray
    return core.array(a, dtype, False)
  File "cupy/core/core.pyx", line 2174, in cupy.core.core.array
  File "cupy/core/core.pyx", line 2207, in cupy.core.core.array
ValueError: Unsupported dtype object
主训练循环中出现异常:不支持的dtype对象 回溯(最近一次呼叫最后一次): 文件“/usr/local/lib/python2.7/dist packages/chainer/training/trainer.py”,第307行,运行中 更新() 文件“/usr/local/lib/python2.7/dist packages/chainer/training/updaters/standard_updater.py”,第165行,在更新中 self.update_core() 文件“/usr/local/lib/python2.7/dist packages/chainer/training/updaters/standard_updater.py”,第171行,在update_core中 in_Array=self.converter(批处理,self.device) concat_示例中的文件“/usr/local/lib/python2.7/dist packages/chainer/dataset/convert.py”,第149行 返回到_设备(设备,_concat_数组(批处理,填充)) 文件“/usr/local/lib/python2.7/dist packages/chainer/dataset/convert.py”,第37行,在to_设备中 将cuda.返回到gpu(x,设备) 文件“/usr/local/lib/python2.7/dist packages/chainer/backends/cuda.py”,第288行,在to_gpu中 返回\u数组\u到\u gpu(数组、设备\u、流) 文件“/usr/local/lib/python2.7/dist packages/chainer/backends/cuda.py”,第336行,在_-array\u-to\u-gpu中 返回cupy.asarray(数组) asarray中的文件“/usr/local/lib/python2.7/dist packages/cupy/creation/from_data.py”,第60行 返回core.array(a,数据类型,False) 文件“cupy/core/core.pyx”,第2174行,位于cupy.core.core.array中 文件“cupy/core/core.pyx”,第2207行,位于cupy.core.core.array中 将在重新引发异常之前完成培训师扩展和更新程序。 回溯(最近一次呼叫最后一次): 文件“train.py”,第126行,在 trainer.run() 文件“/usr/local/lib/python2.7/dist packages/chainer/training/trainer.py”,第321行,运行中 六、重放(*sys.exc_info()) 文件“/usr/local/lib/python2.7/dist packages/chainer/training/trainer.py”,第307行,运行中 更新() 文件“/usr/local/lib/python2.7/dist packages/chainer/training/updaters/standard_updater.py”,第165行,在更新中 self.update_core() 文件“/usr/local/lib/python2.7/dist packages/chainer/training/updaters/standard_updater.py”,第171行,在update_core中 in_Array=self.converter(批处理,self.device) concat_示例中的文件“/usr/local/lib/python2.7/dist packages/chainer/dataset/convert.py”,第149行 返回到_设备(设备,_concat_数组(批处理,填充)) 文件“/usr/local/lib/python2.7/dist packages/chainer/dataset/convert.py”,第37行,在to_设备中 将cuda.返回到gpu(x,设备) 文件“/usr/local/lib/python2.7/dist packages/chainer/backends/cuda.py”,第288行,在to_gpu中 返回\u数组\u到\u gpu(数组、设备\u、流) 文件“/usr/local/lib/python2.7/dist packages/chainer/backends/cuda.py”,第336行,在_-array\u-to\u-gpu中 返回cupy.asarray(数组) asarray中的文件“/usr/local/lib/python2.7/dist packages/cupy/creation/from_data.py”,第60行 返回core.array(a,数据类型,False) 文件“cupy/core/core.pyx”,第2174行,位于cupy.core.core.array中 文件“cupy/core/core.pyx”,第2207行,位于cupy.core.core.array中 ValueError:不支持的数据类型对象 我使用了上面提供的github链接中的数据集。 我是Chainer的新手,请帮帮我

错误说

ValueError:不支持的数据类型对象

Chainer支持
numpy.float32
cupy.float32
数组。试着按如下方式转换数据数组
dtype
怎么样

final_train_set = np.asarray(final_train_set).astype(np.float32)
错误说

ValueError:不支持的数据类型对象

Chainer支持
numpy.float32
cupy.float32
数组。试着按如下方式转换数据数组
dtype
怎么样

final_train_set = np.asarray(final_train_set).astype(np.float32)
这使得
final\u train\u set
成为混合类型的元组列表(
numpy.ndarray
int
)。 因此,
np.asarray(最终列车集)
会产生
dtype=numpy.object
,Chainer不支持该结果

为了将它传递给
SerialIterator
,我认为正确的方法是

# list of tuples of data and labels
final_train_set.append((
    numpy.asarray((temp_slice1[j], temp_slice2[j])).astype(numpy.float32),
    int(la)
))
循环结束后,什么也不做

这使得
final\u train\u set
成为混合类型的元组列表(
numpy.ndarray
int
)。 因此,
np.asarray(最终列车集)
会产生
dtype=numpy.object
,Chainer不支持该结果

为了将它传递给
SerialIterator
,我认为正确的方法是

# list of tuples of data and labels
final_train_set.append((
    numpy.asarray((temp_slice1[j], temp_slice2[j])).astype(numpy.float32),
    int(la)
))

循环后,什么也不做。

应用更改后,培训模块引发了另一个错误。我已经修改了上面的代码。请提供帮助。我认为您应该回复该问题并发布另一个问题,以便其他面临相同错误(即ValueError)的人。应用更改后,培训模块会引发另一个错误。我已经修改了上面的代码。请提供帮助。我认为您应该回复问题并发布另一个问题,以便其他面临相同错误的人(即ValueError)。应用此行会引发另一个错误,即
ValueError:使用序列设置数组元素。
应用此行会引发另一个错误,这是
ValueError:设置带有序列的数组元素。