Python Caffe-使用VGG网络层作为'；结束'；目标并在输入图像上生成梦想

型号

我一直在尝试从头开始训练一个

VGG\u FACE\u 16\u层

Net，遵循本项目的步骤：

然后，我在一个云GPU上，从我的

mdb

文件中训练了一个人脸/情绪识别模型，并生成了一个文件

my\u face.caffemodel.

它有6个标签，虽然预测精度不是最优的，但该模型似乎是可用的

net = caffe.Net('models/vitor_face/deploy.prototxt', 
                 'models/vitor_face/my_face.caffemodel', 
                 caffe.TEST)

W = net.params['fc7'][0].data[...]
b = net.params['fc7'][1].data[...]

所有层的

W

和

b

似乎都打印有效值

以及架构：

[('data', (1, 3, 224, 224)), ('conv1', (1, 96, 111, 111)), ('norm1', (1, 96, 111, 111)), ('pool1', (1, 96, 37, 37)), ('conv2', (1, 256, 37, 37)), ('pool2', (1, 256, 19, 19)), ('conv3', (1, 512, 19, 19)), ('conv4', (1, 512, 19, 19)), ('conv5', (1, 512, 19, 19)), ('pool5', (1, 512, 7, 7)), ('fc6', (1, 4048)), ('fc7', (1, 4048)), ('fc8', (1, 6)), ('prob', (1, 6))]
[('conv1', (96, 3, 7, 7), (96,)), ('conv2', (256, 96, 5, 5), (256,)), ('conv3', (512, 256, 3, 3), (512,)), ('conv4', (512, 512, 3, 3), (512,)), ('conv5', (512, 512, 3, 3), (512,)), ('fc6', (4048, 25088), (4048,)), ('fc7', (4048, 4048), (4048,)), ('fc8_cat', (6, 4048), (6,))]

---

目标

我的目标是在

目标结束时使用
网络中的层
，以生成
梦想
，并使用以下代码使用于训练的人脸“出现”在输入图像上：

model_path = 'happyNet/models/vitor_face/' # substitute your path here
net_fn   = model_path + 'deploy.prototxt'
param_fn = model_path + 'my_face.caffemodel'
MEAN_FILE = model_path + 'mean_training_image.binaryproto'

proto_data = open(MEAN_FILE, "rb").read()
a = caffe.io.caffe_pb2.BlobProto.FromString(proto_data)
MEAN = caffe.io.blobproto_to_array(a)[0]


net = caffe.Classifier(net_fn, 
                        param_fn,
                        mean = np.float32([104.0, 116.0, 122.0]), # ImageNet mean, training set dependent
                        channel_swap = (2,1,0)) # ImageNet mean, training set dependent) # the reference model has channels in BGR order instead of RGB

# a couple of utility functions for converting to and from Caffe's input image layout
def preprocess(net, img):
    return np.float32(np.rollaxis(img, 2)[::-1]) - net.transformer.mean['data']
def deprocess(net, img):
    return np.dstack((img + net.transformer.mean['data'])[::-1])

def objective_L2(dst):
    dst.diff[:] = dst.data 

def make_step(net, step_size=1.5, end='fc7', 
              jitter=32, clip=True, objective=objective_L2):
    '''Basic gradient ascent step.'''

    src = net.blobs['data'] # input image is stored in Net's 'data' blob
    dst = net.blobs[end]
    #print ('src.data', src.data)
    # print  ('PERCENTILE',np.percentile(net.blobs[end].data[0], (0, 10, 50, 90, 100)))

    ox, oy = np.random.randint(-jitter, jitter+1, 2)
    src.data[0] = np.roll(np.roll(src.data[0], ox, -1), oy, -2) # apply jitter shift

    net.forward(end=end)
    objective(dst)  # specify the optimization objective
    net.backward(start=end)
    g = src.diff[0]
    # apply normalized ascent step to the input image
    src.data[:] += step_size/np.abs(g).mean() * g

    src.data[0] = np.roll(np.roll(src.data[0], -ox, -1), -oy, -2) # unshift image

    if clip:
        bias = net.transformer.mean['data']
        src.data[:] = np.clip(src.data, -bias, 255-bias)

def deepdream(net, base_img, iter_n=20, octave_n=4, octave_scale=1.4, 
              end='fc7', clip=True, **step_params):
    # prepare base images for all octaves
    octaves = [preprocess(net, base_img)]

    for i in xrange(octave_n-1):
        octaves.append(nd.zoom(octaves[-1], (1, 1.0/octave_scale,1.0/octave_scale), order=1))

    src = net.blobs['data']
    #print src.data
    # print blobs infos
    print [(k, v.data.shape) for k, v in net.blobs.items()]
    #print weight and bias parameters
    print [(k, v[0].data.shape, v[1].data.shape) for k, v in net.params.items()]

    detail = np.zeros_like(octaves[-1]) # allocate image for network-produced details

    for octave, octave_base in enumerate(octaves[::-1]):
        h, w = octave_base.shape[-2:]

        if octave > 0:
            # upscale details from the previous octave
            h1, w1 = detail.shape[-2:]
            detail = nd.zoom(detail, (1, 1.0*h/h1,1.0*w/w1), order=1)

        src.reshape(1,3,h,w) # resize the network's input image size
        src.data[0] = octave_base+detail

        for i in xrange(20):
            make_step(net, end=end, clip=clip, **step_params)

            # visualization
            vis = deprocess(net, src.data[0])

            if not clip: # adjust image contrast if clipping is disabled
                vis = vis*(255.0/np.percentile(vis, 99.98))
            showarray(vis)
            # save images to disk
            PIL.Image.fromarray(np.uint8(vis)).save('results/{}_{}_{}.png'.format(octave, i, vis.shape))

            print octave, i, end, vis.shape
            clear_output(wait=True)

        # extract details produced on the current octave
        detail = src.data[0]-octave_base
    # returning the resulting image
    return deprocess(net, src.data[0])

然后运行它，期望将经过训练的脸显示在云图上，如下所示：

img = np.float32(PIL.Image.open('img/clouds.jpg'))

_=deepdream(net, img, end)

   dream.py:178: RuntimeWarning: divide by zero encountered in divide
    src.data[:] += step_size/np.abs(g).mean() * g
    dream.py:178: RuntimeWarning: invalid value encountered in multiply
    src.data[:] += step_size/np.abs(g).mean() * g

相关文件

deploy.prototxt

train.prototxt

回溯

上面的代码适用于预训练的网络，但不适用于我的，从头开始训练的网络

不知何故，看起来网络并没有一直被转发，因为
我得到了一些零分的回溯，就像这样：

img = np.float32(PIL.Image.open('img/clouds.jpg'))

_=deepdream(net, img, end)

   dream.py:178: RuntimeWarning: divide by zero encountered in divide
    src.data[:] += step_size/np.abs(g).mean() * g
    dream.py:178: RuntimeWarning: invalid value encountered in multiply
    src.data[:] += step_size/np.abs(g).mean() * g

（结果图像是完全黑色的，如果我手动添加到bias-
g+.1
，我只打印原始图像，根本不转换。）

此外，我还收到以下
InnerProduct
警告，提示我的
input\u dim
在分类层可能会出错，但我没有
理解为什么会这样，也理解为什么会这样，因为原始项目使用
input_dim 224224
：

>275 inner_product_layer.cpp:64] Check failed: K_ == new_K (25088 vs. 20480) Input size incompatible with inner product parameters.

无论如何，我尝试将
deploy.prototxt的
input\u dim
更改为AlexNet的值
227
，但没有效果

OBS
由于我想使用我的模型来生成梦，而不是对图像进行分类，我想知道我是否应该尝试将模型
完全连接到分类层，如下面的回答所示：


如果这是使模型与GoogLeNet架构兼容并从我的模型中激活
=end
层的方法，请有人告诉我怎么做

请，我们非常感谢您的帮助

我很乐意通过请求聊天分享我的模型
 看，是的，我见过。这就是我提到的完全相关的地方。