Keras 非标准输入的神经网络

我想制作一个神经网络，它以图像+图像+值作为输入，对图像执行卷积+池运算，然后对结果进行线性变换。我可以在keras中这样做吗？

假设您的图像是RGB类型，图像的形状是（宽度、高度，3），您可以使用

numpy

组合两个图像，如：

    import numpy as np
    from PIL import Image

    img1 = Image.open('image1.jpg')
    img2 = Image.open('imgae2.jpg')

    img1 = img1.resize((width,height))
    img2 = img2.resize((width,height))

    img1_arr = np.asarray(img1,dtype='int32')
    img2_arr = np.asarray(img2,dtype='int32')

    #shape of img_arr is (width,height,6)
    img_arr = np.concatenate((img1_arr,img2_arr),axis=2)

以这种方式组合两个图像，我们只增加通道，所以我们仍然可以在前两个轴上进行卷积

更新： 我猜你的意思是多任务模型，你想在卷积后合并两个图像，Keras有

concatenate（）

可以做到这一点

    input_tensor = Input(shape=(channels, img_width, img_height))
    # Task1 on image1
    conv_model1 = VGG16(input_tensor=input_tensor, weights=None, include_top=False, classes=classes,
                  input_shape=(channels, img_width, img_height))
    conv_output1 = conv_model1.output
    flatten1 = Flatten()(conv_output1)
    # Task2 on image2
    conv_model2 = VGG16(input_tensor=input_tensor, weights=None, include_top=False, classes=classes,
                  input_shape=(channels, img_width, img_height))
    conv_output2 = conv_model2.output
    flatten2 = Flatten()(conv_output2)
    # Merge the output
    merged = concatenate([conv_output1, conv_output2], axis=1)
    merged = Dense(classes,activation='softmax')(merged)

    # add some Dense layers and Dropout,
    final_model = Model(inputs=[input_tensor,input_tensor],outputs=merged)

---

假设您的图像是RGB类型，图像的形状是（宽度、高度，3），您可以使用

numpy

组合两个图像，如：

    import numpy as np
    from PIL import Image

    img1 = Image.open('image1.jpg')
    img2 = Image.open('imgae2.jpg')

    img1 = img1.resize((width,height))
    img2 = img2.resize((width,height))

    img1_arr = np.asarray(img1,dtype='int32')
    img2_arr = np.asarray(img2,dtype='int32')

    #shape of img_arr is (width,height,6)
    img_arr = np.concatenate((img1_arr,img2_arr),axis=2)

以这种方式组合两个图像，我们只增加通道，所以我们仍然可以在前两个轴上进行卷积

更新： 我猜你的意思是多任务模型，你想在卷积后合并两个图像，Keras有

concatenate（）

可以做到这一点

    input_tensor = Input(shape=(channels, img_width, img_height))
    # Task1 on image1
    conv_model1 = VGG16(input_tensor=input_tensor, weights=None, include_top=False, classes=classes,
                  input_shape=(channels, img_width, img_height))
    conv_output1 = conv_model1.output
    flatten1 = Flatten()(conv_output1)
    # Task2 on image2
    conv_model2 = VGG16(input_tensor=input_tensor, weights=None, include_top=False, classes=classes,
                  input_shape=(channels, img_width, img_height))
    conv_output2 = conv_model2.output
    flatten2 = Flatten()(conv_output2)
    # Merge the output
    merged = concatenate([conv_output1, conv_output2], axis=1)
    merged = Dense(classes,activation='softmax')(merged)

    # add some Dense layers and Dropout,
    final_model = Model(inputs=[input_tensor,input_tensor],outputs=merged)

---

这在架构上与Craig Li的答案类似，但采用图像、图像、值格式，不使用VGG16，只使用普通CNN。这是3个独立的网络，其输出经过单独处理后连接起来，产生的连接向量通过最终层，包括来自所有输入的信息

input_1 = Input(data_1.shape[1:], name = 'input_1')
conv_branch_1 = Conv2D(filters, (kernel_size, kernel_size),
                 activation = LeakyReLU())(conv_branch_1)
conv_branch_1 = MaxPooling2D(pool_size = (2,2))(conv_branch_1)
conv_branch_1 = Flatten()(conv_branch_1)

input_2 = Input(data_2.shape[1:], name = 'input_2')
conv_branch_2 = Conv2D(filters, (kernel_size, kernel_size),
                 activation = LeakyReLU())(conv_branch_2)
conv_branch_2 = MaxPooling2D(pool_size = (2,2))(conv_branch_2)
conv_branch_2 = Flatten()(conv_branch_2)

value_input = Input(value_data.shape[1:], name = 'value_input')
fc_branch = Dense(80, activation=LeakyReLU())(value_input)

merged_branches = concatenate([conv_branch_1, conv_branch_2, fc_branch])
merged_branches = Dense(60, activation=LeakyReLU())(merged_branches)
merged_branches = Dropout(0.25)(merged_branches)
merged_branches = Dense(30, activation=LeakyReLU())(merged_branches)

merged_branches = Dense(1, activation='sigmoid')(merged_branches)

model = Model(inputs=[input_1, input_2, value_input], outputs=[merged_branches])

#if binary classification do this otherwise whatever loss you need

model.compile(loss='binary_crossentropy')

---

这在架构上与Craig Li的答案类似，但采用图像、图像、值格式，不使用VGG16，只使用普通CNN。这是3个独立的网络，其输出经过单独处理后连接起来，产生的连接向量通过最终层，包括来自所有输入的信息

input_1 = Input(data_1.shape[1:], name = 'input_1')
conv_branch_1 = Conv2D(filters, (kernel_size, kernel_size),
                 activation = LeakyReLU())(conv_branch_1)
conv_branch_1 = MaxPooling2D(pool_size = (2,2))(conv_branch_1)
conv_branch_1 = Flatten()(conv_branch_1)

input_2 = Input(data_2.shape[1:], name = 'input_2')
conv_branch_2 = Conv2D(filters, (kernel_size, kernel_size),
                 activation = LeakyReLU())(conv_branch_2)
conv_branch_2 = MaxPooling2D(pool_size = (2,2))(conv_branch_2)
conv_branch_2 = Flatten()(conv_branch_2)

value_input = Input(value_data.shape[1:], name = 'value_input')
fc_branch = Dense(80, activation=LeakyReLU())(value_input)

merged_branches = concatenate([conv_branch_1, conv_branch_2, fc_branch])
merged_branches = Dense(60, activation=LeakyReLU())(merged_branches)
merged_branches = Dropout(0.25)(merged_branches)
merged_branches = Dense(30, activation=LeakyReLU())(merged_branches)

merged_branches = Dense(1, activation='sigmoid')(merged_branches)

model = Model(inputs=[input_1, input_2, value_input], outputs=[merged_branches])

#if binary classification do this otherwise whatever loss you need

model.compile(loss='binary_crossentropy')

---

图像的顺序和数量是固定的吗？@Craig.Li是的，正好是两个图像的顺序和数量是固定的吗？@Craig.Li是的，正好是两个谢谢你的回答，但是我想在两张图像上的卷积上有不同的权重。所以我希望它更像是最终连接的不同层次。再次感谢您的回答。卷积男孩更接近我想要的（因此得名）。谢谢你的回答，但我想在两张图片上的卷积上有不同的权重。所以我希望它更像是最终连接的不同层次。再次感谢您的回答。卷积男孩更接近我想要的（因此得名）。