Keras 与BatchNormalization层关联的参数数量是2048个?
我有以下代码Keras 与BatchNormalization层关联的参数数量是2048个?,keras,batch-normalization,Keras,Batch Normalization,我有以下代码 x = keras.layers.Input(batch_shape = (None, 4096)) hidden = keras.layers.Dense(512, activation = 'relu')(x) hidden = keras.layers.BatchNormalization()(hidden) hidden = keras.layers.Dropout(0.5)(hidden) predictions = keras.layers.Dense(80, acti
x = keras.layers.Input(batch_shape = (None, 4096))
hidden = keras.layers.Dense(512, activation = 'relu')(x)
hidden = keras.layers.BatchNormalization()(hidden)
hidden = keras.layers.Dropout(0.5)(hidden)
predictions = keras.layers.Dense(80, activation = 'sigmoid')(hidden)
mlp_model = keras.models.Model(input = [x], output = [predictions])
mlp_model.summary()
这是模型摘要:
____________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
====================================================================================================
input_3 (InputLayer) (None, 4096) 0
____________________________________________________________________________________________________
dense_1 (Dense) (None, 512) 2097664 input_3[0][0]
____________________________________________________________________________________________________
batchnormalization_1 (BatchNorma (None, 512) 2048 dense_1[0][0]
____________________________________________________________________________________________________
dropout_1 (Dropout) (None, 512) 0 batchnormalization_1[0][0]
____________________________________________________________________________________________________
dense_2 (Dense) (None, 80) 41040 dropout_1[0][0]
====================================================================================================
Total params: 2,140,752
Trainable params: 2,139,728
Non-trainable params: 1,024
____________________________________________________________________________________________________
BatchNormalization(BN)层的输入大小为512。根据所述,BN层的输出的形状和输入的形状相同,即512
那么与BN层关联的参数数量是2048个吗?Keras中的批量规范化实现了 正如您在那里看到的,为了使批处理规范化在培训期间工作,他们需要跟踪每个规范化维度的分布。为此,由于默认情况下您处于
模式=0
,因此它们会计算上一层上每个要素的4个参数。这些参数确保您正确传播和反向传播信息
所以
4*512=2048
,这应该可以回答您的问题 这2048个参数实际上是[伽马权重、贝塔权重、移动平均值(不可训练)、移动方差(不可训练)]
,每个参数都有512个元素(输入层的大小)。有什么理由否决这个问题吗?感谢您解释这两个不可训练的参数!