Keras 如何在具有RepeatVector()层的LSTM自动编码器中屏蔽输入?
我一直在尝试使用LSTM自动编码器获取向量序列的向量表示,以便使用SVM或其他此类监督算法对序列进行分类。数据量使我无法使用完全连接的密集层进行分类 我输入的最短大小是7个时间步,最长的序列是356个时间步。因此,我用零填充较短的序列,以获得最终的x_形状序列(1326356,8),其中1326是训练样本数,8是一个时间步的维数。我试图使用给定的LSTM自动编码器将这些序列编码成单个向量Keras 如何在具有RepeatVector()层的LSTM自动编码器中屏蔽输入?,keras,lstm,masking,Keras,Lstm,Masking,我一直在尝试使用LSTM自动编码器获取向量序列的向量表示,以便使用SVM或其他此类监督算法对序列进行分类。数据量使我无法使用完全连接的密集层进行分类 我输入的最短大小是7个时间步,最长的序列是356个时间步。因此,我用零填充较短的序列,以获得最终的x_形状序列(1326356,8),其中1326是训练样本数,8是一个时间步的维数。我试图使用给定的LSTM自动编码器将这些序列编码成单个向量 model.add(Masking(mask_value=0.0, input_shape=(max_len
model.add(Masking(mask_value=0.0, input_shape=(max_len, 8)))
model.add(LSTM(100, activation='relu'))
model.add(RepeatVector(max_len))
model.add(LSTM(8, activation='relu', return_sequences=True))
model.compile(optimizer='adam', loss='mse')
model.fit(x_train, x_train, batch_size=32, callbacks=[chk], epochs=1000, validation_split=0.05, shuffle=True)
time_steps = 3
n_features = 2
input_layer = tfkl.Input(shape=(time_steps, n_features))
# I want to mask the timestep where all the feature values are 1 (usually we pad by 0)
x = tfk.layers.Masking(mask_value=1)(input_layer)
x = tfkl.LSTM(2, return_sequences=True)(x)
x = tfkl.LSTM(2, return_sequences=False)(x)
x = tfkl.RepeatVector(time_steps)(x)
x = tfkl.LSTM(2, return_sequences=True)(x)
x = tfkl.LSTM(2, return_sequences=True)(x)
x = tfk.layers.Dense(n_features)(x)
lstm_ae = tfk.models.Model(inputs=input_layer, outputs=x)
lstm_ae.compile(optimizer='adam', loss='mse')
print(lstm_ae.summary())
Model: "model_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_3 (InputLayer) [(None, 3, 2)] 0
_________________________________________________________________
masking_2 (Masking) (None, 3, 2) 0
_________________________________________________________________
lstm_8 (LSTM) (None, 3, 2) 40
_________________________________________________________________
lstm_9 (LSTM) (None, 2) 40
_________________________________________________________________
repeat_vector_2 (RepeatVecto (None, 3, 2) 0
_________________________________________________________________
lstm_10 (LSTM) (None, 3, 2) 40
_________________________________________________________________
lstm_11 (LSTM) (None, 3, 2) 40
_________________________________________________________________
dense_2 (Dense) (None, 3, 2) 6
=================================================================
Total params: 166
Trainable params: 166
Non-trainable params: 0
_________________________________________________________________
for i, l in enumerate(lstm_ae.layers):
print(f'layer {i}: {l}')
print(f'has input mask: {l.input_mask}')
print(f'has output mask: {l.output_mask}')
layer 0: <tensorflow.python.keras.engine.input_layer.InputLayer object at 0x645b49cf8>
has input mask: None
has output mask: None
layer 1: <tensorflow.python.keras.layers.core.Masking object at 0x645b49c88>
has input mask: None
has output mask: Tensor("masking_2/Identity_1:0", shape=(None, 3), dtype=bool)
layer 2: <tensorflow.python.keras.layers.recurrent_v2.LSTM object at 0x645b4d0b8>
has input mask: Tensor("masking_2/Identity_1:0", shape=(None, 3), dtype=bool)
has output mask: Tensor("masking_2/Identity_1:0", shape=(None, 3), dtype=bool)
layer 3: <tensorflow.python.keras.layers.recurrent_v2.LSTM object at 0x645b4dba8>
has input mask: Tensor("masking_2/Identity_1:0", shape=(None, 3), dtype=bool)
has output mask: None
layer 4: <tensorflow.python.keras.layers.core.RepeatVector object at 0x645db0390>
has input mask: None
has output mask: None
layer 5: <tensorflow.python.keras.layers.recurrent_v2.LSTM object at 0x6470b5da0>
has input mask: None
has output mask: None
layer 6: <tensorflow.python.keras.layers.recurrent_v2.LSTM object at 0x6471410f0>
has input mask: None
has output mask: None
layer 7: <tensorflow.python.keras.layers.core.Dense object at 0x647dfdf60>
has input mask: None
has output mask: None
我试图掩盖零填充结果,但RepeatVector()层可能会阻碍该过程。因此,经过一段时间后,均方误差损失变得
nan
。有谁能帮我解释一下,如何在计算损失函数时只包含相关的时间步长而忽略其他时间步长?Keras中的每一层都有一个输入掩码和输出掩码,掩码在第一个LSTM
层之后就已经丢失了(当返回\u序列=False
)在你的例子中。让我在下面的示例中对此进行解释,并展示在LSTM自动编码器中实现掩蔽的两种解决方案
model.add(Masking(mask_value=0.0, input_shape=(max_len, 8)))
model.add(LSTM(100, activation='relu'))
model.add(RepeatVector(max_len))
model.add(LSTM(8, activation='relu', return_sequences=True))
model.compile(optimizer='adam', loss='mse')
model.fit(x_train, x_train, batch_size=32, callbacks=[chk], epochs=1000, validation_split=0.05, shuffle=True)
time_steps = 3
n_features = 2
input_layer = tfkl.Input(shape=(time_steps, n_features))
# I want to mask the timestep where all the feature values are 1 (usually we pad by 0)
x = tfk.layers.Masking(mask_value=1)(input_layer)
x = tfkl.LSTM(2, return_sequences=True)(x)
x = tfkl.LSTM(2, return_sequences=False)(x)
x = tfkl.RepeatVector(time_steps)(x)
x = tfkl.LSTM(2, return_sequences=True)(x)
x = tfkl.LSTM(2, return_sequences=True)(x)
x = tfk.layers.Dense(n_features)(x)
lstm_ae = tfk.models.Model(inputs=input_layer, outputs=x)
lstm_ae.compile(optimizer='adam', loss='mse')
print(lstm_ae.summary())
Model: "model_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_3 (InputLayer) [(None, 3, 2)] 0
_________________________________________________________________
masking_2 (Masking) (None, 3, 2) 0
_________________________________________________________________
lstm_8 (LSTM) (None, 3, 2) 40
_________________________________________________________________
lstm_9 (LSTM) (None, 2) 40
_________________________________________________________________
repeat_vector_2 (RepeatVecto (None, 3, 2) 0
_________________________________________________________________
lstm_10 (LSTM) (None, 3, 2) 40
_________________________________________________________________
lstm_11 (LSTM) (None, 3, 2) 40
_________________________________________________________________
dense_2 (Dense) (None, 3, 2) 6
=================================================================
Total params: 166
Trainable params: 166
Non-trainable params: 0
_________________________________________________________________
for i, l in enumerate(lstm_ae.layers):
print(f'layer {i}: {l}')
print(f'has input mask: {l.input_mask}')
print(f'has output mask: {l.output_mask}')
layer 0: <tensorflow.python.keras.engine.input_layer.InputLayer object at 0x645b49cf8>
has input mask: None
has output mask: None
layer 1: <tensorflow.python.keras.layers.core.Masking object at 0x645b49c88>
has input mask: None
has output mask: Tensor("masking_2/Identity_1:0", shape=(None, 3), dtype=bool)
layer 2: <tensorflow.python.keras.layers.recurrent_v2.LSTM object at 0x645b4d0b8>
has input mask: Tensor("masking_2/Identity_1:0", shape=(None, 3), dtype=bool)
has output mask: Tensor("masking_2/Identity_1:0", shape=(None, 3), dtype=bool)
layer 3: <tensorflow.python.keras.layers.recurrent_v2.LSTM object at 0x645b4dba8>
has input mask: Tensor("masking_2/Identity_1:0", shape=(None, 3), dtype=bool)
has output mask: None
layer 4: <tensorflow.python.keras.layers.core.RepeatVector object at 0x645db0390>
has input mask: None
has output mask: None
layer 5: <tensorflow.python.keras.layers.recurrent_v2.LSTM object at 0x6470b5da0>
has input mask: None
has output mask: None
layer 6: <tensorflow.python.keras.layers.recurrent_v2.LSTM object at 0x6471410f0>
has input mask: None
has output mask: None
layer 7: <tensorflow.python.keras.layers.core.Dense object at 0x647dfdf60>
has input mask: None
has output mask: None
第二种解决方案:制作一个定制的瓶颈层,手动通过遮罩
非常感谢。我将尝试这些解决方案,并让您了解结果。如果我的功能值介于0和1之间,您是否可以告诉我使用的最佳激活函数是什么?
# last timestep should be masked because all feature values are 1
x = np.array([1, 2, 1, 2, 1, 1], dtype='float32').reshape(1, 3, 2)
print(x)
array([[[1., 2.],
[1., 2.],
[1., 1.]]], dtype=float32)
y = lstm_ae.predict(x)
print(y)
array([[[ 0.00065455, -0.00294413],
[ 0.00166675, -0.00742249],
[ 0.00166675, -0.00742249]]], dtype=float32)
# the expected loss should be the square error between the first 2 timesteps divided by 6
expected_loss = np.square(x[:, :2, :] - y[:, :2, :]).sum()/6
print(expected_loss)
1.672815163930257
# now the loss is correct with a custom layer
actual_loss_with_masking = lstm_ae.evaluate(x=x, y=x)
print(actual_loss_with_masking)
1.672815203666687