Machine learning keras fit解码器,然后安装自动编码器
我有一个模型:Machine learning keras fit解码器,然后安装自动编码器,machine-learning,keras,lstm,loss-function,autoencoder,Machine Learning,Keras,Lstm,Loss Function,Autoencoder,我有一个模型: class MultiAutoencoder: def decoder_category_brunch(encoder_dense_2): decoded = RepeatVector(max_len_str, name = 'bottleneck-2')(encoder_dense_2) decoder_LSTM_1 = LSTM(25, activation='tanh', return_sequences=True)(decoded) decoder
class MultiAutoencoder:
def decoder_category_brunch(encoder_dense_2):
decoded = RepeatVector(max_len_str, name = 'bottleneck-2')(encoder_dense_2)
decoder_LSTM_1 = LSTM(25, activation='tanh', return_sequences=True)(decoded)
decoder_LSTM_2 = LSTM(50, activation='tanh', return_sequences=True)(decoder_LSTM_1)
decoder_GRU = GRU(200, activation='tanh', return_sequences=True, name='Decoder-GRU')(decoder_LSTM_2)
decoder_drop = Dropout(0.1)(decoder_GRU)
decoder_LSTM_3 = Bidirectional(LSTM(150, return_sequences=False, name='Decoder-LSTM-2'))(decoder_drop)
decoder_output = Dense(max_len_str, activation='softmax', name='decoder-output')(decoder_LSTM_3)
return decoder_output
def encoder_binary_brunch(emb_norm):
encoder_LSTM = Bidirectional(LSTM(100, activation='tanh', return_sequences=True, name='Encoder-LSTM'))(emb_norm)
encoder_drop = Dropout(0.1)(encoder_LSTM)
encoder_GRU = GRU(50, activation='tanh', return_sequences=False, name = 'bottleneck-1')(encoder_drop)
encoder_dense_1 = Dense(25, activation='relu')(encoder_GRU)
encoder_dense_2 = Dense(5, activation='relu')(encoder_dense_1)
encoder_output = Dense(1, activation='softmax', name = 'encoder-output')(encoder_dense_2)
return encoder_dense_2, encoder_output
def build():
encoder_inputs = Input(shape=(max_len_str,), name='Encoder-Input')
emb_layer = Embedding(input_dim=len(word_index)+1, output_dim=300, input_length = max_len_str, name='Embedding', mask_zero=False)(encoder_inputs)
emb_drop = SpatialDropout1D(0.1)(emb_layer)
emb_norm = BatchNormalization()(emb_drop)
encoder_dense_2, encoder_output = MultiAutoencoder.encoder_binary_brunch(emb_norm)
decoder_output = MultiAutoencoder.decoder_category_brunch(encoder_dense_2)
model = Model(inputs=encoder_inputs, outputs=[encoder_output, decoder_output], name="MultiAutoencoder")
return model
一般资料:
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
Encoder-Input (InputLayer) [(None, 699)] 0
__________________________________________________________________________________________________
Embedding (Embedding) (None, 699, 300) 36300 Encoder-Input[0][0]
__________________________________________________________________________________________________
spatial_dropout1d (SpatialDropo (None, 699, 300) 0 Embedding[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 699, 300) 1200 spatial_dropout1d[0][0]
__________________________________________________________________________________________________
bidirectional (Bidirectional) (None, 699, 200) 320800 batch_normalization[0][0]
__________________________________________________________________________________________________
dropout (Dropout) (None, 699, 200) 0 bidirectional[0][0]
__________________________________________________________________________________________________
bottleneck-1 (GRU) (None, 50) 37800 dropout[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 25) 1275 bottleneck-1[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 5) 130 dense[0][0]
__________________________________________________________________________________________________
bottleneck-2 (RepeatVector) (None, 699, 5) 0 dense_1[0][0]
__________________________________________________________________________________________________
lstm (LSTM) (None, 699, 25) 3100 bottleneck-2[0][0]
__________________________________________________________________________________________________
lstm_1 (LSTM) (None, 699, 50) 15200 lstm[0][0]
__________________________________________________________________________________________________
Decoder-GRU (GRU) (None, 699, 200) 151200 lstm_1[0][0]
__________________________________________________________________________________________________
dropout_1 (Dropout) (None, 699, 200) 0 Decoder-GRU[0][0]
__________________________________________________________________________________________________
bidirectional_1 (Bidirectional) (None, 300) 421200 dropout_1[0][0]
__________________________________________________________________________________________________
encoder-output (Dense) (None, 1) 6 dense_1[0][0]
__________________________________________________________________________________________________
decoder-output (Dense) (None, 699) 210399 bidirectional_1[0][0]
==================================================================================================
Total params: 1,198,610
Trainable params: 1,198,010
Non-trainable params: 600
我想在类和解码器产生的内容(潜在表示)上训练编码器模型
当我训练模型时,解码器损耗不断增加:
Epoch 1/100
169/169 [==============================] - 6360s 38s/step - loss: 20740.2245 - encoder-output_loss: 0.8943 - decoder-output_loss: 20739.7780 - encoder-output_categorical_accuracy: 1.0000 - decoder-output_categorical_accuracy: 0.0374 - val_loss: 21028.6387 - val_encoder-output_loss: 0.8073 - val_decoder-output_loss: 21028.2383 - val_encoder-output_categorical_accuracy: 1.0000 - val_decoder-output_categorical_accuracy: 0.0561
Epoch 2/100
60/169 [=========>....................] - ETA: 1:01:58 - loss: 26872 - encoder-output_loss: 0.6741 - decoder-output_loss: 26872.3450 - encoder-output_categorical_accuracy: 1.0000 - decoder-output_categorical_accuracy: 0.0512
....
loss: 226872 - encoder-output_loss: 0.0741 - decoder-output_loss: 226872.3450
是否可以在keras中训练模型的一部分(解码器)一段时间,保存,然后用这些训练的部分初始化权重,然后进一步一起训练(编码器-解码器)
Epoch 1/100
169/169 [==============================] - 6360s 38s/step - loss: 20740.2245 - encoder-output_loss: 0.8943 - decoder-output_loss: 20739.7780 - encoder-output_categorical_accuracy: 1.0000 - decoder-output_categorical_accuracy: 0.0374 - val_loss: 21028.6387 - val_encoder-output_loss: 0.8073 - val_decoder-output_loss: 21028.2383 - val_encoder-output_categorical_accuracy: 1.0000 - val_decoder-output_categorical_accuracy: 0.0561
Epoch 2/100
60/169 [=========>....................] - ETA: 1:01:58 - loss: 26872 - encoder-output_loss: 0.6741 - decoder-output_loss: 26872.3450 - encoder-output_categorical_accuracy: 1.0000 - decoder-output_categorical_accuracy: 0.0512
....
loss: 226872 - encoder-output_loss: 0.0741 - decoder-output_loss: 226872.3450