Python 改变';温度';在RNN中生成文本
我最近遵循了一个生成RNN以生成文本的步骤: 我准确地复制了python代码,并对其进行了理解。 我的模型已经训练了20个时代,它产生了一个由3个单词组成的长循环:Python 改变';温度';在RNN中生成文本,python,tensorflow,keras,recurrent-neural-network,Python,Tensorflow,Keras,Recurrent Neural Network,我最近遵循了一个生成RNN以生成文本的步骤: 我准确地复制了python代码,并对其进行了理解。 我的模型已经训练了20个时代,它产生了一个由3个单词组成的长循环: "and the wour and the wour and the wour..." 我在Andrej Kapathy的书中读到,改变RNN的温度会改变其可信度: 将温度从1降低到某个较低的数值(例如0.5)可以使RNN更加可靠,但其样本也更加保守 我想改变这个温度水平来降低RNN的可信度,以便它创建新的模式,但由于这是我的第一
"and the wour and the wour and the wour..."
我在Andrej Kapathy的书中读到,改变RNN的温度会改变其可信度:
将温度从1降低到某个较低的数值(例如0.5)可以使RNN更加可靠,但其样本也更加保守
我想改变这个温度水平来降低RNN的可信度,以便它创建新的模式,但由于这是我的第一个机器学习项目,我不知道如何进行
以下是我的Python/keras代码:
正在生成文本文件:
# Generate Text
import sys
import numpy
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout
from keras.layers import LSTM
from keras.callbacks import ModelCheckpoint
from keras.utils import np_utils
filename = "king_lear.txt"
raw_text = open(filename).read()
raw_text = raw_text.lower()
chars = sorted(list(set(raw_text)))
char_to_int = dict((c, i) for i, c in enumerate(chars))
int_to_char = dict((i, c) for i, c in enumerate(chars))
n_chars = len(raw_text)
n_vocab = len(chars)
print "Total Characters: ", n_chars
print "Total Vocab: ", n_vocab
seq_length = 100
dataX = []
dataY = []
for i in range(0, n_chars - seq_length, 1):
seq_in = raw_text[i:i + seq_length]
seq_out = raw_text[i + seq_length]
dataX.append([char_to_int[char] for char in seq_in])
dataY.append(char_to_int[seq_out])
n_patterns = len(dataX)
print "Total Patterns: ", n_patterns
X = numpy.reshape(dataX, (n_patterns, seq_length, 1))
X = X / float(n_vocab)
y = np_utils.to_categorical(dataY)
model = Sequential()
model.add(LSTM(256, input_shape=(X.shape[1], X.shape[2]), return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(256))
model.add(Dropout(0.2))
model.add(Dense(y.shape[1], activation='softmax'))
filename = "weights-improvement-08-2.0298-bigger.hdf5"
model.load_weights(filename)
model.compile(loss='categorical_crossentropy', optimizer='adam')
start = numpy.random.randint(0, len(dataX)-1)
pattern = dataX[start]
print "Seed:"
print "\"", ''.join([int_to_char[value] for value in pattern]), "\""
for i in range(60):
x = numpy.reshape(pattern, (1, len(pattern), 1))
x = x / float(n_vocab)
prediction = model.predict(x, verbose=0)
index = numpy.argmax(prediction)
result = int_to_char[index]
seq_in = [int_to_char[value] for value in pattern]
sys.stdout.write(result)
pattern.append(index)
pattern = pattern[1:len(pattern)]
print "\nDone."
学习档案:
# Learn Sentences
import numpy
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout
from keras.layers import LSTM
from keras.callbacks import ModelCheckpoint
from keras.utils import np_utils
filename = "king_lear.txt"
raw_text = open(filename).read()
raw_text = raw_text.lower()
chars = sorted(list(set(raw_text)))
char_to_int = dict((c, i) for i, c in enumerate(chars))
n_chars = len(raw_text)
n_vocab = len(chars)
print "Total Characters: ", n_chars
print "Total Vocab: ", n_vocab
seq_length = 100
dataX = []
dataY = []
for i in range(0, n_chars - seq_length, 1):
seq_in = raw_text[i:i + seq_length]
seq_out = raw_text[i + seq_length]
dataX.append([char_to_int[char] for char in seq_in])
dataY.append(char_to_int[seq_out])
n_patterns = len(dataX)
print "Total Patterns: ", n_patterns
X = numpy.reshape(dataX, (n_patterns, seq_length, 1))
X = X / float(n_vocab)
y = np_utils.to_categorical(dataY)
model = Sequential()
model.add(LSTM(256, input_shape=(X.shape[1], X.shape[2]), return_sequences=True))
model.add(Dropout(0.2))
model.add(LSTM(256))
model.add(Dropout(0.2))
model.add(Dense(y.shape[1], activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam')
filepath="weights-improvement-{epoch:02d}-{loss:.4f}-bigger.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='loss', verbose=1, save_best_only=True, mode='min')
callbacks_list = [checkpoint]
model.fit(X, y, epochs=50, batch_size=64, callbacks=callbacks_list)
请帮我做这个。如果这篇文章有什么问题,请不要犹豫纠正我,因为这是我的第一个问题。
非常感谢。查看Keras GitHub。可以在softmax之前添加Lambda层,以除以温度:
model.add(Lambda(lambda x: x / temp))
根据:
对于高温,所有行为的概率几乎相同,温度越低,预期回报对概率的影响越大。对于低温,预期回报最高的行为概率趋于1
非常感谢。我是否在生成输出的generate_text.py中添加这一行。还是在培训代码中?而且,如果我想让温度高,我是使用一个高值,还是一个低值,因为在训练代码中,在softmax层之前,它除以。高温=高值,低温=低值。