Machine learning PyTorch,简单字符级RNN,can';一个例子不过分
我是PyTorch框架的新手(主要来自Theano和Tensorflow): 我已经按照介绍教程阅读了使用字符级别RNN的名称分类。 现在,我尝试将其应用于char级别的LSTM模型,以便获得使用该框架的一些实际经验。 基本上,我输入了char索引的模型序列,并将相同的序列作为模型的目标,但将来会移动一个。 然而,我不能过分适合一个简单的训练例子,我不知道我做错了什么。 如果有人能发现我的错误,那将非常有帮助。 这是我的密码:Machine learning PyTorch,简单字符级RNN,can';一个例子不过分,machine-learning,char,lstm,recurrent-neural-network,pytorch,Machine Learning,Char,Lstm,Recurrent Neural Network,Pytorch,我是PyTorch框架的新手(主要来自Theano和Tensorflow): 我已经按照介绍教程阅读了使用字符级别RNN的名称分类。 现在,我尝试将其应用于char级别的LSTM模型,以便获得使用该框架的一些实际经验。 基本上,我输入了char索引的模型序列,并将相同的序列作为模型的目标,但将来会移动一个。 然而,我不能过分适合一个简单的训练例子,我不知道我做错了什么。 如果有人能发现我的错误,那将非常有帮助。 这是我的密码: class LSTMTxtGen(nn.Module): def _
class LSTMTxtGen(nn.Module):
def __init__(self, hidden_dim, n_layer, vocab_size):
super(LSTMTxtGen, self).__init__()
self.n_layer = n_layer
self.hidden_dim = hidden_dim
self.vocab_size = vocab_size
self.lstm = nn.LSTM(vocab_size, hidden_dim, n_layer, batch_first=True)
# The linear layer that maps from hidden state space to tag space
#self.hidden = self.init_hidden()
def init_hidden(self, batch_size):
# Before we've done anything, we dont have any hidden state.
# Refer to the Pytorch documentation to see exactly
# why they have this dimensionality.
# The axes semantics are (num_layers, minibatch_size, hidden_dim)
return (autograd.Variable(torch.zeros(self.n_layer, batch_size,
self.hidden_dim)),
autograd.Variable(torch.zeros(self.n_layer, batch_size,
self.hidden_dim)))
def forward(self, seqs):
self.hidden = self.init_hidden(seqs.size()[0])
lstm_out, self.hidden = self.lstm(seqs, self.hidden)
lstm_out = lstm_out.contiguous().view(-1, self.hidden_dim)
lstm_out = nn.Linear(lstm_out.size(1), self.vocab_size)(lstm_out)
return lstm_out
model = LSTMTxtGen (
hidden_dim = 50,
n_layer = 3,
vocab_size = 44,
)
print(Model)
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adamax(model.parameters())
G = Data.batch_generator(5,100)
batch_per_epoch, to_idx, to_char = next(G)
X, Y = next(G)
for epoch in range(10):
losses = []
for batch_count in range(batch_per_epoch):
model.zero_grad()
#mode.hidden = model.init_hidden()
#X, Y = next(G)
X = autograd.Variable(torch.from_numpy(X))
Y = autograd.Variable(torch.from_numpy(Y))
preds = model(X)
loss = criterion(preds.view(-1, model.vocab_size), Y.view(-1))
loss.backward()
optimizer.step()
losses.append(loss)
if (batch_count % 20 == 0):
print('Loss: ', losses[-1])