Python pytorch中隐藏单元的动态添加

我试图在训练时动态地将隐藏单元添加到三层神经网络（输入、隐藏、输出）。当我添加新的隐藏单位时，我想保留网络训练部分的权重。这是我的代码

class my_network(torch.nn.Module):
    def __init__(self,input_dim,hidden_dim,output_dim):
        super(my_network,self).__init__()
        self.I = input_dim
        self.H = hidden_dim
        self.O = output_dim
        self.layer1 = torch.nn.Linear(input_dim,hidden_dim)
        self.layer2 = torch.nn.Linear(hidden_dim,output_dim)

    def add_neurons(self,no_of_neurons,flag):
        if flag == 1:
            weights = [self.layer1.weight.data,self.layer2.weight.data]
            self.layer1 = torch.nn.Linear(self.I,self.H+no_of_neurons)
            self.layer2 = torch.nn.Linear(self.H+no_of_neurons,self.O)
            self.layer1.weight.data[0:-no_of_neurons,:] = weights[0]
            self.layer2.weight.data[:,0:-no_of_neurons] = weights[1]
            self.H = self.H + no_of_neurons
        return self.layer1.weight.shape[0]

    def forward(self,x):
        temp = self.layer1(x)
        out = self.layer2(temp)
        return out

---

我注意到，一旦我调用“add_neurons”方法，权重就会停止更新（同时生成梯度）。任何帮助都将不胜感激。

优化器可能不会被告知您添加到模型中的新参数。最简单的方法可能是使用模型参数的更新列表重新创建optimizer对象