Java 神经网络中的反向传播算法理解困难
我很难理解反向传播算法。我读了很多书，搜索了很多，但我不明白为什么我的神经网络不工作。我想确认我每一部分都做对了 这是我的神经网络，当它初始化时，当第一行输入[1,1]和输出[0]被设置时（正如你所看到的，我正在尝试使用XOR神经网络）： 我有3层：输入、隐藏和输出。第一层（输入）和隐藏层包含2个神经元，每个神经元中有2个突触。最后一层（输出）包含一个神经元和两个突触 一个突触包含一个权重和它之前的增量（在开始时，它是0）。连接到突触的输出可以通过与突触相关联的sourceNeuron找到，如果没有sourceNeuron，则可以在输入数组中找到（如在输入层中） 类Layer.java包含神经元列表。在我的NeuralNetwork.java中，我初始化了神经网络，然后在训练集中循环。在每次迭代中，我替换输入和输出值，并调用反向传播算法上的train，该算法对当前集运行一定数量的时间（目前为1000次） 我使用的激活作用是乙状结肠 培训集和验证集为（输入1、输入2、输出）： 下面是我的Neuron.java实现：Java 神经网络中的反向传播算法理解困难,java,algorithm,artificial-intelligence,neural-network,backpropagation,Java,Algorithm,Artificial Intelligence,Neural Network,Backpropagation,我很难理解反向传播算法。我读了很多书，搜索了很多，但我不明白为什么我的神经网络不工作。我想确认我每一部分都做对了 这是我的神经网络，当它初始化时，当第一行输入[1,1]和输出[0]被设置时（正如你所看到的，我正在尝试使用XOR神经网络）： 我有3层：输入、隐藏和输出。第一层（输入）和隐藏层包含2个神经元，每个神经元中有2个突触。最后一层（输出）包含一个神经元和两个突触 一个突触包含一个权重和它之前的增量（在开始时，它是0）。连接到突触的输出可以通过与突触相关联的sourceNeuron找到，如
public class Neuron {
private IActivation activation;
private ArrayList<Synapse> synapses; // Inputs
private double output; // Output
private double errorToPropagate;
public Neuron(IActivation activation) {
this.activation = activation;
this.synapses = new ArrayList<Synapse>();
this.output = 0;
this.errorToPropagate = 0;
}
public void updateOutput(double[] inputs) {
double sumWeights = this.calculateSumWeights(inputs);
this.output = this.activation.activate(sumWeights);
}
public double calculateSumWeights(double[] inputs) {
double sumWeights = 0;
int index = 0;
for (Synapse synapse : this.getSynapses()) {
if (inputs != null) {
sumWeights += synapse.getWeight() * inputs[index];
} else {
sumWeights += synapse.getWeight() * synapse.getSourceNeuron().getOutput();
}
index++;
}
return sumWeights;
}
public double getDerivative() {
return this.activation.derivative(this.output);
}
[...]
}
this.forwardPropagation(neuralNetwork, inputs);
this.backwardPropagation(neuralNetwork, expectedOutput);
this.updateWeights(neuralNetwork);
Real: 0.0
Current: 0.025012156926937503
Real: 1.0
Current: 0.022566830709341495
Real: 1.0
Current: 0.02768416343491415
Real: 0.0
Current: 0.024903432706154027
=== Input Layer
== Neuron #1
= Synapse #1
Weight: -0.19283583155573614
Input: 1.0
= Synapse #2
Weight: 0.04023817185601586
Input: 1.0
Sum: -0.15259765969972028
Output: 0.461924442180935
== Neuron #2
= Synapse #1
Weight: -0.3281099260608612
Input: 1.0
= Synapse #2
Weight: -0.4388250065958519
Input: 1.0
Sum: -0.7669349326567131
Output: 0.31714251453174147
=== Hidden Layer
== Neuron #1
= Synapse #1
Weight: 0.16703288052854093
Input: 0.461924442180935
= Synapse #2
Weight: 0.31683996162148054
Input: 0.31714251453174147
Sum: 0.17763999229679783
Output: 0.5442935820534444
== Neuron #2
= Synapse #1
Weight: -0.45330313978424686
Input: 0.461924442180935
= Synapse #2
Weight: 0.3287014377113835
Input: 0.31714251453174147
Sum: -0.10514659949771789
Output: 0.47373754172497556
=== Output Layer
== Neuron #1
= Synapse #1
Weight: 0.08643751629154495
Input: 0.5442935820534444
= Synapse #2
Weight: -0.29715579267218695
Input: 0.47373754172497556
Sum: -0.09372646936373039
Output: 0.47658552081912403
我可能有偏见的问题。我将借助以下答案进行调查：。它不会在下一个数据集返回，因此…我最终发现了问题。对于XOR，我不需要任何偏差，它正在收敛到预期值。当你对最终输出进行四舍五入时，我得到了准确的输出。所需要的是训练，然后验证，然后再次训练，直到神经网络令人满意。我一次又一次地训练每一组直到满意为止，但不是整组
// Initialize the Neural Network
algorithm.initialize(this.numberOfInputs);
int index = 0;
double errorRate = 0;
// Loop until satisfaction or after some iterations
do {
// Train the Neural Network
algorithm.train(this.trainingDataSets, this.numberOfInputs);
// Validate the Neural Network and return the error rate
errorRate = algorithm.run(this.validationDataSets, this.numberOfInputs);
index++;
} while (errorRate > minErrorRate && index < numberOfTrainValidateIteration);
public class BackPropagationStrategy implements IStrategy, Serializable {
[...]
public void updateWeightsAndBias(NeuralNetwork neuralNetwork, double[] inputs) {
for (int i = neuralNetwork.getLayers().size() - 1; i >= 0; i--) {
Layer layer = neuralNetwork.getLayers().get(i);
for (Neuron neuron : layer.getNeurons()) {
[...]
Synapse bias = neuron.getBias();
double delta = learning * 1.0;
bias.setWeight(bias.getWeight() + delta + this.momentum * bias.getDelta());
bias.setDelta(delta);
}
}
}
[...]
有了真实数据，网络正在收敛。现在，要找到学习率、动量、错误率、神经元数量、隐藏层数量等的完美变量组合（如果可能的话）是一项修剪工作。您对函数和变量使用了令人困惑的名称。至少，这会让你的代码很难理解，最多也表明你在理解算法方面还有一些不足。例如，您使用
this.errorcalculateSumWeightserrorToPropagateReal: 0.0
Current: 0.025012156926937503
Real: 1.0
Current: 0.022566830709341495
Real: 1.0
Current: 0.02768416343491415
Real: 0.0
Current: 0.024903432706154027
=== Input Layer
== Neuron #1
= Synapse #1
Weight: -0.19283583155573614
Input: 1.0
= Synapse #2
Weight: 0.04023817185601586
Input: 1.0
Sum: -0.15259765969972028
Output: 0.461924442180935
== Neuron #2
= Synapse #1
Weight: -0.3281099260608612
Input: 1.0
= Synapse #2
Weight: -0.4388250065958519
Input: 1.0
Sum: -0.7669349326567131
Output: 0.31714251453174147
=== Hidden Layer
== Neuron #1
= Synapse #1
Weight: 0.16703288052854093
Input: 0.461924442180935
= Synapse #2
Weight: 0.31683996162148054
Input: 0.31714251453174147
Sum: 0.17763999229679783
Output: 0.5442935820534444
== Neuron #2
= Synapse #1
Weight: -0.45330313978424686
Input: 0.461924442180935
= Synapse #2
Weight: 0.3287014377113835
Input: 0.31714251453174147
Sum: -0.10514659949771789
Output: 0.47373754172497556
=== Output Layer
== Neuron #1
= Synapse #1
Weight: 0.08643751629154495
Input: 0.5442935820534444
= Synapse #2
Weight: -0.29715579267218695
Input: 0.47373754172497556
Sum: -0.09372646936373039
Output: 0.47658552081912403
// Initialize the Neural Network
algorithm.initialize(this.numberOfInputs);
int index = 0;
double errorRate = 0;
// Loop until satisfaction or after some iterations
do {
// Train the Neural Network
algorithm.train(this.trainingDataSets, this.numberOfInputs);
// Validate the Neural Network and return the error rate
errorRate = algorithm.run(this.validationDataSets, this.numberOfInputs);
index++;
} while (errorRate > minErrorRate && index < numberOfTrainValidateIteration);
public class Neuron implements Serializable {
[...]
private Synapse bias;
public Neuron(IActivation activation) {
[...]
this.bias = new Synapse(this);
this.bias.setWeight(0.5); // Set initial weight OR keep the random number already set
}
public void updateOutput(double[] inputs) {
double sumWeights = this.calculateSumWeights(inputs);
this.output = this.activation.activate(sumWeights + this.bias.getWeight() * 1.0);
}
[...]
public class BackPropagationStrategy implements IStrategy, Serializable {
[...]
public void updateWeightsAndBias(NeuralNetwork neuralNetwork, double[] inputs) {
for (int i = neuralNetwork.getLayers().size() - 1; i >= 0; i--) {
Layer layer = neuralNetwork.getLayers().get(i);
for (Neuron neuron : layer.getNeurons()) {
[...]
Synapse bias = neuron.getBias();
double delta = learning * 1.0;
bias.setWeight(bias.getWeight() + delta + this.momentum * bias.getDelta());
bias.setDelta(delta);
}
}
}
[...]