20分钟搞懂神经网络BP算法

网络结构和样本数据

BP算法的目标就是优化神经网络的权重使得学习到的模型能够将输入值正确地映射到实际的输出值（也就是，希望模型能够模型真实数据产生的机制。在统计学中就是，我们要学习一个统计模型（统计分布函数），使得真实数据分布与统计模型产生的样本分布尽可能一致）。

前向传播过程

$$\sigma(x) = \frac{1}{1+e^{-x}}$$

$$net_{h1} = 0.15 * 0.05 + 0.2 * 0.1 + 0.35 * 1 = 0.3775$$

$$out_{h1} = \frac{1}{1+e^{-net_{h1}}} = \frac{1}{1+e^{-0.3775}} = 0.593269992$$

$$out_{h2} = 0.596884378$$

$$net_{o1} = w_5 * out_{h1} + w_6 * out_{h2} + b_2 * 1$$

$$net_{o1} = 0.4 * 0.593269992 + 0.45 * 0.596884378 + 0.6 * 1 = 1.105905967$$

$$out_{o1} = \frac{1}{1+e^{-net_{o1}}} = \frac{1}{1+e^{-1.105905967}} = 0.75136507$$

$$out_{o2} = 0.772928465$$

计算模型总误差

$$E_{total} = \sum \frac{1}{2}(target - output)^{2}$$

$$E_{o1} = \frac{1}{2}(target_{o1} - out_{o1})^{2} = \frac{1}{2}(0.01 - 0.75136507)^{2} = 0.274811083$$

$$E_{o2} = 0.023560026$$

$$E_{total} = E_{o1} + E_{o2} = 0.274811083 + 0.023560026 = 0.298371109$$

后向传播过程

输出层（output layer)

$$\frac{\partial E_{total}}{\partial w_{5}} = \frac{\partial E_{total}}{\partial out_{o1}} * \frac{\partial out_{o1}}{\partial net_{o1}} * \frac{\partial net_{o1}}{\partial w_{5}}$$

$$E_{total} = \frac{1}{2}(target_{o1} - out_{o1})^{2} + \frac{1}{2}(target_{o2} - out_{o2})^{2}$$

$$\frac{\partial E_{total}}{\partial out_{o1}} = 2 * \frac{1}{2}(target_{o1} - out_{o1})^{2 - 1} * -1 + 0$$

$$\frac{\partial E_{total}}{\partial out_{o1}} = -(target_{o1} - out_{o1}) = -(0.01 - 0.75136507) = 0.74136507$$

$$out_{o1} = \frac{1}{1+e^{-net_{o1}}}$$

$$\frac{\partial out_{o1}}{\partial net_{o1}} = out_{o1}(1 - out_{o1}) = 0.75136507(1 - 0.75136507) = 0.186815602$$

logistic函数对自变量求导，可参考：https://en.wikipedia.org/wiki/Logistic_function#Derivative

$$net_{o1} = w_5 * out_{h1} + w_6 * out_{h2} + b_2 * 1$$

$$\frac{\partial net_{o1}}{\partial w_{5}} = 1 * out_{h1} * w_5^{(1 - 1)} + 0 + 0 = out_{h1} = 0.593269992$$

$$\frac{\partial E_{total}}{\partial w_{5}} = \frac{\partial E_{total}}{\partial out_{o1}} * \frac{\partial out_{o1}}{\partial net_{o1}} * \frac{\partial net_{o1}}{\partial w_{5}}$$

$$\frac{\partial E_{total}}{\partial w_{5}} = 0.74136507 * 0.186815602 * 0.593269992 = 0.082167041$$

$$w_5^{+} = w_5 - \eta * \frac{\partial E_{total}}{\partial w_{5}} = 0.4 - 0.5 * 0.082167041 = 0.35891648$$

$w_6^{+} = 0.408666186$
$w_7^{+} = 0.511301270$
$w_8^{+} = 0.561370121$

隐藏层 （hidden layer）

$$\frac{\partial E_{total}}{\partial w_{1}} = \frac{\partial E_{total}}{\partial out_{h1}} * \frac{\partial out_{h1}}{\partial net_{h1}} * \frac{\partial net_{h1}}{\partial w_{1}}$$

$$\frac{\partial E_{total}}{\partial out_{h1}} = \frac{\partial E_{o1}}{\partial out_{h1}} + \frac{\partial E_{o2}}{\partial out_{h1}}$$

$$\frac{\partial E_{o1}}{\partial out_{h1}} = \frac{\partial E_{o1}}{\partial net_{o1}} * \frac{\partial net_{o1}}{\partial out_{h1}}$$

$$\frac{\partial E_{o1}}{\partial net_{o1}} = \frac{\partial E_{o1}}{\partial out_{o1}} * \frac{\partial out_{o1}}{\partial net_{o1}} = 0.74136507 * 0.186815602 = 0.138498562$$

$$net_{o1} = w_5 * out_{h1} + w_6 * out_{h2} + b_2 * 1$$

$$\frac{\partial net_{o1}}{\partial out_{h1}} = w_5 = 0.40$$

$$\frac{\partial E_{o1}}{\partial out_{h1}} = \frac{\partial E_{o1}}{\partial net_{o1}} * \frac{\partial net_{o1}}{\partial out_{h1}} = 0.138498562 * 0.40 = 0.055399425$$

$$\frac{\partial E_{o2}}{\partial out_{h1}} = -0.019049119$$

$$\frac{\partial E_{total}}{\partial out_{h1}} = \frac{\partial E_{o1}}{\partial out_{h1}} + \frac{\partial E_{o2}}{\partial out_{h1}} = 0.055399425 + -0.019049119 = 0.036350306$$

$$out_{h1} = \frac{1}{1+e^{-net_{h1}}}$$

$$\frac{\partial out_{h1}}{\partial net_{h1}} = out_{h1}(1 - out_{h1}) = 0.59326999(1 - 0.59326999 ) = 0.241300709$$

$$net_{h1} = w_1 * i_1 + w_3 * i_2 + b_1 * 1$$

$$\frac{\partial net_{h1}}{\partial w_1} = i_1 = 0.05$$

$$\frac{\partial E_{total}}{\partial w_{1}} = \frac{\partial E_{total}}{\partial out_{h1}} * \frac{\partial out_{h1}}{\partial net_{h1}} * \frac{\partial net_{h1}}{\partial w_{1}}$$

$$\frac{\partial E_{total}}{\partial w_{1}} = 0.036350306 * 0.241300709 * 0.05 = 0.000438568$$

$w_1$的更新值为：

$$w_1^{+} = w_1 - \eta * \frac{\partial E_{total}}{\partial w_{1}} = 0.15 - 0.5 * 0.000438568 = 0.149780716$$

$$w_2^{+} = 0.19956143$$

$$w_3^{+} = 0.24975114$$

$$w_4^{+} = 0.29950229$$

|
5天前
|

【MATLAB】BiGRU神经网络回归预测算法
【MATLAB】BiGRU神经网络回归预测算法
45 0
|
21小时前
|

【MATLAB】CEEMDAN_ MFE_SVM_LSTM 神经网络时序预测算法
【MATLAB】CEEMDAN_ MFE_SVM_LSTM 神经网络时序预测算法
10 4
|
1天前
|

【MATLAB】CEEMD_ MFE_SVM_LSTM 神经网络时序预测算法
【MATLAB】CEEMD_ MFE_SVM_LSTM 神经网络时序预测算法
7 0
|
1天前
|

8 1
|
2天前
|

【MATLAB】EEMD_ MFE_SVM_LSTM 神经网络时序预测算法
【MATLAB】EEMD_ MFE_SVM_LSTM 神经网络时序预测算法
10 1
|
3天前
|

【MATLAB】EMD_MFE_SVM_LSTM神经网络时序预测算法
【MATLAB】EMD_MFE_SVM_LSTM神经网络时序预测算法
18 1
|
4天前
|

9 0
|
3天前
|

m基于码率兼容打孔LDPC码nms最小和译码算法的LDPC编译码matlab误码率仿真
m基于码率兼容打孔LDPC码nms最小和译码算法的LDPC编译码matlab误码率仿真
9 0
|
5天前
|

11 0
|
5天前
|

【MATLAB 】 EEMD-ARIMA联合时序预测算法，科研创新优选算法
【MATLAB 】 EEMD-ARIMA联合时序预测算法，科研创新优选算法
18 0

• 机器翻译
• 工业大脑

更多

更多

更多