当前位置: 首页 > news >正文

机器学习:反向神经元传播公式推导

首先,我们有正向传播的公式:
qk+1,i=∑j=1nkwk+1,i,j⋅rk,j+bk+1,i q_{k+1,i}=\sum_{j=1}^{n_{k}} w_{k+1,i,j}\cdot r_{k,j}+b_{k+1,i} qk+1,i=j=1nkwk+1,i,jrk,j+bk+1,i

∂l∂wk,i,j=∂l∂qk,i⋅∂qk,i∂wk,i,j=∂l∂qk,i⋅rk−1,j \begin{aligned} \frac{\partial l}{\partial w_{k,i,j}}&= \frac{\partial l}{\partial q_{k,i}}\cdot\frac{\partial q_{k,i}}{\partial w_{k,i,j}}\\ &=\frac{\partial l}{\partial q_{k,i}}\cdot r_{k-1,j} \end{aligned} wk,i,jl=qk,ilwk,i,jqk,i=qk,ilrk1,j

∂l∂bk,i=∂l∂qk,i⋅∂qk,i∂bk,i=∂l∂qk,i \begin{aligned} \frac{\partial l}{\partial b_{k,i}} &=\frac{\partial l}{\partial q_{k,i}}\cdot\frac{\partial q_{k,i}}{\partial b_{k,i}}\\ &=\frac{\partial l}{\partial q_{k,i}} \end{aligned} bk,il=qk,ilbk,iqk,i=qk,il

观察这个式子:

qk+1,i=∑j=1nkwk+1,i,j⋅rk,j+bk+1,i q_{k+1,i}=\sum_{j=1}^{n_{k}} w_{k+1,i,j}\cdot r_{k,j}+b_{k+1,i} qk+1,i=j=1nkwk+1,i,jrk,j+bk+1,i

我们考察 rk,jr_{k,j}rk,jqk+1,iq_{k+1,i}qk+1,i 的影响,发现:
∂qk+1,i∂rk,j=wk+1,i,j \frac{\partial q_{k+1,i}}{\partial r_{k,j}} =w_{k+1,i,j} rk,jqk+1,i=wk+1,i,j

进而:

∂qk+1,i∂qk,j=∂qk+1,i∂rk,j⋅∂rk,j∂qk,j=wk+1,i,j⋅fk′(qk,j) \begin{aligned} \frac{\partial q_{k+1,i}}{\partial q_{k,j}} &= \frac{\partial q_{k+1,i}}{\partial r_{k,j}} \cdot \frac{\partial r_{k,j}}{\partial q_{k,j}} \\ &= w_{k+1,i,j} \cdot f_k^{'}(q_{k,j}) \end{aligned} qk,jqk+1,i=rk,jqk+1,iqk,jrk,j=wk+1,i,jfk(qk,j)

因此:

δk,j=∂l∂qk,j=∂l∂qk+1,i⋅∂qk+1,i∂qk,j=δk+1,i⋅∂qk+1,i∂qk,j \begin{aligned} \delta_{k,j} = \frac{\partial l}{\partial q_{k,j}} &= \frac{\partial l}{\partial q_{k+1,i}} \cdot \frac{\partial q_{k+1,i}}{\partial q_{k,j}} \\ &= \delta_{k+1,i} \cdot \frac{\partial q_{k+1,i}}{\partial q_{k,j}} \\ \end{aligned} δk,j=qk,jl=qk+1,ilqk,jqk+1,i=δk+1,iqk,jqk+1,i

最后,由于每一个神经元对下一层有多条影响路径,所以对其求和,并带入
∂qk+1,i∂qk,j\frac{\partial q_{k+1,i}}{\partial q_{k,j}}qk,jqk+1,i

δk,j=∂l∂qk,j=∑i=1nk+1∂l∂qk+1,i⋅∂qk+1,i∂qk,j=fk′(qk,j)⋅∑i=1nk+1δk+1,i⋅wk+1,i,j \begin{aligned} \delta_{k,j}= \frac{\partial l}{\partial q_{k,j}} &= \sum_{i=1}^{n_{k+1}} \frac{\partial l}{\partial q_{k+1,i}} \cdot \frac{\partial q_{k+1,i}}{\partial q_{k,j}} \\ &= f_k^{'}(q_{k,j}) \cdot \sum_{i=1}^{n_{k+1}} \delta_{k+1,i} \cdot w_{k+1,i,j} \end{aligned} δk,j=qk,jl=i=1nk+1qk+1,ilqk,jqk+1,i=fk(qk,j)i=1nk+1δk+1,iwk+1,i,j

l=L(rT1,rT2,...rTnT,y1,y2,...ynT)l=L(r_{T1}, r_{T2}, ... r_{Tn_T}, y_1, y_2, ... y_{n_T})l=L(rT1,rT2,...rTnT,y1,y2,...ynT)

∂l∂qTi=∂l∂rTi⋅∂rTi∂qTi=∂l∂rTi⋅fT′(qTi) \begin{aligned} \dfrac{\partial l}{\partial q_{Ti}}&=\dfrac{\partial l}{\partial r_{Ti}}\cdot\dfrac{\partial r_{Ti}}{\partial q_{Ti}}\\ &=\dfrac{\partial l}{\partial r_{Ti}}\cdot f_T^{'}(q_{Ti}) \end{aligned} qTil=rTilqTirTi=rTilfT(qTi)

http://www.dtcms.com/a/272745.html

相关文章:

  • 记录今天学习Comfyui的感受
  • python正则表达式(小白五分钟从入门到精通)
  • 智能化时代下的门店运营:AI的深刻影响
  • 2025年第十五届APMCM亚太地区大学生数学建模竞赛(中文赛项)
  • 【C++】红黑树的底层思想 and 大厂面试常问
  • BootStrap
  • 售前:该站高位思考还是站低位思考
  • Codeforces Round 1034 (Div. 3) G题题解记录
  • 创建本地软件仓库(rhel7与rhel9)
  • HighReport报表工具开始支持BS报表设计器
  • SW-CA(多平台产品上架系统)
  • uni-app 途径站点组件开发与实现分享
  • 体积超过2MB?uniapp小程序分包上传
  • [论文阅读]Text Compression for Efficient Language Generation
  • Go语言包管理完全指南:从基础到最佳实践
  • BM12 单链表的排序
  • 东土科技智能塔机系统亮相南京,助力智能建造高质量发展
  • HOOK专题
  • web前端面试笔记
  • 北京一家IPO业绩持续性存疑,关联交易频繁独立性堪忧
  • 24、企业设备清单管理(Equipment)详解:从分类到管理,设备全生命周期把控
  • etf期权到期的风险大不大怎么看?
  • MySQL中使用GROUP_CONCAT数据丢失问题的原因和处理方案
  • 深入理解区块链 | 去中心化架构与密码学保障
  • springboot数据脱敏(接口级别)
  • Uni-app 生命周期与钩子:程序的“生命”旅程
  • 企业电商平台搭建:ZKmall开源商城服务器部署与容灾方案
  • Spring--04--1--AOP自定义注解,记录用户操作日志
  • 第35周—————糖尿病预测模型优化探索
  • 网络资源模板--基于Android Studio 实现的健身系统App