当前位置：首页 > news >正文

关于最简单的机器学习的梯度下降公式的求导过程

news 2025/10/19 6:18:55

$loss = \frac{\sum(\hat{y} - y)^2}{sampleNum}$

那么对该式求导可得

$loss' = \frac{2}{num}\sum(\hat{y} - y)$

然后由

$y = kx +b\Rightarrow y' = x$

由求导的连续法则

那么loss对k求导得

$\frac{\mathrm{d}(loss)}{\mathrm{d}(k)} = \frac{2(\sum(\hat{y} - y)^2\times x)}{num}$

用python的numpy表示为

    grad_w = 2/sample_num * np.dot(-(y - y_pred),X)

对b求导同理

完整代码如下

import numpy as np
import matplotlib.pyplot as plt
# import tensorflow as tf
# from tensorflow.examples.tutorials.mnist import input_data
true_w = 2.5
true_b = 1.0
sample_num = 199
X = np.linspace(0,10,sample_num)
noise = np.random.normal(0,2.0,sample_num)
#第一个参数是均值，第二个参数是标准差，第三个值是样本数量
y = true_w*X + true_b +noise
plt.figure(figsize = (10,6))
#宽10英寸，高6英寸
plt.scatter(X,y)#画的散点图
# plt.show()
def linear_model(X,w,b):return X*w +b
def loss_f(y_true,y_pred):#MSEreturn np.mean((y_true - y_pred)**2)
w = 0
b = 0
learning_rate = 0.01 #是自己试的，通过经验来
loses = []#观察每一步损失有多大
epoches = 1000
#因为样本量小，所以不用分batch
for epoch in range(epoches):y_pred = linear_model(X,w,b)#X是药物的计量，X，w,b都是向量loss = loss_f(y,y_pred)loses.append(loss)#现在就要开始进行梯度下降啦grad_w = 2/sample_num * np.dot(-(y - y_pred),X)grad_b = 2/sample_num * np.sum(-y + y_pred)w = w - learning_rate * grad_wb = b - learning_rate * grad_bif (epoch+1)%100 == 0:print("{}{}{}{}".format(epoch+1,loss,w,",",b))
print("final w = ",w)
print("final b = ",b)

查看全文

http://www.dtcms.com/a/498694.html