loss function, cost function, and objective function

最新推荐文章于 2025-08-22 09:54:53 发布

转载最新推荐文章于 2025-08-22 09:54:53 发布 · 234 阅读

CC 4.0 BY-SA版权

原文链接：https://stats.stackexchange.com/questions/179026/objective-function-cost-function-loss-function-are-they-the-same-thing

文章标签：

#深度学习

深度学习专栏收录该内容

4 篇文章

订阅专栏

本文详细阐述了损失函数、成本函数和目标函数在机器学习中的区别和联系。损失函数衡量单个数据点上的预测误差，如平方损失和 hinge 损失；成本函数是对整个训练集损失的总和加上正则化项，如均方误差和 SVM 成本函数；而目标函数是最通用的概念，涵盖训练中优化的所有函数，包括最大似然估计等。这三者在机器学习优化过程中扮演着不同角色。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

https://stats.stackexchange.com/questions/179026/objective-function-cost-function-loss-function-are-they-the-same-thing

These are not very strict terms and they are highly related. However:

Loss function is usually a function defined on a data point, prediction and label, and measures the penalty. For example:
square loss $l(f(xi∣θ),yi)=(f(xi∣θ)−yi)2l(f(x_i|\theta),y_i) = \left (f(x_i|\theta)-y_i \right )^2$ , used in linear regression
hinge loss $l(f(xi∣θ),yi)=max⁡(0,1−f(xi∣θ)yi)l(f(x_i|\theta), y_i) = \max(0, 1-f(x_i|\theta)y_i)$ , used in SVM
0/1 loss $f(xi∣θ)≠yil(f(x_i|\theta), y_i) = 1 \iff f(x_i|\theta) \neq y_i$ , used in theoretical analysis and definition of accuracy
Cost function is usually more general. It might be a sum of loss functions over your training set plus some model complexity penalty (regularization). For example:
Mean Squared Error $MSE(θ)=1N∑i=1N(f(xi∣θ)−yi)2MSE(\theta) = \frac{1}{N} \sum_{i=1}^N \left (f(x_i|\theta)-y_i \right )^2$
SVM cost function $SVM(θ)=∥θ∥2+C∑i=1NξiSVM(\theta) = \|\theta\|^2 + C \sum_{i=1}^N \xi_i$ (there are additional constraints connecting $ξi\xi_i$ with $C$ and with training set)
Objective function is the most general term for any function that you optimize during training. For example, a probability of generating training set in maximum likelihood approach is a well defined objective function, but it is not a loss function nor cost function (however you could define an equivalent cost function). For example:
MLE is a type of objective function (which you maximize)
Divergence between classes can be an objective function but it is barely a cost function, unless you define something artificial, like 1-Divergence, and name it a cost
Long story short, I would say that: