ch07-逻辑回归/02-损失函数的梯度推导.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 先算出`损失函数`\n",
    "\n",
    "$$\\widehat{p}=\\sigma(\\theta^{T}\\cdot x_{b})=\\frac{1}{1+e^{\\theta^{T}\\cdot x_{b}}}$$\n",
    "\n",
    "估计的代表概率\n",
    "\n",
    "MSE：损失函数？\n",
    "\n",
    "不能推导出正规方程解、没有数学公式解；只能使用梯度下降法\n",
    "\n",
    "不能直接套公式，算出数学解\n",
    "\n",
    "也是个凸函数，没有局部最优解，只有唯一的一个全局最优解\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 梯度下降法\n",
    "$$J(\\theta)=-\\frac{1}{m} \\sum_{i=1}^{m}y^{(i)}log(\\sigma(X_{b}^{(i)}\\theta))+(1-y^{(i)})log(1-\\sigma(X_{b}^{(i)}\\theta))$$\n",
    "\n",
    "- 对 $\\theta$ 的每一维度求导\n",
    "\n",
    "### 先解决 $\\sigma$函数的导数\n",
    "\n",
    "$$\\frac{J(\\theta)}{\\theta_{j}}=\\frac{1}{m} \\sum_{i=1}^{m}(\\sigma(X_{b}^{(i)}\\theta)-y^{(i)})X_{j}^{(i)}$$\n",
    "$$=\\frac{1}{m} \\sum_{i=1}^{m}(\\widehat{y}^{(i)}-y^{(i)})X_{j}^{(i)}$$\n",
    "\n",
    "$\\sigma(X_{b}^{(i)}\\theta)$ 第i行乘以 $\\theta$ 就是逻辑回归中预测第i行对应的概率是多少！\n",
    "\n",
    "计算梯度，搜索出 `$\\sigma$` 值就可以了\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/09.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/10.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/11.png)\n",
    "![](images/12.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](images/06.png)\n",
    "![](images/07.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4rc1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}