Skip to content

Commit

Permalink
Fix typos/wording
Browse files Browse the repository at this point in the history
  • Loading branch information
andreasgrv committed Dec 17, 2024
1 parent f979fc7 commit f83359a
Showing 1 changed file with 5 additions and 5 deletions.
10 changes: 5 additions & 5 deletions notebooks/generative-vs-discriminative-circuit.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -190,8 +190,8 @@
"Can we do better?\n",
"\n",
"\n",
"### Parameter Sharing\n",
"Yes! We can use **parameter sharing**, i.e., we can share the early layers of our circuit and only adapt the output layer to produce our conditional distributions.\n",
"### Shared Representations\n",
"Yes! We can share the early layers of our circuit and only adapt the output layer to produce our conditional distributions.\n",
"Since there are $10$ classes, we change the last layer of our circuit to output $10$ log-probabilities: the probability mass over images conditioned on each class, $p(\\mathbf{x} \\mid y)$.\n",
"We can do so by specifying the `num_classes` option in `data_modalities.image_data`.\n",
"\n",
Expand Down Expand Up @@ -347,7 +347,7 @@
"\n",
"</div>\n",
"\n",
"The RAT SPN papepr also considers training using a trade-off between the two objectives.\n",
"The RAT SPN paper also considers training using a trade-off between the two objectives.\n",
"To do so, it introduces a parameter $\\lambda \\in [0, 1]$ and minimize a combined loss:\n",
"\n",
"$$\n",
Expand All @@ -357,7 +357,7 @@
"where we divide $\\mathcal{L}_{gen}(\\theta)$ by the number of pixels in the image, $|X|$, such that the losses are on a comparable scale[<sup>2</sup>](#fn2).\n",
"\n",
"* $\\lambda = 0 \\rightarrow$ Model trained only generatively\n",
"* $\\lambda = 1 \\rightarrow$ Model trained only contrastively\n",
"* $\\lambda = 1 \\rightarrow$ Model trained only discriminatively\n",
"* $\\lambda \\in (0, 1) \\rightarrow$ Trade-off between losses\n",
"\n",
"<span style=\"font-size: .75em;\"> [<sup id=\"fn2\">2</sup>](#fn2-back) Note that $\\mathcal{L}_{gen}$ and $\\mathcal{L}_{dis}$ are on different scales, primarily because they are defined on a different number of random variables.\n",
Expand Down Expand Up @@ -1222,7 +1222,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.7"
"version": "3.10.12"
}
},
"nbformat": 4,
Expand Down

0 comments on commit f83359a

Please sign in to comment.