progress commit

NishantBaheti · Jan 1, 2025 · 618e32b · 618e32b
1 parent 1ee339e
commit 618e32b
Showing 1 changed file with 27 additions and 22 deletions.
diff --git a/notebooks/transformers/explore.ipynb b/notebooks/transformers/explore.ipynb
@@ -2598,33 +2598,38 @@
     "\n",
     "```sh\n",
     "\n",
-    "        input\n",
-    "          |\n",
-    "          |\n",
-    "          |\\\n",
-    "          |  \\\n",
-    "          |    \\  \n",
-    "          |     BN\n",
-    "          |      |\n",
-    "          |     ReLU\n",
-    "          |      |\n",
-    "          |    Weight\n",
-    "          |      |\n",
-    "          |      BN\n",
-    "          |      |\n",
-    "          |    ReLU\n",
-    "          |     /\n",
-    "          |   weight\n",
-    "          |  /\n",
-    "          ADD\n",
-    "           |\n",
-    "           |\n",
-    "          Output\n",
+    "                     input\n",
+    "                     |\n",
+    "                     o Fork\n",
+    "                     |\\\n",
+    "            |        |  \\            A\n",
+    "            |        |    \\          |\n",
+    "            |        |     BN        |\n",
+    "Forward     |        |      |        |  Backward\n",
+    "            |        |     ReLU      |\n",
+    "            |        |      |        |\n",
+    "            V        |    Weight\n",
+    "                     |      |\n",
+    "                     |      BN\n",
+    "                     |      |\n",
+    "                     |    ReLU\n",
+    "                     |     /\n",
+    "                     |   weight\n",
+    "                     |  /\n",
+    "                    addition\n",
+    "                     |\n",
+    "                     |\n",
+    "                     Output\n",
     "```\n",
     "\n",
     "Because of skip connection the gradients can travel faster to initial layers and initial layers can learn as fast as final layers, This helps when we are building very Deep Neural Networks"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": []
+  },
   {
    "cell_type": "code",
    "execution_count": null,