Skip to content

Commit

Permalink
Update chapters/en/Unit 3 - Vision Transformers/KnowledgeDistillation…
Browse files Browse the repository at this point in the history
….mdx

Co-authored-by: Merve Noyan <[email protected]>
  • Loading branch information
asusevski and merveenoyan authored Feb 10, 2024
1 parent 801776e commit 73316e1
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Presumably, we've all had teachers who "teach" by simply providing us the correc
of machine learning models where we provide a labelled dataset to train on. Instead of having a model train on labels, however,
we can pursue [Knowledge Distillation](https://arxiv.org/abs/1503.02531) as an alternative to arrive at a much smaller model that can perform comparably to the larger model, and much faster to boot.

## For some intuition,
## Intuition Behind Knowledge Distillation

Imagine you were given this multiple choice question:

Expand Down

0 comments on commit 73316e1

Please sign in to comment.