Unit 3 : Vision Transformers / Transfer Learning & Fine-Tuning Chapter Content #204
Conversation
added mdx
add colab button
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Transfer learning sub section in the fine tuning notebook
chapters/en/Unit 3 - Vision Transformers/KnowledgeDistillation.mdx
Outdated
Show resolved
Hide resolved
chapters/en/Unit 3 - Vision Transformers/KnowledgeDistillation.mdx
Outdated
Show resolved
Hide resolved
chapters/en/Unit 3 - Vision Transformers/KnowledgeDistillation.mdx
Outdated
Show resolved
Hide resolved
chapters/en/Unit 3 - Vision Transformers/KnowledgeDistillation.mdx
Outdated
Show resolved
Hide resolved
|
|
||
|  | ||
|
|
||
| If you had someone just tell you "the answer is Dracoy Malfoy", that doesn't teach you a whole lot about each of the characters' relative relationships with Harry Potter. |
There was a problem hiding this comment.
I think the original is correct
There was a problem hiding this comment.
It's "Draco Malfoy", the typo is in the first name :)
| predicted_panoptic_map = result["segmentation"] | ||
| ``` | ||
|
|
||
| ## Applying Transfer learning/Finetuning to ViT-based segmentation models |
There was a problem hiding this comment.
| ## Applying Transfer learning/Finetuning to ViT-based segmentation models | |
| ## Fine-tuning Vision Transformer-based Segmentation Models |
| With many pretrained segmentation models available, transfer learning or finetuning are used to adapt these models to specific use cases, especially since ViT-based segmentation models, like MaskFormer, are data-hungry and challenging to train from scratch. | ||
| these techniques leverages pre-trained representations to adapt these models efficiently to new tasks. Typically for MaskFormer, the backbone, the pixel decoder and the transformer decoder are kept frozen to leverage their learned general features, while the transformer module is finetuned to adapt its class prediction and mask generation capabilities to new segmentation tasks. | ||
|
|
||
| This notebook will walk you through a transfer learning tutorial on image segmentation using MaskFormer: |
There was a problem hiding this comment.
| This notebook will walk you through a transfer learning tutorial on image segmentation using MaskFormer: | |
| [This notebook](https://colab.research.google.com/github/johko/computer-vision-course/blob/main/notebooks/Unit%203%20-%20Vision%20Transformers/transfer-learning-segmentation.ipynb) will walk you through a transfer learning tutorial on image segmentation using MaskFormer. |
|
|
||
| This notebook will walk you through a transfer learning tutorial on image segmentation using MaskFormer: | ||
|
|
||
| <a |
There was a problem hiding this comment.
You can remove the HTML link
|
|
||
| ## References | ||
|
|
||
| 1. [MaskFormer Hugging Face documentation](https://huggingface.co/docs/transformers/en/model_doc/maskformer) |
There was a problem hiding this comment.
| 1. [MaskFormer Hugging Face documentation](https://huggingface.co/docs/transformers/en/model_doc/maskformer) | |
| - [MaskFormer Hugging Face documentation](https://huggingface.co/docs/transformers/en/model_doc/maskformer) |
| ## References | ||
|
|
||
| 1. [MaskFormer Hugging Face documentation](https://huggingface.co/docs/transformers/en/model_doc/maskformer) | ||
| 2. [Image Segmentation Hugging Face Task Guide](https://huggingface.co/docs/transformers/en/tasks/semantic_segmentation) |
There was a problem hiding this comment.
| 2. [Image Segmentation Hugging Face Task Guide](https://huggingface.co/docs/transformers/en/tasks/semantic_segmentation) | |
| - [Image Segmentation Hugging Face Task Guide](https://huggingface.co/docs/transformers/en/tasks/semantic_segmentation) |
….mdx Co-authored-by: Merve Noyan <[email protected]>
….mdx Co-authored-by: Merve Noyan <[email protected]>
….mdx Co-authored-by: Merve Noyan <[email protected]>
….mdx Co-authored-by: Merve Noyan <[email protected]>
….mdx Co-authored-by: Merve Noyan <[email protected]>
|
|
||
|  | ||
|
|
||
| If you had someone just tell you "the answer is Dracoy Malfoy", that doesn't teach you a whole lot about each of the characters' relative relationships with Harry Potter. |
There was a problem hiding this comment.
I think the original is correct
….mdx Co-authored-by: Merve Noyan <[email protected]>
…or Image Classification.mdx Co-authored-by: Merve Noyan <[email protected]>
Very sorry about that @merveenoyan !! We were just concerned about timelines, that we wouldn't get all of our individual changes merged in time. We realize this must have been super annoying to go through! |
|
Looks good to me 👍🏽 (granted noted grammatical correction occur) |
|
Hi I am adding the reference for PR #218 where I added the mdx file for fine-tuning for object detection. |
|
@shreydan @asusevski I think the table of contents is not up to date, can you merge the main of this repository into your branch? then we can merge |
|
@merveenoyan merged and update table of contents! Thank you! |
|
@asusevski can you approve so we can merge? I'll fix CI afterwards, the previously merged PR broke that |
|
@asusevski I fixed bunch of grammar errors and made the CI green, if you can approve we can merge |
chapters/en/Unit 3 - Vision Transformers/KnowledgeDistillation.mdx
Outdated
Show resolved
Hide resolved
|
@asusevski had to solve merge conflict, sorry, can you approve again? |
hello, this PR adds the following chapter content for Unit 3: Vision Transformers : Transfer Learning and Fine-Tuning
Contributions:
@asusevski
KnowledgeDistillation.mdxKnowledgeDistillation.ipynb: took a large part of the intro and moved it into the MDX so that the MDX for the chapter is a better introduction to the topic and the notebook is more succinctly focused on the example with code.@hanouticelina
Vision Transformers for Image Segmentation.mdx@shreydan
Vision Transformers for Image Classification.mdx@sezan92
Fine-tuning Vision Transformers for Object detection.ipynb: included an additional section on transfer learning.@Anindyadeep
Fine-tuning Vision Transformers for Object Detection.ipynbas per reviews.