Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates README #3

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 25 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,32 @@
VTON-IT: Virtual Try-On using Image Translation
</h1>

<div align="center">
<a target="_blank" href="https://arxiv.org/pdf/2310.04558.pdf"><b>Paper</b></a>&nbsp&nbsp |
<a href="https://github.com/shuntos/VITON-IT/">&nbsp&nbsp<b>Github</b></a>&nbsp&nbsp |
<a target="_blank" href="https://www.youtube.com/watch?v=sYdoLNQOzsk">&nbsp&nbsp<b>Video</b></a>
</div>

<p align="center">
<br/>
<p>
This paper introduces VTON-IT, a novel Virtual Try-On application that uses semantic segmentation and a generative adversarial network to produce high-resolution, natural-looking images of clothes overlaid onto segmented body regions, addressing the challenges of body size, pose, and occlusions.
</p>

<p>
Below is the overview of proposed VTON-IT. First, the human body is detected and cropped. Then, the desired body region is segmented through U2-Net architecture and the segmented mask is fed to the image translation network to generate wrapped cloth. Finally, the wrapped cloth is overlayed over the input image.
</p>

<div align="center">
<a href="https://github.com/shuntos/VITON-IT/"><b>Project Page</b></a> |
<a href="https://arxiv.org/pdf/2310.04558.pdf"><b>Paper</b></a> |
<a href="https://www.youtube.com/watch?v=sYdoLNQOzsk"><b>Video</b></a>
</div>

<p align="center">
<img src="assets/overview.png" width=100%>
</p>

## Result

For evaluating the performance of VTON-IT through visual observation, we compared the final overlayed images with the output of CP-VTON+. Image below shows that the proposed virtual try-on application produces more realistic and convincing results in terms of texture transfer quality and pose preservation.

<div align="center">
<img src=final_overlay.jpg width="800">
</div>
<p align="center">
<img src="assets/comparision.png" width=100%>
</p>

## Requirements

Expand All @@ -30,29 +38,31 @@
- opencv

## Training Pix2pix:

```
python3 train.py --label_nc 0 --no_instance --name vd2.0_2 --dataroot ./datasets/vd2.0_2 --continue_train --gpu_ids 0,1 --batchSize 2
python3 train.py --label_nc 0 --no_instance --name vd2.0_2 --dataroot ./datasets/vd2.0_2 --continue_train --gpu_ids 0,1 --batchSize 2
```

## Train Segmentation model

```
u2net_train.py

```


## Inference

```
Inference.py
```

## Reference

If you find this repo helpful, please consider citing:
If you find our paper and code useful in your research, please consider giving a star ⭐ and citation 📝 :)

```
@misc{adhikari2023vtonit,
title={VTON-IT: Virtual Try-On using Image Translation},
title={VTON-IT: Virtual Try-On using Image Translation},
author={Santosh Adhikari and Bishnu Bhusal and Prashant Ghimire and Anil Shrestha},
year={2023},
eprint={2310.04558},
Expand All @@ -62,10 +72,5 @@ If you find this repo helpful, please consider citing:
```

## Acknowledgements
The authors would like to thank IKebana Solutions LLC for providing them with constant support for this research project.






The authors would like to thank IKebana Solutions LLC for providing them with constant support for this research project.
Binary file added assets/comparision.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
Binary file added assets/overview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.