Skip to content

Commit

Permalink
Added README.md for text_to_image_reference_using_torch_loadgen
Browse files Browse the repository at this point in the history
  • Loading branch information
maria-18-git committed Jan 25, 2024
1 parent 096ab28 commit 74ad697
Showing 1 changed file with 43 additions and 0 deletions.
43 changes: 43 additions & 0 deletions text_to_image_reference_using_torch_loadgen/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@

# MLPerf Inference - Text to Image - ifferent backend(pytorch, pytorch-dist, debug)
This implementation runs text to image model with the different backend.

Currently it supports the following models:
- stable-diffusion-xl

## Accuracy Full Run(5000 images):
- SingleStream
Accuracy Command:
```
axs byquery loadgen_output,task=text_to_image,framework=torch,profile=stable-diffusion-xl-pytorch,dtype=fp16,loadgen_scenario=SingleStream,accuracy+
```
Accuracy:
```
axs byquery loadgen_output,task=text_to_image,framework=torch,profile=stable-diffusion-xl-pytorch,dtype=fp16,loadgen_scenario=SingleStream,accuracy+ , get accuracy_dict
{'fid': 23.61146873078252, 'clip': 31.749096275568007}
````
```
axs byquery loadgen_output,task=text_to_image,framework=torch,profile=stable-diffusion-xl-pytorch,dtype=fp16,loadgen_scenario=SingleStream,accuracy+ , get fid
23.61146873078252
``

```
axs byquery loadgen_output,task=text_to_image,framework=torch,profile=stable-diffusion-xl-pytorch,dtype=fp16,loadgen_scenario=SingleStream,accuracy+ , get clip
31.749096275568007
```

## Perfomance Full Run
- SingleStream
Performance Command
```
axs byquery loadgen_output,task=text_to_image,framework=torch,profile=stable-diffusion-xl-pytorch,dtype=fp16,loadgen_scenario=SingleStream,loadgen_target_latency=6,loadgen_mode=PerformanceOnly,sut_name=7920t-kilt-onnxruntime_gpu
```
Performance
```
axs byquery loadgen_output,task=text_to_image,framework=torch,profile=stable-diffusion-xl-pytorch,dtype=fp16,loadgen_scenario=SingleStream,loadgen_target_latency=6,loadgen_mode=PerformanceOnly,sut_name=7920t-kilt-onnxruntime_gpu , get performance
VALID : _Early_stopping_90th_percentile_estimate=4621.172 (milliseconds)
```

0 comments on commit 74ad697

Please sign in to comment.