Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[onert] Reallocated tensor when size becomes larger #13658

Merged
merged 1 commit into from
Aug 14, 2024

Conversation

hseok-oh
Copy link
Contributor

This commit updates dynamic allocator usage on basic tensor to reallocate when shape is changed to larger size.
It will improve performance and reduce memory usage when tensor size is lesser by dynamic shape on inference than prepare phase.

ONE-DCO-1.0-Signed-off-by: Hyeongseok Oh [email protected]

This commit updates dynamic allocator usage on basic tensor to reallocate when shape is changed to larger size.
It will improve performance and reduce memory usage when tensor size is lesser by dynamic shape on inference than prepare phase.

ONE-DCO-1.0-Signed-off-by: Hyeongseok Oh <[email protected]>
@hseok-oh hseok-oh added the PR/ready for review It is ready to review. Please review it. label Aug 13, 2024
@hseok-oh hseok-oh requested a review from a team August 13, 2024 04:53
@hseok-oh
Copy link
Contributor Author

Before this PR with #13654

$ ./Product/out/bin/onert_run -r 1 --shape_prepare [0,[4,299,299,3]] --shape_run [0,[3,299,299,3]] -m /home/nfs/model/inception_v3/inception_v3.tflite
Model Filename /home/nfs/model/inception_v3/inception_v3.tflite
===================================
MODEL_LOAD   takes 38.136 ms
PREPARE      takes 64.846 ms
EXECUTE      takes 127.666 ms
- MEAN     :  127.666 ms
- MAX      :  127.666 ms
- MIN      :  127.666 ms
- GEOMEAN  :  127.666 ms
===================================
RSS
- MODEL_LOAD   takes 108516 kb
- PREPARE      takes 143860 kb
- EXECUTE      takes 177556 kb
- PEAK         takes 177556 kb
===================================
HWM
- MODEL_LOAD   takes 110244 kb
- PREPARE      takes 145764 kb
- EXECUTE      takes 177712 kb
- PEAK         takes 177712 kb
===================================
PSS
- MODEL_LOAD   takes 195033 kb
- PREPARE      takes 274669 kb
- EXECUTE      takes 345730 kb
- PEAK         takes 345730 kb
===================================
Used Peak Memory : 177384 kb
- HWM after run  : 186024 kb
- HWM before init: 8640 kb
===================================

After this PR with #13654

$ ./Product/out/bin/onert_run -r 1 --shape_prepare [0,[4,299,299,3]] --shape_run [0,[3,299,299,3]] -m /home/nfs/model/inception_v3/inception_v3.tflite
Model Filename /home/nfs/model/inception_v3/inception_v3.tflite
===================================
MODEL_LOAD   takes 37.572 ms
PREPARE      takes 65.058 ms
EXECUTE      takes 107.753 ms
- MEAN     :  107.753 ms
- MAX      :  107.753 ms
- MIN      :  107.753 ms
- GEOMEAN  :  107.753 ms
===================================
RSS
- MODEL_LOAD   takes 105168 kb
- PREPARE      takes 146448 kb
- EXECUTE      takes 156376 kb
- PEAK         takes 156376 kb
===================================
HWM
- MODEL_LOAD   takes 110352 kb
- PREPARE      takes 146448 kb
- EXECUTE      takes 157236 kb
- PEAK         takes 157236 kb
===================================
PSS
- MODEL_LOAD   takes 194697 kb
- PREPARE      takes 278701 kb
- EXECUTE      takes 301738 kb
- PEAK         takes 301738 kb
===================================
Used Peak Memory : 153112 kb
- HWM after run  : 161944 kb
- HWM before init: 8832 kb
===================================

127.666 ms -> 107.753 ms
Peak memory: 177384 kb -> 153112 kb

Copy link
Contributor

@jyoungyun jyoungyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jyoungyun jyoungyun requested a review from a team August 13, 2024 10:26
Copy link
Contributor

@ragmani ragmani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@glistening glistening merged commit c31c280 into Samsung:master Aug 14, 2024
9 checks passed
@hseok-oh hseok-oh deleted the alloc_when_size_up branch August 14, 2024 02:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PR/ready for review It is ready to review. Please review it.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants