[CharTensor] Enable memory data to store scale factors based on quantization schemes #2844
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
djeong20
requested review from
myungjoo,
jijoongmoon,
again4you,
jaeyun-jung,
leemgs,
wooksong,
gichan-jang,
anyj0527,
lhs8928,
songgot,
jihochu,
DonghakPark,
SeoHyungjun,
baek2sm,
skykongkong8 and
EunjuYang
as code owners
EunjuYang
reviewed
Dec 27, 2024
nntrainer/tensor/char_tensor.cpp
Outdated
| MemoryData *mem_data = | ||
| new MemoryData((void *)(new int8_t[dim.getDataLen()]())); | ||
| new MemoryData((void *)(new int8_t[dim.getDataLen() + 4 * scale_size()]())); |
There was a problem hiding this comment.
What about updating the code like:
Suggested change
| new MemoryData((void *)(new int8_t[dim.getDataLen() + 4 * scale_size()]())); | |
| new MemoryData((void *)(new int8_t[dim.getDataLen() + sizeof(float) * scale_size()]())); |
If you have a plan to support different type of scales, it would be better to update it.
There was a problem hiding this comment.
thank you for sharing your thoughts! sizeof(float) makes more sense :)
nntrainer/tensor/char_tensor.cpp
Outdated
| return nullptr; | ||
|
|
||
| data->validate(); | ||
| return ((int8_t *)getScale()) + idx; |
There was a problem hiding this comment.
Isn't it ((float *)getScale()) + idx?
There was a problem hiding this comment.
yes, you're right! thank you 👍
djeong20
force-pushed
the
update/char_tensor/qinfo
branch
from
c55db72 to
a5ec4e6
Compare
…ization schemes This pull request aims to modify the existing codebase such that the memory data of CharTensor can now store scale factors based on different quantization schemes. Additionally, this change allows the Tensor class to specify the desired quantization scheme while creating a new CharTensor instance. The scale factors are determined either during the quantization process using a specific quantizer or they can be manually initialized if both the quantized data and the corresponding scale factors are provided as inputs. **Self-evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: Donghyeon Jeong <[email protected]>
djeong20
force-pushed
the
update/char_tensor/qinfo
branch
from
a5ec4e6 to
44fbaf6
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request aims to modify the existing codebase such that the memory data of CharTensor can now store scale factors based on different quantization schemes.
Additionally, this change allows the Tensor class to specify the desired quantization scheme while creating a new CharTensor instance.
The scale factors are determined either during the quantization process using a specific quantizer or they can be manually initialized if both the quantized data and the corresponding scale factors are provided as inputs.
Self-evaluation: