#16871: Unify conditional typecasting in BinaryNg #17148
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mcw-anasuya
force-pushed
the
anasuya/unify_typecast_in_binary_ng
branch
from
5b8870c to
19cb5d8
Compare
KalaivaniMCW
force-pushed
the
anasuya/unify_typecast_in_binary_ng
branch
2 times, most recently
from
f31e04e to
01adab6
Compare
Aswinmcw
reviewed
Jan 28, 2025
umadevimcw
approved these changes
Jan 28, 2025
mcw-anasuya
force-pushed
the
anasuya/unify_typecast_in_binary_ng
branch
2 times, most recently
from
2e6a288 to
5de8ec1
Compare
KalaivaniMCW
force-pushed
the
anasuya/unify_typecast_in_binary_ng
branch
2 times, most recently
from
373803a to
51535a1
Compare
mcw-anasuya
force-pushed
the
anasuya/unify_typecast_in_binary_ng
branch
from
51535a1 to
271df47
Compare
mcw-anasuya
requested review from
Aswinmcw,
VirdhatchaniKN,
KalaivaniMCW and
mouliraj-mcw
KalaivaniMCW
approved these changes
Jan 29, 2025
mcw-anasuya
force-pushed
the
anasuya/unify_typecast_in_binary_ng
branch
from
271df47 to
7f95220
Compare
mcw-anasuya
requested review from
patrickroberts and
yan-zaretskiy
as code owners
VirdhatchaniKN
approved these changes
Jan 30, 2025
mcw-anasuya
force-pushed
the
anasuya/unify_typecast_in_binary_ng
branch
2 times, most recently
from
6039cf2 to
4675ee3
Compare
patrickroberts
requested changes
Jan 30, 2025
patrickroberts
approved these changes
Jan 30, 2025
mcw-anasuya
force-pushed
the
anasuya/unify_typecast_in_binary_ng
branch
2 times, most recently
from
37a278c to
51386b9
Compare
mcw-anasuya
force-pushed
the
anasuya/unify_typecast_in_binary_ng
branch
from
51386b9 to
50a3cb2
Compare
nikileshx
pushed a commit
to nikileshx/tt-metal
that referenced
this pull request
Feb 3, 2025
…rent#17148) ### Ticket tenstorrent#16871 ### Problem description Move conditional logic that exists in inplace binary_ng to binary_ng and delegate inplace binary_ng to binary_ng struct. ### What's changed Unified the logic for binary_ng and inplace binary_ng. ### Explanation bfloat8_b and bfloat4_b tensors need to be typecasted to bfloat16 for binary_ng. In this unified logic, inplace operation is delegated to BinaryNg by passing `input_tensor_a` as the `optional_output_tensor`. This combined logic takes care of the following: **1. Operation with no optional output provided:** Eg: output_tensor = ttnn.experimental.sub(input_tensor_a, input_tensor_b). In this case, if none of the inputs require typecasting, the program will return the result. The resulting tensor will have the dtype of `input_tensor_a`. If `input_tensor_a` alone or both input tensors require typecasting, we typecast the output_tensor back to the original dtype and return result. <img width="344" alt="Screenshot 2025-01-28 at 19 37 57" src="https://github.com/user-attachments/assets/de301b9f-e8fc-435f-8032-223297ac9d12" /> **2. Operation with optional output tensor provided:** Eg: ttnn.experimental.sub(input_tensor_a, input_tensor_b, output_tensor = out_tt) where out_tt is a tensor of provided output shape. In this case, if none of the inputs require typecasting, the program will return the result. The resulting tensor will have the dtype of `out_tt`. If `out_tt` is of dtype bfloat8_b or bfloat4_b, we typecast it to bfloat16 before passing it to the invoke function. We then typecast the result back to the original dtype and copy it back to `out_tt`, which is then returned. <img width="338" alt="Screenshot 2025-01-29 at 17 57 38" src="https://github.com/user-attachments/assets/4e4ddda5-e0ea-48f5-8e91-73154efcf4b6" /> **3. Inplace operation:** Eg: ttnn.experimental.sub_(input_tensor_a, input_tensor_b). In this case, the optional output tensor is `input_tensor_a`. if none of the inputs require typecasting, the program will return the result with the dtype of `input_tensor_a`. If `input_tensor_a` alone or both input tensors require typecasting, we typecast result from bfloat16 back to to `input_tensor_a`'s dtype. ### Checklist - [ ] Post commit CI passes https://github.com/tenstorrent/tt-metal/actions/runs/13072107561 - [x] Blackhole Post commit tests https://github.com/tenstorrent/tt-metal/actions/runs/13072118352 - [ ] (Single-card) Tests for new models https://github.com/tenstorrent/tt-metal/actions/runs/13072147663 - [x] New/Existing tests provide coverage for changes --------- Co-authored-by: KalaivaniMCW <[email protected]>
hschoi4448
pushed a commit
that referenced
this pull request
Feb 20, 2025
### Ticket #16871 ### Problem description Move conditional logic that exists in inplace binary_ng to binary_ng and delegate inplace binary_ng to binary_ng struct. ### What's changed Unified the logic for binary_ng and inplace binary_ng. ### Explanation bfloat8_b and bfloat4_b tensors need to be typecasted to bfloat16 for binary_ng. In this unified logic, inplace operation is delegated to BinaryNg by passing `input_tensor_a` as the `optional_output_tensor`. This combined logic takes care of the following: **1. Operation with no optional output provided:** Eg: output_tensor = ttnn.experimental.sub(input_tensor_a, input_tensor_b). In this case, if none of the inputs require typecasting, the program will return the result. The resulting tensor will have the dtype of `input_tensor_a`. If `input_tensor_a` alone or both input tensors require typecasting, we typecast the output_tensor back to the original dtype and return result. <img width="344" alt="Screenshot 2025-01-28 at 19 37 57" src="https://github.com/user-attachments/assets/de301b9f-e8fc-435f-8032-223297ac9d12" /> **2. Operation with optional output tensor provided:** Eg: ttnn.experimental.sub(input_tensor_a, input_tensor_b, output_tensor = out_tt) where out_tt is a tensor of provided output shape. In this case, if none of the inputs require typecasting, the program will return the result. The resulting tensor will have the dtype of `out_tt`. If `out_tt` is of dtype bfloat8_b or bfloat4_b, we typecast it to bfloat16 before passing it to the invoke function. We then typecast the result back to the original dtype and copy it back to `out_tt`, which is then returned. <img width="338" alt="Screenshot 2025-01-29 at 17 57 38" src="https://github.com/user-attachments/assets/4e4ddda5-e0ea-48f5-8e91-73154efcf4b6" /> **3. Inplace operation:** Eg: ttnn.experimental.sub_(input_tensor_a, input_tensor_b). In this case, the optional output tensor is `input_tensor_a`. if none of the inputs require typecasting, the program will return the result with the dtype of `input_tensor_a`. If `input_tensor_a` alone or both input tensors require typecasting, we typecast result from bfloat16 back to to `input_tensor_a`'s dtype. ### Checklist - [ ] Post commit CI passes https://github.com/tenstorrent/tt-metal/actions/runs/13072107561 - [x] Blackhole Post commit tests https://github.com/tenstorrent/tt-metal/actions/runs/13072118352 - [ ] (Single-card) Tests for new models https://github.com/tenstorrent/tt-metal/actions/runs/13072147663 - [x] New/Existing tests provide coverage for changes --------- Co-authored-by: KalaivaniMCW <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Ticket
#16871
Problem description
Move conditional logic that exists in inplace binary_ng to binary_ng and delegate inplace binary_ng to binary_ng struct.
What's changed
Unified the logic for binary_ng and inplace binary_ng.
Explanation
bfloat8_b and bfloat4_b tensors need to be typecasted to bfloat16 for binary_ng.
In this unified logic, inplace operation is delegated to BinaryNg by passing
input_tensor_aas theoptional_output_tensor. This combined logic takes care of the following:1. Operation with no optional output provided:
Eg: output_tensor = ttnn.experimental.sub(input_tensor_a, input_tensor_b). In this case, if none of the inputs require typecasting, the program will return the result. The resulting tensor will have the dtype of
input_tensor_a. Ifinput_tensor_aalone or both input tensors require typecasting, we typecast the output_tensor back to the original dtype and return result.2. Operation with optional output tensor provided:
Eg: ttnn.experimental.sub(input_tensor_a, input_tensor_b, output_tensor = out_tt) where out_tt is a tensor of provided output shape. In this case, if none of the inputs require typecasting, the program will return the result. The resulting tensor will have the dtype of
out_tt. Ifout_ttis of dtype bfloat8_b or bfloat4_b, we typecast it to bfloat16 before passing it to the invoke function. We then typecast the result back to the original dtype and copy it back toout_tt, which is then returned.3. Inplace operation:
Eg: ttnn.experimental.sub_(input_tensor_a, input_tensor_b). In this case, the optional output tensor is
input_tensor_a. if none of the inputs require typecasting, the program will return the result with the dtype ofinput_tensor_a. Ifinput_tensor_aalone or both input tensors require typecasting, we typecast result from bfloat16 back to toinput_tensor_a's dtype.Checklist