[CK_TILE] Add Various Fusion Functions to RMSNorm #1802
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ruanjm
requested review from
junliume,
illsilin,
carlushuang,
qianfengz,
aosewski,
poyenc,
geyyer,
bartekxk,
andriy-ca and
afagaj
as code owners
ruanjm
force-pushed
the
amd/dev/jruan/tile_rmsnorm_fusion
branch
2 times, most recently
from
1714df2 to
c91b9b7
Compare
rocking5566
reviewed
Jan 9, 2025
rocking5566
reviewed
Jan 9, 2025
rocking5566
reviewed
Jan 9, 2025
ruanjm
force-pushed
the
amd/dev/jruan/tile_rmsnorm_fusion
branch
from
03a4be5 to
b14a991
Compare
rocking5566
reviewed
Jan 13, 2025
…rmsnorm2d_fwd_api.cpp and rmsnorm2d_fwd_instance_common.hpp
remove dbg code
Bug fix after rename Remove files
ruanjm
force-pushed
the
amd/dev/jruan/tile_rmsnorm_fusion
branch
from
a376459 to
180e739
Compare
rocking5566
reviewed
Jan 14, 2025
include/ck_tile/ops/rmsnorm2d/pipeline/rmsnorm2d_fwd_pipeline_two_pass.hpp
Outdated
Show resolved
Hide resolved
|
passed in ater unit test |
carlushuang
approved these changes
Jan 15, 2025
kylasa
pushed a commit
to kylasa/composable_kernel
that referenced
this pull request
Jan 19, 2025
* Add shortcut to RMSNorm * Modify test for adding shortcut for RMSNorm * Add fused parameter into tests * 1. Add YDataType. 2. rmsnorm2d_fwd_traits_ from rmsnorm2d_fwd.hpp to rmsnorm2d_fwd_api.cpp and rmsnorm2d_fwd_instance_common.hpp * 1. Supports various stride and percisions. * Add support of Epilogue * Add fuse and epilogue support to rmsnorm ref * Modify rmsnorm example * Refactor tests/examples * Bug fix for newly added tests/examples * Bug fix for new tests 2 * Modify smoke test scripts remove dbg code * Supports non-smooth dyanmic quant * Update Rmsnorm2dFwd::GetName() * rename xscale and prec_sx to smoothscale and prec_sm Bug fix after rename Remove files * change example_rmsnorm2d_fwd.cpp * update performance calculator * Fix issue in two-pass when fuse add is enabled * Remove comment of beta --------- Co-authored-by: rocking <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added following functions to RMSNorm. In summary, the interface and function of RMSNorm is very similar to LayerNorm now:
XScaletoSmoothScalein quant epilogue. This change affects layernorm, smoothquant and moe smoothquant as well.