[ Tensor ] Apply SIMD in matrix transpose fp32 @open sesame 12/18 10:16 #2832
Conversation
- Implement NEON SIMD kernel for matrix transpose for fp32 datatype - Connect such kernel with current function template - This expands SIMD coverage for matrix transpose datatype : fp16 only -> fp16, fp32 **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <[email protected]>
- With NEON support (arm), apply matrix transpose with SIMD add BLAS level. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <[email protected]>
…ber function - For fp32, channel-first, 0:2:1 transpose case, use transpose from blas interface instead of current loop implementation. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <[email protected]>
- Verify transpose function with A.T.T = A **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <[email protected]>
skykongkong8
requested review from
myungjoo,
jijoongmoon,
again4you,
jaeyun-jung,
leemgs,
wooksong,
gichan-jang,
anyj0527,
lhs8928,
songgot,
jihochu,
DonghakPark,
SeoHyungjun,
baek2sm,
djeong20 and
EunjuYang
as code owners
skykongkong8
changed the title
skykongkong8
changed the title
| #else | ||
| transpose_fallback<float>(M, N, src, ld_src, dst, ld_dst); | ||
| #endif | ||
| } |
There was a problem hiding this comment.
Someday, all of such #ifdef (NEON, FP16, ...) need to be migrated to header files (preferably centralized into a single header) and code lines depending on such need to be separated as a class or a file (e.g., defined in blas_neon.cpp and whether to use functions/sub-classes in that file is determined at a header and build-script/option.
But for today, lets move on.
There was a problem hiding this comment.
Seems this issue is related to #2549
I am aware of it. Think we urgently have to discuss when to apply this
There was a problem hiding this comment.
In fact, one of the biggest problems of applying #2549 is that all android make file / tizen spec file in the current nntr is including blas_interface.h since this pr substitutes blas_interface into single cpu_backend.h, this affects quite a lot of files...
Nice approach! Gotta try that one too |
This PR proposes neon simd kernel in fp32 matirix transpose.
Table below is conducted with Galaxy S24U, TC = 100.
Note that this kernel is more effective for sufficiently big matrices, but still better than before anyway.
I added additional unittest TC for validating this function with simple idea : (A.T.T = A) but please suggest any idea if there's more efficient way.
This will instantly impact fp32 BCQ Tensor usage
Self evaluation: