NTT Evaulation #962
Open
NTT Evaulation #962
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
sequencer
commented
Jan 26, 2025
sequencer
commented
- [eval] add build
- [eval] add eval.ntt*
Co-Authored-By: Lucas-Wye <[email protected]>
|
@Avimitin Please add CI to run all evaluation cases under |
sequencer
commented
Jan 26, 2025
There was a problem hiding this comment.
I think we should do some thing like this:
- For
ntt_*_main.cit should detect theVLENat runtime and delegate to different algorithm. - For
static const int arr[xxx]and its layers, a script should be added to generate those these vector(maybe fromref.ws?) rather than hardcode. - Neither do I like
gen_data.py, it should also be rewritten intowsfor simplicity. - How does
twiddleget calculated? Fromgen_vector_ntt_tw.py? I think this python should be rewrite into ws as a function, and codegen corresponding headers.
Please clarify the build stage flows:
- Put all user inputs(maybe
arr?) into acsvorjsonfile. - At the compile time, a
wsshould read thecsvand generatedataandtwiddle, the output should be store asjsonandcsv - A codegen python/Scala should read output from
wsand generate a corresponding C header file. - The test entry should include the C header file and execute to get the result.
Some random thing I can think about:
- How do software programmer toggle performance counters though C-API?
- What's the SW engineer API for benchmark? I think we may need a generic post-evaluation flow for the benchmark.
Comment on lines
+5
to
+11
| 9997, 6362, 7134, 11711, 5849, 9491, 5972, 4164, 5894, 11069, 7697, | ||
| 8319, 2077, 12086, 10239, 5394, 4898, 1370, 1205, 2997, 5274, | ||
| 4625, 11983, 1789, 3645, 7666, 12128, 10883, 7376, 8883, 2321, | ||
| 1889, 2026, 8059, 2741, 865, 1785, 9955, 2395, 9330, 11465, 7383, | ||
| 9649, 11285, 3647, 578, 1158, 9936, 12019, 11114, 7894, 4832, | ||
| 10148, 10363, 11388, 9122, 10758, 2642, 4171, 10586, 1194, 5280, | ||
| 3055, 9220 |
There was a problem hiding this comment.
Q: How to choose these numbers? Is this configurable?
tests/eval/_ntt/ntt_512_main.c
Outdated
Comment on lines
97
to
104
| // for (int i = 0; i < n; i++) { | ||
| // printf("%d", dst[i]); | ||
| // if ((i + 1) % 8 == 0) { | ||
| // printf("\n"); | ||
| // } else { | ||
| // printf(" "); | ||
| // } | ||
| // } |
There was a problem hiding this comment.
Don't include comment here, you can add #IFDEF DEBUG macro for these codes.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.