Problem with header files and static type of functions

[mahdifani14]

Hello,
It has been a long time that I'm using Orio to annotate CHStone benchmark codes toward my MSc thesis and really grateful for your valuable tool. Meanwhile, I have faced with lots of errors and could figure out how to solve them or what is the source of the issue in Orio but, I still have some doubt and questions though.

• I noticed that Orio does not work when you have the prototype of your functions in header files or at the beginning of your code and you have to comment them before S2S and uncomment after.
• Moreover, Orio has a problem with the static type of function definition and you have to remove this type to make it works.

I wanted to know is there any workaround regards to those issues?

Also, What is your suggestion about the best search algorithm to choose compatible with Orio? Because I have been transferring the code for several times with the same search algorithm while Orio gave me different result in each iteration.

Thanks,

[brnorris03]

Orio is only meant for annotating key loops, it really does not handle functions. Are you trying to annotate portions of code that contain multiple functions? In addition there should not be declarations in the annotated region. I can suggest other ways to do this if you cannot identify a specific loop suitable for Orio annotation. Results in search will be different depending on the search strategy and also because of system noise. Any search strategy (random, exhaustive, simplex, etc) is applicable to any code, with different tradeoff of accuracy and runtime. There is no way to know in advance which is best.

[mahdifani14]

@brnorris03 Thanks professor for replying and your explanation,
Actually, I have already created GitHub repository of my annotated CHStone codes and you can access them by the following link:
https://github.com/mahdifani14/CHStone-Codes-Annotated-By-Orio
The JPEG code is the one that I have a problem to annotate because the loop that I want to annotate is out of the main file and located in another one.
It would be my pleasure if you have time and could check my repository to see my annotated codes and let me know your comments about.
Moreover, regards to identifying the suitable loop, I would be so thankful if you help me and let me know the possible ways.

Thanks,
Sincerely Yours,

[brnorris03]

I will take a look! Sometimes you may provide a special initialization function (just for the autotuning), which is fairly straightforward. I will let you know if that's the case.

[mahdifani14]

@brnorris03 Thank you so much, I look forward to hearing from you.
Also, if you would like, it could be my honor to collaborate with you for completing the Orio tests with the new testbench (CHStone) that I'm working with in the future.

[mahdifani14]

@brnorris03 Dear professor, I have another question, how can I substitute the default GCC compiler with LLVM or CLANG? Because I couldn't find any document or example about this matter and when I'm changing the default compiler to CLANG in the spec file, it returns some compiler errors while in the normal compilation, I don't have those.
Error sample:

{'UF': 1, 'VEC': False, 'CFLAGS': '-O0'}
DEBUG[OptDriver]:__optimizeCodeFrag: code_frag type is NonAnn
DEBUG[OptDriver]:OptDriver::__optimizeCodeFrag line 106
DEBUG[OptDriver]:__optimizeCodeFrag: code_frag type is AnnCodeRegion
DEBUG[OptDriver]:OptDriver line 113: Loop
DEBUG[OptDriver]:OptDriver line 142, detected code annotated for tuning
DEBUG[OptDriver]:__optimizeCodeFrag: code_frag type is NonAnn
DEBUG[OptDriver]:OptDriver::__optimizeCodeFrag line 106
DEBUG[OptDriver]:about to load module.class orio.module.loop.loop.Loop corresponding to annotation Loop
DEBUG[OptDriver]:about to instantiate transformation class: orio.module.loop.loop.Loop
DEBUG[OptDriver]:successfully instantiated transformation class: orio.module.loop.loop.Loop
DEBUG[]:applying array copy
Code stats:
        Number of loops: 	3
        Max loop nest depth: 	2
        Additions: 		9
        Multiplications: 	0
        Divisions: 		0
        Logical: 		0
        Reads: 			36
        Writes: 		9
        Comparisons:		5
        Gotos: 			0
        
digraph CFG {
	ForLoop136 [label="ForLoop136  for (; k<=KEYSIZE-1; ) {...",shape=box];
	ForLoop136 -> 114;
	ForLoop136 -> _JOIN_136;
	ForLoop179 [label="ForLoop179  for (j=0; j<=i-1; j=j+1) {...",shape=box];
	ForLoop179 -> 171;
	ForLoop179 -> _JOIN_179;
	ForLoop185 [label="ForLoop185  for (k=0; k<=KEYSIZE-1; ) {...",shape=box];
	ForLoop185 -> ForLoop136;
	ForLoop185 -> _JOIN_185;
	IfStmt134 [label="IfStmt134  if (i<=N-1) ...",shape=box];
	IfStmt134 -> 133;
	IfStmt134 -> _JOIN_134;
	183 [label="183  i=0;...",shape=box];
	183 -> _JOIN_185;
	_JOIN_136 [label="_JOIN_136None...",shape=box];
	_JOIN_136 -> 145;
	_JOIN_134 [label="_JOIN_134None...",shape=box];
	_JOIN_134 -> _JOIN_136;
	133 [label="133  break;...",shape=box];
	133 -> _JOIN_134;
	114 [label="114  indata[i]=in_key[k];...",shape=box];
	_JOIN_179 [label="_JOIN_179None...",shape=box];
	_JOIN_179 -> 183;
	145 [label="145  BF_cfb64_encrypt(indata,outdata,i,ivec,numAdd,encordec);...",shape=box];
	145 -> ForLoop179;
	171 [label="171  check=check+(outdata[j]==out_key[l]);...",shape=box];
	_JOIN_185 [label="_JOIN_185None...",shape=box];
	_TOP_ [label="_TOP_None...",shape=box];
	_TOP_ -> ForLoop185;

}

DEBUG[OptDriver]:__optimizeCodeFrag: code_frag type is NonAnn
DEBUG[OptDriver]:OptDriver::__optimizeCodeFrag line 106
 compiling timer:
	clang -O0 -c -o timer_cpu.o timer_cpu.c
 building the original code:
	clang  -DORIO_REPS=500 -DORIGINAL -o __orio_perftest_original.exe __orio_perftest1.c timer_cpu.o -lrt
In file included from __orio_perftest1.c:70:
./bf_skey.c:79:1: error: function definition is not allowed here
{
^
./bf_skey.c:96:1: error: function definition is not allowed here
{
^
In file included from __orio_perftest1.c:71:
./bf_cfb64.c:82:19: error: a parameter list without types is only allowed in a function definition
BF_cfb64_encrypt (in, out, length, ivec, num, encrypt)
                  ^
./bf_cfb64.c:82:55: error: expected ';' at end of declaration
BF_cfb64_encrypt (in, out, length, ivec, num, encrypt)
                                                      ^
                                                      ;
In file included from __orio_perftest1.c:72:
./bf_enc.c:82:13: error: a parameter list without types is only allowed in a function definition
BF_encrypt (data, encrypt)
            ^
./bf_enc.c:82:27: error: expected ';' at end of declaration
BF_encrypt (data, encrypt)
                          ^
                          ;
./bf_enc.c:84:10: error: redefinition of 'encrypt'
     int encrypt;
         ^
./bf_cfb64.c:88:10: note: previous definition is here
     int encrypt;
         ^
__orio_perftest1.c:800:1: error: function definition is not allowed here
{
^
__orio_perftest1.c:873:1: error: function definition is not allowed here
{
^
9 errors generated.
ERROR: orio.main.tuner.ptest_driver:  failed to compile the original version of the code: "clang  -DORIO_REPS=500 -DORIGINAL -o __orio_perftest_original.exe __orio_perftest1.c timer_cpu.o -lrt"
  File "/usr/local/bin/orcc", line 27, in <module>
    orio.main.orio_main.start(sys.argv, orio.main.orio_main.C_CPP)
  File "/usr/local/lib/python2.7/dist-packages/orio/main/orio_main.py", line 109, in start
    optimized_code_seq = odriver.optimizeCodeFrags(cfrags, True)
  File "/usr/local/lib/python2.7/dist-packages/orio/main/opt_driver.py", line 45, in optimizeCodeFrags
    cur_seq = self.__optimizeCodeFrag(cf, perf_params)
  File "/usr/local/lib/python2.7/dist-packages/orio/main/opt_driver.py", line 120, in __optimizeCodeFrag
    cfrag.cfrags)
  File "/usr/local/lib/python2.7/dist-packages/orio/main/tuner/tuner.py", line 152, in tune
    best_perf_params, best_perf_cost = search_eng.search()
  File "/usr/local/lib/python2.7/dist-packages/orio/main/tuner/search/search.py", line 119, in search
    best_coord,best_perf,search_time,runs = self.searchBestCoord(startCoord)
  File "/usr/local/lib/python2.7/dist-packages/orio/main/tuner/search/randomsearch/randomsearch.py", line 164, in searchBestCoord
    perf_costs = self.getPerfCosts([coord])
  File "/usr/local/lib/python2.7/dist-packages/orio/main/tuner/search/search.py", line 307, in getPerfCosts
    new_perf_costs = self.ptdriver.run(test_code, perf_params=perf_params,coord=coord_key)
  File "/usr/local/lib/python2.7/dist-packages/orio/main/tuner/ptest_driver.py", line 481, in run
    self.__build(perf_params=perf_params,coord=coord)
  File "/usr/local/lib/python2.7/dist-packages/orio/main/tuner/ptest_driver.py", line 256, in __build
    err('orio.main.tuner.ptest_driver:  failed to compile the original version of the code: "%s"' % cmd)
  File "/usr/local/lib/python2.7/dist-packages/orio/main/util/globals.py", line 319, in err
    traceback.print_stack()

[amirjamez]

I guess the issue is that LLVM's clang as of now doesn't support nested functions and this is the issue that gcc can handle by default, if not invoked by its standard c compiler (i.e. std=c99, etc).
@brnorris03, Is there an easy way to control the way orcc generates its __orio_perftest_original.c to be able to pair it with clang as well?

[brnorris03]

Sorry for the delay, it's a busy week. I think you may want to change the compiler options in the tuning spec, what is your compiler command line now (that you have in the tuning spec)? AFAIK clang can handle c99, e.g., clang -std=c99 should be supported in clang 5.

So something like this in your tuning spec should fix the problem:
def build {
arg build_command = 'clang -std=c99';
}

[amirjamez]

Thanks @brnorris03 for the reply.
Yes. thats true, actually the issue is the opposite. That is Orio's orcc is not generating its perftest generated c files (namely, __orio_perftest.c) in standard c. i.e.: ___orio_perftest1.c_has nested functions that only gcc can handle.
If you activate -std=c99 with clang it will complain, same goes without passing this to clang. The bottomline is that clang doesn't support nested functions as of now and you will face compilation errors as bellow:

building the original code:
	clang -std=c99  -DORIO_REPS=50 -DORIGINAL -o __orio_perftest_original.exe __orio_perftest1.c timer_cpu.o 
__orio_perftest1.c:230:1: error: function definition is not allowed here
{
^
__orio_perftest1.c:241:1: error: function definition is not allowed here
{

I was wondering if there is an easy workaround to fix this issue when we use clang with Orio. Thanks

[brnorris03]

Hm, I am pretty sure we can get rid of function nesting, can you point me to one specific example in your repo in which I can see exactly where this is happening?

[amirjamez]

For instance here, if you change gcc with 'clang' or 'clang -std=c99' or , I guess you should be able to reproduce that error as the first ___orio_perftest.c_is generated.

[brnorris03]

Will try that. I noticed that you used 500 reps -- are you sure you need that many? Typically even for short-running tests, 100 or fewer should be fine.

[amirjamez]

Thanks.
Yes, that was there as an example, should be around 50 I guess :)

[mahdifani14]

@brnorris03 Did you have time to check the problem and test it with clang -std=c99?

[brnorris03]

No, sorry, this is an incredibly busy time of the term, lots of Ph.D. defenses, comprehensive exams and conference organizing on top of usual teaching/research. It will be over in about 2 weeks, when I hope to devote my attention to this!

[mahdifani14]

@brnorris03 No problem, I see that you are fully busy with academic matters and I'm really grateful for your support about our problem. Actually, in our thesis, we are creating an optimization toolchain from high-level code to hardware synthesis in which we are using Orio for S2S level, LLVM compiler, and LegUp which is compatible with LLVM, not GCC. That's why, for consistency in the entire chain, it's crucial for us to have the LLVM in Orio transformation instead of GCC.
So, I'm looking forward to hearing from you, thanks in advance.

[mahdifani14]

@brnorris03 Dear Professor, did you have time to check the issue and fix the problem?

[brnorris03]

I did look at it. I think this is not the best way to organize the code in terms of where the annotations are. Instead of enclosing everything in the tuning spec, the most minimal tunable code should be the only input to Orio, with the rest specified as part of initialization (separately compiled file). I can reorganize one of the examples in this manner.

[mahdifani14]

@brnorris03
First and foremost, Thank you so much for following up the issue and your support,
It would be great and I will be so grateful if you could do so.
I'm waiting for your example to apply the idea to the rest of codes, thanks in advance.

[mahdifani14]

@brnorris03
Dear Professor,
I wish that you had an amazing summer vacation.
As we discussed before, I would like to know that did you have time to test one of our sample codes with LLVM? Or could you help us to configure Orio with LLVM instead of GCC by overcoming the stated issue above?
Unfortunately, we urgently need this feature in order to defend our MSc thesis within 2 weeks.

Thanks a million in advance for your valuable help and look forward to your response