-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathdocumentation.html
268 lines (181 loc) · 17.5 KB
/
documentation.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
<html><head><title>PROGRAM benchark-blender.pl</title>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" >
</head>
<body class='pod'>
<!--
generated by Pod::Simple::HTML v3.28,
using Pod::Simple::PullParser v3.28,
under Perl v5.018002 at Tue Feb 24 18:54:15 2015 GMT.
If you want to change this HTML document, you probably shouldn't do that
by changing it directly. Instead, see about changing the calling options
to Pod::Simple::HTML, and/or subclassing Pod::Simple::HTML,
then reconverting this document from the Pod source.
When in doubt, email the author of Pod::Simple::HTML for advice.
See 'perldoc Pod::Simple::HTML' for more info.
-->
<!-- start doc -->
<a name='___top' class='dummyTopAnchor' ></a>
<h1><a class='u'
name="PROGRAM_benchark-blender.pl"
>PROGRAM benchark-blender.pl</a></h1>
<pre> Blender Automated Benchmark Suite (BABS)
[cc-by] Piotr Arlukowicz, <[email protected]></pre>
<p>Documentation: read at least config section below :)</p>
<h2><a class='u'
name="USAGE"
>USAGE</a></h2>
<pre> benchmark-blender.pl [options] -b blendfile
benchmark-blender.pl -h</pre>
<h2><a class='u'
name="OPTIONS"
>OPTIONS</a></h2>
<p>Options are specified via command line using a typical getopt notation. Each option should start with <code>-</code> and is one letter long. Do not join options with their parameters, always add spaces around. Bad: -i2, good: -i 2.</p>
<pre> -e file - blender executable (default: blender)</pre>
<p>The blender executable file itself. In Linux, this will be <code>blender</code>, while in windows this can be <code>blender.exe</code> or <code>blender.lnk</code>.</p>
<pre> -p path - blender path (default: .)</pre>
<p>Path to the blender. Can be absolute or relative to the place where you are run the script. Usually I keep Blender in quite a long path, and this could be a typical case. The location of: <code>/home/piotao/bin/blender-git/install/linux</code> can be expressed by several shorter terms: <code>%home/%git/%install</code>. Then, if you are using this terms, you have to put them into CONFIG section inside the script, like this:</p>
<pre> %home => '/home/piotao',
%git => 'bin/blender-git',
%install => 'install/linux',</pre>
<p>The script will substitute all config variables from those information. In this way you can make shortcuts and aliases, and run some additional commands along with blender testing.</p>
<pre> -b file - blender file (HAS to be specified)</pre>
<p>Normal BLEND file to be used for testing. This file will be run through blender with options given in another sections of config (or by <code>-c</code> option). Usually this has to be specified, or written into config section.</p>
<pre> -c string - the entire command to run blender with options</pre>
<p>This is the command the script executes. The output of this command has to have a string: <code>Time: (xx:yy.zz)</code>. The script can catch this time and collect all results for each rendered frame and calculate the average time. If you have to run Blender for an animation, and you want to catch times of render per frame, you have to run Blender as a standalone program, and then you have to catch the log. The info about time is in this log. You can then put this log to be parsed by this script with <code>-l</code> option.</p>
<p>The typical command is by default composed from another config parts, for example:</p>
<pre> blenderCommand => '%blenderPath/%blenderExe %options',</pre>
<p>The script is building the right command from this %-something pieces. Each of them has to be specified in config sections, and you can have your own defs and strings defined there too. All tokens with <code>%</code> sign are replaced by their values defined somewhere in another parts of the config, so finally you can get the whole command. If there is a recursion problem, script will catch this and stop. So, if you define <code>%blenderPath</code> and <code>%blenderExe</code> like this:</p>
<pre> blenderPath => '/bin',
blenderExe => 'blender_x86',</pre>
<p>Then, if you have your options like this:</p>
<pre> options => '%options2',
options1 => '-noaudio -noglsl -nojoystick ',
options2 => '%options1 -b %blenderFile %thr -f 1 2>&1 ',</pre>
<p>Then this will give you the whole options like build as:</p>
<pre> options=>'-noaudio -noglsl -nojoystick -b %blenderFile %thr -f1 2>&1 '</pre>
<p>...which will then have the <code>%blenderFile</code> and <code>%thr</code> token replaced by their corresponding values. For blender file you have to specify <code>-b</code> option, and for threads number to use you have to put <code>-t</code> option.</p>
<pre> -a float - maximal change of averaged time</pre>
<p>This is the number, which stands for precision of testing. Each new test which was carried out by running blender is measured in terms of time. This measured time is different (probably) from each other time, so it is averaged, to gain the best approximation. But, how you can tell the average time is good enough? How many runs you really have to do, in order to get the good average time? Please notice, that running blender three, or even ten times, and averaging all times is not enough, if your times are very different from each other. So, this value can force program to run tests ENOUGH times, and after each time the new average is calculated from all gained tests. This new average is compared to the old one, and IF the old one differs no more than this value, then all tests are finished. To understand this, consider this example:</p>
<p>Here, we have just a few times measured during some blender runs. They have diffrent values. How many of such runs is enough to have a good average value which do not change much with next tests? Let's assume we can accept 0.1 change between two consecutive averaged results.</p>
<pre> No Time measurements Average of times so far Diffrence of average
1 10 10 (of course!) (nothing yet)
2 11 10.50 0.50
3 10 10.33 0.16
4 12 10.75 0.42
5 10 10.60 0.15
6 10 10.50 0.1 -> this is the point
7 10 10.43 0.07
8 10 10.38 0.05
9 </pre>
<p>As you can see, during tests, we renderd 8 times, and the results were collected. For each new result, we calculated average value of all measurements and this new average value is put into the third column. Now, we can calculate the diffrence between our new average and the previous one. This value is our <i>epsilon</i> - the change which tells us, how much the average is changed between tests. If this change is small enough, we can stop further testing, because our result will be no better that current one! So, in this example we know, that there is only 6 run sufficient instead of 8, which can save some time. In fact, this procedure is executed as long as the change of averages is smaller than the epsilon. In our case we assumed that we are acceptig 0.1 value, so change smaller that that do not count much. This value is set by default to 0.01 (you have to specify a float number, not percent).</p>
<p>For lower epsilon values the computations can lasts longer and some of them can take a really long time. On my 6-cores cpu, under linux, where only 10-20 processes were run in parallel (even cron was killed tho), this procedure has taken 47 times, until the level of 0.01 was reached. For the same computer, in the save environment, the GPU has to use only 8-10 iterations.</p>
<pre> -i int - additional runs after time is stable</pre>
<p>This is the number which tells how many additional runs the script should run AFTER the epsilon limit is reached. Look at the above example: we got the demanded minimum value in 6 iteration. Instead of stopping the calculation process, there were two additional runs performed to ensure the stability of average value. It seems that this is necessary, because there are some random fluctuations and some tests can raise the average above the limit. If that is the case, the value of repetitions is reset, and after gaining the minimum again, this number of additional runs still has to be performed.</p>
<pre> -t int - number of threads blender use</pre>
<p>This option is used to specify how many threads should blender use in order to render on CPU. For GPU this value is ignored. If this number is set to 0, which is the default, then all threads are used in auto-detect mode and blender will decide itself. If a value is given here, then this option appears in command line and specifies the desired number of threads to use, by applying -t NUM option to blender command line.</p>
<pre> -h, --help - prints this help and exits.</pre>
<p>This is simple: it displays a short help about the program.</p>
<pre> -v - verbose output and some debug</pre>
<p>This option helped me to design this program and debug it. You do not have to use it, and I leaved it here just for those, who are curious :)</p>
<pre> -l txtfile - blender log file to analyze only</pre>
<p>If you do not want to run blender with any blender file, but you have a log output from some runs, you can just analyze that log file with no blender at all. To generate results in the same format like those generated during a real tests, specify just a single <code>-l logname</code> option, where <code>logname</code> is just a plan text file. This text file should be the complete and exact log file right from blender. It can contain multiple frames renderings, etc. but only time and last read blend file are catched. Only one log file is processed at a time and it has a precedence over blend file and blender run.</p>
<h2><a class='u'
name="CONFIG_SECTION"
>CONFIG SECTION</a></h2>
<p>In the config section you can define all configuration you need to run this script without any parameters. However, configuration have it's own limitations so do not expect miracles :) This is perl, and I made this script flexible enough to suit well my needs. So, here it is.</p>
<p>All config definition is build as KEY => VALUE. KEY has to be unique string. VALUE can be a number, a string, etc. It can be even another structure, but then you have to know what you are doing (learn Perl! - this is very good language). So, if you define something like:</p>
<pre> script => 'perl',</pre>
<p>notice that what is on the right side HAS to be in apostrophes. The left side is automatically cited, so no apostrophes are needed. However, if you like this:</p>
<pre> big idea => 'my mind',</pre>
<p>This will not work, because KEY has to be a single string, so you have to cite it this way:</p>
<pre> 'big idea' => 'my mind',</pre>
<p>This will not work either, because keys with spaces won't be parsed correctly. When you have some such keys and values defined, you can use them to define new values. Note however that you can't use them as mixed key names. So this works:</p>
<pre> newidea => 'this is %script',</pre>
<p>but this doesn't work:</p>
<pre> %script => 'something',</pre>
<p>Note that newidea key will be translated to 'this is something' because of substitutions. So you can make some even more, nested substitutions, and they are handy if you have to keep diffrent configurations inside one script. Look at this:</p>
<pre> blenderCommand => '%setup1',
setup1 => '.....',
setup2 => '.....',</pre>
<p>In this case one can change only one option at the start of the script, and all is changed respectively. One can then specify new configuration option via the command line, putting <code>-c '%setup2'</code> parameter, and this will use a whole new, and different definition from the script config. I think this is so handy, that should suffice my needs and maybe even yours.</p>
<p>Remember that some of keys in the config file are reseved and they are used in the code directly (however this may change in the Future). There are:</p>
<pre> blenderCommand
maxAverageChange
maxRunsToEnsure
verbose
blenderFile
version</pre>
<p>Also, the script will create some new config keys during its work, so they are also kind-of reserved, because even if you defined them, they can be overwritten. They are:</p>
<pre> framecount
- this is the numer of tests performed so far
start
- time in seconds from EPOCH when script just started
end
- time in seconds when script is about finish
totalsum
- sum of all time measurements
average
- current average value
frames
- a list of all collected time measurements (and more)</pre>
<p>All other keys are not mandatory and you can change them, use them, etc.</p>
<h3><a class='u'
name="Recursion"
>Recursion</a></h3>
<p>One can mistakenly made a recursive definitions in config, for example:</p>
<pre> one => '%two',
two => '%one',</pre>
<p>This naturally can lead to infinite recursion, but I thought a bit and coded simple thing to prevent this. Script will, however, fail when such a recursive definition is found. This applays also to cases where you can build token with recursive calls which at some point can lead to situation where token should be explained by itself. I'm not sure if my security code is enough, so better try not to build recursive definitions.</p>
<h3><a class='u'
name="Output_of_the_blender_command"
>Output of the blender command</a></h3>
<p>This script works with the assumption that blender will print a log of it's rendering process to the standard output. Script is then reading this output using normal redirection pipe. If you change this behaviour, the results can be unpredictable. So, you have to assure that <code>blenderCommand</code> will be executing a command (or commands) which are able to print to the standard output. The text which is printed doesn't really matter - in the whole stream of log there is only one string searched. This is Time: xx:yy.yy value. Blender prints this time for both internal and Cycles renderers, if you run it with console output. If you plan to include some intermediate scripts between this script and blender, you should ensure that at least this line with the text Time ... is properly outputted to the output stream.</p>
<p>Also, to avoid messy output, the <code>blenderCommand</code> is defined with a stderr redirection to stdout. Thanks to this you don't have to see any warnings or messy error messages printed by Blender itself. This of course hides some info so you should test your blend files and render to see if everything works.</p>
<h2><a class='u'
name="DEFAULT_CONFIG"
>DEFAULT CONFIG</a></h2>
<p>My default config section is here (in case you've lost it):</p>
<pre> maxAverageChange => '0.01',
blenderExe => 'blender',
blenderPath => '%bin/%git/%install',
blenderFile => 'any.blend',
blenderCommand => '%blenderPath/%blenderExe %options2',
options1 => '-noaudio -noglsl -nojoystick ',
options2 => '%options1 -b %blenderFile %thr -f 1 2>&1 ',
threads => 0,
verbose => 0,
version => '0.1',
start => time,
home => '/home/piotao',
bin => '%home/bin',
git => 'blender-git',
install => 'install/linux',</pre>
<h2><a class='u'
name="DEFAULT_OPTIONS"
>DEFAULT OPTIONS</a></h2>
<p>If you would like to enter command line to redefine all config sections, here is the complete set of options:</p>
<pre> -e 'blender' # or 'blender.exe'
-p '/path/to/blender' # or 'c:\path\under\windoze'
-c '%blenderPath/%blenderExe -b %blenderFile %thr -f 1 '
-b blendfile.blen # HAS TO BE!
-a 0.01
-i 3
-t 0</pre>
<h2><a class='u'
name="AUTHOR"
>AUTHOR</a></h2>
<pre> Piotr Arlukowicz,
Blender Foundation Certified Trainer
<[email protected]>
Twitter: /piotao
Blender Network: /piotr-arlukowicz</pre>
<h2><a class='u'
name="BUGS"
>BUGS</a></h2>
<p>If there are some bugs in this code, then sorry. Nobody is perfect :) However, if you are unable to repair this script by yourself, you can try to bother me using my email. I do not promise to help, but you can always try.</p>
<h2><a class='u'
name="DISCLAIMER"
>DISCLAIMER</a></h2>
<p>Hereby, I do not have any responsibility and will not take it if you were pissed off, harmed, killed or injured in any way using this script. Remember, this script is a one-day project made just for fun, and for testing blender on a new computer, so it is not intended to be bulletproof. It is not a rocket-science either. So, try to not shoot yourself in the foot, please :)</p>
<!-- end doc -->
</body></html>