Encoding Test Results in Return Code

When you invoke vvtest with --encode-exit-status vvtest will only ever exit with a zero return code when all the tests exited with "pass", otherwise it will be nonzero.

The return code of 1 is special and reserved for internal vvtest failures. The other return codes are encoded with each binary digit of the number communicating if tests exit with other labels.

The --encode-exit-status option will encode non-passing and not-run tests into the program exit status. That is, the exit status integer is filled like this

exitstatus  = ( 2^1 if num diff > 0 )
exitstatus |= ( 2^2 if num fail > 0 )
exitstatus |= ( 2^3 if num timeout > 0 )
exitstatus |= ( 2^4 if num notdone > 0 )
exitstatus |= ( 2^5 if num notrun > 0 )

For example, the following return codes mean:

• 00000010 Some tests diffed
• 00000100 Some tests failed
• 00000110 Some tests diffed and some failed
• 00001000 Some tests timed out
• 00001010 Some tests diffed and some timed out
• etc.