Introduce pedantic mode for tester
Motivation
----------
External tools need to know whether or not any test in suite failed. For
example buildbot, or packaging tool would like to stop the execution and
mark it as failed if some tests does not pass.
Modification
------------
Add `--pedantic` switch to tester program which will force it to quit
with non-zero exit code if at least one failure detected. Also update
`tests()` proc in koch to inspect result from tester and propagate it to
user.
Result
------
Nothing has changed in default behaviour. But following invocations will
exit with non-zero code if there failed tests:
./koch tests --pedantic all
./tests/testament/tester --pedantic all
This commit is contained in:
parent
71561bef58
commit
411e602d13
2 changed files with 9 additions and 2 deletions
|
|
@ -31,6 +31,7 @@ Arguments:
|
|||
Options:
|
||||
--print also print results to the console
|
||||
--failing only show failing/ignored tests
|
||||
--pedantic return non-zero status code if there are failures
|
||||
""" % resultsFile
|
||||
|
||||
type
|
||||
|
|
@ -310,12 +311,14 @@ proc main() =
|
|||
backend.open()
|
||||
var optPrintResults = false
|
||||
var optFailing = false
|
||||
var optPedantic = false
|
||||
var p = initOptParser()
|
||||
p.next()
|
||||
while p.kind == cmdLongoption:
|
||||
case p.key.string.normalize
|
||||
of "print", "verbose": optPrintResults = true
|
||||
of "failing": optFailing = true
|
||||
of "pedantic": optPedantic = true
|
||||
else: quit Usage
|
||||
p.next()
|
||||
if p.kind != cmdArgument: quit Usage
|
||||
|
|
@ -348,8 +351,10 @@ proc main() =
|
|||
if action == "html": openDefaultBrowser(resultsFile)
|
||||
else: echo r, r.data
|
||||
backend.close()
|
||||
if optPedantic:
|
||||
var failed = r.total - r.passed - r.skipped
|
||||
if failed > 0 : quit(QuitFailure)
|
||||
|
||||
if paramCount() == 0:
|
||||
quit Usage
|
||||
main()
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue