Nim/tests
Euan 7828199827
#12103 - CI for OpenBSD (#12105)
* Working on OpenBSD CI
* Condense steps into 2 steps to make output easier to follow.
* Move up one directory after csources build.
* Remove FreeBSD build manifest and add OpenBSD test ignores for coroutines and hot code reloading.
* If runCI fails, run the test results script.
* Add email trigger for build failure
* Remove .git from repository URL
* Disable SFML test on OpenBSD
* Disable tgetaddrinfo on OpenBSD as only UDP and TCP protocols are supported.
* Remove getFilePermissions as it causes CI test failures with NimScript.
* Set clang as cc in nim.cfg and use gmake to build csources.
* Add getCurrentDir to nimscript.
* Remove duplicate getCurrentDir and check for not weirdTarget.
* Add CI badge for OpenBSD.
* Disable tests which allocate lots of memory for OpenBSD.
* Use `CORO_BACKEND_SETJMP` on OpenBSD rather than ucontext.
* Simplify building of koch
* Disable t8657 on OpenBSD. See issue #13760.
* Fix #12142 - tarray_of_channels fails on OpenBSD
* Disable thhtpclient_ssl and tosprocterminate on OpenBSD. These tests can be enabled at a later date after fixing them.
* Install libffi.
* Set path to libc for openbsd.
* Disable tevalffi for now.
* Remove tevalffi.nim.
* Use ncpuonline sysctl rather than ncpu.
* Disable tacceptcloserace and tasynchttpserver on OpenBSD.
* Enable tacceptcloserace and tasynchttpserver.
* Fix #13775 as suggested by @alaviss - use /bin/cat on OpenBSD rather than /bin/sh.
* Enable test on OpenBSD.
* Disable tflowvar on OpenBSD.
2020-04-21 15:05:21 +02:00
..
actiontable Merge tests into a larger file (part 1 of ∞) (#9318) 2018-10-12 17:02:46 +02:00
alias Remove expr/stmt (#5857) 2017-07-25 09:28:23 +02:00
align fixes #13122 (#13126) 2020-01-13 14:11:51 +01:00
ambsym require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
arc fixes #14003 (#14006) [backport:1.2] 2020-04-17 13:23:04 +02:00
arithm fix operators containing percent for VM usage (#13536) 2020-03-11 01:01:25 +01:00
array guards.nim:sameTree(): handle uint literals correctly (#12483) [backport] 2019-10-22 18:18:33 +02:00
assert stacktraces can now show custom runtime msgs per frame (#13351) 2020-03-30 13:45:32 +02:00
assign fix rtti sizeof for varargs in global scope (#13125) [backport] 2020-01-13 10:17:21 +01:00
astoverload
astspec Fix spellings (#12277) [backport] 2019-09-27 07:02:54 +02:00
async Fix #13889 with testcase (#13896) [backport] 2020-04-06 16:25:24 +02:00
bind Remove deprecated modules (asyncio, sockets, ftpclient) (#10401) 2019-01-22 16:06:44 +01:00
borrow require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
casestmt EndsInNoReturn in expressions extension, fixes #13490 (#13520) 2020-02-28 09:55:06 +00:00
ccgbugs fix #13902 distinct uint64 type corruption on 32-bit with borrow (#13907) [backport:1.2] 2020-04-08 09:56:16 +02:00
clearmsg require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
closure faster CIs (#13803) 2020-03-30 13:18:12 +02:00
cnstseq require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
collections fix mapIt issues #12625 & #12639 (#14041) 2020-04-21 14:50:16 +02:00
compilepragma fixes #10299 (#11324) 2019-05-25 07:33:33 +02:00
compiler Fixes #12536 (#12568) [backport] 2019-11-02 08:21:49 +01:00
compilerapi make case-object transitions explicit, make unknownLineInfo a const, replace a few magic numbers with consts (#13170) 2020-01-17 10:34:31 +01:00
compilerfeatures new compiler feature: --expandMacro 2019-05-29 20:47:28 +02:00
compiles delete list comprehension (#12392) 2019-11-22 19:57:43 +01:00
concat
concepts fixes #12741 (#14005) 2020-04-19 11:59:01 +02:00
constr fixes #10943 (#10947) 2019-04-05 14:50:57 +02:00
constraints require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
constructors Turn some of the errors back into warnings 2020-04-01 19:38:44 +02:00
controlflow Testament: refactoring; makes the test joiner green 2018-12-11 21:23:23 +01:00
converter Fix spellings (#12277) [backport] 2019-09-27 07:02:54 +02:00
coroutines #12103 - CI for OpenBSD (#12105) 2020-04-21 15:05:21 +02:00
cpp fix typos and deprecation warnings for tconvariancerules.nim (#13772) 2020-03-27 08:36:21 +01:00
defaultprocparam updated tests to be executed 2018-11-23 11:58:28 +01:00
deprecated fix #8063 by adding a testcase for: Deprecation warnings for enum values print twice (#10508) 2019-01-31 11:44:11 +01:00
deps [backport] Fix spelling typos (#12755) 2019-11-28 08:30:55 +01:00
destructor Implements RFCs #209 (#13995) 2020-04-16 21:04:05 +02:00
dir with space fixes #10042 (allow spaces in import) (#10504) 2019-01-31 08:20:00 +01:00
discard Hrm, the new errors highlighted some code that seems to be broken 2020-04-01 19:38:44 +02:00
distinct fixes #11715 (#11721) 2019-07-13 07:55:56 +02:00
dll #12103 - CI for OpenBSD (#12105) 2020-04-21 15:05:21 +02:00
effects '.push raises: []' now also affects proc types (#13776) 2020-03-29 22:00:18 +02:00
enum fix enumtostr crash for enum-range (#13035) 2020-01-05 09:18:14 +01:00
errmsgs fixes #12741 (#14005) 2020-04-19 11:59:01 +02:00
exception #12103 - CI for OpenBSD (#12105) 2020-04-21 15:05:21 +02:00
exprs require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
fields make tests green again 2019-05-27 21:29:02 +02:00
flags require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
float Revert "printing float values will have one more digit. (#13276) [backport]" (#13363) 2020-02-08 09:30:24 +01:00
fragmentation tests: disable flaky tests for now; closes #9421 2018-10-18 12:12:15 +02:00
friends
gc faster CIs (#13803) 2020-03-30 13:18:12 +02:00
generics faster CIs (#13803) 2020-03-30 13:18:12 +02:00
gensym fixes yet another gensym regression (#12145) 2019-09-06 00:45:00 +02:00
global require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
ic Incremental compilation (IC): Improvements (#11881) 2019-08-08 08:41:05 +02:00
implicit lots of small changes 2018-12-11 21:23:22 +01:00
import_in_config fixes a regression that caused --import in a config file to break things 2016-08-29 11:33:41 +02:00
init activated more tests, allow input in test spec 2018-11-23 11:58:28 +01:00
iter fix #13739 (#13742) 2020-04-07 14:42:59 +02:00
js relativePath("foo", "foo") is now ".", not "" (#13452) 2020-02-22 09:07:53 +01:00
lent fixes #12224 (#12225) 2019-09-21 06:45:27 +02:00
let require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
lexer megatest: make it green on Linux 2018-12-11 21:23:25 +01:00
lookups Allow void macro result (#11286) 2019-05-21 21:31:40 +02:00
macros Fix typeSym.getImpl for ref types (#13752) 2020-03-26 09:26:19 +01:00
magics run runnableExamples in the module scope (#11732) [feature] 2019-07-22 16:22:57 +02:00
manyloc fix deprecations and other warnings (#13748) 2020-03-25 19:15:34 +01:00
metatype allow generic typedesc field access (#12220) 2020-04-20 15:36:57 +02:00
method faster CIs (#13803) 2020-03-30 13:18:12 +02:00
misc Fix semfold handling of {.str/int/bool-define.} (#13964) 2020-04-13 14:22:33 +02:00
modules fixes #12420 [backport] (#12456) 2019-10-18 22:13:24 +02:00
namedparams fixes #12196 2019-09-16 12:12:44 +02:00
newconfig [RFC] 'walkDir' now has a new 'checkDir' flag, to mimic behaviour of other languages (#13642) 2020-03-20 16:39:55 +01:00
nimble Fixes #12767 (#12768) 2019-11-29 10:18:44 +01:00
niminaction #12103 - CI for OpenBSD (#12105) 2020-04-21 15:05:21 +02:00
notnil More precise error messages for uninitialized fields in the presence of inheritance 2020-04-01 19:38:44 +02:00
objects Turn some of the errors back into warnings 2020-04-01 19:38:44 +02:00
objvariant Better case coverage error message for alias and range enum (#12913) 2019-12-18 10:29:01 +01:00
openarray clean up deprecated stuff and unused imports in tests (#13059) 2020-01-07 09:49:08 +01:00
osproc fix #13449 texitcode flaky on windows (#13487) 2020-02-25 17:49:16 +01:00
overflw make fullpaths the default in error messages and stack traces for mor… (#11385) 2019-06-05 08:02:54 +02:00
overload fixes #11375 (#11376) 2019-06-02 06:33:35 +02:00
package_level_objects new feature: package level objects 2017-10-29 07:54:39 +01:00
parallel #12103 - CI for OpenBSD (#12105) 2020-04-21 15:05:21 +02:00
parser Remove immediate pragma (#11308) 2019-05-29 22:21:51 +02:00
pragmas fix #13909 (#13914) [backport:1.2] 2020-04-08 11:28:09 +02:00
proc allow typed/untyped in magic procs (#12911) 2019-12-17 11:07:32 +01:00
procvar Merge tests into a larger file (part 5 of ∞) (#9368) 2018-10-16 10:50:10 +02:00
range catchable defects (#13626) 2020-03-12 23:44:33 +01:00
rational require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
realtimeGC WIP, most tests are now skipped 2018-12-11 21:23:18 +01:00
sets tables/sharedtables/intsets/etc: fix #13496, #13504, #13505; add lots of tests (#13498) [backport] 2020-02-26 22:07:09 +01:00
showoff Revert "printing float values will have one more digit. (#13276) [backport]" (#13363) 2020-02-08 09:30:24 +01:00
specialops dotops: add a simple object delegation test 2017-12-01 02:05:48 +01:00
statictypes fix #12864 static params were mutating arg types during sigmatch; fix #12713 ; refs #13529 (#13976) 2020-04-14 15:00:02 +02:00
stdlib #12103 - CI for OpenBSD (#12105) 2020-04-21 15:05:21 +02:00
system #12103 - CI for OpenBSD (#12105) 2020-04-21 15:05:21 +02:00
template Fix spellings (#12277) [backport] 2019-09-27 07:02:54 +02:00
testament test suite: rename tests containing 'fail' for easier search in logs 2019-11-26 16:04:28 +01:00
testdata Fix for #5134 (parseJson(string) should fail on leftover data) (#5203) 2017-01-11 06:50:18 +01:00
threads faster CIs (#13803) 2020-03-30 13:18:12 +02:00
tools new feature: --staticBoundChecks:on to enforce static array index checking (#10965) 2020-03-18 14:25:10 +01:00
trmacros fixes #7524 2019-01-14 12:15:40 +01:00
tuples fix several typos in documentation and comments (#12553) 2019-10-30 09:08:45 +01:00
typerel fix tests 2019-07-08 15:24:20 -07:00
types fix #14007 (#14012) [backport] 2020-04-18 16:15:17 +02:00
untestable SSL certificate verify GitHub action (#13697) 2020-03-20 17:11:39 +01:00
usingstmt Testament: refactoring; makes the test joiner green 2018-12-11 21:23:23 +01:00
valgrind attempt to add valgrind support to the CIs and testament (#12646) 2019-11-18 09:10:48 +01:00
varres fix #13848: make var result work with nim cpp (#13959) 2020-04-13 14:16:06 +02:00
varstmt Testament: refactoring; makes the test joiner green 2018-12-11 21:23:23 +01:00
vm Add hashWangYi1 (#13823) 2020-04-15 20:11:18 +02:00
ast_pattern_matching.nim add tastspec (and ast_pattern_matching) (#10863) 2019-03-19 11:45:29 +01:00
config.nims followup on #10573: prevent common user config to interfere with testament (excessiveStackTrace) (#10590) 2019-02-07 12:29:31 +01:00
dummy.txt Remove install.txt and readme.txt (#9521) 2018-10-28 13:36:52 +01:00
js.html
js.nim
mmaptest.nim Fix spellings (#12277) [backport] 2019-09-27 07:02:54 +02:00
readme.md Fix reference to parseSpec proc in readme (#12359) 2019-10-04 19:30:58 +02:00
rectest.nim
stckovfl.nim
test_nimscript.nims relativePath("foo", "foo") is now ".", not "" (#13452) 2020-02-22 09:07:53 +01:00
trunner.nim fix some codegen bugs: NIM_BOOL, NIM_STATIC_ASSERT, --passc:-std=... (etc) (#13798) 2020-04-07 15:17:30 +02:00

This directory contains the test cases.

Each test must have a filename of the form: t*.nim

Note: Tests are only compiled by default. In order to get the tester to execute the compiled binary, you need to specify a spec with an action key (see below for details).

Specs

Each test can contain a spec in a discard """ ... """ block.

Check out the parseSpec procedure in the specs module for a full and reliable reference

action

Specifies what action this test should take.

Default: compile

Options:

  • compile - compiles the module and fails the test if compilations fails.
  • run - compiles and runs the module, fails the test if compilation or execution of test code fails.
  • reject - compiles the module and fails the test if compilation succeeds.

There are certain spec keys that imply run, including output and outputsub.

cmd

Specifies the Nim command to use for compiling the test.

There are a number of variables that are replaced in this spec option:

  • $target - the compilation target, e.g. c.
  • $options - the options for the compiler.
  • $file - the filename of the test.
  • $filedir - the directory of the test file.

Example:

discard """
  cmd: "nim $target --nimblePath:./nimbleDir/simplePkgs $options $file"
"""

Categories

Each folder under this directory represents a test category, which can be tested by running koch tests cat <category>.

The folder rodfiles contains special tests that test incremental compilation via symbol files.

The folder dll contains simple DLL tests.

The folder realtimeGC contains a test for validating that the realtime GC can run properly without linking against the nimrtl.dll/so. It includes a C client and platform specific build files for manual compilation.