jyapayne/Nim
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Nim/tests
History
alaviss c1ca06b452
 tfdleak: fix flakyness on Windows (#14550) 
* tfdleak_multiple: introduce stress tester for tfdleak

Imported from #14548 and tweaked for consumption by testament.

This test seems to be really good at bringing out the flakyness of
tfdleadk.

Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>

* tfdleak: increase accuracy of the test on Windows

This commit implements a new testing strategy for Windows:
1. We duplicate the handle that will be tested and enable inheritance.
   This duplicate will serve as a reference handle.
2. In addition to checking whether the handle is valid, we also verify
   whether the handle is the same as the reference. This gives us
   complete certainty on whether the handle in question is inherited
   from the parent.
   A side effect is that this uses Windows 10+ APIs. But since
   this is just for the test, we don't have to be picky about it.

Ideally we would want to do something like this for other POSIX-based
system, but most of them lack a facility to do this, and as of writing
there isn't any false positive for them, so we won't need the additional
checks.

MemFile.fHandle will also no longer be tested, as this handle defaults
to being invalid.

Co-authored-by: Timothee Cour <timothee.cour2@gmail.com>
2020-06-04 13:25:38 +02:00
..
actiontable Merge tests into a larger file (part 1 of ∞) (#9318) 2018-10-12 17:02:46 +02:00
alias Remove expr/stmt (#5857) 2017-07-25 09:28:23 +02:00
align fixes #13122 (#13126) 2020-01-13 14:11:51 +01:00
ambsym require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
arc fixes #14495 [backport:1.2] (#14496) 2020-05-29 23:35:57 +02:00
arithm fix operators containing percent for VM usage (#13536) 2020-03-11 01:01:25 +01:00
array disallow typedesc in arrays & move existing checks to types.typeAllowedAux (#13261) 2020-05-29 11:48:15 +02:00
assert Error -> Defect for defects (#13908) 2020-04-28 19:56:01 +02:00
assign fix rtti sizeof for varargs in global scope (#13125) [backport] 2020-01-13 10:17:21 +01:00
astoverload
astspec Fix spellings (#12277) [backport] 2019-09-27 07:02:54 +02:00
async fix #14320 (tasyncawait.nim is recently very flaky) + avoid hardcoding service ports everywhere + flakyAssert (#14327) 2020-05-13 11:05:47 +02:00
bind Remove deprecated modules (asyncio, sockets, ftpclient) (#10401) 2019-01-22 16:06:44 +01:00
borrow require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
casestmt EndsInNoReturn in expressions extension, fixes #13490 (#13520) 2020-02-28 09:55:06 +00:00
ccgbugs warn about observerable stores but don't prevent them for 1.2.2 [backport:1.2]; refs https://github.com/nim-lang/RFCs/issues/230 (#14510) 2020-05-30 19:38:51 +02:00
clearmsg require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
closure Make the fields of times.DateTime private (#14197) 2020-05-06 12:20:34 +02:00
cnstseq require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
collections fix mapIt issues #12625 & #12639 (#14041) 2020-04-21 14:50:16 +02:00
compilepragma fixes #10299 (#11324) 2019-05-25 07:33:33 +02:00
compiler trunner was not actually being tested in non-CTFFI mode; minor testament cleanups (#14377) 2020-05-19 09:41:31 +02:00
compilerapi hotfix: make tcompilerapi green again 2020-04-30 22:48:10 +02:00
compilerfeatures new compiler feature: --expandMacro 2019-05-29 20:47:28 +02:00
compiles delete list comprehension (#12392) 2019-11-22 19:57:43 +01:00
concat
concepts fix #14217 (#14218) 2020-05-05 07:26:32 +02:00
constr fixes #10943 (#10947) 2019-04-05 14:50:57 +02:00
constraints require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
constructors Turn some of the errors back into warnings 2020-04-01 19:38:44 +02:00
controlflow Testament: refactoring; makes the test joiner green 2018-12-11 21:23:23 +01:00
converter Fix spellings (#12277) [backport] 2019-09-27 07:02:54 +02:00
coroutines #12103 - CI for OpenBSD (#12105) 2020-04-21 15:05:21 +02:00
cpp fix typos and deprecation warnings for tconvariancerules.nim (#13772) 2020-03-27 08:36:21 +01:00
defaultprocparam updated tests to be executed 2018-11-23 11:58:28 +01:00
deprecated fix #8063 by adding a testcase for: Deprecation warnings for enum values print twice (#10508) 2019-01-31 11:44:11 +01:00
deps Remove some deprecated procs from std/times (#14129) 2020-04-27 14:42:29 +02:00
destructor fix #14421 items uses lent T (#14447) 2020-05-29 17:10:59 +02:00
dir with space fixes #10042 (allow spaces in import) (#10504) 2019-01-31 08:20:00 +01:00
discard improve the 'has to be discarded' error message 2020-05-02 22:31:19 +02:00
distinct fixes #11715 (#11721) 2019-07-13 07:55:56 +02:00
distros fix tdistros test which was failing on Nightlies 2020-04-28 15:59:32 +02:00
dll HCR: properly handling complex const objects in the codegen - fixes #13915 (#14115) 2020-04-25 20:17:33 +02:00
effects do not track 'raise Defect' in the .raises: [] clause anymore (#14298) 2020-05-11 16:25:56 +02:00
enum fix enumtostr crash for enum-range (#13035) 2020-01-05 09:18:14 +01:00
errmsgs Implement rendering of []=, {}, {}= braces (#14539) 2020-06-01 19:56:17 +02:00
exception fix #14369 (#14386) 2020-05-18 09:38:36 +02:00
exprs update tests that tested for the 'discard' error messages 2020-05-02 22:31:19 +02:00
fields make tests green again 2019-05-27 21:29:02 +02:00
flags require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
float Error -> Defect for defects (#13908) 2020-04-28 19:56:01 +02:00
fragmentation tests: disable flaky tests for now; closes #9421 2018-10-18 12:12:15 +02:00
friends tests: Trim .nim files trailing whitespace 2015-09-04 23:04:32 +02:00
gc new implementations for --gc:orc (#14121) 2020-04-27 11:57:26 +02:00
generics Remove the uses of {.procvar.} pragma (#14359) 2020-05-15 23:37:24 +02:00
gensym fixes yet another gensym regression (#12145) 2019-09-06 00:45:00 +02:00
global require errormsg to be specified before file. 2018-12-11 21:23:21 +01:00
ic Incremental compilation (IC): Improvements (#11881) 2019-08-08 08:41:05 +02:00
implicit lots of small changes 2018-12-11 21:23:22 +01:00
import_in_config fixes a regression that caused --import in a config file to break things 2016-08-29 11:33:41 +02:00
init activated more tests, allow input in test spec 2018-11-23 11:58:28 +01:00
iter fix #13739 (#13742) 2020-04-07 14:42:59 +02:00
js fix #14350, cstrings in JS init as null (#14355) 2020-05-15 10:07:30 +02:00
lent fixes #14498 [backport:1.2] (#14503) 2020-05-30 15:32:31 +02:00
let Allow let to not have value when using importc (#14258) 2020-05-12 20:28:18 +02:00
lexer megatest: make it green on Linux 2018-12-11 21:23:25 +01:00
lookups Allow void macro result (#11286) 2019-05-21 21:31:40 +02:00
macros Implement rendering of []=, {}, {}= braces (#14539) 2020-06-01 19:56:17 +02:00
magics close #12746; minor cleanup (#14379) 2020-05-17 13:28:18 +02:00
manyloc Remove the uses of {.procvar.} pragma (#14359) 2020-05-15 23:37:24 +02:00
metatype disallow typedesc in arrays & move existing checks to types.typeAllowedAux (#13261) 2020-05-29 11:48:15 +02:00
method Error -> Defect for defects (#13908) 2020-04-28 19:56:01 +02:00
misc walkDirRecFilter, update doc CI filter, compiler/index.nim for docs + various other fixes (#14501) 2020-06-01 19:21:41 +02:00
modules fixes #12420 [backport] (#12456) 2019-10-18 22:13:24 +02:00
namedparams fixes #12196 2019-09-16 12:12:44 +02:00
newconfig --hint:processing (+friends) is now supported and means --hint:processing:on, like all other bool flags (#14271) 2020-05-08 18:54:16 +02:00
nimble tnimblepathdollarfail.nim -> tests/nimble/tnimblepathdollar_fault to reduce false positives when searching for fail in CI logs (#14450) 2020-05-25 13:54:30 -07:00
nimdoc fix #6583, fix #14376, index+search now generated for all projects, many bug fixes with nim doc (#14324) 2020-05-25 13:25:40 +02:00
niminaction update tests that tested for the 'discard' error messages 2020-05-02 22:31:19 +02:00
notnil Add tests for #8481, #6490 and #4061 (#14083) 2020-04-23 19:52:33 +02:00
objects Error -> Defect for defects (#13908) 2020-04-28 19:56:01 +02:00
objvariant Better case coverage error message for alias and range enum (#12913) 2019-12-18 10:29:01 +01:00
openarray clean up deprecated stuff and unused imports in tests (#13059) 2020-01-07 09:49:08 +01:00
osproc fix #13449 texitcode flaky on windows (#13487) 2020-02-25 17:49:16 +01:00
overflw Error -> Defect for defects (#13908) 2020-04-28 19:56:01 +02:00
overload exp. features now in correct manual, closes #11932 (#14195) 2020-05-02 22:30:08 +02:00
package_level_objects new feature: package level objects 2017-10-29 07:54:39 +01:00
parallel #12103 - CI for OpenBSD (#12105) 2020-04-21 15:05:21 +02:00
parser make from an operator (#14241) 2020-05-08 07:54:46 +02:00
pragmas add test for define, undef (#14443) 2020-05-25 11:17:25 +02:00
proc allow typed/untyped in magic procs (#12911) 2019-12-17 11:07:32 +01:00
procvar Merge tests into a larger file (part 5 of ∞) (#9368) 2018-10-16 10:50:10 +02:00
range Error -> Defect for defects (#13908) 2020-04-28 19:56:01 +02:00
rational Error -> Defect for defects (#13908) 2020-04-28 19:56:01 +02:00
realtimeGC WIP, most tests are now skipped 2018-12-11 21:23:18 +01:00
sets tables/sharedtables/intsets/etc: fix #13496, #13504, #13505; add lots of tests (#13498) [backport] 2020-02-26 22:07:09 +01:00
showoff Revert "printing float values will have one more digit. (#13276) [backport]" (#13363) 2020-02-08 09:30:24 +01:00
specialops dotops: add a simple object delegation test 2017-12-01 02:05:48 +01:00
statictypes fix #12864 static params were mutating arg types during sigmatch; fix #12713 ; refs #13529 (#13976) 2020-04-14 15:00:02 +02:00
stdlib tfdleak: fix flakyness on Windows (#14550) 2020-06-04 13:25:38 +02:00
system fix #14350, cstrings in JS init as null (#14355) 2020-05-15 10:07:30 +02:00
template Fix spellings (#12277) [backport] 2019-09-27 07:02:54 +02:00
testament test suite: rename tests containing 'fail' for easier search in logs 2019-11-26 16:04:28 +01:00
testdata Fix for #5134 (parseJson(string) should fail on leftover data) (#5203) 2017-01-11 06:50:18 +01:00
threads faster CIs (#13803) 2020-03-30 13:18:12 +02:00
tools new feature: --staticBoundChecks:on to enforce static array index checking (#10965) 2020-03-18 14:25:10 +01:00
trmacros fixes #7524 2019-01-14 12:15:40 +01:00
tuples fixes #14112, tests for #12892, #12671, #11697 (#14125) 2020-04-26 10:12:16 +02:00
typerel disallow typedesc in arrays & move existing checks to types.typeAllowedAux (#13261) 2020-05-29 11:48:15 +02:00
types fix #14007 (#14012) [backport] 2020-04-18 16:15:17 +02:00
untestable fix nim CI; fix local testament (#14102) 2020-04-24 10:24:30 +02:00
usingstmt Testament: refactoring; makes the test joiner green 2018-12-11 21:23:23 +01:00
valgrind attempt to add valgrind support to the CIs and testament (#12646) 2019-11-18 09:10:48 +01:00
varres fix #13848: make var result work with nim cpp (#13959) 2020-04-13 14:16:06 +02:00
varstmt Testament: refactoring; makes the test joiner green 2018-12-11 21:23:23 +01:00
vm nim doc --backend:js, nim doc --doccmd:-d:foo, nim r --backend:js, --doccmd:skip + other improvements (#14278) 2020-05-11 12:01:18 +02:00
ast_pattern_matching.nim add tastspec (and ast_pattern_matching) (#10863) 2019-03-19 11:45:29 +01:00
config.nims fix #14320 (tasyncawait.nim is recently very flaky) + avoid hardcoding service ports everywhere + flakyAssert (#14327) 2020-05-13 11:05:47 +02:00
dummy.txt Remove install.txt and readme.txt (#9521) 2018-10-28 13:36:52 +01:00
js.html
js.nim tests: Trim .nim files trailing whitespace 2015-09-04 23:04:32 +02:00
mmaptest.nim Fix spellings (#12277) [backport] 2019-09-27 07:02:54 +02:00
readme.md trunner was not actually being tested in non-CTFFI mode; minor testament cleanups (#14377) 2020-05-19 09:41:31 +02:00
rectest.nim tests: Trim .nim files trailing whitespace 2015-09-04 23:04:32 +02:00
stckovfl.nim tests: Trim .nim files trailing whitespace 2015-09-04 23:04:32 +02:00
test_nimscript.nims don't close #14142 2020-05-02 22:31:19 +02:00

readme.md

This directory contains the test cases.

Each test must have a filename of the form: t*.nim

Note: Testament is only aware of tests under a directory (eg tests/foo/) and will ignore top-level tests like tests/tbar.nim.

Specs

Each test can contain a spec in a discard """ ... """ block.

Check out the parseSpec procedure in the specs module for a full and reliable reference

action

Specifies what action this test should take.

Default: run

Options:

  • compile - compiles the module and fails the test if compilations fails.
  • run - compiles and runs the module, fails the test if compilation or execution of test code fails.
  • reject - compiles the module and fails the test if compilation succeeds.

There are certain spec keys that imply run, including output and outputsub.

cmd

Specifies the Nim command to use for compiling the test.

There are a number of variables that are replaced in this spec option:

  • $target - the compilation target, e.g. c.
  • $options - the options for the compiler.
  • $file - the filename of the test.
  • $filedir - the directory of the test file.

Example:

discard """
  cmd: "nim $target --nimblePath:./nimbleDir/simplePkgs $options $file"
"""

Categories

Each folder under this directory represents a test category, which can be tested by running koch tests pcat <category> (or cat to avoid parallel testing, which is slower).

The folder dll contains simple DLL tests.

The folder realtimeGC contains a test for validating that the realtime GC can run properly without linking against the nimrtl.dll/so. It includes a C client and platform specific build files for manual compilation.