Nim/lib/pure
havardjohn f4bbf3bf0b
Add use of Windows Wide CRT API for env. vars (#20084)
* Add use of Windows Wide CRT API for env. vars

Replaces use of CRT API `getenv` and `putenv` with respectively
`_wgetenv` and `_wputenv`. Motivation is to reliably convert environment
variables to UTF-8, and the wide API is best there, because it's
reliably UTF-16.

Changed the hack in `lib/std/private/win_setenv.nim` by switching the
order of the Unicode and MBCS environment update; Unicode first, MBCS
second. Because `_wgetenv`/`_wputenv` is now used, the Unicode
environment will be initialized, so it should always be updated.

Stop updating MBCS environment with the name of `getEnv`. It's not
necessarily true that MBCS encoding and the `string` encoding is the
same. Instead convert UTF-16 to current Windows code page with
`wcstombs`, and use that string to update MBCS.

Fixes regression in `6b3c77e` that caused `std/envvars.getEnv` or
`std/os.getEnv` on Windows to return non-UTF-8 encoded strings.

Add tests that test environment variables with Unicode characters in
their name or value.

* Fix test issues

Fixes

* `nim cpp` didn't compile the tests
* Nimscript import of `tosenv.nim` from `test_nimscript.nims` failed
  with "cannot importc"

* Fix missing error check on `wcstombs`

* Fix ANSI testing errors

* Separate ANSI-related testing to their own tests, and only executing
  them if running process has a specific code page
  * Setting locale with `setlocale` was not reliable and didn't work on
    certain machines
* Add handling of a "no character representation" error in second
  `wcstombs` call

* tests/newruntime_misc: Increment allocCount

Increments overall allocations in `tnewruntime_misc` test. This is
because `getEnv` now does an additional allocation: allocation of the
UTF-16 string used as parameter to `c_wgetenv`.

* Revert "tests/newruntime_misc: Increment allocCount"

This reverts commit 4d4fe8bd3edb1bfc6d600f247af797c7552f5477.

* tests/newruntime_misc: Increment allocCount on Windows

Increments overall allocations in `tnewruntime_misc` test for Windows.
This is because `getEnv` on Windows now does an additional allocation:
allocation of the UTF-16 string used as parameter to `c_wgetenv`.

* Refactor, adding suggestions from code review

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* Document, adding suggestions

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
2022-08-20 04:30:11 -04:00
..
collections bootstrap the compiler with nimPreviewSlimSystem (#20176) 2022-08-09 16:32:29 +08:00
concurrency Resolve cross file resolution errors in atomics (#19422) [backport:1.6] 2022-01-20 13:58:59 +01:00
includes Add use of Windows Wide CRT API for env. vars (#20084) 2022-08-20 04:30:11 -04:00
ioselects enable style:usages for stdlib tests [backport: 1.6] (#19715) 2022-04-13 11:53:02 +02:00
unidecode fix shebangs #! /xxx => #!/xxx (#18444) 2021-07-07 06:00:59 +02:00
algorithm.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
async.nim [formatter] async.nim uses two spaces (#19264) 2021-12-17 10:16:59 +00:00
asyncdispatch.nim make AsyncSocket.getPeerAddr work ; fix #15022 (#20038) 2022-07-16 10:59:58 +01:00
asyncdispatch.nim.cfg Rename *.nimrod.cfg to *.nim.cfg 2015-02-03 09:04:24 +01:00
asyncfile.nim Fix #15150 (#18730) 2021-08-31 08:14:05 +02:00
asyncftpclient.nim further progress on rst roles & directives (fix #17646) (#17659) 2021-04-08 19:00:14 +02:00
asyncfutures.nim Fix type mismatch with future logging (#19131) 2021-11-12 08:00:19 +01:00
asynchttpserver.nim remove shallowCopy for ARC/ORC (#20070) 2022-07-26 16:51:01 +02:00
asyncmacro.nim Better error message and tests for bad await (#19622) 2022-04-13 23:03:46 +02:00
asyncnet.nim enable style:usages for stdlib tests [backport: 1.6] (#19715) 2022-04-13 11:53:02 +02:00
asyncstreams.nim Fix FutureStream memory usage (#17395) [backport:1.2] 2021-03-16 21:46:22 +01:00
base64.nim Documentation: Fix word usage (#19529) 2022-02-14 22:15:59 +00:00
bitops.nim typetraits: add toSigned, toUnsigned (#18445) 2022-04-07 17:38:01 -04:00
browsers.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
cgi.nim CIs: attempt to use csources_v1 (#16282) 2021-04-21 07:41:33 +02:00
colors.nim Update colors.nim (#19274) 2021-12-20 10:40:32 +01:00
complex.nim CIs: attempt to use csources_v1 (#16282) 2021-04-21 07:41:33 +02:00
cookies.nim enable style:usages for stdlib tests [backport: 1.6] (#19715) 2022-04-13 11:53:02 +02:00
coro.nim RST: improve simple tables (#19859) 2022-06-04 07:03:03 +02:00
coro.nimcfg fix test failure in coro.nim 2015-10-01 12:05:45 -07:00
cstrutils.nim Improve documentation for cstrutils (#17004) 2021-02-10 20:38:18 +01:00
db_common.nim use single backtick (#17166) 2021-02-23 21:05:39 -08:00
distros.nim distros.foreignDeps made public (#18830) 2021-09-10 10:20:32 +02:00
dynlib.nim Update documentation for dynlib (#17356) 2021-07-26 19:15:21 -04:00
encodings.nim workaround #18481 (#18482) 2021-07-13 09:22:33 +02:00
endians.nim Improve endians module (#17674) 2021-04-08 14:08:58 +02:00
fenv.nim use lowercase --define switches (#17283) 2021-03-07 12:08:41 +01:00
future.nim renamed future.nim to sugar.nim 2018-04-06 11:59:49 +02:00
hashes.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
htmlgen.nim nnkArglist => nnkArgList + special case stylecheck:error (#17529) 2021-03-27 10:28:11 +01:00
htmlparser.nim move io out of system (#19442) 2022-02-02 17:10:11 +01:00
httpclient.nim Add client.close() in httpclient examples. (#20118) 2022-08-04 06:46:08 +02:00
httpcore.nim Removed deprecated httpcore func (#19550) 2022-03-18 07:54:20 +01:00
json.nim replace shallowcopy with move in ARC/ORC (#20061) 2022-07-18 21:23:39 +02:00
lenientops.nim Use func in lenientops (#16641) 2021-01-09 00:24:41 +01:00
lexbase.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
logging.nim make implicit cstring conversions explicit (#19488) 2022-08-19 15:40:53 -04:00
marshal.nim remove shallowCopy for ARC/ORC (#20070) 2022-07-26 16:51:01 +02:00
math.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
md5.nim Documentation: Fix word usage (#19529) 2022-02-14 22:15:59 +00:00
memfiles.nim move io out of system (#19442) 2022-02-02 17:10:11 +01:00
mersenne.nim make deprecated message better (#18737) 2021-08-23 11:13:10 -07:00
mimetypes.nim use OrderedTable instead of OrderedTableRef for mimedb (#19522) 2022-02-14 08:56:35 +01:00
nativesockets.nim make implicit cstring conversions explicit (#19488) 2022-08-19 15:40:53 -04:00
net.nim make implicit cstring conversions explicit (#19488) 2022-08-19 15:40:53 -04:00
nimprof.nim close #4451 (#17627) 2021-04-06 22:08:59 +02:00
nimprof.nim.cfg Rename *.nimrod.cfg to *.nim.cfg 2015-02-03 09:04:24 +01:00
nimtracker.nim make memory tracker work without onThreadCreation 2017-03-14 15:45:35 +01:00
oids.nim Remove Deprecated oids.oidsToString (#19519) 2022-02-14 09:09:21 +01:00
options.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
os.nim make implicit cstring conversions explicit (#19488) 2022-08-19 15:40:53 -04:00
osproc.nim make implicit cstring conversions explicit (#19488) 2022-08-19 15:40:53 -04:00
oswalkdir.nim use single backtick (#17181) 2021-02-24 21:47:27 -08:00
parsecfg.nim move io out of system (#19442) 2022-02-02 17:10:11 +01:00
parsecsv.nim move io out of system (#19442) 2022-02-02 17:10:11 +01:00
parsejson.nim make implicit cstring conversions explicit (#19488) 2022-08-19 15:40:53 -04:00
parseopt.nim No need to export pos from OptParser (#19688) 2022-04-06 20:28:58 +02:00
parsesql.nim enable style:usages for stdlib tests [backport: 1.6] (#19715) 2022-04-13 11:53:02 +02:00
parseutils.nim Change stdlib imports to use std prefix in most examples (#17202) 2021-02-28 13:17:19 -08:00
parsexml.nim Fix crash in parsexml (#15582) (#15583) 2020-10-15 12:45:55 +02:00
pathnorm.nim use single backtick (#17133) 2021-02-21 00:27:39 -08:00
pegs.nim enable style:usages for stdlib tests [backport: 1.6] (#19715) 2022-04-13 11:53:02 +02:00
prelude.nim make strformat part of the prelude (#18046) 2021-05-19 19:18:16 +02:00
punycode.nim Fix punycode.decode function (#19136) 2021-11-12 22:51:58 -05:00
random.nim prevent cache thrashing (#20129) 2022-08-02 00:06:27 +08:00
rationals.nim CIs: attempt to use csources_v1 (#16282) 2021-04-21 07:41:33 +02:00
reservedmem.nim add std/private/win_getsysteminfo; refactor the usage of GetSystemInfo (#19310) 2022-01-04 11:49:54 +01:00
ropes.nim move io out of system (#19442) 2022-02-02 17:10:11 +01:00
segfaults.nim enable style:usages for stdlib tests [backport: 1.6] (#19715) 2022-04-13 11:53:02 +02:00
selectors.nim enable maxDescriptors on Illumos/Solaris (#19295) 2021-12-30 21:10:36 +01:00
smtp.nim Added support for sending "ehlo" and receiving multiline "ehlo" response. (#19077) 2021-11-03 09:14:59 +01:00
smtp.nim.cfg revert moving oids and smtp to graveyard 2019-01-11 08:51:19 +01:00
ssl_certs.nim termux ssl (#18520) 2021-07-20 12:50:43 +02:00
ssl_config.nim net: use a secure cipher list by default 2020-06-06 21:11:53 +02:00
stats.nim Fix #18718 (#18731) 2021-08-23 00:13:39 +02:00
streams.nim [Orc] fixes "streams.readDataStr segafaults" when accepting a string literal (#20019) [backport] 2022-07-15 09:42:54 +02:00
streamwrapper.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
strformat.nim string is missing formatting when calling fmt (#19780) 2022-05-10 17:45:57 +02:00
strmisc.nim Improve documentation of strmisc (#16665) 2021-01-11 10:53:15 +01:00
strscans.nim bootstrap the compiler with nimPreviewSlimSystem (#20176) 2022-08-09 16:32:29 +08:00
strtabs.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
strutils.nim Improve error message for strutils.addf (#20157) 2022-08-05 13:44:21 -04:00
sugar.nim fixes #18954 (#18955) 2021-10-05 12:29:59 +02:00
terminal.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
times.nim fix broken runnableExamples for getWeeksInIsoYear (#20193) 2022-08-11 09:39:18 +08:00
typetraits.nim typetraits: add toSigned, toUnsigned (#18445) 2022-04-07 17:38:01 -04:00
unicode.nim fixes #18858 [backport] (#18868) 2021-09-19 23:35:50 +02:00
unittest.nim Pass check condition directly to if (#20217) 2022-08-15 18:37:10 -04:00
uri.nim bootstrap the compiler with nimPreviewSlimSystem (#20176) 2022-08-09 16:32:29 +08:00
volatile.nim use typeof instead type (#16962) 2021-02-08 09:46:07 +01:00
xmlparser.nim move io out of system (#19442) 2022-02-02 17:10:11 +01:00
xmltree.nim bootstrap the compiler with nimPreviewSlimSystem (#20176) 2022-08-09 16:32:29 +08:00