Nim/lib/std
havardjohn f4bbf3bf0b
Add use of Windows Wide CRT API for env. vars (#20084)
* Add use of Windows Wide CRT API for env. vars

Replaces use of CRT API `getenv` and `putenv` with respectively
`_wgetenv` and `_wputenv`. Motivation is to reliably convert environment
variables to UTF-8, and the wide API is best there, because it's
reliably UTF-16.

Changed the hack in `lib/std/private/win_setenv.nim` by switching the
order of the Unicode and MBCS environment update; Unicode first, MBCS
second. Because `_wgetenv`/`_wputenv` is now used, the Unicode
environment will be initialized, so it should always be updated.

Stop updating MBCS environment with the name of `getEnv`. It's not
necessarily true that MBCS encoding and the `string` encoding is the
same. Instead convert UTF-16 to current Windows code page with
`wcstombs`, and use that string to update MBCS.

Fixes regression in `6b3c77e` that caused `std/envvars.getEnv` or
`std/os.getEnv` on Windows to return non-UTF-8 encoded strings.

Add tests that test environment variables with Unicode characters in
their name or value.

* Fix test issues

Fixes

* `nim cpp` didn't compile the tests
* Nimscript import of `tosenv.nim` from `test_nimscript.nims` failed
  with "cannot importc"

* Fix missing error check on `wcstombs`

* Fix ANSI testing errors

* Separate ANSI-related testing to their own tests, and only executing
  them if running process has a specific code page
  * Setting locale with `setlocale` was not reliable and didn't work on
    certain machines
* Add handling of a "no character representation" error in second
  `wcstombs` call

* tests/newruntime_misc: Increment allocCount

Increments overall allocations in `tnewruntime_misc` test. This is
because `getEnv` now does an additional allocation: allocation of the
UTF-16 string used as parameter to `c_wgetenv`.

* Revert "tests/newruntime_misc: Increment allocCount"

This reverts commit 4d4fe8bd3edb1bfc6d600f247af797c7552f5477.

* tests/newruntime_misc: Increment allocCount on Windows

Increments overall allocations in `tnewruntime_misc` test for Windows.
This is because `getEnv` on Windows now does an additional allocation:
allocation of the UTF-16 string used as parameter to `c_wgetenv`.

* Refactor, adding suggestions from code review

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* Document, adding suggestions

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
2022-08-20 04:30:11 -04:00
..
private Add use of Windows Wide CRT API for env. vars (#20084) 2022-08-20 04:30:11 -04:00
assertions.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
compilesettings.nim add mm to compilesettings; deprecate gc (#19394) 2022-01-16 21:10:35 +01:00
decls.nim New/better macro pragmas, mark some as experimental (#19406) 2022-01-20 20:57:50 +01:00
editdistance.nim use single backtick (#17141) 2021-02-21 19:53:28 -08:00
effecttraits.nim .forbids pragma: defining forbidden tags (#20050) 2022-07-26 07:40:49 +02:00
enumerate.nim move tests to testament (#16101) 2020-11-24 19:06:41 +01:00
enumutils.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
envvars.nim Add use of Windows Wide CRT API for env. vars (#20084) 2022-08-20 04:30:11 -04:00
exitprocs.nim Freeing critical sections via atexit in system/alloc and system/io (#19062) 2021-10-29 13:42:44 +02:00
genasts.nim new genAst as replacement for quote do (#17426) 2021-04-02 13:11:03 +02:00
importutils.nim make privateAccess work with generic types and generic instantiations; fix a SIGSEGV (#18260) 2021-06-19 11:24:46 -07:00
isolation.nim Rename = to =copy in stdlib (#17781) 2021-04-19 13:16:10 +02:00
jsbigints.nim add number literal jsbigints.big (#17707) 2021-04-13 09:33:28 +02:00
jsfetch.nim Pass headers and body correctly to FetchOptions (#19884) [backport] 2022-06-13 08:03:40 +02:00
jsformdata.nim Use .. warning:: (#17320) 2021-03-10 10:39:23 -08:00
jsheaders.nim Use .. warning:: (#17320) 2021-03-10 10:39:23 -08:00
jsonutils.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
logic.nim use single backtick (#17141) 2021-02-21 19:53:28 -08:00
monotimes.nim Implement zephyr urandom and monotime (#19142) 2021-11-14 12:49:30 +01:00
oserrors.nim Added std/oserrors for OS error reporting (#19390) 2022-01-17 13:06:31 +01:00
packedsets.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
setutils.nim correct licence header (#18935) 2021-10-01 13:57:06 +02:00
sha1.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
socketstreams.nim Implements streams for sockets (#15729) 2021-01-07 16:09:57 +01:00
stackframes.nim init checks and 'out' parameters (#14521) 2020-06-23 10:53:57 +02:00
strbasics.nim bootstrap the compiler with nimPreviewSlimSystem (#20176) 2022-08-09 16:32:29 +08:00
sums.nim Deprecate std/sums (#18439) 2022-06-09 08:27:31 +02:00
syncio.nim move io out of system (#19442) 2022-02-02 17:10:11 +01:00
sysrand.nim sysrand: fix syscall signature [backport] (#19982) 2022-07-07 21:26:58 +08:00
tasks.nim enable style:usages for stdlib tests [backport: 1.6] (#19715) 2022-04-13 11:53:02 +02:00
tempfiles.nim [std/tempfiles] docs improvement (#18936) 2021-10-01 20:14:10 +02:00
time_t.nim use single backtick (#17141) 2021-02-21 19:53:28 -08:00
varints.nim Make unused code into actual test, replace echo with doassert (#13952) 2020-04-11 22:16:26 +02:00
vmutils.nim enable VM tracing in user code via {.define(nimVmTrace).} (#18244) 2021-06-24 11:55:31 +02:00
with.nim Small doc improvements for std/with (#17556) 2021-03-29 12:44:31 +02:00
wordwrap.nim move rest of tests to testament (#16140) 2020-11-27 20:47:49 +01:00
wrapnils.nim major improvements to std/wrapnils: optimal codegen, case objects, lvalue semantics (#18435) 2021-07-07 06:04:36 +02:00