Nim/lib/std/private
havardjohn f4bbf3bf0b
Add use of Windows Wide CRT API for env. vars (#20084)
* Add use of Windows Wide CRT API for env. vars

Replaces use of CRT API `getenv` and `putenv` with respectively
`_wgetenv` and `_wputenv`. Motivation is to reliably convert environment
variables to UTF-8, and the wide API is best there, because it's
reliably UTF-16.

Changed the hack in `lib/std/private/win_setenv.nim` by switching the
order of the Unicode and MBCS environment update; Unicode first, MBCS
second. Because `_wgetenv`/`_wputenv` is now used, the Unicode
environment will be initialized, so it should always be updated.

Stop updating MBCS environment with the name of `getEnv`. It's not
necessarily true that MBCS encoding and the `string` encoding is the
same. Instead convert UTF-16 to current Windows code page with
`wcstombs`, and use that string to update MBCS.

Fixes regression in `6b3c77e` that caused `std/envvars.getEnv` or
`std/os.getEnv` on Windows to return non-UTF-8 encoded strings.

Add tests that test environment variables with Unicode characters in
their name or value.

* Fix test issues

Fixes

* `nim cpp` didn't compile the tests
* Nimscript import of `tosenv.nim` from `test_nimscript.nims` failed
  with "cannot importc"

* Fix missing error check on `wcstombs`

* Fix ANSI testing errors

* Separate ANSI-related testing to their own tests, and only executing
  them if running process has a specific code page
  * Setting locale with `setlocale` was not reliable and didn't work on
    certain machines
* Add handling of a "no character representation" error in second
  `wcstombs` call

* tests/newruntime_misc: Increment allocCount

Increments overall allocations in `tnewruntime_misc` test. This is
because `getEnv` now does an additional allocation: allocation of the
UTF-16 string used as parameter to `c_wgetenv`.

* Revert "tests/newruntime_misc: Increment allocCount"

This reverts commit 4d4fe8bd3edb1bfc6d600f247af797c7552f5477.

* tests/newruntime_misc: Increment allocCount on Windows

Increments overall allocations in `tnewruntime_misc` test for Windows.
This is because `getEnv` on Windows now does an additional allocation:
allocation of the UTF-16 string used as parameter to `c_wgetenv`.

* Refactor, adding suggestions from code review

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* Document, adding suggestions

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

Co-authored-by: ringabout <43030857+ringabout@users.noreply.github.com>
Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
2022-08-20 04:30:11 -04:00
..
asciitables.nim move asciitables to std/private/ (#16498) 2020-12-29 16:20:47 +01:00
bitops_utils.nim typetraits: add toSigned, toUnsigned (#18445) 2022-04-07 17:38:01 -04:00
dbutils.nim fix a sqlite bug (#18669) 2021-08-12 09:21:01 -07:00
decode_helpers.nim make oids module better (#16704) 2021-01-25 07:39:25 +01:00
digitsutils.nim fixes #19078 [backport] (#19090) 2021-11-02 16:30:59 +01:00
gitutils.nim gitutils: add diffStrings, diffFiles, and use it in testament to compare expected vs gotten (#17892) 2021-04-30 11:00:33 +02:00
globs.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
jsutils.nim improve jsutils docs (#17421) 2021-03-21 10:35:00 +01:00
miscdollars.nim improvements to addInt and $ for integer types (#18592) 2021-08-19 11:33:52 +02:00
since.nim document since (#17048) 2021-02-16 20:20:01 +01:00
strimpl.nim refactor cmpNimIdentifier (#16611) 2021-01-13 12:29:30 +01:00
underscored_calls.nim nnkArglist => nnkArgList + special case stylecheck:error (#17529) 2021-03-27 10:28:11 +01:00
win_getsysteminfo.nim add std/private/win_getsysteminfo; refactor the usage of GetSystemInfo (#19310) 2022-01-04 11:49:54 +01:00
win_setenv.nim Add use of Windows Wide CRT API for env. vars (#20084) 2022-08-20 04:30:11 -04:00