Nim/lib/std
havardjohn 23e0160af2
Add improved Windows UNC path support in std/os (#20281)
* Add improved Windows UNC path support in std/os

Original issue: `std/os.createDir` tries to create every component of
the given path as a directory. The problem is that `createDir`
interprets every backslash/slash as a path separator. For a UNC path
this is incorrect. E.g. one UNC form is `\\Server\Volume\Path`. It's an
error to create the `\\Server` directory, as well as creating
`\\Server\Volume`.

Add `ntpath.nim` module with `splitDrive` proc. This implements UNC path
parsing as implemented in the Python `ntpath.py` module. The following
UNC forms are supported:

* `\\Server\Volume\Path`
* `\\?\Volume\Path`
* `\\?\UNC\Server\Volume\Path`

Improves support for UNC paths in various procs in `std/os`:
---

* pathnorm.addNormalizePath
  * Issue: This had incomplete support for UNC paths
    * The UNC prefix (first 2 characters of a UNC path) was assumed to
      be exactly `\\`, but it can be `//` and `\/`, etc. as well
    * Also, the UNC prefix must be normalized to the `dirSep` argument
      of `addNormalizePath`
  * Resolution: Changed to account for different UNC prefixes, and
    normalizing the prefixes according to `dirSep`
    * Affected procs that get tests: `relativePath`, `joinPath`
  * Issue: The server/volume part of UNC paths can be stripped when
    normalizing `..` path components
    * This error should be negligable, so ignoring this
* splitPath
  * Now make sure the UNC drive is not split; return the UNC drive as
    `head` if the UNC drive is the only component of the path
  * Consequently fixes `extractFilename`, `lastPathPart`
* parentDir / `/../`
  * Strip away drive before working on the path, prepending the drive
    after all work is done - prevents stripping UNC components
  * Return empty string if drive component is the only component; this
    is the behavior for POSIX paths as well
  * Alternative implementation: Just call something like
    `pathnorm.normalizePath(path & "/..")` for the whole proc - maybe
    too big of a change
* tailDir
  * If drive is present in path, just split that from path and return
    path
* parentDirs iterator
  * Uses `parentDir` for going backwards
  * When going forwards, first `splitDrive`, yield the drive field, and
    then iterate over path field as normal
* splitFile
  * Make sure path parsing stops at end of drive component
* createDir
  * Fixed by skipping drive part before creating directories
  * Alternative implementation: use `parentDirs` iterator instead of
    iterating over characters
    * Consequence is that it will try to create the root directory
* isRootDir
  * Changed to treat UNC drive alone as root (e.g. "//?/c:" is root)
  * This change prevents the empty string being yielded by the
    `parentDirs` iterator with `fromRoot = false`
* Internal `sameRoot`
  * The "root" refers to the drive, so `splitDrive` can be used here

This adds UNC path support to all procs that could use it in std/os. I
don't think any more work has to be done to support UNC paths. For the
future, I believe the path handling code can be refactored due to
duplicate code. There are multiple ways of manipulating paths, such as
manually searching string for path separator and also having a path
normalizer (pathnorm.nim). If all path manipulation used `pathnorm.nim`,
and path component splitting used `parentDirs` iterator, then a lot of
code could be removed.

Tests
---

Added test file for `pathnorm.nim` and `ntpath.nim`.
`pathnorm.normalizePath` has no tests, so I'm adding a few unit tests.
`ntpath.nim` contains tests copied from Python's test suite.

Added integration tests to `tos.nim` that tests UNC paths.

Removed incorrect `relativePath` runnableExamples from being tested on Windows:
---

`relativePath("/Users///me/bar//z.nim", "//Users/", '/') == "me/bar/z.nim"`

This is incorrect on Windows because the `/` and `//` are not the same
root. `/` (or `\`) is expanded to the drive in the current working
directory (e.g. `C:\`). `//` (or `\\`), however, are the first two
characters of a UNC path. The following holds true for normal Windows
installations:

* `dirExists("/Users") != dirExists("//Users")`
* `dirExists("\\Users") != dirExists("\\\\Users")`

Fixes #19103

Questions:
---

* Should the `splitDrive` proc be in `os.nim` instead with copyright
  notice above the proc?
* Is it fine to put most of the new tests into the `runnableExamples`
  section of the procs in std/os?

* [skipci] Apply suggestions from code review

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* [skip ci] Update lib/pure/os.nim

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>

* Move runnableExamples tests in os.nim to tos.nim

* tests/topt_no_cursor: Change from using splitFile to splitDrive

`splitFile` can no longer be used in the test, because it generates
different ARC code on Windows and Linux. This replaces `splitFile` with
`splitDrive`, because it generates same ARC code on Windows and Linux,
and returns a tuple. I assume the test wants a proc that returns a
tuple.

* Drop copyright attribute to Python

Co-authored-by: Clay Sweetser <Varriount@users.noreply.github.com>
2022-09-03 20:47:09 -04:00
..
private Add improved Windows UNC path support in std/os (#20281) 2022-09-03 20:47:09 -04:00
assertions.nim new .redefine pragma for templates, warn on redefinition without it (#20211) 2022-08-23 21:41:30 +02:00
compilesettings.nim add mm to compilesettings; deprecate gc (#19394) 2022-01-16 21:10:35 +01:00
decls.nim New/better macro pragmas, mark some as experimental (#19406) 2022-01-20 20:57:50 +01:00
editdistance.nim use single backtick (#17141) 2021-02-21 19:53:28 -08:00
effecttraits.nim .forbids pragma: defining forbidden tags (#20050) 2022-07-26 07:40:49 +02:00
enumerate.nim move tests to testament (#16101) 2020-11-24 19:06:41 +01:00
enumutils.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
envvars.nim Add use of Windows Wide CRT API for env. vars (#20084) 2022-08-20 04:30:11 -04:00
exitprocs.nim Freeing critical sections via atexit in system/alloc and system/io (#19062) 2021-10-29 13:42:44 +02:00
formatfloat.nim move formatfloat out of system (#20195) 2022-08-24 13:38:30 +02:00
genasts.nim new genAst as replacement for quote do (#17426) 2021-04-02 13:11:03 +02:00
importutils.nim make privateAccess work with generic types and generic instantiations; fix a SIGSEGV (#18260) 2021-06-19 11:24:46 -07:00
isolation.nim Rename = to =copy in stdlib (#17781) 2021-04-19 13:16:10 +02:00
jsbigints.nim add number literal jsbigints.big (#17707) 2021-04-13 09:33:28 +02:00
jsfetch.nim Pass headers and body correctly to FetchOptions (#19884) [backport] 2022-06-13 08:03:40 +02:00
jsformdata.nim Use .. warning:: (#17320) 2021-03-10 10:39:23 -08:00
jsheaders.nim Use .. warning:: (#17320) 2021-03-10 10:39:23 -08:00
jsonutils.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
logic.nim use single backtick (#17141) 2021-02-21 19:53:28 -08:00
monotimes.nim Implement zephyr urandom and monotime (#19142) 2021-11-14 12:49:30 +01:00
objectdollar.nim test removing dollar for objects out of system (#20242) 2022-08-24 09:44:16 +02:00
oserrors.nim Added std/oserrors for OS error reporting (#19390) 2022-01-17 13:06:31 +01:00
packedsets.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
setutils.nim correct licence header (#18935) 2021-10-01 13:57:06 +02:00
sha1.nim move assertions out of system (#19599) 2022-03-23 20:34:53 +01:00
socketstreams.nim Implements streams for sockets (#15729) 2021-01-07 16:09:57 +01:00
stackframes.nim init checks and 'out' parameters (#14521) 2020-06-23 10:53:57 +02:00
strbasics.nim bootstrap the compiler with nimPreviewSlimSystem (#20176) 2022-08-09 16:32:29 +08:00
sums.nim Deprecate std/sums (#18439) 2022-06-09 08:27:31 +02:00
syncio.nim Revert "fix #19600 No error checking on fclose (#19836)" (#20297) 2022-09-02 19:04:58 +02:00
sysrand.nim sysrand: fix syscall signature [backport] (#19982) 2022-07-07 21:26:58 +08:00
tasks.nim enable style:usages for stdlib tests [backport: 1.6] (#19715) 2022-04-13 11:53:02 +02:00
tempfiles.nim [std/tempfiles] docs improvement (#18936) 2021-10-01 20:14:10 +02:00
time_t.nim use single backtick (#17141) 2021-02-21 19:53:28 -08:00
varints.nim Make unused code into actual test, replace echo with doassert (#13952) 2020-04-11 22:16:26 +02:00
vmutils.nim enable VM tracing in user code via {.define(nimVmTrace).} (#18244) 2021-06-24 11:55:31 +02:00
with.nim Small doc improvements for std/with (#17556) 2021-03-29 12:44:31 +02:00
wordwrap.nim move rest of tests to testament (#16140) 2020-11-27 20:47:49 +01:00
wrapnils.nim major improvements to std/wrapnils: optimal codegen, case objects, lvalue semantics (#18435) 2021-07-07 06:04:36 +02:00