Compare commits

..

27 commits

Author SHA1 Message Date
Joey Yakimowich-Payne
b61d37040f Modify removeStatic to comment out body, then re-comment 2018-07-11 13:41:58 +09:00
Joey Yakimowich-Payne
410977bb71 Move remove static and reset after processing 2018-07-11 07:42:34 +09:00
Joey Yakimowich-Payne
797fa20fca Make static bodies replacement use regex 2018-07-10 20:09:51 +09:00
Joey Yakimowich-Payne
f5110d9da1 Add n.post section 2018-07-10 15:19:58 +09:00
Joey Yakimowich-Payne
d1813414e3 Add check for absolute directories 2018-07-08 22:56:56 +09:00
Joey Yakimowich-Payne
0918f02479 Remove dir sep from replace 2018-07-08 22:51:03 +09:00
Joey Yakimowich-Payne
c96b2f9709 Combine passC and absolute imports 2018-07-08 22:43:40 +09:00
Joey Yakimowich-Payne
cc0999b70b Add section for resetting files 2018-07-08 13:05:09 +09:00
Joey Yakimowich-Payne
e0ea3d09ac Make env vars automatically grab from config 2018-07-08 12:53:12 +09:00
Joey Yakimowich-Payne
2259a51d4e Add test for nim-libnx 2018-07-08 10:07:10 +09:00
Joey Yakimowich-Payne
eca296cf11 Add readme section about remove_static 2018-07-08 10:03:33 +09:00
Joey Yakimowich-Payne
09e5ea8f12 Add preliminary support for removing static funcs 2018-07-07 19:16:16 +09:00
Joey Yakimowich-Payne
8975db2cc9 Rename execute to pipe 2018-06-27 22:24:46 +09:00
Joey Yakimowich-Payne
b148bba4b2 Reset files after processing 2018-06-24 11:26:38 +09:00
Joey Yakimowich-Payne
ea6fa57670 Always do sparse checkout in case new folders are needed in nimgen cfg 2018-06-24 08:37:28 +09:00
Joey Yakimowich-Payne
153591397e Fix issue if cfg file is local gProject dir will be an empty string 2018-06-24 08:22:29 +09:00
Joey Yakimowich-Payne
14d8eca24b Change project dir based on nimgen cfg file 2018-06-24 07:52:36 +09:00
Joey Yakimowich-Payne
941a530885 Add env vars to readme 2018-06-20 14:04:14 +09:00
Joey Yakimowich-Payne
883c79115d Add ability to use environment vars 2018-06-20 13:59:46 +09:00
Joey Yakimowich-Payne
a55d71b02b Version bump 2018-06-18 20:22:47 +09:00
Joey Yakimowich-Payne
a71ddc6478 Fix ctags 2018-06-18 19:56:34 +09:00
Joey Yakimowich-Payne
0449f548dc Only include necessary file for include libs
The previous behavior caused a "Too many files open" error when
referrencing lots of libs because of the call to "gorge". This
modification retains the old behavior and also works if the library is
compiled locally.
2018-06-18 19:53:54 +09:00
Joey Yakimowich-Payne
a3d48fb1ac Use exceptions instead of quitting to get stacktrace 2018-06-18 19:47:13 +09:00
Joey Yakimowich-Payne
acb07ba3e1 Implement relative import support in header files.
You must currently replace relative imports using nimgen's commands,
but relative files don't crash nimgen anymore.
2018-06-18 19:44:45 +09:00
Joey Yakimowich-Payne
77e28545ef Add after command that operates on files after nimgen has run 2018-06-18 19:41:12 +09:00
Joey Yakimowich-Payne
0e0eadde37 Merge branch 'configurable_compiler' into all_merged 2018-06-16 15:39:21 +09:00
Joey Yakimowich-Payne
d2e3f41c5f Merge branch 'allow_deeper_output' into all_merged 2018-06-16 15:39:05 +09:00
25 changed files with 977 additions and 4616 deletions

View file

@ -1,27 +0,0 @@
os:
- linux
- osx
language: c
env:
- BRANCH=devel
addons:
apt:
packages:
- libssh2-1-dev
before_install:
- if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then HOMEBREW_NO_AUTO_UPDATE=1 brew install libssh2; fi
install:
- export CHOOSENIM_CHOOSE_VERSION=$BRANCH
- |
curl https://nim-lang.org/choosenim/init.sh -sSf > init.sh
sh init.sh -y
- export PATH=$HOME/.nimble/bin:$PATH
script:
- nimble install -y
- nimble test

129
README.md
View file

@ -1,7 +1,3 @@
[![Chat on Gitter](https://badges.gitter.im/gitterHQ/gitter.png)](https://gitter.im/nimgen/Lobby)
[![Build status](https://ci.appveyor.com/api/projects/status/05t5ja88lmv1rt3r/branch/master?svg=true)](https://ci.appveyor.com/project/genotrance/nimgen/branch/master)
[![Build Status](https://travis-ci.org/genotrance/nimgen.svg?branch=master)](https://travis-ci.org/genotrance/nimgen)
Nimgen is a helper for [c2nim](https://github.com/nim-lang/c2nim/) to simplify and automate the wrapping of C libraries. Nimgen is a helper for [c2nim](https://github.com/nim-lang/c2nim/) to simplify and automate the wrapping of C libraries.
Nimgen can be used to automate the process of manipulating C files so that c2nim can be run on them without issues. This includes adding/removing code snippets, removal of complex preprocessor definitions that c2nim doesn't yet comprehend and recursively running on #include files. Nimgen can be used to automate the process of manipulating C files so that c2nim can be run on them without issues. This includes adding/removing code snippets, removal of complex preprocessor definitions that c2nim doesn't yet comprehend and recursively running on #include files.
@ -32,7 +28,15 @@ Nimble already requires Git so those commands can be assumed to be present to do
__Capabilities & Limitations__ __Capabilities & Limitations__
Nimgen supports compiling in C/C++ sources and static libraries as well as loading in dynamic libraries. Nimgen supports compiling in C/C++ sources as well as loading in dynamic libraries at this time. Support for static libraries (.a, .lib) are still to come.
Environment variables are supported via Nim's string interpolation `%` symbol imported from the `strutils` module. Simply use double quotes to enclose any value and put `${}` around the environment variable name. In addition, the `output` var from the n.global section is available as ${output}. For example:
[n.global]
output="src/path"
[n.include]
"${output}/library/include"
To see examples of nimgen in action check out the following wrappers:- To see examples of nimgen in action check out the following wrappers:-
* Link with a dynamic library * Link with a dynamic library
@ -40,38 +44,26 @@ To see examples of nimgen in action check out the following wrappers:-
* download ZIP with headers and binary * download ZIP with headers and binary
* [nimlibxlsxwriter](https://github.com/KeepCoolWithCoolidge/nimlibxlsxwriter) - libxlsxwriter wrapper * [nimlibxlsxwriter](https://github.com/KeepCoolWithCoolidge/nimlibxlsxwriter) - libxlsxwriter wrapper
* git checkout * git checkout
* [nim-clblast](https://github.com/numforge/nim-clblast) - OpenCL BLAS wrapper
* static
* Compile C/C++ code into binary * Compile C code into binary
* [nim7z](https://github.com/genotrance/nim7z) - 7z decoder wrapper: [docs](http://nimgen.genotrance.com/nim7z) * [nim7z](https://github.com/genotrance/nim7z) - 7z decoder wrapper: [docs](http://nimgen.genotrance.com/nim7z)
* git sparse checkout * git sparse checkout
* [nimarchive](https://github.com/genotrance/nimarchive) - libarchive wrapper: [docs](http://nimgen.genotrance.com/nimarchive) * [nimarchive](https://github.com/genotrance/nimarchive) - libarchive wrapper: [docs](http://nimgen.genotrance.com/nimarchive)
* git sparse checkout * git sparse checkout
* [nimbigwig](https://github.com/genotrance/nimbigwig) - libbigWig wrapper: [docs](http://nimgen.genotrance.com/nimbigwig) * [nimbigwig](https://github.com/genotrance/nimbigwig) - libbigWig wrapper: [docs](http://nimgen.genotrance.com/nimbigwig)
* git checkout * git checkout
* [nimclipboard](https://github.com/genotrance/nimclipboard) - libclipboard wrapper: [docs](http://nimgen.genotrance.com/nimclipboard) * [nimfuzz](https://github.com/genotrance/nimfuzz) - fts_fuzzy_match wrapper: [docs](http://nimgen.genotrance.com/nimfuzz)
* git checkout
* [nimfastText](https://github.com/genotrance/nimfastText) - fastText wrapper: [docs](http://nimgen.genotrance.com/nimfastText)
* git sparse checkout
* [nimfuzzy](https://github.com/genotrance/nimfuzzy) - fts_fuzzy_match wrapper: [docs](http://nimgen.genotrance.com/nimfuzzy)
* download header file * download header file
* [nimkerberos](https://github.com/genotrance/nimkerberos) - WinKerberos wrapper: [docs](http://nimgen.genotrance.com/nimkerberos) * [nimkerberos](https://github.com/genotrance/nimkerberos) - WinKerberos wrapper: [docs](http://nimgen.genotrance.com/nimkerberos)
* git sparse checkout * git sparse checkout
* [nimmonocypher](https://github.com/genotrance/nimmonocypher) - monocypher wrapper: [docs](http://nimgen.genotrance.com/nimmonocypher)
* git sparse checkout
* [nimnuklear](https://github.com/genotrance/nimnuklear) - nuklear wrapper: [docs](https://nimgen.genotrance.com/nimnuklear)
* git sparse checkout
* [nimpcre](https://github.com/genotrance/nimpcre) - PCRE wrapper: [docs](http://nimgen.genotrance.com/nimpcre) * [nimpcre](https://github.com/genotrance/nimpcre) - PCRE wrapper: [docs](http://nimgen.genotrance.com/nimpcre)
* git checkout * git checkout
* [nimrax](https://github.com/genotrance/nimrax) - Radix tree wrapper: [docs](http://nimgen.genotrance.com/nimrax) * [nimrax](https://github.com/genotrance/nimrax) - Radix tree wrapper: [docs](http://nimgen.genotrance.com/nimrax)
* git checkout * git checkout
* [nimssl](https://github.com/genotrance/nimssl) - OpenSSL wrapper: [docs](http://nimgen.genotrance.com/nimssl) * [nimssl](https://github.com/genotrance/nimssl) - OpenSSL wrapper: [docs](http://nimgen.genotrance.com/nimssl)
* git sparse checkout * git sparse checkout
* [nimtess2](https://github.com/genotrance/nimtess2) - libtess2 wrapper: [docs](http://nimgen.genotrance.com/nimtess2) * [libsvm](https://github.com/genotrance/libsvm) - libsvm wrapper: [docs](http://nimgen.genotrance.com/libsvm)
* git checkout * git sparse checkout
* [duktape-nim](https://github.com/manguluka/duktape-nim) - Duktape wrapper
* static
* Compile in as static binary * Compile in as static binary
* [nimssh2](https://github.com/genotrance/nimssh2) - libssh2 wrapper: [docs](http://nimgen.genotrance.com/nimssh2) * [nimssh2](https://github.com/genotrance/nimssh2) - libssh2 wrapper: [docs](http://nimgen.genotrance.com/nimssh2)
@ -81,25 +73,14 @@ Nimgen only supports the ```gcc``` preprocessor at this time. Support for detect
__Config file__ __Config file__
In all sections below, environment variables are supported via Nim's string interpolation `%` symbol imported from the `strutils` module. Simply use double quotes to enclose any value and put `$` or `${}` around the environment variable name. In addition, the `output` var from the n.global section is available as ${output}. For example:
[n.global]
c_compiler="$CC"
cpp_compiler="${CPP}-arm"
output="src/path"
[n.include]
"${output}/library/include"
"${MY_INCLUDE_PATH}/include"
Append -win, -lin and -mac/osx for OS specific sections and tasks. E.g. n.global-win, n.post-lin, download-win, execute-lin-mac.unique1. -unix can be used as a shortcut for -lin-mac.
_[n.global]_ _[n.global]_
```output``` = name of the Nimble project once installed, also location to place generated .nim files ```output``` = name of the Nimble project once installed, also location to place generated .nim files
```quotes``` = pick up any headers included using "" (and not <> which is typically used for standard headers) [default: true] ```quotes``` = pick up any headers included using "" (and not <> which is typically used for standard headers) [default: true]
```reset``` = whether or not to reset all files after processing [default: false]
```filter``` = string to identify and recurse into library .h files in #include statements and exclude standard headers ```filter``` = string to identify and recurse into library .h files in #include statements and exclude standard headers
```cpp_compiler``` = string to specify a CPP compiler executable. [default: g++] ```cpp_compiler``` = string to specify a CPP compiler executable. [default: g++]
@ -120,21 +101,17 @@ List of all directories or files to exclude from all parsing. If an entry here m
_[n.prepare]_ _[n.prepare]_
The following keys can be used to prepare dependencies such as downloading ZIP files, cloning Git repositories, etc. Multiple entries are possible by appending any .string to the key. E.g. download.file1. The following keys can be used to prepare dependencies such as downloading ZIP files, cloning Git repositories, etc. Multiple entries are possible by appending any .string to the key. E.g. download.file1. -win, -lin and -osx can be used for OS specific tasks. E.g. download-win
```download``` = url to download to the output directory. ZIP files are automatically extracted. Files are not redownloaded if already present but re-extracted ```download``` = url to download to the output directory. ZIP files are automatically extracted. Files are not redownloaded if already present but re-extracted
```extract``` = ZIP file to extract in case they are local and don't need to be downloaded. Path is relative to output directory. ```extract``` = ZIP file to extract in case they are local and don't need to be downloaded. Path is relative to output directory.
```gitcheckout``` = branch, commit or tag of repository to checkout in following Git command, resets after each use. Use "-b name" for branches ```git``` = url of Git repository to clone. Full repo is pulled so gitremote + gitsparse is preferable. Resets to HEAD if already present
```gitoutput``` = directory for all following Git commands relative to `n.global:output` [default: `n.global:output` directory]
```git``` = url of Git repository to clone. Full repo is pulled so gitremote + gitsparse is preferable. Resets if already present
```gitremote``` = url of Git repository to partially checkout. Use with gitsparse to pull only files and dirs of interest ```gitremote``` = url of Git repository to partially checkout. Use with gitsparse to pull only files and dirs of interest
```gitsparse``` = list of files and/or dirs to include in partial checkout, one per line. Resets if already present ```gitsparse``` = list of files and/or dirs to include in partial checkout, one per line. Resets to HEAD if already present
```execute``` = command to run during preparation ```execute``` = command to run during preparation
@ -144,9 +121,7 @@ _[n.post]_
This section is the same as the prepare section, but for performing actions after the project has been processed. This section is the same as the prepare section, but for performing actions after the project has been processed.
```gitoutput``` = output directory for Git reset [default: `n.global:output` directory] ```reset``` = whether or not to perform a git reset on all files after processing [default: false]
```reset``` = perform a Git reset on all files after processing [default: false]
```execute``` = command to run after processing ```execute``` = command to run after processing
@ -156,13 +131,14 @@ File wildcards such as *.nim, ssl*.h, etc. can be used to perform tasks across a
```wildcard``` = pattern to match against. All keys following the wildcard declaration will apply to matched files ```wildcard``` = pattern to match against. All keys following the wildcard declaration will apply to matched files
_[n.sourcefile]_ _[n.after]_
The same as n.wildcard, but instead of operating on the files before nimgen is done generating, it operates after, so you can modify Nim files or other files that result from nimgen or c2nim.
This section allows selection of multiple sourcefiles without requiring a detailed section for each file. Each specific file can be listed one line at a time and file wildcards can be used to include multiple source files. E.g. `$output/include/*/h`. `[n.wildcard]` definitions can be used to perform common operations on these source files if required. ```wildcard``` = pattern to match against. All keys following the wildcard declaration will apply to matched files
_[sourcefile]_ _[sourcefile]_
The following keys apply to library source code and help with generating the .nim files. -win, -lin and -osx can be used for OS specific tasks. E.g. dynlib-win, pragma-win. The following keys apply to library source code and help with generating the .nim files. -win, -lin and -osx can be used for OS specific tasks. E.g. dynlib-win, pragma-win
```recurse``` = find #include files and process them [default: false] ```recurse``` = find #include files and process them [default: false]
@ -174,19 +150,17 @@ The following keys apply to library source code and help with generating the .ni
```defines``` = pulls out simple #defines of ints, floats and hex values for separate conversion [default: false] - works only when preprocess or ctags is used and helps include useful definitions in generated .nim file ```defines``` = pulls out simple #defines of ints, floats and hex values for separate conversion [default: false] - works only when preprocess or ctags is used and helps include useful definitions in generated .nim file
```remove_static``` = pulls out the bodies of inline static functions [default: false]
```flags``` = flags to pass to the c2nim process in "quotes" [default: --stdcall]. --cdecl, --assumedef, --assumendef may be useful ```flags``` = flags to pass to the c2nim process in "quotes" [default: --stdcall]. --cdecl, --assumedef, --assumendef may be useful
```ppflags``` = flags to pass to the preprocessor [default: ""]. -D for gcc and others may be useful ```ppflags``` = flags to pass to the preprocessor [default: ""]. -D for gcc and others may be useful
```noprocess``` = do not process this source file with c2nim [default: false] - this is useful if a file only needs to be manipulated ```noprocess``` = do not process this source file with c2nim [default: false] - this is useful if a file only needs to be manipulated
```nowildcard``` = ignore any wildcard definitions for this sourcefile
```reset``` = reset the file back to original state after all processing [default: false]
Multiple entries for the all following keys are possible by appending any .string to the key. E.g. dynlib.win, compile.dir Multiple entries for the all following keys are possible by appending any .string to the key. E.g. dynlib.win, compile.dir
```compile``` = file or dir of files of source code to {.compile.} into generated .nim. If directory, picks *.c if C mode and *.cxx, *.cpp, *.cc, *.c++ and *.C for cpp mode. Dir can also include wildcards. e.g. compile = """dir/A*.cxx""" ```compile``` = file or dir of files of source code to {.compile.} into generated .nim
```pragma``` = pragmas to define in generated .nim file. E.g. pragma = "passL: \"-lssl\"" => {.passL: "-lssl".} ```pragma``` = pragmas to define in generated .nim file. E.g. pragma = "passL: \"-lssl\"" => {.passL: "-lssl".}
@ -196,19 +170,15 @@ The following keys apply to library source code (before processing) and generate
```create``` = create a file at exact location with contents specified. File needs to be in the _[n.exclude]_ list in order to be created. ```create``` = create a file at exact location with contents specified. File needs to be in the _[n.exclude]_ list in order to be created.
```pipe``` = execute a command on a file and store the output of the command as the new file contents. E.g. pipe = "cat $file | grep 'static inline'"
```search``` = search string providing context for following prepend/append/replace directives ```search``` = search string providing context for following prepend/append/replace directives
```regex``` = regex search string providing context for the following replace directive. Specify using """ to avoid regex parsing issues ```pipe``` = execute a command on a file and store the output of the command as the new file contents. Ex: pipe = "cat $file | grep 'static inline'"
```prepend``` = string value to prepend into file at beginning or before search ```prepend``` = string value to prepend into file at beginning or before search
```append``` = string value to append into file at the end or after search ```append``` = string value to append into file at the end or after search
```replace``` = string value to replace search string in file. Regex captures can be referred to using $1, $2, etc. ```replace``` = string value to replace search string in file
```move``` = search string providing context for location to move the results of a preceding search or regex match
```comment``` = number of lines to comment from search location ```comment``` = number of lines to comment from search location
@ -222,47 +192,6 @@ The following key only applies before processing and allows renaming the generat
`$replace(srch1=repl1, srch2=reply2)` = rename specific portions in `$nimout` `$replace(srch1=repl1, srch2=reply2)` = rename specific portions in `$nimout`
__Command Line__
A subset of capabilities are available through the command line to enable quick tests using nimgen. Command line flags only apply to source files specified on the command line and do not influence any ```cfg``` files which are expected to be self-sufficient.
```
Usage:
nimgen [options] file.cfg|file.h ...
Params:
-C<compile> add compile entry *
-E<exclude> add n.exclude entry *
-F<flags> set c2nim flags *
-I<include> add n.include dir *
-O<outdir> set output directory
-P<ppflags> set preprocessor flags *
Options:
-c set ctags = true
-d set defines = true
-i set inline = true
-n set noprocess = true
-p set preprocess = true
-r set recurse = true
Editing:
-a<append> append string *
-e<prepend> prepend string *
-l<replace> replace string *
-o#lines comment X lines *
-s<search> search string *
-x<regex> regex search string *
* supports multiple instances
```
__Feedback__ __Feedback__
Nimgen is a work in progress and any feedback or suggestions are welcome. It is hosted on [GitHub](https://github.com/genotrance/nimgen) with an MIT license so issues, forks and PRs are most appreciated. Also join us at https://gitter.im/nimgen/Lobby to chat about nimgen and the future of Nim wrappers. Nimgen is a work in progress and any feedback or suggestions are welcome. It is hosted on [GitHub](https://github.com/genotrance/nimgen) with an MIT license so issues, forks and PRs are most appreciated.
_Credits_
Thank you to the following contributors for their hard work!
https://github.com/jyapayne

View file

@ -1,103 +0,0 @@
version: '{build}'
image:
- Ubuntu
- Visual Studio 2017
matrix:
fast_finish: true
environment:
matrix:
- NIM_VERSION: 0.20.0
- NIM_VERSION: 0.19.6
for:
-
matrix:
only:
- image: Visual Studio 2017
environment:
ARCH: 32
MINGW_URL: https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/8.1.0/threads-posix/dwarf
MINGW_ARCHIVE: i686-8.1.0-release-posix-dwarf-rt_v6-rev0.7z
SFNET_URL: https://sourceforge.net/projects/msys2/files/REPOS/MINGW/i686
LIBSSH2_ARCHIVE: mingw-w64-i686-libssh2-1.8.0-1-any.pkg
LIBCRYPTO_ARCHIVE: mingw-w64-i686-openssl-1.0.2.o-1-any.pkg
install:
- CD c:\
- IF not exist "binaries" (
echo %NIM_VERSION% &&
MKDIR binaries &&
CD binaries &&
appveyor DownloadFile "%MINGW_URL%/%MINGW_ARCHIVE%/download" -FileName "%MINGW_ARCHIVE%" &&
7z x -y "%MINGW_ARCHIVE%"> nul &&
del "%MINGW_ARCHIVE%" &&
appveyor DownloadFile "%SFNET_URL%/%LIBSSH2_ARCHIVE%.tar.xz/download" -FileName "%LIBSSH2_ARCHIVE%.tar.xz" &&
7z x -y "%LIBSSH2_ARCHIVE%.tar.xz"> nul &&
del "%LIBSSH2_ARCHIVE%.tar.xz" &&
7z x -y "%LIBSSH2_ARCHIVE%.tar"> nul &&
del "%LIBSSH2_ARCHIVE%.tar" &&
appveyor DownloadFile "%SFNET_URL%/%LIBCRYPTO_ARCHIVE%.tar.xz/download" -FileName "%LIBCRYPTO_ARCHIVE%.tar.xz" &&
7z x -y "%LIBCRYPTO_ARCHIVE%.tar.xz"> nul &&
del "%LIBCRYPTO_ARCHIVE%.tar.xz" &&
7z x -y "%LIBCRYPTO_ARCHIVE%.tar"> nul &&
del "%LIBCRYPTO_ARCHIVE%.tar" &&
appveyor DownloadFile "https://nim-lang.org/download/nim-%NIM_VERSION%_x%ARCH%.zip" -FileName "nim-%NIM_VERSION%_x%ARCH%.zip" &&
7z x -y "nim-%NIM_VERSION%_x%ARCH%.zip"> nul &&
del "nim-%NIM_VERSION%_x%ARCH%.zip")
- SET PATH=c:\binaries\mingw%ARCH%\bin;c:\binaries\nim-%NIM_VERSION%\bin;%USERPROFILE%\.nimble\bin;%PATH%
- CD c:\projects\nimgen
on_finish:
- 7z a -r buildlogs-win-pkgs.zip %USERPROFILE%\.nimble\pkgs
- appveyor PushArtifact buildlogs-win-pkgs.zip
- 7z a -r buildlogs-win-projects.zip c:\projects\*
- appveyor PushArtifact buildlogs-win-projects.zip
- 7z a -r nimgen-docs.zip c:\projects\nimgen\web
- appveyor PushArtifact nimgen-docs.zip
cache:
- c:\binaries
-
matrix:
only:
- image: Ubuntu
install:
- sudo apt -qq update
- sudo apt -qq install --yes libssh2-1-dev libgcrypt20-dev libgpg-error-dev
- if [ ! -e /home/appveyor/binaries ]; then
echo $NIM_VERSION &&
mkdir /home/appveyor/binaries &&
cd /home/appveyor/binaries &&
curl -s -o nim-$NIM_VERSION.tar.xz https://nim-lang.org/download/nim-$NIM_VERSION.tar.xz &&
tar xJf nim-$NIM_VERSION.tar.xz &&
cd nim-$NIM_VERSION &&
sh build.sh &&
bin/nim c -d:release koch &&
./koch boot -d:release &&
./koch nimble -d:release;
fi
- export PATH=/home/appveyor/binaries/nim-$NIM_VERSION/bin:~/.nimble/bin:$PATH
- cd /home/appveyor/projects/nimgen
on_finish:
- zip -r -q buildlogs-lin-pkgs.zip ~/.nimble/pkgs
- appveyor PushArtifact buildlogs-lin-pkgs.zip
- zip -r -q buildlogs-lin-projects.zip /home/appveyor/projects
- appveyor PushArtifact buildlogs-lin-projects.zip
cache:
- /home/appveyor/binaries
build_script:
- nimble install -y
test_script:
- nimble test
deploy: off

908
nimgen.nim Normal file
View file

@ -0,0 +1,908 @@
import nre, os, ospaths, osproc, parsecfg, pegs, ropes, sequtils, streams, strutils, tables
var
gDoneRecursive: seq[string] = @[]
gDoneAfter: seq[string] = @[]
gDoneInline: seq[string] = @[]
gProjectDir = ""
gConfig: Config
gFilter = ""
gQuotes = true
gReset = false
gCppCompiler = "g++"
gCCompiler = "gcc"
gOutput = ""
gIncludes: seq[string] = @[]
gExcludes: seq[string] = @[]
gRenames = initTable[string, string]()
gWildcards = newConfig()
gAfter = newConfig()
type
c2nimConfigObj = object
flags, ppflags: string
recurse, inline, preprocess, ctags, defines: bool
dynlib, compile, pragma: seq[string]
const DOC = """
Nimgen is a helper for c2nim to simpilfy and automate the wrapping of C libraries
Usage:
nimgen [options] <file.cfg>...
Options:
-f delete all artifacts and regenerate
"""
# ###
# Helpers
proc addEnv(str: string): string =
var newStr = str
for pair in envPairs():
try:
newStr = newStr % [pair.key, pair.value.string]
except ValueError:
# Ignore if there are no values to replace. We
# want to continue anyway
discard
try:
newStr = newStr % ["output", gOutput]
except ValueError:
# Ignore if there are no values to replace. We
# want to continue anyway
discard
# if there are still format args, print a warning
if newStr.contains("${"):
echo "WARNING: \"", newStr, "\" still contains an uninterpolated value!"
return newStr
proc `[]`(table: OrderedTableRef[string, string], key: string): string =
## Gets table values with env vars inserted
tables.`[]`(table, key).addEnv
proc execProc(cmd: string): string =
result = ""
var
p = startProcess(cmd, options = {poStdErrToStdOut, poUsePath, poEvalCommand})
outp = outputStream(p)
line = newStringOfCap(120).TaintedString
while true:
if outp.readLine(line):
result.add(line)
result.add("\n")
elif not running(p): break
var x = p.peekExitCode()
if x != 0:
raise newException(
Exception,
"Command failed: " & $x &
"\nCMD: " & $cmd &
"\nRESULT: " & $result
)
proc extractZip(zipfile: string) =
var cmd = "unzip -o $#"
if defined(Windows):
cmd = "powershell -nologo -noprofile -command \"& { Add-Type -A " &
"'System.IO.Compression.FileSystem'; " &
"[IO.Compression.ZipFile]::ExtractToDirectory('$#', '.'); }\""
setCurrentDir(gOutput)
defer: setCurrentDir(gProjectDir)
echo "Extracting " & zipfile
discard execProc(cmd % zipfile)
proc downloadUrl(url: string) =
let
file = url.extractFilename()
ext = file.splitFile().ext.toLowerAscii()
var cmd = "curl $# -o $#"
if defined(Windows):
cmd = "powershell wget $# -OutFile $#"
if not (ext == ".zip" and fileExists(gOutput/file)):
echo "Downloading " & file
discard execProc(cmd % [url, gOutput/file])
if ext == ".zip":
extractZip(file)
proc gitReset() =
echo "Resetting Git repo"
setCurrentDir(gOutput)
defer: setCurrentDir(gProjectDir)
discard execProc("git reset --hard HEAD")
proc gitCheckout(filename: string) {.used.} =
echo "Resetting file: $#" % [filename]
setCurrentDir(gOutput)
defer: setCurrentDir(gProjectDir)
let adjustedFile = filename.replace(gOutput & $DirSep, "")
discard execProc("git checkout $#" % [adjustedFile])
proc gitRemotePull(url: string, pull=true) =
if dirExists(gOutput/".git"):
if pull:
gitReset()
return
setCurrentDir(gOutput)
defer: setCurrentDir(gProjectDir)
echo "Setting up Git repo"
discard execProc("git init .")
discard execProc("git remote add origin " & url)
if pull:
echo "Checking out artifacts"
discard execProc("git pull --depth=1 origin master")
proc gitSparseCheckout(plist: string) =
let sparsefile = ".git/info/sparse-checkout"
if fileExists(gOutput/sparsefile):
gitReset()
setCurrentDir(gOutput)
defer: setCurrentDir(gProjectDir)
discard execProc("git config core.sparsecheckout true")
writeFile(sparsefile, plist)
echo "Checking out artifacts"
discard execProc("git pull --depth=1 origin master")
proc doCopy(flist: string) =
for pair in flist.split(","):
let spl = pair.split("=")
if spl.len() != 2:
raise newException(Exception, "Bad copy syntax: " & flist)
let
lfile = spl[0].strip()
rfile = spl[1].strip()
copyFile(lfile, rfile)
echo "Copied $# to $#" % [lfile, rfile]
proc getKey(ukey: string): tuple[key: string, val: bool] =
var kv = ukey.replace(re"\..*", "").split("-", 1)
if kv.len() == 1:
kv.add("")
if (kv[1] == "") or
(kv[1] == "win" and defined(Windows)) or
(kv[1] == "lin" and defined(Linux)) or
(kv[1] == "osx" and defined(MacOSX)):
return (kv[0], true)
return (kv[0], false)
# ###
# File loction
proc getNimout(file: string, rename=true): string =
if file == "":
return ""
result = file.splitFile().name.replace(re"[\-\.]", "_") & ".nim"
if gOutput != "":
result = gOutput/result
if not rename:
return
if gRenames.hasKey(file):
result = gRenames[file]
if not dirExists(parentDir(result)):
createDir(parentDir(result))
proc exclude(file: string): bool =
for excl in gExcludes:
if excl in file:
return true
return false
proc search(file: string): string =
if exclude(file):
return ""
result = file
if file.splitFile().ext == ".nim":
result = getNimout(file)
elif not fileExists(result) and not dirExists(result):
var found = false
for inc in gIncludes:
result = inc/file
if fileExists(result) or dirExists(result):
found = true
break
if not found:
raise newException(Exception, "File doesn't exist: " & file)
# Only keep relative directory
result = result.replace(gProjectDir & $DirSep, "")
return result.replace(re"[\\/]", $DirSep)
# ###
# Loading / unloading
template withFile(file: string, body: untyped): untyped =
if fileExists(file):
var f: File
while true:
try:
f = open(file)
break
except:
sleep(100)
var contentOrig = f.readAll()
f.close()
var content {.inject.} = contentOrig
body
if content != contentOrig:
var f = open(file, fmWrite)
write(f, content)
f.close()
else:
echo "Missing file " & file
# ###
# Manipulating content
proc prepend(file: string, data: string, search="") =
withFile(file):
if search == "":
content = data & content
else:
let idx = content.find(search)
if idx != -1:
content = content[0..<idx] & data & content[idx..<content.len()]
proc pipe(file: string, command: string) =
let cmd = command % ["file", file]
let commandResult = execProc(cmd).strip()
if commandResult != "":
withFile(file):
content = commandResult
proc append(file: string, data: string, search="") =
withFile(file):
if search == "":
content &= data
else:
let idx = content.find(search)
let idy = idx + search.len()
if idx != -1:
content = content[0..<idy] & data & content[idy..<content.len()]
proc freplace(file: string, pattern: string, repl="") =
withFile(file):
if pattern in content:
content = content.replace(pattern, repl)
proc freplace(file: string, pattern: Regex, repl="") =
withFile(file):
if content.find(pattern).isSome():
if "$#" in repl:
for m in content.findIter(pattern):
content = content.replace(m.match, repl % m.captures[0])
else:
content = content.replace(pattern, repl)
proc comment(file: string, pattern: string, numlines: string) =
let
ext = file.splitFile().ext.toLowerAscii()
cmtchar = if ext == ".nim": "#" else: "//"
withFile(file):
var
idx = content.find(pattern)
num = 0
try:
num = numlines.parseInt()
except ValueError:
echo "Bad comment value, should be integer: " & numlines
if idx != -1:
for i in 0 .. num-1:
if idx >= content.len():
break
content = content[0..<idx] & cmtchar & content[idx..<content.len()]
while idx < content.len():
idx += 1
if content[idx] == '\L':
idx += 1
break
proc removeStatic(filename: string) =
## Replace static function bodies with a semicolon and commented
## out body
withFile(filename):
content = content.replace(
re"(?m)(static inline.*?\))(\s*\{(\s*?.*?$)*[\n\r]\})",
proc (match: RegexMatch): string =
let funcDecl = match.captures[0]
let body = match.captures[1].strip()
result = ""
result.add("$#;" % [funcDecl])
result.add(body.replace(re"(?m)^", "//"))
)
proc reAddStatic(filename: string) =
## Uncomment out the body and remove the semicolon. Undoes
## removeStatic
withFile(filename):
content = content.replace(
re"(?m)(static inline.*?\));(\/\/\s*\{(\s*?.*?$)*[\n\r]\/\/\})",
proc (match: RegexMatch): string =
let funcDecl = match.captures[0]
let body = match.captures[1].strip()
result = ""
result.add("$# " % [funcDecl])
result.add(body.replace(re"(?m)^\/\/", ""))
)
proc rename(file: string, renfile: string) =
if file.splitFile().ext == ".nim":
return
var
nimout = getNimout(file, false)
newname = renfile.replace("$nimout", extractFilename(nimout))
if newname =~ peg"(!\$.)*{'$replace'\s*'('\s*{(!\)\S)+}')'}":
var final = nimout.extractFilename()
for entry in matches[1].split(","):
let spl = entry.split("=")
if spl.len() != 2:
raise newException(Exception, "Bad replace syntax: " & renfile)
let
srch = spl[0].strip()
repl = spl[1].strip()
final = final.replace(srch, repl)
newname = newname.replace(matches[0], final)
gRenames[file] = gOutput/newname
proc compile(dir="", file=""): string =
proc fcompile(file: string): string =
return "{.compile: \"$#\".}" % file.replace("\\", "/")
var data = ""
if dir != "" and dirExists(dir):
for f in walkFiles(dir / "*.c"):
data &= fcompile(f) & "\n"
if file != "" and fileExists(file):
data &= fcompile(file) & "\n"
return data
proc fixFuncProtos(file: string) =
withFile(file):
for fp in content.findIter(re"(?m)(^.*?)[ ]*\(\*(.*?)\((.*?)\)\)[ \r\n]*\((.*?[\r\n]*.*?)\);"):
var tdout = "typedef $# (*type_$#)($#);\n" % [fp.captures[0], fp.captures[1], fp.captures[3]] &
"type_$# $#($#);" % [fp.captures[1], fp.captures[1], fp.captures[2]]
content = content.replace(fp.match, tdout)
# ###
# Convert to Nim
proc getIncls(file: string, inline=false): seq[string] =
result = @[]
if inline and file in gDoneInline:
return
var curPath = splitFile(expandFileName(file))[0]
withFile(file):
for f in content.findIter(re"(?m)^\s*#\s*include\s+(.*?)$"):
var inc = f.captures[0].strip()
if ((gQuotes and inc.contains("\"")) or (gFilter != "" and gFilter in inc)) and (not exclude(inc)):
var addInc = inc.replace(re"""[<>"]""", "").replace(re"\/[\*\/].*$", "").strip()
try:
# Try searching for a local library
let finc = expandFileName(curPath & "/" & addInc)
let fname = extractFileName(finc)
result.add(fname.search())
except OSError:
# If it's a system library
result.add(addInc)
result = result.deduplicate()
gDoneInline.add(file)
if inline:
var sres = newSeq[string]()
for incl in result:
let sincl = search(incl)
if sincl == "":
continue
sres.add(getIncls(sincl, inline))
result.add(sres)
result = result.deduplicate()
proc getDefines(file: string, inline=false): string =
result = ""
if inline:
var incls = getIncls(file, inline)
for incl in incls:
let sincl = search(incl)
if sincl != "":
echo "Inlining " & sincl
result &= getDefines(sincl)
withFile(file):
for def in content.findIter(re"(?m)^(\s*#\s*define\s+[\w\d_]+\s+[\d\-.xf]+)(?:\r|//|/*).*?$"):
result &= def.captures[0] & "\n"
proc runPreprocess(file, ppflags, flags: string, inline: bool): string =
var
pproc = if flags.contains("cpp"): gCppCompiler else: gCCompiler
cmd = "$# -E $# $#" % [pproc, ppflags, file]
for inc in gIncludes:
cmd &= " -I " & inc
# Run preprocessor
var data = execProc(cmd)
# Include content only from file
var
rdata: Rope
start = false
sfile = file.replace("\\", "/")
if inline:
sfile = sfile.parentDir()
for line in data.splitLines():
if line.strip() != "":
if line[0] == '#' and not line.contains("#pragma"):
start = false
if sfile in line.replace("\\", "/").replace("//", "/"):
start = true
if not ("\\" in line) and not ("/" in line) and extractFilename(sfile) in line:
start = true
else:
if start:
rdata.add(
line.replace("_Noreturn", "")
.replace("(())", "")
.replace("WINAPI", "")
.replace("__attribute__", "")
.replace("extern \"C\"", "")
.replace(re"\(\([_a-z]+?\)\)", "")
.replace(re"\(\(__format__[\s]*\(__[gnu_]*printf__, [\d]+, [\d]+\)\)\);", ";") & "\n"
)
return $rdata
proc runCtags(file: string): string =
var
cmd = "ctags -o - --fields=+S+K --c-kinds=+p --file-scope=no " & file
fps = execProc(cmd)
fdata = ""
for line in fps.splitLines():
var spl = line.split(re"\t")
if spl.len() > 4:
if spl[0] != "main":
var fn = ""
var match = spl[2].find(re"/\^(.*?)\(")
if match.isSome():
fn = match.get().captures[0]
fn &= spl[4].replace("signature:", "") & ";"
fdata &= fn & "\n"
return fdata
proc runFile(file: string, cfgin: OrderedTableRef)
proc c2nim(fl, outfile: string, c2nimConfig: c2nimConfigObj) =
var file = search(fl)
if file == "":
return
if file in gDoneRecursive:
return
echo "Processing $# => $#" % [file, outfile]
gDoneRecursive.add(file)
fixFuncProtos(file)
var incout = ""
if c2nimConfig.recurse:
var
incls = getIncls(file)
cfg = newOrderedTable[string, string]()
for name, value in c2nimConfig.fieldPairs:
when value is string:
cfg[name] = value
when value is bool:
cfg[name] = $value
for i in c2nimConfig.dynlib:
cfg["dynlib." & i] = i
for inc in incls:
runFile(inc, cfg)
let nimout = inc.search().getNimout()
if nimout.len() > 0:
incout &= "import $#\n" % nimout[0 .. ^5]
var cfile = file
if c2nimConfig.preprocess:
cfile = "temp-$#.c" % [outfile.extractFilename()]
writeFile(cfile, runPreprocess(file, c2nimConfig.ppflags, c2nimConfig.flags, c2nimConfig.inline))
elif c2nimConfig.ctags:
cfile = "temp-$#.c" % [outfile.extractFilename()]
writeFile(cfile, runCtags(file))
if c2nimConfig.defines and (c2nimConfig.preprocess or c2nimConfig.ctags):
prepend(cfile, getDefines(file, c2nimConfig.inline))
var
extflags = ""
passC = ""
outlib = ""
outpragma = ""
passC = "import strutils\n"
passC &= "import ospaths\n"
for inc in gIncludes:
if inc.isAbsolute:
passC &= ("""{.passC: "-I\"$#\"".}""" % [inc]) & "\n"
else:
let relativeInc = inc.replace(gOutput, "")
passC &= (
"""{.passC: "-I\"" & currentSourcePath().splitPath().head & "/$#\"".}""" %
[relativeInc]
) & "\n"
for prag in c2nimConfig.pragma:
outpragma &= "{." & prag & ".}\n"
let fname = file.splitFile().name.replace(re"[\.\-]", "_")
let fincl = file.replace(gOutput, "")
if c2nimConfig.dynlib.len() != 0:
let
win = "when defined(Windows):\n"
lin = "when defined(Linux):\n"
osx = "when defined(MacOSX):\n"
var winlib, linlib, osxlib: string = ""
for dl in c2nimConfig.dynlib:
let
lib = " const dynlib$# = \"$#\"\n" % [fname, dl]
ext = dl.splitFile().ext
if ext == ".dll":
winlib &= lib
elif ext == ".so":
linlib &= lib
elif ext == ".dylib":
osxlib &= lib
if winlib != "":
outlib &= win & winlib & "\n"
if linlib != "":
outlib &= lin & linlib & "\n"
if osxlib != "":
outlib &= osx & osxlib & "\n"
if outlib != "":
extflags &= " --dynlib:dynlib$#" % fname
else:
if file.isAbsolute():
passC &= "const header$# = \"$#\"\n" % [fname, fincl]
else:
passC &= "const header$# = currentSourcePath().splitPath().head & \"$#\"\n" % [fname, fincl]
extflags = "--header:header$#" % fname
# Run c2nim on generated file
var cmd = "c2nim $# $# --out:$# $#" % [c2nimConfig.flags, extflags, outfile, cfile]
when defined(windows):
cmd = "cmd /c " & cmd
discard execProc(cmd)
if c2nimConfig.preprocess or c2nimConfig.ctags:
try:
removeFile(cfile)
except:
discard
# Import nim modules
if c2nimConfig.recurse:
prepend(outfile, incout)
# Nim doesn't like {.cdecl.} for type proc()
freplace(outfile, re"(?m)(.*? = proc.*?){.cdecl.}", "$#")
freplace(outfile, " {.cdecl.})", ")")
# Include {.compile.} directives
for cpl in c2nimConfig.compile:
let fcpl = search(cpl)
if getFileInfo(fcpl).kind == pcFile:
prepend(outfile, compile(file=fcpl))
else:
prepend(outfile, compile(dir=fcpl))
# Add any pragmas
if outpragma != "":
prepend(outfile, outpragma)
# Add header file and include paths
if passC != "":
prepend(outfile, passC)
# Add dynamic library
if outlib != "":
prepend(outfile, outlib)
proc doActions(file: string, c2nimConfig: var c2nimConfigObj, cfg: OrderedTableRef) =
var
srch = ""
sfile = search(file)
for act in cfg.keys():
let (action, val) = getKey(act)
if val == true:
if action == "create":
createDir(file.splitPath().head)
writeFile(file, cfg[act])
elif action in @["prepend", "append", "replace", "comment",
"rename", "compile", "dynlib", "pragma",
"pipe"] and sfile != "":
if action == "prepend":
if srch != "":
prepend(sfile, cfg[act], cfg[srch])
else:
prepend(sfile, cfg[act])
elif action == "append":
if srch != "":
append(sfile, cfg[act], cfg[srch])
else:
append(sfile, cfg[act])
elif action == "replace":
if srch != "":
freplace(sfile, cfg[srch], cfg[act])
elif action == "comment":
if srch != "":
comment(sfile, cfg[srch], cfg[act])
elif action == "rename":
rename(sfile, cfg[act])
elif action == "compile":
c2nimConfig.compile.add(cfg[act])
elif action == "dynlib":
c2nimConfig.dynlib.add(cfg[act])
elif action == "pragma":
c2nimConfig.pragma.add(cfg[act])
elif action == "pipe":
pipe(sfile, cfg[act])
srch = ""
elif action == "search":
srch = act
proc processAfter(nimFile: string, c2nimConfig: var c2nimConfigObj) =
var file = search(nimFile)
if file == "":
return
if file in gDoneAfter:
return
gDoneAfter.add(file)
var afterConfig = newOrderedTable[string, string]()
for pattern in gAfter.keys():
let pat = pattern.replace(".", "\\.").replace("*", ".*").replace("?", ".?")
if nimFile.find(re(pat)).isSome():
for key in gAfter[pattern].keys():
let value = gAfter[pattern][key]
afterConfig[key & "." & pattern] = value
doActions(nimFile, c2nimConfig, afterConfig)
# ###
# Processor
proc runFile(file: string, cfgin: OrderedTableRef) =
var
cfg = cfgin
sfile = search(file)
for pattern in gWildcards.keys():
let pat = pattern.replace(".", "\\.").replace("*", ".*").replace("?", ".?")
if file.find(re(pat)).isSome():
echo "Appending " & file & " " & pattern
for key in gWildcards[pattern].keys():
cfg[key & "." & pattern] = gWildcards[pattern][key]
var
c2nimConfig = c2nimConfigObj(
flags: "--stdcall", ppflags: "",
recurse: false, inline: false, preprocess: false, ctags: false, defines: false,
dynlib: @[], compile: @[], pragma: @[]
)
doActions(file, c2nimConfig, cfg)
if file.splitFile().ext != ".nim":
var noprocess = false
for act in cfg.keys():
let (action, val) = getKey(act)
if val == true:
if cfg[act] == "true":
if action == "recurse":
c2nimConfig.recurse = true
elif action == "inline":
c2nimConfig.inline = true
elif action == "preprocess":
c2nimConfig.preprocess = true
elif action == "ctags":
c2nimConfig.ctags = true
elif action == "defines":
c2nimConfig.defines = true
elif action == "noprocess":
noprocess = true
elif action == "flags":
c2nimConfig.flags = cfg[act]
elif action == "ppflags":
c2nimConfig.ppflags = cfg[act]
if c2nimConfig.recurse and c2nimConfig.inline:
echo "Cannot use recurse and inline simultaneously"
quit(1)
# Remove static inline function bodies
removeStatic(sfile)
if not noprocess:
let nimFile = getNimout(sfile)
c2nim(file, nimFile, c2nimConfig)
processAfter(nimFile, c2nimConfig)
# Add them back for compilation
reAddStatic(sfile)
proc runCfg(cfg: string) =
if not fileExists(cfg):
raise newException(Exception, "Config doesn't exist: " & cfg)
gProjectDir = parentDir(cfg.expandFilename())
gConfig = loadConfig(cfg)
if gConfig.hasKey("n.global"):
if gConfig["n.global"].hasKey("output"):
gOutput = gConfig["n.global"]["output"]
if dirExists(gOutput):
if "-f" in commandLineParams():
try:
removeDir(gOutput)
except OSError:
raise newException(Exception, "Directory in use: " & gOutput)
else:
for f in walkFiles(gOutput/"*.nim"):
try:
removeFile(f)
except OSError:
raise newException(Exception, "Unable to delete: " & f)
createDir(gOutput)
if gConfig["n.global"].hasKey("cpp_compiler"):
gCppCompiler = gConfig["n.global"]["cpp_compiler"]
if gConfig["n.global"].hasKey("c_compiler"):
gCCompiler = gConfig["n.global"]["c_compiler"]
if gConfig["n.global"].hasKey("filter"):
gFilter = gConfig["n.global"]["filter"]
if gConfig["n.global"].hasKey("quotes"):
if gConfig["n.global"]["quotes"] == "false":
gQuotes = false
if gConfig["n.global"].hasKey("reset"):
if gConfig["n.global"]["reset"] == "true":
gReset = true
if gConfig.hasKey("n.include"):
for inc in gConfig["n.include"].keys():
gIncludes.add(inc.addEnv())
if gConfig.hasKey("n.exclude"):
for excl in gConfig["n.exclude"].keys():
gExcludes.add(excl.addEnv())
if gConfig.hasKey("n.prepare"):
for prep in gConfig["n.prepare"].keys():
let (key, val) = getKey(prep)
if val == true:
let prepVal = gConfig["n.prepare"][prep]
if key == "download":
downloadUrl(prepVal)
elif key == "extract":
extractZip(prepVal)
elif key == "git":
gitRemotePull(prepVal)
elif key == "gitremote":
gitRemotePull(prepVal, false)
elif key == "gitsparse":
gitSparseCheckout(prepVal)
elif key == "execute":
discard execProc(prepVal)
elif key == "copy":
doCopy(prepVal)
if gConfig.hasKey("n.wildcard"):
var wildcard = ""
for wild in gConfig["n.wildcard"].keys():
let (key, val) = getKey(wild)
if val == true:
if key == "wildcard":
wildcard = gConfig["n.wildcard"][wild]
else:
gWildcards.setSectionKey(wildcard, wild,
gConfig["n.wildcard"][wild])
if gConfig.hasKey("n.after"):
var wildcard = ""
for afterKey in gConfig["n.after"].keys():
let (key, val) = getKey(afterKey)
if val == true:
if key == "wildcard":
wildcard = gConfig["n.after"][afterKey]
else:
gAfter.setSectionKey(wildcard, afterKey,
gConfig["n.after"][afterKey])
for file in gConfig.keys():
if file in @["n.global", "n.include", "n.exclude",
"n.prepare", "n.wildcard", "n.after", "n.post"]:
continue
runFile(file, gConfig[file])
if gConfig.hasKey("n.post"):
for post in gConfig["n.post"].keys():
let (key, val) = getKey(post)
if val == true:
let postVal = gConfig["n.post"][post]
if key == "reset":
gitReset()
elif key == "execute":
discard execProc(postVal)
# ###
# Main loop
for i in commandLineParams():
if i != "-f":
runCfg(i)

View file

@ -1,18 +1,17 @@
# Package # Package
version = "0.5.1" version = "0.2.3"
author = "genotrance" author = "genotrance"
description = "c2nim helper to simplify and automate the wrapping of C libraries" description = "c2nim helper to simplify and automate the wrapping of C libraries"
license = "MIT" license = "MIT"
bin = @["nimgen"] skipDirs = @["tests"]
srcDir = "src"
skipDirs = @["nimgen", "tests", "web"]
# Dependencies # Dependencies
requires "nim >= 0.19.0", "c2nim >= 0.9.14", "regex >= 0.10.0" requires "nim >= 0.17.2", "c2nim >= 0.9.13"
bin = @["nimgen"]
task test, "Test nimgen": task test, "Test nimgen":
exec "nim c -r tests/rununittests.nim"
exec "nim e tests/nimgentest.nims" exec "nim e tests/nimgentest.nims"

View file

@ -1,5 +0,0 @@
import os
import nimgen/runcfg
runCli()

View file

@ -1,131 +0,0 @@
import os, regex, strutils
when (NimMajor, NimMinor, NimPatch) < (0, 19, 9):
import ospaths
import external, file, fileops, gencore, globals
proc relativePath(path: string): string =
if gOutput.len() == 0:
result = path
else:
# multiReplace() bug - #9557
result = path.replace(gOutput, "")
return result.multiReplace([("\\", "/"), ("//", "/")])
proc c2nim*(fl, outfile: string, c2nimConfig: c2nimConfigObj) =
var file = search(fl)
if file.len() == 0:
return
echo "Generating " & outfile
var cfile = file
if c2nimConfig.preprocess:
cfile = "temp-$#.c" % [outfile.extractFilename()]
writeFileFlush(cfile, runPreprocess(file, c2nimConfig.ppflags, c2nimConfig.flags, c2nimConfig.inline))
elif c2nimConfig.ctags:
cfile = "temp-$#.c" % [outfile.extractFilename()]
writeFileFlush(cfile, runCtags(file))
if c2nimConfig.removeBodies:
removeBodies(cfile)
if c2nimConfig.defines and (c2nimConfig.preprocess or c2nimConfig.ctags):
prepend(cfile, getDefines(file, c2nimConfig.inline))
removeStatic(cfile)
var
extflags = ""
passC = ""
outlib = ""
outpragma = ""
passC = "import strutils\n"
passC &= """const sourcePath = currentSourcePath().split({'\\', '/'})[0..^2].join("/")""" & "\n"
for inc in gIncludes:
if inc.isAbsolute():
passC &= ("""{.passC: "-I\"$#\"".}""" % [inc.sanitizePath()]) & "\n"
else:
passC &= (
"""{.passC: "-I\"" & sourcePath & "$#\"".}""" %
inc.relativePath()
) & "\n"
for prag in c2nimConfig.pragma:
outpragma &= "{." & prag & ".}\n"
let fname = file.splitFile().name.multiReplace([(".", "_"), ("-", "_")])
if c2nimConfig.dynlib.len() != 0:
let
win = "when defined(Windows):\n"
lin = "when defined(Linux):\n"
osx = "when defined(MacOSX):\n"
var winlib, linlib, osxlib: string = ""
for dl in c2nimConfig.dynlib:
let
lib = " const dynlib$# = \"$#\"\n" % [fname, dl]
ext = dl.splitFile().ext
if ext == ".dll":
winlib &= lib
elif ext == ".so":
linlib &= lib
elif ext == ".dylib":
osxlib &= lib
if winlib != "":
outlib &= win & winlib & "\n"
if linlib != "":
outlib &= lin & linlib & "\n"
if osxlib != "":
outlib &= osx & osxlib & "\n"
if outlib != "":
extflags &= " --dynlib:dynlib$#" % fname
else:
if file.isAbsolute():
passC &= "const header$# = \"$#\"\n" % [fname, file]
else:
passC &= "const header$# = sourcePath & \"$#\"\n" %
[fname, file.relativePath()]
extflags = "--header:header$#" % fname
# Run c2nim on generated file
var cmd = "c2nim $# $# --out:$# $#" % [c2nimConfig.flags, extflags, outfile, cfile]
when defined(windows):
cmd = "cmd /c " & cmd.quoteShell
discard execProc(cmd)
if c2nimConfig.preprocess or c2nimConfig.ctags:
try:
removeFile(cfile)
except:
discard
else:
if c2nimConfig.removeBodies:
reAddBodies(cfile)
reAddStatic(cfile)
# Nim doesn't like {.cdecl.} for type proc()
freplace(outfile, re"(?m)(.*? = proc.*?)\{.cdecl.\}", "$1")
freplace(outfile, " {.cdecl.})", ")")
# Include {.compile.} directives
for cpl in c2nimConfig.compile:
prepend(outfile, compile(cpl, c2nimConfig.flags))
# Add any pragmas
if outpragma != "":
prepend(outfile, outpragma)
# Add header file and include paths
if passC != "":
prepend(outfile, passC)
# Add dynamic library
if outlib != "":
prepend(outfile, outlib)

View file

@ -1,174 +0,0 @@
import os, osproc, regex, streams, strutils
import globals
proc sanitizePath*(path: string): string =
path.multiReplace([("\\", "/"), ("//", "/")])
proc execProc*(cmd: string): string =
var ret: int
(result, ret) = execCmdEx(cmd)
if ret != 0:
echo "Command failed: " & $ret
echo cmd
echo result
quit(1)
proc execAction*(cmd: string): string =
var ccmd = ""
when defined(Windows):
ccmd = "cmd /c " & cmd.replace("/", "\\")
when defined(Linux) or defined(MacOSX):
ccmd = "bash -c '" & cmd & "'"
echo "Running '" & ccmd[0..<min(64, len(ccmd))] & "'"
return execProc(ccmd)
proc extractZip*(zipfile: string) =
var cmd = "unzip -o $#"
if defined(Windows):
cmd = "powershell -nologo -noprofile -command \"& { Add-Type -A " &
"'System.IO.Compression.FileSystem'; " &
"[IO.Compression.ZipFile]::ExtractToDirectory('$#', '.'); }\""
setCurrentDir(gOutput)
defer: setCurrentDir(gProjectDir)
echo "Extracting " & zipfile
discard execProc(cmd % zipfile)
proc downloadUrl*(url: string) =
let
file = url.extractFilename()
ext = file.splitFile().ext.toLowerAscii()
var cmd = if defined(Windows):
"powershell [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; wget $# -OutFile $#"
else:
"curl -L $# -o $#"
if not (ext == ".zip" and fileExists(gOutput/file)):
echo "Downloading " & file
discard execProc(cmd % [url, gOutput/file])
if ext == ".zip":
extractZip(file)
template setGitDir() =
setCurrentDir(gGitOutput)
defer: setCurrentDir(gProjectDir)
proc gitReset*() =
echo "Resetting " & gGitOutput
setGitDir()
let cmd = "git reset --hard"
while execCmdEx(cmd)[0].contains("Permission denied"):
sleep(1000)
echo " Retrying ..."
proc gitCheckout*(file: string) =
echo "Resetting " & file
setGitDir()
let cmd = "git checkout $#" % file.replace(gGitOutput & "/", "")
while execCmdEx(cmd)[0].contains("Permission denied"):
sleep(500)
echo " Retrying ..."
proc gitPull() =
if gGitCheckout.len() != 0:
echo "Checking out " & gGitCheckout
discard execProc("git pull --tags origin master")
discard execProc("git checkout " & gGitCheckout)
gGitCheckout = ""
else:
echo "Pulling repository"
discard execProc("git pull --depth=1 origin master")
proc gitRemotePull*(url: string, pull=true) =
if dirExists(gGitOutput/".git"):
if pull:
gitReset()
return
setGitDir()
echo "Setting up Git repo: " & url
discard execProc("git init .")
discard execProc("git remote add origin " & url)
if pull:
gitPull()
proc gitSparseCheckout*(plist: string) =
let sparsefile = ".git/info/sparse-checkout"
if fileExists(gGitOutput/sparsefile):
gitReset()
return
setGitDir()
discard execProc("git config core.sparsecheckout true")
writeFile(sparsefile, plist)
gitPull()
proc runPreprocess*(file, ppflags, flags: string, inline: bool): string =
var
pproc = if flags.contains("cpp"): gCppCompiler else: gCCompiler
cmd = "$# -E $# $#" % [pproc, ppflags, file]
for inc in gIncludes:
cmd &= " -I " & inc.quoteShell
# Run preprocessor
var data = execProc(cmd)
# Include content only from file
var
rdata: seq[string] = @[]
start = false
sfile = file.sanitizePath
if inline:
sfile = sfile.parentDir()
for line in data.splitLines():
if line.strip() != "":
if line[0] == '#' and not line.contains("#pragma"):
start = false
if sfile in line.sanitizePath:
start = true
if not ("\\" in line) and not ("/" in line) and extractFilename(sfile) in line:
start = true
else:
if start:
rdata.add(
line.multiReplace([("_Noreturn", ""), ("(())", ""), ("WINAPI", ""),
("__attribute__", ""), ("extern \"C\"", "")])
.replace(re"\(\([_a-z]+?\)\)", "")
.replace(re"\(\(__format__[\s]*\(__[gnu_]*printf__, [\d]+, [\d]+\)\)\);", ";")
)
return rdata.join("\n")
proc runCtags*(file: string): string =
var
cmd = "ctags -o - --fields=+S+K --c-kinds=+p --file-scope=no " & file
fps = execProc(cmd)
fdata = ""
for line in fps.splitLines():
var spl = line.split('\t')
if spl.len() > 4:
if spl[0] != "main" and spl[3] != "member":
var fn = ""
var match: RegexMatch
if spl[2].find(re"/\^(.*?)\(", match):
fn = spl[2][match.group(0)[0]]
fn &= spl[4].replace("signature:", "") & ";"
fdata &= fn & "\n"
return fdata

View file

@ -1,120 +0,0 @@
import os, pegs, regex, strutils, tables
when (NimMajor, NimMinor, NimPatch) < (0, 19, 9):
import ospaths
import globals, external
# ###
# File loction
proc getNimout*(file: string, rename=true): string =
result = file.splitFile().name.multiReplace([("-", "_"), (".", "_")]) & ".nim"
if gOutput != "":
result = gOutput & "/" & result
if not rename:
return
if gRenames.hasKey(file):
result = gRenames[file]
createDir(parentDir(result))
proc exclude*(file: string): bool =
for excl in gExcludes:
if excl in file:
return true
return false
proc search*(file: string): string =
if exclude(file):
return ""
result = file
if file.splitFile().ext == ".nim":
result = getNimout(file)
elif not fileExists(result) and not dirExists(result):
var found = false
for inc in gIncludes:
result = inc & "/" & file
if fileExists(result) or dirExists(result):
found = true
break
if not found:
echo "File doesn't exist: " & file
quit(1)
# Only keep relative directory
return result.sanitizePath.replace(gProjectDir & "/", "")
proc rename*(file: string, renfile: string) =
if file.splitFile().ext == ".nim":
return
var
nimout = getNimout(file, false)
newname = renfile.replace("$nimout", extractFilename(nimout))
if newname =~ peg"(!\$.)*{'$replace'\s*'('\s*{(!\)\S)+}')'}":
var final = nimout.extractFilename()
for entry in matches[1].split(","):
let spl = entry.split("=")
if spl.len() != 2:
echo "Bad replace syntax: " & renfile
quit(1)
let
srch = spl[0].strip()
repl = spl[1].strip()
final = final.replace(srch, repl)
newname = newname.replace(matches[0], final)
gRenames[file] = gOutput & "/" & newname
# ###
# Actions
proc openRetry*(file: string, mode: FileMode = fmRead): File =
while true:
try:
result = open(file, mode)
break
except IOError:
sleep(100)
template writeFileFlush*(file, content: string): untyped =
let f = openRetry(file, fmWrite)
f.write(content)
f.flushFile()
f.close()
template withFile*(file: string, body: untyped): untyped =
if fileExists(file):
var f = openRetry(file)
var contentOrig = f.readAll()
f.close()
var content {.inject.} = contentOrig
body
if content != contentOrig:
writeFileFlush(file, content)
else:
echo "Missing file " & file
proc doCopy*(flist: string) =
for pair in flist.split(","):
let spl = pair.split("=")
if spl.len() != 2:
echo "Bad copy syntax: " & flist
quit(1)
let
lfile = spl[0].strip()
rfile = spl[1].strip()
copyFile(lfile, rfile)
echo "Copied $# to $#" % [lfile, rfile]

View file

@ -1,145 +0,0 @@
import os, regex, strutils
import external, file
# ###
# Manipulating content
proc prepend*(file: string, data: string, search="") =
withFile(file):
if search == "":
content = data & content
else:
let idx = content.find(search)
if idx != -1:
content = content[0..<idx] & data & content[idx..<content.len()]
proc pipe*(file: string, command: string) =
let cmd = command % ["file", file]
let commandResult = execProc(cmd).strip()
if commandResult != "":
withFile(file):
content = commandResult
proc append*(file: string, data: string, search="") =
withFile(file):
if search == "":
content &= data
else:
let idx = content.find(search)
let idy = idx + search.len()
if idx != -1:
content = content[0..<idy] & data & content[idy..<content.len()]
proc freplace*(file: string, pattern: string|Regex, repl="") =
withFile(file):
content = content.replace(pattern, repl)
proc move*(file: string, pattern: string|Regex, move: string) =
var tomove: seq[string] = @[]
withFile(file):
when pattern is string:
tomove.add(pattern)
when pattern is Regex:
var ms = content.findAll(pattern)
for i, m in ms:
tomove.add(content[m.group(0)[0]])
content = content.replace(pattern, "")
for i in tomove:
append(file, i, move)
proc comment*(file: string, pattern: string, numlines: string) =
let
ext = file.splitFile().ext.toLowerAscii()
cmtchar = if ext == ".nim": "#" else: "//"
withFile(file):
var
idx = content.find(pattern)
num = 0
try:
num = numlines.parseInt()
except ValueError:
echo "Bad comment value, should be integer: " & numlines
if idx != -1:
for i in 0 .. num-1:
if idx >= content.len():
break
content = content[0..<idx] & cmtchar & content[idx..<content.len()]
while idx < content.len():
idx += 1
if content[idx] == '\L':
idx += 1
break
proc removeStatic*(filename: string) =
## Replace static function bodies with a semicolon and commented
## out body
withFile(filename):
content = content.replace(
re"(?m)(static inline.*?\))(\s*\{(\s*?.*?$)*?[\n\r]\})",
proc (m: RegexMatch, s: string): string =
let funcDecl = s[m.group(0)[0]]
let body = s[m.group(1)[0]].strip()
result = ""
result.add("$#;" % [funcDecl])
result.add(body.replace(re"(?m)^(.*\n?)", "//$1"))
)
proc removeBodies*(filename: string) =
## Replace function bodies with a semicolon and commented
## out body
withFile(filename):
content = content.replace(
re"(?m)(.*?\))(\s*\{(\s*?.*?$)*?[\n\r]\})",
proc (m: RegexMatch, s: string): string =
let funcDecl = s[m.group(0)[0]]
let body = s[m.group(1)[0]].strip()
result = ""
result.add("$#;" % [funcDecl])
result.add(body.replace(re"(?m)^(.*\n?)", "//$1"))
)
proc reAddBodies*(filename: string) =
## Uncomment out the body and remove the semicolon. Undoes
## removeBodies
withFile(filename):
content = content.replace(
re"(?m)(.*?\))(\s*\{(\s*?.*?$)*?[\n\r]\})",
proc (m: RegexMatch, s: string): string =
let funcDecl = s[m.group(0)[0]]
let body = s[m.group(1)[0]].strip()
result = ""
result.add("$# " % [funcDecl])
result.add(body.replace(re"(?m)^\/\/(.*\n?)", "$1"))
)
proc reAddStatic*(filename: string) =
## Uncomment out the body and remove the semicolon. Undoes
## removeStatic
withFile(filename):
content = content.replace(
re"(?m)(static inline.*?\));(\/\/\s*\{(\s*?.*?$)*?[\n\r]\/\/\})",
proc (m: RegexMatch, s: string): string =
let funcDecl = s[m.group(0)[0]]
let body = s[m.group(1)[0]].strip()
result = ""
result.add("$# " % [funcDecl])
result.add(body.replace(re"(?m)^\/\/(.*\n?)", "$1"))
)
proc fixFuncProtos*(file: string) =
withFile(file):
content = content.replace(re"(?m)(^.*?)[ ]*\(\*(.*?)\((.*?)\)\)[ \r\n]*\((.*?[\r\n]*.*?)\);",
proc (m: RegexMatch, s: string): string =
(("typedef $# (*type_$#)($#);\n" % [s[m.group(0)[0]], s[m.group(1)[0]], s[m.group(3)[0]]]) &
("type_$# $#($#);" % [s[m.group(1)[0]], s[m.group(1)[0]], s[m.group(2)[0]]]))
)

View file

@ -1,108 +0,0 @@
import os, regex, sequtils, strutils
import file, globals
proc addEnv*(str: string): string =
var newStr = str
if "$output" in newStr or "${output}" in newStr:
newStr = newStr % ["output", gOutput]
for pair in envPairs():
if pair.key.len() == 0:
continue
if ("$" & pair.key) in newStr or ("${" & pair.key & "}") in newStr:
newStr = newStr % [pair.key, pair.value.string]
return newStr
proc compile*(cpl, flags: string): string =
var data = ""
proc fcompile(file: string): string =
let fn = file.splitFile().name
var
ufn = fn
uniq = 1
while ufn in gCompile:
ufn = fn & $uniq
uniq += 1
gCompile.add(ufn)
if fn == ufn:
return "{.compile: \"$#\".}" % file.replace("\\", "/")
else:
return "{.compile: (\"../$#\", \"$#.o\").}" % [file.replace("\\", "/"), ufn]
proc dcompile(dir: string) =
for f in walkFiles(dir):
data &= fcompile(f) & "\n"
if cpl.contains("*") or cpl.contains("?"):
dcompile(cpl)
else:
let fcpl = search(cpl)
if getFileInfo(fcpl).kind == pcFile:
data &= fcompile(fcpl) & "\n"
elif getFileInfo(fcpl).kind == pcDir:
if flags.contains("cpp"):
for i in @["*.C", "*.cpp", "*.c++", "*.cc", "*.cxx"]:
dcompile(fcpl / i)
else:
dcompile(fcpl / "*.c")
return data
proc getIncls*(file: string, inline=false): seq[string] =
result = @[]
if inline and file in gDoneInline:
return
let curPath = splitFile(expandFileName(file)).dir
withFile(file):
for f in content.findAll(re"(?m)^\s*#\s*include\s+(.*?)$"):
var inc = content[f.group(0)[0]].strip()
if ((gQuotes and inc.contains("\"")) or (gFilter != "" and gFilter in inc)) and (not exclude(inc)):
let addInc = inc.replace(re"""[<>"]""", "").replace(re"\/[\*\/].*$", "").strip()
try:
# Try searching for a local library. expandFilename will throw
# OSError if the file does not exist
let
finc = expandFileName(curPath / addInc)
fname = finc.replace(curPath & $DirSep, "")
if fname.len() > 0:
# only add if the file is non-empty
result.add(fname.search())
except OSError:
# If it's a system library
result.add(addInc)
result = result.deduplicate()
gDoneInline.add(file)
if inline:
var sres = newSeq[string]()
for incl in result:
let sincl = search(incl)
if sincl == "":
continue
sres.add(getIncls(sincl, inline))
result.add(sres)
result = result.deduplicate()
proc getDefines*(file: string, inline=false): string =
result = ""
if inline:
var incls = getIncls(file, inline)
for incl in incls:
let sincl = search(incl)
if sincl != "":
result &= getDefines(sincl)
withFile(file):
for def in content.findAll(re"(?m)^(\s*#\s*define\s+[\w\d_]+\s+[\d\-.xf]+)(?:\r|//|/*).*?$"):
result &= content[def.group(0)[0]] & "\n"

View file

@ -1,47 +0,0 @@
import os, parsecfg, tables
const
cCompilerEnv* = "CC"
cppCompilerEnv* = "CPP"
defaultCCompiler* = "gcc"
defaultCppCompiler* = "g++"
var
# Config
gConfig*: Config
gExcludes*: seq[string] = @[]
gIncludes*: seq[string] = @[]
gRenames* = initTable[string, string]()
gWildcards* = newConfig()
# n.global
gOutput* = "."
gQuotes* = true
gFilter* = ""
gCppCompiler* = getEnv(cppCompilerEnv, defaultCCompiler)
gCCompiler* = getEnv(cCompilerEnv, defaultCppCompiler)
# State tracking
gGitCheckout* = ""
gGitOutput* = ""
gProjectDir* = ""
gCompile*: seq[string] = @[]
gDoneInline*: seq[string] = @[]
gDoneRecursive*: seq[string] = @[]
type
c2nimConfigObj* = object
flags*, ppflags*: string
recurse*, inline*, preprocess*, removeBodies*, ctags*, defines*: bool
dynlib*, compile*, pragma*: seq[string]
const gDoc* = """
Nimgen is a helper for c2nim to simpilfy and automate the wrapping of C libraries
Usage:
nimgen [options] <file.cfg>...
Options:
-f delete all artifacts and regenerate
"""

View file

@ -1,411 +0,0 @@
import os, parsecfg, regex, strutils, tables
import c2nim, external, file, fileops, gencore, globals
proc `[]`*(table: OrderedTableRef[string, string], key: string): string =
## Gets table values with env vars inserted
tables.`[]`(table, key).addEnv
proc getKey(ukey: string, section = false): tuple[key: string, val: bool] =
var kv = if not section: ukey.replace(re"\..*", "").split("-", 1) else: ukey.split("-", 1)
if kv.len() == 1:
kv.add("")
if kv[1] == "":
return (kv[0], true)
for ostyp in kv[1].split("-"):
if (ostyp == "win" and defined(Windows)) or
(ostyp == "lin" and defined(Linux)) or
((ostyp == "osx" or ostyp == "mac") and defined(MacOSX)) or
(ostyp == "unix" and (defined(Linux) or defined(MacOSX))):
return (kv[0], true)
return (kv[0], false)
proc runFile*(file: string, cfgin: OrderedTableRef = newOrderedTable[string, string]()) =
var
cfg = cfgin
sfile = search(file)
nowildcard = false
if sfile in gDoneRecursive:
return
if sfile.len() != 0:
echo "Processing " & sfile
gDoneRecursive.add(sfile)
for act in cfg.keys():
let (action, val) = getKey(act)
if val == true and action == "nowildcard" and cfg[act] == "true":
nowildcard = true
break
if not nowildcard:
for pattern in gWildcards.keys():
var m: RegexMatch
let pat = pattern.replace(".", "\\.").replace("*", ".*").replace("?", ".?")
if file.find(toPattern(pat), m):
echo " Appending keys for wildcard " & pattern
for key in gWildcards[pattern].keys():
cfg[key & "." & pattern] = gWildcards[pattern][key]
var
srch = ""
rgx = ""
c2nimConfig = c2nimConfigObj(
flags: "--stdcall", ppflags: "",
recurse: false, inline: false, preprocess: false, ctags: false, defines: false,
dynlib: @[], compile: @[], pragma: @[]
)
for act in cfg.keys():
let (action, val) = getKey(act)
if val == true:
if action == "create":
echo "Creating " & file
createDir(file.splitPath().head)
writeFileFlush(file, cfg[act])
if file in gExcludes:
gExcludes.delete(gExcludes.find(file))
sfile = file
gDoneRecursive.add(sfile)
elif action in @["prepend", "append", "replace", "move", "comment",
"rename", "compile", "dynlib", "pragma", "pipe"] and
sfile != "":
if action == "prepend":
if srch != "":
prepend(sfile, cfg[act], cfg[srch])
else:
prepend(sfile, cfg[act])
elif action == "append":
if srch != "":
append(sfile, cfg[act], cfg[srch])
else:
append(sfile, cfg[act])
elif action == "replace":
if srch != "":
freplace(sfile, cfg[srch], cfg[act])
elif rgx != "":
freplace(sfile, toPattern(cfg[rgx]), cfg[act])
elif action == "move":
if srch != "":
move(sfile, cfg[srch], cfg[act])
elif rgx != "":
move(sfile, toPattern(cfg[rgx]), cfg[act])
elif action == "comment":
if srch != "":
comment(sfile, cfg[srch], cfg[act])
elif action == "rename":
rename(sfile, cfg[act])
elif action == "compile":
c2nimConfig.compile.add(cfg[act])
elif action == "dynlib":
c2nimConfig.dynlib.add(cfg[act])
elif action == "pragma":
c2nimConfig.pragma.add(cfg[act])
elif action == "pipe":
pipe(sfile, cfg[act])
srch = ""
rgx = ""
elif action == "search":
srch = act
elif action == "regex":
rgx = act
if file.splitFile().ext != ".nim":
var
noprocess = false
reset = false
for act in cfg.keys():
let (action, val) = getKey(act)
if val == true:
if cfg[act] == "true":
if action == "recurse":
c2nimConfig.recurse = true
elif action == "inline":
c2nimConfig.inline = true
elif action == "preprocess":
c2nimConfig.preprocess = true
elif action == "removeBodies":
c2nimConfig.removeBodies = true
elif action == "ctags":
c2nimConfig.ctags = true
elif action == "defines":
c2nimConfig.defines = true
elif action == "noprocess":
noprocess = true
elif action == "reset":
reset = true
elif action == "flags":
c2nimConfig.flags = cfg[act]
elif action == "ppflags":
c2nimConfig.ppflags = cfg[act]
if c2nimConfig.recurse and c2nimConfig.inline:
echo "Cannot use recurse and inline simultaneously"
quit(1)
removeStatic(sfile)
fixFuncProtos(sfile)
let outfile = getNimout(sfile)
var incout = ""
if c2nimConfig.recurse or c2nimConfig.inline:
var
cfg = newOrderedTable[string, string]()
incls = getIncls(sfile)
for name, value in c2nimConfig.fieldPairs:
when value is string:
cfg[name] = value
when value is bool:
cfg[name] = $value
for i in c2nimConfig.dynlib:
cfg["dynlib." & i] = i
if c2nimConfig.inline:
cfg["noprocess"] = "true"
for inc in incls:
runFile(inc, cfg)
if c2nimConfig.recurse:
incout &= "import $#\n" % inc.search().getNimout()[0 .. ^5]
if not noprocess:
c2nim(file, outfile, c2nimConfig)
if c2nimConfig.recurse and incout.len() != 0:
prepend(outfile, incout)
if reset:
gitCheckout(sfile)
proc setOutputDir(dir: string) =
gOutput = dir.sanitizePath
if dirExists(gOutput):
if "-f" in commandLineParams():
try:
removeDir(gOutput)
except OSError:
echo "Directory in use: " & gOutput
quit(1)
else:
for f in walkFiles(gOutput/"*.nim"):
try:
removeFile(f)
except OSError:
echo "Unable to delete: " & f
quit(1)
createDir(gOutput)
gGitOutput = gOutput
proc runCfg*(cfg: string) =
if not fileExists(cfg):
echo "Config doesn't exist: " & cfg
quit(1)
gProjectDir = parentDir(cfg.expandFilename()).sanitizePath
gConfig = loadConfig(cfg)
gCppCompiler = getEnv(cppCompilerEnv, defaultCppCompiler).quoteShell
gCCompiler = getEnv(cCompilerEnv, defaultCCompiler).quoteShell
gGitOutput = gOutput
for section in gConfig.keys():
let (sname, sval) = getKey(section, true)
if not sval:
continue
case sname:
of "n.global":
for glob in gConfig[section].keys():
let (key, val) = getKey(glob)
if val == true:
let globVal = gConfig[section][glob]
case key:
of "output":
setOutputDir(globVal)
of "cpp_compiler":
gCppCompiler = globVal.quoteShell
of "c_compiler":
gCCompiler = globVal.quoteShell
of "filter":
gFilter = globVal
of "quotes":
if globVal == "false":
gQuotes = false
of "n.include":
for inc in gConfig[section].keys():
gIncludes.add(inc.addEnv().sanitizePath)
of "n.exclude":
for excl in gConfig[section].keys():
gExcludes.add(excl.addEnv().sanitizePath)
of "n.prepare":
for prep in gConfig[section].keys():
let (key, val) = getKey(prep)
if val == true:
let prepVal = gConfig[section][prep]
case key:
of "download":
downloadUrl(prepVal)
of "extract":
extractZip(prepVal)
of "gitcheckout":
gGitCheckout = prepVal
of "gitoutput":
gGitOutput = gOutput/prepVal
createDir(gGitOutput)
of "git":
gitRemotePull(prepVal)
of "gitremote":
gitRemotePull(prepVal, false)
of "gitsparse":
gitSparseCheckout(prepVal)
of "execute":
discard execAction(prepVal)
of "copy":
doCopy(prepVal)
of "n.wildcard":
var wildcard = ""
for wild in gConfig[section].keys():
let (key, val) = getKey(wild)
if val == true:
if key == "wildcard":
wildcard = gConfig[section][wild]
else:
gWildcards.setSectionKey(wildcard, wild,
gConfig[section][wild])
of "n.sourcefile":
for pattern in gConfig[section].keys():
for file in walkFiles(pattern.addEnv):
runFile(file)
of "n.post":
for post in gConfig[section].keys():
let (key, val) = getKey(post)
if val == true:
let postVal = gConfig[section][post]
case key:
of "gitoutput":
gGitOutput = gOutput/postVal
of "reset":
gitReset()
of "execute":
discard execAction(postVal)
else:
runFile(section, gConfig[section])
let gHelp = """
Nimgen is a helper for c2nim to simplify and automate the wrapping of C libraries
Usage:
nimgen [options] file.cfg|file.h ...
Params:
-C<compile> add compile entry *
-E<exclude> add n.exclude entry *
-F<flags> set c2nim flags *
-I<include> add n.include dir *
-O<outdir> set output directory
-P<ppflags> set preprocessor flags *
Options:
-c set ctags = true
-d set defines = true
-i set inline = true
-n set noprocess = true
-p set preprocess = true
-r set recurse = true
Editing:
-a<append> append string *
-e<prepend> prepend string *
-l<replace> replace string *
-o#lines comment X lines *
-s<search> search string *
-x<regex> regex search string *
* supports multiple instances
"""
proc runCli*() =
var
cfg = newOrderedTable[string, string]()
files: seq[string]
uniq = 1
gProjectDir = getCurrentDir().sanitizePath
for param in commandLineParams():
let flag = if param.len() <= 2: param else: param[0..<2]
if fileExists(param):
if param.splitFile().ext.toLowerAscii() == ".cfg":
runCfg(param)
else:
files.add(param)
elif flag == "-C":
cfg["compile." & $uniq] = param[2..^1]
elif flag == "-E":
gExcludes.add(param[2..^1].addEnv().sanitizePath)
elif flag == "-F":
if cfg.hasKey("flags"):
cfg["flags"] = cfg["flags"] & " " & param[2..^1]
else:
cfg["flags"] = param[2..^1]
elif flag == "-I":
gIncludes.add(param[2..^1].addEnv().sanitizePath)
elif flag == "-O":
setOutputDir(param[2..^1])
elif flag == "-P":
if cfg.hasKey("ppflags"):
cfg["ppflags"] = cfg["ppflags"] & " " & param[2..^1]
else:
cfg["ppflags"] = param[2..^1]
elif flag == "-c":
cfg["ctags"] = "true"
elif flag == "-d":
cfg["defines"] = "true"
elif flag == "-i":
cfg["inline"] = "true"
elif flag == "-n":
cfg["noprocess"] = "true"
elif flag == "-p":
cfg["preprocess"] = "true"
elif flag == "-r":
cfg["recurse"] = "true"
elif flag == "-a":
cfg["append." & $uniq] = param[2..^1]
elif flag == "-e":
cfg["prepend." & $uniq] = param[2..^1]
elif flag == "-l":
cfg["replace." & $uniq] = param[2..^1]
elif flag == "-o":
cfg["comment." & $uniq] = param[2..^1]
elif flag == "-s":
cfg["search." & $uniq] = param[2..^1]
elif flag == "-x":
cfg["regex." & $uniq] = param[2..^1]
elif param == "-h" or param == "-?" or param == "--help":
echo gHelp
quit(0)
uniq += 1
for file in files:
runFile(file, cfg)

View file

@ -1,78 +1,58 @@
import distros, ospaths, strutils import distros
import ospaths
import strutils
var var
pygonly = false full = false
comps = @["nim7z", #"nimarchive", comps = @[
"nimbass", "nimbigwig", "genotrance/libsvm", "genotrance/nim7z", "genotrance/nimarchive",
"nimclipboard", "nimfuzzy", "nimmonocypher", "genotrance/nimbass", "genotrance/nimbigwig", "genotrance/nimfuzz",
#"nimnuklear", "genotrance/nimpcre", "genotrance/nimrax", "genotrance/nimssl",
"nimpcre", "nimrax", "nimssl", "nimssh2", "genotrance/nimssh2", "jyapayne/nim-libnx"
"nimtess2" ]
]
let
gccver = staticExec("gcc --version").split("\n")[0].split(" ")[^1]
nimver = staticExec("nim -v").split("\n")[0].split(" ")[3]
if nimver >= "0.19.0" and (gccver >= "5.0.0" or detectOs(MacOSX)):
comps.add("nimfastText")
if detectOs(Windows): if detectOs(Windows):
comps.add("nimkerberos") comps.add("nimkerberos")
if not detectOs(MacOSX):
comps.add("nimzbar")
echo "Nim version: " & nimver
echo "GCC version: " & gccver
echo "Testing comps:"
for comp in comps:
echo " " & comp
if paramCount() > 2: if paramCount() > 2:
for i in 3 .. paramCount(): for i in 3 .. paramCount():
if paramStr(i) == "--pygonly": if paramStr(i) == "--full":
pygonly = true full = true
elif paramStr(i).len() > 10 and "--comps=" in paramStr(i)[0 ..< 8]: elif paramStr(i).len() > 10 and "--comps=" in paramStr(i)[0 ..< 8]:
comps = paramStr(i)[8 .. ^1].split(",") comps = paramStr(i)[8 .. ^1].split(",")
for comp in comps: for comp in comps:
if not pygonly: if not dirExists(".."/comp):
if not dirExists(".."/comp): withDir(".."):
withDir(".."): exec "git clone --depth=1 https://github.com/" & comp
exec "git clone --depth=1 https://github.com/genotrance/" & comp
exec "nimble uninstall -y " & comp, "", "" exec "nimble uninstall -y " & comp, "", ""
withDir(".."/comp): withDir(".."/comp):
exec "git pull" exec "git pull"
if full:
rmDir(comp) rmDir(comp)
exec "nimble install -y" exec "nimble install -y"
exec "nimble test" exec "nimble test"
exec "nimble install -y" exec "nimble install -y"
exec "nimble test" exec "nimble test"
when defined(windows): if dirExists("web"/comp):
if not pygonly: rmDir("web"/comp)
if dirExists("web"/comp):
rmDir("web"/comp)
mkDir("web"/comp) mkDir("web"/comp)
for file in listFiles(".."/comp/comp) & listFiles(".."/comp): for file in listFiles(".."/comp/comp) & listFiles(".."/comp):
if file.splitFile().ext == ".nim": if file.splitFile().ext == ".nim":
cpFile(file, "web"/comp/extractFilename(file)) cpFile(file, "web"/comp/extractFilename(file))
cpFile("web"/"nimdoc.cfg", "web"/comp/"nimdoc.cfg") cpFile("web"/"nimdoc.cfg", "web"/comp/"nimdoc.cfg")
withDir("web"/comp): withDir("web"/comp):
for file in listFiles("."): for file in listFiles("."):
if file.splitFile().ext == ".nim": if file.splitFile().ext == ".nim":
if not pygonly: exec "nim doc --git.url:. --index:on -o:" & file.changeFileExt("html") & " " & file
exec "nim doc --git.url:. --index:on -o:" & file.changeFileExt("html") & " " & file exec "pygmentize -f html -O full,linenos=1,anchorlinenos=True,lineanchors=L,style=vs -o " & file & ".html " & file
exec "pygmentize -f html -O full,linenos=1,anchorlinenos=True,lineanchors=L,style=vs -o " & file & ".html " & file
if not pygonly: exec "nim buildIndex -o:index.html ."
exec "nim buildIndex -o:index.html ." rmFile("web"/comp/"nimdoc.cfg")
rmFile("web"/comp/"nimdoc.cfg")

View file

@ -1,10 +0,0 @@
import os, osproc, strutils
proc main() =
var failures = 0
for file in walkFiles(currentSourcePath().splitPath().head / "unittests/*.nim"):
let (path, fname, ext) = file.splitFile()
if fname.startswith("test"):
failures += execCmd "nim c -r " & file
quit(failures)
main()

View file

@ -1,5 +0,0 @@
import unittest
proc checkFile*(filepath, expected: string) =
let result = readFile(filepath)
check result == expected

View file

@ -1,675 +0,0 @@
/**
* @file ipc.h
* @brief Inter-process communication handling
* @author plutoo
* @copyright libnx Authors
*/
#pragma once
#include "../result.h"
#include "../arm/tls.h"
#include "../kernel/svc.h"
/// IPC input header magic
#define SFCI_MAGIC 0x49434653
/// IPC output header magic
#define SFCO_MAGIC 0x4f434653
/// IPC invalid object ID
#define IPC_INVALID_OBJECT_ID UINT32_MAX
///@name IPC request building
///@{
/// IPC command (request) structure.
#define IPC_MAX_BUFFERS 8
#define IPC_MAX_OBJECTS 8
typedef enum {
BufferType_Normal=0, ///< Regular buffer.
BufferType_Type1=1, ///< Allows ProcessMemory and shared TransferMemory.
BufferType_Invalid=2,
BufferType_Type3=3 ///< Same as Type1 except remote process is not allowed to use device-mapping.
} BufferType;
typedef enum {
BufferDirection_Send=0,
BufferDirection_Recv=1,
BufferDirection_Exch=2,
} BufferDirection;
typedef enum {
IpcCommandType_Invalid = 0,
IpcCommandType_LegacyRequest = 1,
IpcCommandType_Close = 2,
IpcCommandType_LegacyControl = 3,
IpcCommandType_Request = 4,
IpcCommandType_Control = 5,
IpcCommandType_RequestWithContext = 6,
IpcCommandType_ControlWithContext = 7,
} IpcCommandType;
typedef enum {
DomainMessageType_Invalid = 0,
DomainMessageType_SendMessage = 1,
DomainMessageType_Close = 2,
} DomainMessageType;
/// IPC domain message header.
typedef struct {
u8 Type;
u8 NumObjectIds;
u16 Length;
u32 ThisObjectId;
u32 Pad[2];
} DomainMessageHeader;
typedef struct {
size_t NumSend; // A
size_t NumRecv; // B
size_t NumExch; // W
const void* Buffers[IPC_MAX_BUFFERS];
size_t BufferSizes[IPC_MAX_BUFFERS];
BufferType BufferTypes[IPC_MAX_BUFFERS];
size_t NumStaticIn; // X
size_t NumStaticOut; // C
const void* Statics[IPC_MAX_BUFFERS];
size_t StaticSizes[IPC_MAX_BUFFERS];
u8 StaticIndices[IPC_MAX_BUFFERS];
bool SendPid;
size_t NumHandlesCopy;
size_t NumHandlesMove;
Handle Handles[IPC_MAX_OBJECTS];
size_t NumObjectIds;
u32 ObjectIds[IPC_MAX_OBJECTS];
} IpcCommand;
/**
* @brief Initializes an IPC command structure.
* @param cmd IPC command structure.
*/
static inline void ipcInitialize(IpcCommand* cmd);//{
// *cmd = (IpcCommand){0};
//}
/// IPC buffer descriptor.
typedef struct {
u32 Size; ///< Size of the buffer.
u32 Addr; ///< Lower 32-bits of the address of the buffer
u32 Packed; ///< Packed data (including higher bits of the address)
} IpcBufferDescriptor;
/// IPC static send-buffer descriptor.
typedef struct {
u32 Packed; ///< Packed data (including higher bits of the address)
u32 Addr; ///< Lower 32-bits of the address
} IpcStaticSendDescriptor;
/// IPC static receive-buffer descriptor.
typedef struct {
u32 Addr; ///< Lower 32-bits of the address of the buffer
u32 Packed; ///< Packed data (including higher bits of the address)
} IpcStaticRecvDescriptor;
/**
* @brief Adds a buffer to an IPC command structure.
* @param cmd IPC command structure.
* @param buffer Address of the buffer.
* @param size Size of the buffer.
* @param type Buffer type.
*/
static inline void ipcAddSendBuffer(IpcCommand* cmd, const void* buffer, size_t size, BufferType type);//{
// size_t off = cmd->NumSend;
// cmd->Buffers[off] = buffer;
// cmd->BufferSizes[off] = size;
// cmd->BufferTypes[off] = type;
// cmd->NumSend++;
//}
/**
* @brief Adds a receive-buffer to an IPC command structure.
* @param cmd IPC command structure.
* @param buffer Address of the buffer.
* @param size Size of the buffer.
* @param type Buffer type.
*/
static inline void ipcAddRecvBuffer(IpcCommand* cmd, void* buffer, size_t size, BufferType type);//{
// size_t off = cmd->NumSend + cmd->NumRecv;
// cmd->Buffers[off] = buffer;
// cmd->BufferSizes[off] = size;
// cmd->BufferTypes[off] = type;
// cmd->NumRecv++;
//}
/**
* @brief Adds an exchange-buffer to an IPC command structure.
* @param cmd IPC command structure.
* @param buffer Address of the buffer.
* @param size Size of the buffer.
* @param type Buffer type.
*/
static inline void ipcAddExchBuffer(IpcCommand* cmd, void* buffer, size_t size, BufferType type);//{
// size_t off = cmd->NumSend + cmd->NumRecv + cmd->NumExch;
// cmd->Buffers[off] = buffer;
// cmd->BufferSizes[off] = size;
// cmd->BufferTypes[off] = type;
// cmd->NumExch++;
//}
/**
* @brief Adds a static-buffer to an IPC command structure.
* @param cmd IPC command structure.
* @param buffer Address of the buffer.
* @param size Size of the buffer.
* @param index Index of buffer.
*/
static inline void ipcAddSendStatic(IpcCommand* cmd, const void* buffer, size_t size, u8 index);//{
// size_t off = cmd->NumStaticIn;
// cmd->Statics[off] = buffer;
// cmd->StaticSizes[off] = size;
// cmd->StaticIndices[off] = index;
// cmd->NumStaticIn++;
//}
/**
* @brief Adds a static-receive-buffer to an IPC command structure.
* @param cmd IPC command structure.
* @param buffer Address of the buffer.
* @param size Size of the buffer.
* @param index Index of buffer.
*/
static inline void ipcAddRecvStatic(IpcCommand* cmd, void* buffer, size_t size, u8 index);//{
// size_t off = cmd->NumStaticIn + cmd->NumStaticOut;
// cmd->Statics[off] = buffer;
// cmd->StaticSizes[off] = size;
// cmd->StaticIndices[off] = index;
// cmd->NumStaticOut++;
//}
/**
* @brief Adds a smart-buffer (buffer + static-buffer pair) to an IPC command structure.
* @param cmd IPC command structure.
* @param ipc_buffer_size IPC buffer size.
* @param buffer Address of the buffer.
* @param size Size of the buffer.
* @param index Index of buffer.
*/
static inline void ipcAddSendSmart(IpcCommand* cmd, size_t ipc_buffer_size, const void* buffer, size_t size, u8 index);//{
// if (ipc_buffer_size != 0 && size <= ipc_buffer_size) {
// ipcAddSendBuffer(cmd, NULL, 0, BufferType_Normal);
// ipcAddSendStatic(cmd, buffer, size, index);
// } else {
// ipcAddSendBuffer(cmd, buffer, size, BufferType_Normal);
// ipcAddSendStatic(cmd, NULL, 0, index);
// }
//}
/**
* @brief Adds a smart-receive-buffer (buffer + static-receive-buffer pair) to an IPC command structure.
* @param cmd IPC command structure.
* @param ipc_buffer_size IPC buffer size.
* @param buffer Address of the buffer.
* @param size Size of the buffer.
* @param index Index of buffer.
*/
static inline void ipcAddRecvSmart(IpcCommand* cmd, size_t ipc_buffer_size, void* buffer, size_t size, u8 index);//{
// if (ipc_buffer_size != 0 && size <= ipc_buffer_size) {
// ipcAddRecvBuffer(cmd, NULL, 0, BufferType_Normal);
// ipcAddRecvStatic(cmd, buffer, size, index);
// } else {
// ipcAddRecvBuffer(cmd, buffer, size, BufferType_Normal);
// ipcAddRecvStatic(cmd, NULL, 0, index);
// }
//}
/**
* @brief Tags an IPC command structure to send the PID.
* @param cmd IPC command structure.
*/
static inline void ipcSendPid(IpcCommand* cmd);//{
// cmd->SendPid = true;
//}
/**
* @brief Adds a copy-handle to be sent through an IPC command structure.
* @param cmd IPC command structure.
* @param h Handle to send.
* @remark The receiving process gets a copy of the handle.
*/
static inline void ipcSendHandleCopy(IpcCommand* cmd, Handle h);//{
// cmd->Handles[cmd->NumHandlesCopy++] = h;
//}
/**
* @brief Adds a move-handle to be sent through an IPC command structure.
* @param cmd IPC command structure.
* @param h Handle to send.
* @remark The sending process loses ownership of the handle, which is transferred to the receiving process.
*/
static inline void ipcSendHandleMove(IpcCommand* cmd, Handle h);//{
// cmd->Handles[cmd->NumHandlesCopy + cmd->NumHandlesMove++] = h;
//}
/**
* @brief Prepares the header of an IPC command structure.
* @param cmd IPC command structure.
* @param sizeof_raw Size in bytes of the raw data structure to embed inside the IPC request
* @return Pointer to the raw embedded data structure in the request, ready to be filled out.
*/
static inline void* ipcPrepareHeader(IpcCommand* cmd, size_t sizeof_raw);//{
// u32* buf = (u32*)armGetTls();
// size_t i;
// *buf++ = IpcCommandType_Request | (cmd->NumStaticIn << 16) | (cmd->NumSend << 20) | (cmd->NumRecv << 24) | (cmd->NumExch << 28);
//
// u32* fill_in_size_later = buf;
//
// if (cmd->NumStaticOut > 0) {
// *buf = (cmd->NumStaticOut + 2) << 10;
// }
// else {
// *buf = 0;
// }
//
// if (cmd->SendPid || cmd->NumHandlesCopy > 0 || cmd->NumHandlesMove > 0) {
// *buf++ |= 0x80000000;
// *buf++ = (!!cmd->SendPid) | (cmd->NumHandlesCopy << 1) | (cmd->NumHandlesMove << 5);
//
// if (cmd->SendPid)
// buf += 2;
//
// for (i=0; i<(cmd->NumHandlesCopy + cmd->NumHandlesMove); i++)
// *buf++ = cmd->Handles[i];
// }
// else {
// buf++;
// }
//
// for (i=0; i<cmd->NumStaticIn; i++, buf+=2) {
// IpcStaticSendDescriptor* desc = (IpcStaticSendDescriptor*) buf;
//
// uintptr_t ptr = (uintptr_t) cmd->Statics[i];
// desc->Addr = ptr;
// desc->Packed = cmd->StaticIndices[i] | (cmd->StaticSizes[i] << 16) |
// (((ptr >> 32) & 15) << 12) | (((ptr >> 36) & 15) << 6);
// }
//
// for (i=0; i<(cmd->NumSend + cmd->NumRecv + cmd->NumExch); i++, buf+=3) {
// IpcBufferDescriptor* desc = (IpcBufferDescriptor*) buf;
// desc->Size = cmd->BufferSizes[i];
//
// uintptr_t ptr = (uintptr_t) cmd->Buffers[i];
// desc->Addr = ptr;
// desc->Packed = cmd->BufferTypes[i] |
// (((ptr >> 32) & 15) << 28) | ((ptr >> 36) << 2);
// }
//
// u32 padding = ((16 - (((uintptr_t) buf) & 15)) & 15) / 4;
// u32* raw = (u32*) (buf + padding);
//
// size_t raw_size = (sizeof_raw/4) + 4;
// buf += raw_size;
//
// u16* buf_u16 = (u16*) buf;
//
// for (i=0; i<cmd->NumStaticOut; i++) {
// size_t off = cmd->NumStaticIn + i;
// size_t sz = (uintptr_t) cmd->StaticSizes[off];
//
// buf_u16[i] = (sz > 0xFFFF) ? 0 : sz;
// }
//
// size_t u16s_size = ((2*cmd->NumStaticOut) + 3)/4;
// buf += u16s_size;
// raw_size += u16s_size;
//
// *fill_in_size_later |= raw_size;
//
// for (i=0; i<cmd->NumStaticOut; i++, buf+=2) {
// IpcStaticRecvDescriptor* desc = (IpcStaticRecvDescriptor*) buf;
// size_t off = cmd->NumStaticIn + i;
//
// uintptr_t ptr = (uintptr_t) cmd->Statics[off];
// desc->Addr = ptr;
// desc->Packed = (ptr >> 32) | (cmd->StaticSizes[off] << 16);
// }
//
// return (void*) raw;
//}
/**
* @brief Dispatches an IPC request.
* @param session IPC session handle.
* @return Result code.
*/
static inline Result ipcDispatch(Handle session);//{
// return svcSendSyncRequest(session);
//}
///@}
///@name IPC response parsing
///@{
/// IPC parsed command (response) structure.
typedef struct {
IpcCommandType CommandType; ///< Type of the command
bool HasPid; ///< true if the 'Pid' field is filled out.
u64 Pid; ///< PID included in the response (only if HasPid is true)
size_t NumHandles; ///< Number of handles copied.
Handle Handles[IPC_MAX_OBJECTS]; ///< Handles.
bool WasHandleCopied[IPC_MAX_OBJECTS]; ///< true if the handle was moved, false if it was copied.
bool IsDomainMessage; ///< true if the the message is a Domain message.
DomainMessageType MessageType; ///< Type of the domain message.
u32 MessageLength; ///< Size of rawdata (for domain messages).
u32 ThisObjectId; ///< Object ID to call the command on (for domain messages).
size_t NumObjectIds; ///< Number of object IDs (for domain messages).
u32 ObjectIds[IPC_MAX_OBJECTS]; ///< Object IDs (for domain messages).
size_t NumBuffers; ///< Number of buffers in the response.
void* Buffers[IPC_MAX_BUFFERS]; ///< Pointers to the buffers.
size_t BufferSizes[IPC_MAX_BUFFERS]; ///< Sizes of the buffers.
BufferType BufferTypes[IPC_MAX_BUFFERS]; ///< Types of the buffers.
BufferDirection BufferDirections[IPC_MAX_BUFFERS]; ///< Direction of each buffer.
size_t NumStatics; ///< Number of statics in the response.
void* Statics[IPC_MAX_BUFFERS]; ///< Pointers to the statics.
size_t StaticSizes[IPC_MAX_BUFFERS]; ///< Sizes of the statics.
u8 StaticIndices[IPC_MAX_BUFFERS]; ///< Indices of the statics.
size_t NumStaticsOut; ///< Number of output statics available in the response.
void* Raw; ///< Pointer to the raw embedded data structure in the response.
void* RawWithoutPadding; ///< Pointer to the raw embedded data structure, without padding.
size_t RawSize; ///< Size of the raw embedded data.
} IpcParsedCommand;
/**
* @brief Parse an IPC command response into an IPC parsed command structure.
* @param IPC parsed command structure to fill in.
* @return Result code.
*/
static inline Result ipcParse(IpcParsedCommand* r);//{
// u32* buf = (u32*)armGetTls();
// u32 ctrl0 = *buf++;
// u32 ctrl1 = *buf++;
// size_t i;
//
// r->IsDomainMessage = false;
//
// r->CommandType = (IpcCommandType) (ctrl0 & 0xffff);
// r->HasPid = false;
// r->RawSize = (ctrl1 & 0x1ff) * 4;
// r->NumHandles = 0;
//
// r->NumStaticsOut = (ctrl1 >> 10) & 15;
// if (r->NumStaticsOut >> 1) r->NumStaticsOut--; // Value 2 -> Single descriptor
// if (r->NumStaticsOut >> 1) r->NumStaticsOut--; // Value 3+ -> (Value - 2) descriptors
//
// if (ctrl1 & 0x80000000) {
// u32 ctrl2 = *buf++;
//
// if (ctrl2 & 1) {
// r->HasPid = true;
// r->Pid = *buf++;
// r->Pid |= ((u64)(*buf++)) << 32;
// }
//
// size_t num_handles_copy = ((ctrl2 >> 1) & 15);
// size_t num_handles_move = ((ctrl2 >> 5) & 15);
//
// size_t num_handles = num_handles_copy + num_handles_move;
// u32* buf_after_handles = buf + num_handles;
//
// if (num_handles > IPC_MAX_OBJECTS)
// num_handles = IPC_MAX_OBJECTS;
//
// for (i=0; i<num_handles; i++)
// {
// r->Handles[i] = *(buf+i);
// r->WasHandleCopied[i] = (i < num_handles_copy);
// }
//
// r->NumHandles = num_handles;
// buf = buf_after_handles;
// }
//
// size_t num_statics = (ctrl0 >> 16) & 15;
// u32* buf_after_statics = buf + num_statics*2;
//
// if (num_statics > IPC_MAX_BUFFERS)
// num_statics = IPC_MAX_BUFFERS;
//
// for (i=0; i<num_statics; i++, buf+=2) {
// IpcStaticSendDescriptor* desc = (IpcStaticSendDescriptor*) buf;
// u64 packed = (u64) desc->Packed;
//
// r->Statics[i] = (void*) (desc->Addr | (((packed >> 12) & 15) << 32) | (((packed >> 6) & 15) << 36));
// r->StaticSizes[i] = packed >> 16;
// r->StaticIndices[i] = packed & 63;
// }
//
// r->NumStatics = num_statics;
// buf = buf_after_statics;
//
// size_t num_bufs_send = (ctrl0 >> 20) & 15;
// size_t num_bufs_recv = (ctrl0 >> 24) & 15;
// size_t num_bufs_exch = (ctrl0 >> 28) & 15;
//
// size_t num_bufs = num_bufs_send + num_bufs_recv + num_bufs_exch;
// r->Raw = (void*)(((uintptr_t)(buf + num_bufs*3) + 15) &~ 15);
// r->RawWithoutPadding = (void*)((uintptr_t)(buf + num_bufs*3));
//
// if (num_bufs > IPC_MAX_BUFFERS)
// num_bufs = IPC_MAX_BUFFERS;
//
// for (i=0; i<num_bufs; i++, buf+=3) {
// IpcBufferDescriptor* desc = (IpcBufferDescriptor*) buf;
// u64 packed = (u64) desc->Packed;
//
// r->Buffers[i] = (void*) (desc->Addr | ((packed >> 28) << 32) | (((packed >> 2) & 15) << 36));
// r->BufferSizes[i] = desc->Size;
// r->BufferTypes[i] = (BufferType) (packed & 3);
//
// if (i < num_bufs_send)
// r->BufferDirections[i] = BufferDirection_Send;
// else if (i < (num_bufs_send + num_bufs_recv))
// r->BufferDirections[i] = BufferDirection_Recv;
// else
// r->BufferDirections[i] = BufferDirection_Exch;
// }
//
// r->NumBuffers = num_bufs;
// return 0;
//}
/**
* @brief Queries the size of an IPC pointer buffer.
* @param session IPC session handle.
* @param size Output variable in which to store the size.
* @return Result code.
*/
static inline Result ipcQueryPointerBufferSize(Handle session, size_t *size);//{
// u32* buf = (u32*)armGetTls();
//
// buf[0] = IpcCommandType_Control;
// buf[1] = 8;
// buf[2] = 0;
// buf[3] = 0;
// buf[4] = SFCI_MAGIC;
// buf[5] = 0;
// buf[6] = 3;
// buf[7] = 0;
//
// Result rc = ipcDispatch(session);
//
// if (R_SUCCEEDED(rc)) {
// IpcParsedCommand r;
// ipcParse(&r);
//
// struct ipcQueryPointerBufferSizeResponse {
// u64 magic;
// u64 result;
// u32 size;
// } *raw = (struct ipcQueryPointerBufferSizeResponse*)r.Raw;
//
// rc = raw->result;
//
// if (R_SUCCEEDED(rc)) {
// *size = raw->size & 0xffff;
// }
// }
//
// return rc;
//}
/**
* @brief Closes the IPC session with proper clean up.
* @param session IPC session handle.
* @return Result code.
*/
static inline Result ipcCloseSession(Handle session);//{
// u32* buf = (u32*)armGetTls();
// buf[0] = IpcCommandType_Close;
// buf[1] = 0;
// return ipcDispatch(session);
//}
///@}
///@name IPC domain handling
///@{
/**
* @brief Converts an IPC session handle into a domain.
* @param session IPC session handle.
* @param object_id_out Output variable in which to store the object ID.
* @return Result code.
*/
static inline Result ipcConvertSessionToDomain(Handle session, u32* object_id_out);//{
// u32* buf = (u32*)armGetTls();
//
// buf[0] = IpcCommandType_Control;
// buf[1] = 8;
// buf[4] = SFCI_MAGIC;
// buf[5] = 0;
// buf[6] = 0;
// buf[7] = 0;
//
// Result rc = ipcDispatch(session);
//
// if (R_SUCCEEDED(rc)) {
// IpcParsedCommand r;
// ipcParse(&r);
//
// struct ipcConvertSessionToDomainResponse {
// u64 magic;
// u64 result;
// u32 object_id;
// } *raw = (struct ipcConvertSessionToDomainResponse*)r.Raw;
//
// rc = raw->result;
//
// if (R_SUCCEEDED(rc)) {
// *object_id_out = raw->object_id;
// }
// }
//
// return rc;
//}
/**
* @brief Adds an object ID to be sent through an IPC domain command structure.
* @param cmd IPC domain command structure.
* @param object_id Object ID to send.
*/
static inline void ipcSendObjectId(IpcCommand* cmd, u32 object_id);//{
// cmd->ObjectIds[cmd->NumObjectIds++] = object_id;
//}
/**
* @brief Prepares the header of an IPC command structure (domain version).
* @param cmd IPC command structure.
* @param sizeof_raw Size in bytes of the raw data structure to embed inside the IPC request
* @oaram object_id Domain object ID.
* @return Pointer to the raw embedded data structure in the request, ready to be filled out.
*/
static inline void* ipcPrepareHeaderForDomain(IpcCommand* cmd, size_t sizeof_raw, u32 object_id);//{
// void* raw = ipcPrepareHeader(cmd, sizeof_raw + sizeof(DomainMessageHeader));
// DomainMessageHeader* hdr = (DomainMessageHeader*) raw;
// u32 *object_ids = (u32*)(((uintptr_t) raw) + sizeof(DomainMessageHeader) + sizeof_raw);
//
// hdr->Type = DomainMessageType_SendMessage;
// hdr->NumObjectIds = (u8)cmd->NumObjectIds;
// hdr->Length = sizeof_raw;
// hdr->ThisObjectId = object_id;
// hdr->Pad[0] = hdr->Pad[1] = 0;
//
// for(size_t i = 0; i < cmd->NumObjectIds; i++)
// object_ids[i] = cmd->ObjectIds[i];
// return (void*)(((uintptr_t) raw) + sizeof(DomainMessageHeader));
//}
/**
* @brief Parse an IPC command response into an IPC parsed command structure (domain version).
* @param IPC parsed command structure to fill in.
* @return Result code.
*/
static inline Result ipcParseForDomain(IpcParsedCommand* r);//{
// Result rc = ipcParse(r);
// DomainMessageHeader *hdr;
// u32 *object_ids;
// if(R_FAILED(rc))
// return rc;
//
// hdr = (DomainMessageHeader*) r->Raw;
// object_ids = (u32*)(((uintptr_t) hdr) + sizeof(DomainMessageHeader) + hdr->Length);
// r->Raw = (void*)(((uintptr_t) r->Raw) + sizeof(DomainMessageHeader));
//
// r->IsDomainMessage = true;
// r->MessageType = (DomainMessageType)(hdr->Type);
// switch (r->MessageType) {
// case DomainMessageType_SendMessage:
// case DomainMessageType_Close:
// break;
// default:
// return MAKERESULT(Module_Libnx, LibnxError_DomainMessageUnknownType);
// }
// r->ThisObjectId = hdr->ThisObjectId;
// r->NumObjectIds = hdr->NumObjectIds > 8 ? 8 : hdr->NumObjectIds;
// if ((uintptr_t)object_ids + sizeof(u32) * r->NumObjectIds - (uintptr_t)armGetTls() >= 0x100) {
// return MAKERESULT(Module_Libnx, LibnxError_DomainMessageTooManyObjectIds);
// }
// for(size_t i = 0; i < r->NumObjectIds; i++)
// r->ObjectIds[i] = object_ids[i];
//
// return rc;
//}
/**
* @brief Closes a domain object by ID.
* @param session IPC session handle.
* @param object_id ID of the object to close.
* @return Result code.
*/
static inline Result ipcCloseObjectById(Handle session, u32 object_id);//{
// IpcCommand c;
// DomainMessageHeader* hdr;
//
// ipcInitialize(&c);
// hdr = (DomainMessageHeader*)ipcPrepareHeader(&c, sizeof(DomainMessageHeader));
//
// hdr->Type = 2;
// hdr->NumObjectIds = 0;
// hdr->Length = 0;
// hdr->ThisObjectId = object_id;
// hdr->Pad[0] = hdr->Pad[1] = 0;
//
// return ipcDispatch(session); // this command has no associated response
//}
///@}

View file

@ -1,68 +0,0 @@
/**
* @file condvar.h
* @brief Condition variable synchronization primitive.
* @author plutoo
* @copyright libnx Authors
*/
#pragma once
#include "../types.h"
#include "../kernel/mutex.h"
/// Condition variable structure.
typedef struct {
u32 tag;
Mutex* mutex;
} CondVar;
/**
* @brief Initializes a condition variable.
* @param[in] c Condition variable object.
* @param[in] m Mutex object to use inside the condition variable.
*/
void condvarInit(CondVar* c, Mutex* m);
/**
* @brief Waits on a condition variable with a timeout.
* @param[in] c Condition variable object.
* @param[in] timeout Timeout in nanoseconds.
* @return Result code (0xEA01 on timeout).
* @remark On function return, the underlying mutex is acquired.
*/
Result condvarWaitTimeout(CondVar* c, u64 timeout);
/**
* @brief Waits on a condition variable.
* @param[in] c Condition variable object.
* @return Result code.
* @remark On function return, the underlying mutex is acquired.
*/
static inline Result condvarWait(CondVar* c);//{
// return condvarWaitTimeout(c, -1ull);
//}
/**
* @brief Wakes up up to the specified number of threads waiting on a condition variable.
* @param[in] c Condition variable object.
* @param[in] num Maximum number of threads to wake up (or -1 to wake them all up).
* @return Result code.
*/
Result condvarWake(CondVar* c, int num);
/**
* @brief Wakes up a single thread waiting on a condition variable.
* @param[in] c Condition variable object.
* @return Result code.
*/
static inline Result condvarWakeOne(CondVar* c);//{
// return condvarWake(c, 1);
//}
/**
* @brief Wakes up all thread waiting on a condition variable.
* @param[in] c Condition variable object.
* @return Result code.
*/
static inline Result condvarWakeAll(CondVar* c);//{
// return condvarWake(c, -1);
//}

View file

@ -1,675 +0,0 @@
/**
* @file ipc.h
* @brief Inter-process communication handling
* @author plutoo
* @copyright libnx Authors
*/
#pragma once
#include "../result.h"
#include "../arm/tls.h"
#include "../kernel/svc.h"
/// IPC input header magic
#define SFCI_MAGIC 0x49434653
/// IPC output header magic
#define SFCO_MAGIC 0x4f434653
/// IPC invalid object ID
#define IPC_INVALID_OBJECT_ID UINT32_MAX
///@name IPC request building
///@{
/// IPC command (request) structure.
#define IPC_MAX_BUFFERS 8
#define IPC_MAX_OBJECTS 8
typedef enum {
BufferType_Normal=0, ///< Regular buffer.
BufferType_Type1=1, ///< Allows ProcessMemory and shared TransferMemory.
BufferType_Invalid=2,
BufferType_Type3=3 ///< Same as Type1 except remote process is not allowed to use device-mapping.
} BufferType;
typedef enum {
BufferDirection_Send=0,
BufferDirection_Recv=1,
BufferDirection_Exch=2,
} BufferDirection;
typedef enum {
IpcCommandType_Invalid = 0,
IpcCommandType_LegacyRequest = 1,
IpcCommandType_Close = 2,
IpcCommandType_LegacyControl = 3,
IpcCommandType_Request = 4,
IpcCommandType_Control = 5,
IpcCommandType_RequestWithContext = 6,
IpcCommandType_ControlWithContext = 7,
} IpcCommandType;
typedef enum {
DomainMessageType_Invalid = 0,
DomainMessageType_SendMessage = 1,
DomainMessageType_Close = 2,
} DomainMessageType;
/// IPC domain message header.
typedef struct {
u8 Type;
u8 NumObjectIds;
u16 Length;
u32 ThisObjectId;
u32 Pad[2];
} DomainMessageHeader;
typedef struct {
size_t NumSend; // A
size_t NumRecv; // B
size_t NumExch; // W
const void* Buffers[IPC_MAX_BUFFERS];
size_t BufferSizes[IPC_MAX_BUFFERS];
BufferType BufferTypes[IPC_MAX_BUFFERS];
size_t NumStaticIn; // X
size_t NumStaticOut; // C
const void* Statics[IPC_MAX_BUFFERS];
size_t StaticSizes[IPC_MAX_BUFFERS];
u8 StaticIndices[IPC_MAX_BUFFERS];
bool SendPid;
size_t NumHandlesCopy;
size_t NumHandlesMove;
Handle Handles[IPC_MAX_OBJECTS];
size_t NumObjectIds;
u32 ObjectIds[IPC_MAX_OBJECTS];
} IpcCommand;
/**
* @brief Initializes an IPC command structure.
* @param cmd IPC command structure.
*/
static inline void ipcInitialize(IpcCommand* cmd) {
*cmd = (IpcCommand){0};
}
/// IPC buffer descriptor.
typedef struct {
u32 Size; ///< Size of the buffer.
u32 Addr; ///< Lower 32-bits of the address of the buffer
u32 Packed; ///< Packed data (including higher bits of the address)
} IpcBufferDescriptor;
/// IPC static send-buffer descriptor.
typedef struct {
u32 Packed; ///< Packed data (including higher bits of the address)
u32 Addr; ///< Lower 32-bits of the address
} IpcStaticSendDescriptor;
/// IPC static receive-buffer descriptor.
typedef struct {
u32 Addr; ///< Lower 32-bits of the address of the buffer
u32 Packed; ///< Packed data (including higher bits of the address)
} IpcStaticRecvDescriptor;
/**
* @brief Adds a buffer to an IPC command structure.
* @param cmd IPC command structure.
* @param buffer Address of the buffer.
* @param size Size of the buffer.
* @param type Buffer type.
*/
static inline void ipcAddSendBuffer(IpcCommand* cmd, const void* buffer, size_t size, BufferType type) {
size_t off = cmd->NumSend;
cmd->Buffers[off] = buffer;
cmd->BufferSizes[off] = size;
cmd->BufferTypes[off] = type;
cmd->NumSend++;
}
/**
* @brief Adds a receive-buffer to an IPC command structure.
* @param cmd IPC command structure.
* @param buffer Address of the buffer.
* @param size Size of the buffer.
* @param type Buffer type.
*/
static inline void ipcAddRecvBuffer(IpcCommand* cmd, void* buffer, size_t size, BufferType type) {
size_t off = cmd->NumSend + cmd->NumRecv;
cmd->Buffers[off] = buffer;
cmd->BufferSizes[off] = size;
cmd->BufferTypes[off] = type;
cmd->NumRecv++;
}
/**
* @brief Adds an exchange-buffer to an IPC command structure.
* @param cmd IPC command structure.
* @param buffer Address of the buffer.
* @param size Size of the buffer.
* @param type Buffer type.
*/
static inline void ipcAddExchBuffer(IpcCommand* cmd, void* buffer, size_t size, BufferType type) {
size_t off = cmd->NumSend + cmd->NumRecv + cmd->NumExch;
cmd->Buffers[off] = buffer;
cmd->BufferSizes[off] = size;
cmd->BufferTypes[off] = type;
cmd->NumExch++;
}
/**
* @brief Adds a static-buffer to an IPC command structure.
* @param cmd IPC command structure.
* @param buffer Address of the buffer.
* @param size Size of the buffer.
* @param index Index of buffer.
*/
static inline void ipcAddSendStatic(IpcCommand* cmd, const void* buffer, size_t size, u8 index) {
size_t off = cmd->NumStaticIn;
cmd->Statics[off] = buffer;
cmd->StaticSizes[off] = size;
cmd->StaticIndices[off] = index;
cmd->NumStaticIn++;
}
/**
* @brief Adds a static-receive-buffer to an IPC command structure.
* @param cmd IPC command structure.
* @param buffer Address of the buffer.
* @param size Size of the buffer.
* @param index Index of buffer.
*/
static inline void ipcAddRecvStatic(IpcCommand* cmd, void* buffer, size_t size, u8 index) {
size_t off = cmd->NumStaticIn + cmd->NumStaticOut;
cmd->Statics[off] = buffer;
cmd->StaticSizes[off] = size;
cmd->StaticIndices[off] = index;
cmd->NumStaticOut++;
}
/**
* @brief Adds a smart-buffer (buffer + static-buffer pair) to an IPC command structure.
* @param cmd IPC command structure.
* @param ipc_buffer_size IPC buffer size.
* @param buffer Address of the buffer.
* @param size Size of the buffer.
* @param index Index of buffer.
*/
static inline void ipcAddSendSmart(IpcCommand* cmd, size_t ipc_buffer_size, const void* buffer, size_t size, u8 index) {
if (ipc_buffer_size != 0 && size <= ipc_buffer_size) {
ipcAddSendBuffer(cmd, NULL, 0, BufferType_Normal);
ipcAddSendStatic(cmd, buffer, size, index);
} else {
ipcAddSendBuffer(cmd, buffer, size, BufferType_Normal);
ipcAddSendStatic(cmd, NULL, 0, index);
}
}
/**
* @brief Adds a smart-receive-buffer (buffer + static-receive-buffer pair) to an IPC command structure.
* @param cmd IPC command structure.
* @param ipc_buffer_size IPC buffer size.
* @param buffer Address of the buffer.
* @param size Size of the buffer.
* @param index Index of buffer.
*/
static inline void ipcAddRecvSmart(IpcCommand* cmd, size_t ipc_buffer_size, void* buffer, size_t size, u8 index) {
if (ipc_buffer_size != 0 && size <= ipc_buffer_size) {
ipcAddRecvBuffer(cmd, NULL, 0, BufferType_Normal);
ipcAddRecvStatic(cmd, buffer, size, index);
} else {
ipcAddRecvBuffer(cmd, buffer, size, BufferType_Normal);
ipcAddRecvStatic(cmd, NULL, 0, index);
}
}
/**
* @brief Tags an IPC command structure to send the PID.
* @param cmd IPC command structure.
*/
static inline void ipcSendPid(IpcCommand* cmd) {
cmd->SendPid = true;
}
/**
* @brief Adds a copy-handle to be sent through an IPC command structure.
* @param cmd IPC command structure.
* @param h Handle to send.
* @remark The receiving process gets a copy of the handle.
*/
static inline void ipcSendHandleCopy(IpcCommand* cmd, Handle h) {
cmd->Handles[cmd->NumHandlesCopy++] = h;
}
/**
* @brief Adds a move-handle to be sent through an IPC command structure.
* @param cmd IPC command structure.
* @param h Handle to send.
* @remark The sending process loses ownership of the handle, which is transferred to the receiving process.
*/
static inline void ipcSendHandleMove(IpcCommand* cmd, Handle h) {
cmd->Handles[cmd->NumHandlesCopy + cmd->NumHandlesMove++] = h;
}
/**
* @brief Prepares the header of an IPC command structure.
* @param cmd IPC command structure.
* @param sizeof_raw Size in bytes of the raw data structure to embed inside the IPC request
* @return Pointer to the raw embedded data structure in the request, ready to be filled out.
*/
static inline void* ipcPrepareHeader(IpcCommand* cmd, size_t sizeof_raw) {
u32* buf = (u32*)armGetTls();
size_t i;
*buf++ = IpcCommandType_Request | (cmd->NumStaticIn << 16) | (cmd->NumSend << 20) | (cmd->NumRecv << 24) | (cmd->NumExch << 28);
u32* fill_in_size_later = buf;
if (cmd->NumStaticOut > 0) {
*buf = (cmd->NumStaticOut + 2) << 10;
}
else {
*buf = 0;
}
if (cmd->SendPid || cmd->NumHandlesCopy > 0 || cmd->NumHandlesMove > 0) {
*buf++ |= 0x80000000;
*buf++ = (!!cmd->SendPid) | (cmd->NumHandlesCopy << 1) | (cmd->NumHandlesMove << 5);
if (cmd->SendPid)
buf += 2;
for (i=0; i<(cmd->NumHandlesCopy + cmd->NumHandlesMove); i++)
*buf++ = cmd->Handles[i];
}
else {
buf++;
}
for (i=0; i<cmd->NumStaticIn; i++, buf+=2) {
IpcStaticSendDescriptor* desc = (IpcStaticSendDescriptor*) buf;
uintptr_t ptr = (uintptr_t) cmd->Statics[i];
desc->Addr = ptr;
desc->Packed = cmd->StaticIndices[i] | (cmd->StaticSizes[i] << 16) |
(((ptr >> 32) & 15) << 12) | (((ptr >> 36) & 15) << 6);
}
for (i=0; i<(cmd->NumSend + cmd->NumRecv + cmd->NumExch); i++, buf+=3) {
IpcBufferDescriptor* desc = (IpcBufferDescriptor*) buf;
desc->Size = cmd->BufferSizes[i];
uintptr_t ptr = (uintptr_t) cmd->Buffers[i];
desc->Addr = ptr;
desc->Packed = cmd->BufferTypes[i] |
(((ptr >> 32) & 15) << 28) | ((ptr >> 36) << 2);
}
u32 padding = ((16 - (((uintptr_t) buf) & 15)) & 15) / 4;
u32* raw = (u32*) (buf + padding);
size_t raw_size = (sizeof_raw/4) + 4;
buf += raw_size;
u16* buf_u16 = (u16*) buf;
for (i=0; i<cmd->NumStaticOut; i++) {
size_t off = cmd->NumStaticIn + i;
size_t sz = (uintptr_t) cmd->StaticSizes[off];
buf_u16[i] = (sz > 0xFFFF) ? 0 : sz;
}
size_t u16s_size = ((2*cmd->NumStaticOut) + 3)/4;
buf += u16s_size;
raw_size += u16s_size;
*fill_in_size_later |= raw_size;
for (i=0; i<cmd->NumStaticOut; i++, buf+=2) {
IpcStaticRecvDescriptor* desc = (IpcStaticRecvDescriptor*) buf;
size_t off = cmd->NumStaticIn + i;
uintptr_t ptr = (uintptr_t) cmd->Statics[off];
desc->Addr = ptr;
desc->Packed = (ptr >> 32) | (cmd->StaticSizes[off] << 16);
}
return (void*) raw;
}
/**
* @brief Dispatches an IPC request.
* @param session IPC session handle.
* @return Result code.
*/
static inline Result ipcDispatch(Handle session) {
return svcSendSyncRequest(session);
}
///@}
///@name IPC response parsing
///@{
/// IPC parsed command (response) structure.
typedef struct {
IpcCommandType CommandType; ///< Type of the command
bool HasPid; ///< true if the 'Pid' field is filled out.
u64 Pid; ///< PID included in the response (only if HasPid is true)
size_t NumHandles; ///< Number of handles copied.
Handle Handles[IPC_MAX_OBJECTS]; ///< Handles.
bool WasHandleCopied[IPC_MAX_OBJECTS]; ///< true if the handle was moved, false if it was copied.
bool IsDomainMessage; ///< true if the the message is a Domain message.
DomainMessageType MessageType; ///< Type of the domain message.
u32 MessageLength; ///< Size of rawdata (for domain messages).
u32 ThisObjectId; ///< Object ID to call the command on (for domain messages).
size_t NumObjectIds; ///< Number of object IDs (for domain messages).
u32 ObjectIds[IPC_MAX_OBJECTS]; ///< Object IDs (for domain messages).
size_t NumBuffers; ///< Number of buffers in the response.
void* Buffers[IPC_MAX_BUFFERS]; ///< Pointers to the buffers.
size_t BufferSizes[IPC_MAX_BUFFERS]; ///< Sizes of the buffers.
BufferType BufferTypes[IPC_MAX_BUFFERS]; ///< Types of the buffers.
BufferDirection BufferDirections[IPC_MAX_BUFFERS]; ///< Direction of each buffer.
size_t NumStatics; ///< Number of statics in the response.
void* Statics[IPC_MAX_BUFFERS]; ///< Pointers to the statics.
size_t StaticSizes[IPC_MAX_BUFFERS]; ///< Sizes of the statics.
u8 StaticIndices[IPC_MAX_BUFFERS]; ///< Indices of the statics.
size_t NumStaticsOut; ///< Number of output statics available in the response.
void* Raw; ///< Pointer to the raw embedded data structure in the response.
void* RawWithoutPadding; ///< Pointer to the raw embedded data structure, without padding.
size_t RawSize; ///< Size of the raw embedded data.
} IpcParsedCommand;
/**
* @brief Parse an IPC command response into an IPC parsed command structure.
* @param IPC parsed command structure to fill in.
* @return Result code.
*/
static inline Result ipcParse(IpcParsedCommand* r) {
u32* buf = (u32*)armGetTls();
u32 ctrl0 = *buf++;
u32 ctrl1 = *buf++;
size_t i;
r->IsDomainMessage = false;
r->CommandType = (IpcCommandType) (ctrl0 & 0xffff);
r->HasPid = false;
r->RawSize = (ctrl1 & 0x1ff) * 4;
r->NumHandles = 0;
r->NumStaticsOut = (ctrl1 >> 10) & 15;
if (r->NumStaticsOut >> 1) r->NumStaticsOut--; // Value 2 -> Single descriptor
if (r->NumStaticsOut >> 1) r->NumStaticsOut--; // Value 3+ -> (Value - 2) descriptors
if (ctrl1 & 0x80000000) {
u32 ctrl2 = *buf++;
if (ctrl2 & 1) {
r->HasPid = true;
r->Pid = *buf++;
r->Pid |= ((u64)(*buf++)) << 32;
}
size_t num_handles_copy = ((ctrl2 >> 1) & 15);
size_t num_handles_move = ((ctrl2 >> 5) & 15);
size_t num_handles = num_handles_copy + num_handles_move;
u32* buf_after_handles = buf + num_handles;
if (num_handles > IPC_MAX_OBJECTS)
num_handles = IPC_MAX_OBJECTS;
for (i=0; i<num_handles; i++)
{
r->Handles[i] = *(buf+i);
r->WasHandleCopied[i] = (i < num_handles_copy);
}
r->NumHandles = num_handles;
buf = buf_after_handles;
}
size_t num_statics = (ctrl0 >> 16) & 15;
u32* buf_after_statics = buf + num_statics*2;
if (num_statics > IPC_MAX_BUFFERS)
num_statics = IPC_MAX_BUFFERS;
for (i=0; i<num_statics; i++, buf+=2) {
IpcStaticSendDescriptor* desc = (IpcStaticSendDescriptor*) buf;
u64 packed = (u64) desc->Packed;
r->Statics[i] = (void*) (desc->Addr | (((packed >> 12) & 15) << 32) | (((packed >> 6) & 15) << 36));
r->StaticSizes[i] = packed >> 16;
r->StaticIndices[i] = packed & 63;
}
r->NumStatics = num_statics;
buf = buf_after_statics;
size_t num_bufs_send = (ctrl0 >> 20) & 15;
size_t num_bufs_recv = (ctrl0 >> 24) & 15;
size_t num_bufs_exch = (ctrl0 >> 28) & 15;
size_t num_bufs = num_bufs_send + num_bufs_recv + num_bufs_exch;
r->Raw = (void*)(((uintptr_t)(buf + num_bufs*3) + 15) &~ 15);
r->RawWithoutPadding = (void*)((uintptr_t)(buf + num_bufs*3));
if (num_bufs > IPC_MAX_BUFFERS)
num_bufs = IPC_MAX_BUFFERS;
for (i=0; i<num_bufs; i++, buf+=3) {
IpcBufferDescriptor* desc = (IpcBufferDescriptor*) buf;
u64 packed = (u64) desc->Packed;
r->Buffers[i] = (void*) (desc->Addr | ((packed >> 28) << 32) | (((packed >> 2) & 15) << 36));
r->BufferSizes[i] = desc->Size;
r->BufferTypes[i] = (BufferType) (packed & 3);
if (i < num_bufs_send)
r->BufferDirections[i] = BufferDirection_Send;
else if (i < (num_bufs_send + num_bufs_recv))
r->BufferDirections[i] = BufferDirection_Recv;
else
r->BufferDirections[i] = BufferDirection_Exch;
}
r->NumBuffers = num_bufs;
return 0;
}
/**
* @brief Queries the size of an IPC pointer buffer.
* @param session IPC session handle.
* @param size Output variable in which to store the size.
* @return Result code.
*/
static inline Result ipcQueryPointerBufferSize(Handle session, size_t *size) {
u32* buf = (u32*)armGetTls();
buf[0] = IpcCommandType_Control;
buf[1] = 8;
buf[2] = 0;
buf[3] = 0;
buf[4] = SFCI_MAGIC;
buf[5] = 0;
buf[6] = 3;
buf[7] = 0;
Result rc = ipcDispatch(session);
if (R_SUCCEEDED(rc)) {
IpcParsedCommand r;
ipcParse(&r);
struct ipcQueryPointerBufferSizeResponse {
u64 magic;
u64 result;
u32 size;
} *raw = (struct ipcQueryPointerBufferSizeResponse*)r.Raw;
rc = raw->result;
if (R_SUCCEEDED(rc)) {
*size = raw->size & 0xffff;
}
}
return rc;
}
/**
* @brief Closes the IPC session with proper clean up.
* @param session IPC session handle.
* @return Result code.
*/
static inline Result ipcCloseSession(Handle session) {
u32* buf = (u32*)armGetTls();
buf[0] = IpcCommandType_Close;
buf[1] = 0;
return ipcDispatch(session);
}
///@}
///@name IPC domain handling
///@{
/**
* @brief Converts an IPC session handle into a domain.
* @param session IPC session handle.
* @param object_id_out Output variable in which to store the object ID.
* @return Result code.
*/
static inline Result ipcConvertSessionToDomain(Handle session, u32* object_id_out) {
u32* buf = (u32*)armGetTls();
buf[0] = IpcCommandType_Control;
buf[1] = 8;
buf[4] = SFCI_MAGIC;
buf[5] = 0;
buf[6] = 0;
buf[7] = 0;
Result rc = ipcDispatch(session);
if (R_SUCCEEDED(rc)) {
IpcParsedCommand r;
ipcParse(&r);
struct ipcConvertSessionToDomainResponse {
u64 magic;
u64 result;
u32 object_id;
} *raw = (struct ipcConvertSessionToDomainResponse*)r.Raw;
rc = raw->result;
if (R_SUCCEEDED(rc)) {
*object_id_out = raw->object_id;
}
}
return rc;
}
/**
* @brief Adds an object ID to be sent through an IPC domain command structure.
* @param cmd IPC domain command structure.
* @param object_id Object ID to send.
*/
static inline void ipcSendObjectId(IpcCommand* cmd, u32 object_id) {
cmd->ObjectIds[cmd->NumObjectIds++] = object_id;
}
/**
* @brief Prepares the header of an IPC command structure (domain version).
* @param cmd IPC command structure.
* @param sizeof_raw Size in bytes of the raw data structure to embed inside the IPC request
* @oaram object_id Domain object ID.
* @return Pointer to the raw embedded data structure in the request, ready to be filled out.
*/
static inline void* ipcPrepareHeaderForDomain(IpcCommand* cmd, size_t sizeof_raw, u32 object_id) {
void* raw = ipcPrepareHeader(cmd, sizeof_raw + sizeof(DomainMessageHeader));
DomainMessageHeader* hdr = (DomainMessageHeader*) raw;
u32 *object_ids = (u32*)(((uintptr_t) raw) + sizeof(DomainMessageHeader) + sizeof_raw);
hdr->Type = DomainMessageType_SendMessage;
hdr->NumObjectIds = (u8)cmd->NumObjectIds;
hdr->Length = sizeof_raw;
hdr->ThisObjectId = object_id;
hdr->Pad[0] = hdr->Pad[1] = 0;
for(size_t i = 0; i < cmd->NumObjectIds; i++)
object_ids[i] = cmd->ObjectIds[i];
return (void*)(((uintptr_t) raw) + sizeof(DomainMessageHeader));
}
/**
* @brief Parse an IPC command response into an IPC parsed command structure (domain version).
* @param IPC parsed command structure to fill in.
* @return Result code.
*/
static inline Result ipcParseForDomain(IpcParsedCommand* r) {
Result rc = ipcParse(r);
DomainMessageHeader *hdr;
u32 *object_ids;
if(R_FAILED(rc))
return rc;
hdr = (DomainMessageHeader*) r->Raw;
object_ids = (u32*)(((uintptr_t) hdr) + sizeof(DomainMessageHeader) + hdr->Length);
r->Raw = (void*)(((uintptr_t) r->Raw) + sizeof(DomainMessageHeader));
r->IsDomainMessage = true;
r->MessageType = (DomainMessageType)(hdr->Type);
switch (r->MessageType) {
case DomainMessageType_SendMessage:
case DomainMessageType_Close:
break;
default:
return MAKERESULT(Module_Libnx, LibnxError_DomainMessageUnknownType);
}
r->ThisObjectId = hdr->ThisObjectId;
r->NumObjectIds = hdr->NumObjectIds > 8 ? 8 : hdr->NumObjectIds;
if ((uintptr_t)object_ids + sizeof(u32) * r->NumObjectIds - (uintptr_t)armGetTls() >= 0x100) {
return MAKERESULT(Module_Libnx, LibnxError_DomainMessageTooManyObjectIds);
}
for(size_t i = 0; i < r->NumObjectIds; i++)
r->ObjectIds[i] = object_ids[i];
return rc;
}
/**
* @brief Closes a domain object by ID.
* @param session IPC session handle.
* @param object_id ID of the object to close.
* @return Result code.
*/
static inline Result ipcCloseObjectById(Handle session, u32 object_id) {
IpcCommand c;
DomainMessageHeader* hdr;
ipcInitialize(&c);
hdr = (DomainMessageHeader*)ipcPrepareHeader(&c, sizeof(DomainMessageHeader));
hdr->Type = 2;
hdr->NumObjectIds = 0;
hdr->Length = 0;
hdr->ThisObjectId = object_id;
hdr->Pad[0] = hdr->Pad[1] = 0;
return ipcDispatch(session); // this command has no associated response
}
///@}

View file

@ -1,71 +0,0 @@
/**
* @file condvar.h
* @brief Condition variable synchronization primitive.
* @author plutoo
* @copyright libnx Authors
*/
#pragma once
#include "../types.h"
#include "../kernel/mutex.h"
/// Condition variable structure.
typedef struct {
u32 tag;
Mutex* mutex;
} CondVar;
/**
* @brief Initializes a condition variable.
* @param[in] c Condition variable object.
* @param[in] m Mutex object to use inside the condition variable.
*/
void condvarInit(CondVar* c, Mutex* m);
/**
* @brief Waits on a condition variable with a timeout.
* @param[in] c Condition variable object.
* @param[in] timeout Timeout in nanoseconds.
* @return Result code (0xEA01 on timeout).
* @remark On function return, the underlying mutex is acquired.
*/
Result condvarWaitTimeout(CondVar* c, u64 timeout);
/**
* @brief Waits on a condition variable.
* @param[in] c Condition variable object.
* @return Result code.
* @remark On function return, the underlying mutex is acquired.
*/
static inline Result condvarWait(CondVar* c)
{
return condvarWaitTimeout(c, -1ull);
}
/**
* @brief Wakes up up to the specified number of threads waiting on a condition variable.
* @param[in] c Condition variable object.
* @param[in] num Maximum number of threads to wake up (or -1 to wake them all up).
* @return Result code.
*/
Result condvarWake(CondVar* c, int num);
/**
* @brief Wakes up a single thread waiting on a condition variable.
* @param[in] c Condition variable object.
* @return Result code.
*/
static inline Result condvarWakeOne(CondVar* c)
{
return condvarWake(c, 1);
}
/**
* @brief Wakes up all thread waiting on a condition variable.
* @param[in] c Condition variable object.
* @return Result code.
*/
static inline Result condvarWakeAll(CondVar* c)
{
return condvarWake(c, -1);
}

View file

@ -1,68 +0,0 @@
/**
* @file condvar.h
* @brief Condition variable synchronization primitive.
* @author plutoo
* @copyright libnx Authors
*/
#pragma once
#include "../types.h"
#include "../kernel/mutex.h"
/// Condition variable structure.
typedef struct {
u32 tag;
Mutex* mutex;
} CondVar;
/**
* @brief Initializes a condition variable.
* @param[in] c Condition variable object.
* @param[in] m Mutex object to use inside the condition variable.
*/
void condvarInit(CondVar* c, Mutex* m);
/**
* @brief Waits on a condition variable with a timeout.
* @param[in] c Condition variable object.
* @param[in] timeout Timeout in nanoseconds.
* @return Result code (0xEA01 on timeout).
* @remark On function return, the underlying mutex is acquired.
*/
Result condvarWaitTimeout(CondVar* c, u64 timeout);
/**
* @brief Waits on a condition variable.
* @param[in] c Condition variable object.
* @return Result code.
* @remark On function return, the underlying mutex is acquired.
*/
static inline Result condvarWait(CondVar* c) {
return condvarWaitTimeout(c, -1ull);
}
/**
* @brief Wakes up up to the specified number of threads waiting on a condition variable.
* @param[in] c Condition variable object.
* @param[in] num Maximum number of threads to wake up (or -1 to wake them all up).
* @return Result code.
*/
Result condvarWake(CondVar* c, int num);
/**
* @brief Wakes up a single thread waiting on a condition variable.
* @param[in] c Condition variable object.
* @return Result code.
*/
static inline Result condvarWakeOne(CondVar* c) {
return condvarWake(c, 1);
}
/**
* @brief Wakes up all thread waiting on a condition variable.
* @param[in] c Condition variable object.
* @return Result code.
*/
static inline Result condvarWakeAll(CondVar* c) {
return condvarWake(c, -1);
}

View file

@ -1 +0,0 @@
--path="../../src"

View file

@ -1,209 +0,0 @@
import nimgen/fileops, common, regex, os
import unittest
let testFileContent = """
this is text
this is text
replace me
prepend me
end
"""
let prependMiddleExpected = """
this is text
this is text
replace me
prepended data
prepend me
end
"""
let prependEndExpected = """
this is text
this is text
replace me
prepend me
data
end
"""
let appendEndExpected = """
this is text
this is text
replace me
prepend me
end
data
"""
let appendMiddleExpected = """
this is data
text
this is text
replace me
prepend me
end
"""
let freplaceDefaultExpected = """
replace me
prepend me
end
"""
let freplaceWithExpected = """
this is text
this is text
foobar
prepend me
end
"""
let freplaceRegexExpected = """
foobar
foobar
replace me
prepend me
end
"""
let commentExpected = """
this is text
this is text
//replace me
//prepend me
//end
"""
let commentMiddleExpected = """
this //is text
//this is text
replace me
prepend me
end
"""
let dataDir = currentSourcePath().splitPath().head / "data"
let testfilename = dataDir / "testing.txt"
suite "test file ops":
if not dataDir.dirExists():
dataDir.createDir()
setup:
writeFile(testfilename, testFileContent)
################### Prepend #######################
test "prepend at beginning of file":
prepend(testfilename, "data\n")
let expected = "data\n" & testFileContent
testfilename.checkFile(expected)
test "prepend at middle of file":
prepend(testfilename, "prepended data\n", "prepend me")
testfilename.checkFile(prependMiddleExpected)
test "prepend at end of file":
prepend(testfilename, "data\n", "end\n")
testfilename.checkFile(prependEndExpected)
################### Pipe #########################
test "pipe command into file":
when defined(windows):
pipe(testfilename, "ECHO foo")
testfilename.checkFile("foo")
else:
pipe(testfilename, "cat $file | grep 'this is text'")
testfilename.checkFile("this is text\nthis is text")
################# Append #########################
test "append file end":
append(testfilename, "data\n")
testfilename.checkFile(appendEndExpected)
test "append file middle":
append(testfilename, " data\n", "this is")
testfilename.checkFile(appendMiddleExpected)
################# FReplace #########################
test "freplace default empty":
freplace(testfilename, "this is text")
testfilename.checkFile(freplaceDefaultExpected)
test "freplace with content":
freplace(testfilename, "replace me", "foobar")
testfilename.checkFile(freplaceWithExpected)
test "freplace regex":
freplace(testfilename, re"this .*", "foobar")
testfilename.checkFile(freplaceRegexExpected)
####################### Comment ######################
test "comment":
comment(testfilename, "replace me", "3")
testfilename.checkFile(commentExpected)
test "comment over length":
comment(testfilename, "replace me", "10")
testfilename.checkFile(commentExpected)
test "comment negative":
comment(testfilename, "replace me", "-3")
testfilename.checkFile(testFileContent)
test "comment zero":
comment(testfilename, "replace me", "0")
testfilename.checkFile(testFileContent)
test "comment middle":
comment(testfilename, "is text", "2")
testfilename.checkFile(commentMiddleExpected)
############### Static inline removal ################
test "replace static inline with front braces at end of line":
let
file = dataDir / "teststaticfrontbraces.h"
resFile = dataDir / "teststaticexpectedfrontbraces.h"
test = readFile(file)
expected = readFile(resFile)
writeFile(testfilename, test)
removeStatic(testfilename)
testfilename.checkFile(expected)
reAddStatic(testfilename)
testfilename.checkFile(test)
test "replace static inline with newline before brace":
let
file = dataDir / "teststaticnewlinebraces.h"
resFile = dataDir / "teststaticexpectednewlinebraces.h"
reAddedFile = dataDir / "teststaticnewlinebracesreadded.h"
test = readFile(file)
expected = readFile(resFile)
reAdded = readFile(reAddedFile)
writeFile(testfilename, test)
removeStatic(testfilename)
testfilename.checkFile(expected)
reAddStatic(testfilename)
testfilename.checkFile(reAdded)

View file

@ -1 +0,0 @@
nimgen.genotrance.com

File diff suppressed because it is too large Load diff