jyapayne/nimgenEx
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Compare commits

base: jyapayne:master
Branches Tags
jyapayne:master
jyapayne:fix_unittest_continue
jyapayne:fix_another_issue
jyapayne:fix_cross_compile_sourcepath
jyapayne:fix_windows_issues
jyapayne:temp_regex_fix
jyapayne:add_static_remove
jyapayne:master_backup
jyapayne:allow_relative_imports
jyapayne:reset_files
jyapayne:add_post_wildcard
jyapayne:configurable_compiler
jyapayne:fix_includes
jyapayne:add_execute_for_files
jyapayne:allow_deeper_output
jyapayne:v0.2.0
jyapayne:v0.1.5
jyapayne:v0.1.3
jyapayne:v0.1.4
jyapayne:v0.1.2
jyapayne:v0.1.1
jyapayne:v0.1.0
..
compare: jyapayne:fix_cross_compile_sourcepath
Branches Tags
jyapayne:master
jyapayne:fix_unittest_continue
jyapayne:fix_another_issue
jyapayne:fix_cross_compile_sourcepath
jyapayne:fix_windows_issues
jyapayne:temp_regex_fix
jyapayne:add_static_remove
jyapayne:master_backup
jyapayne:allow_relative_imports
jyapayne:reset_files
jyapayne:add_post_wildcard
jyapayne:configurable_compiler
jyapayne:fix_includes
jyapayne:add_execute_for_files
jyapayne:allow_deeper_output
jyapayne:v0.2.0
jyapayne:v0.1.5
jyapayne:v0.1.3
jyapayne:v0.1.4
jyapayne:v0.1.2
jyapayne:v0.1.1
jyapayne:v0.1.0

No commits in common. "master" and "fix_cross_compile_sourcepath" have entirely different histories.
master ... fix_cross_

17 changed files with 372 additions and 2102 deletions
Download patch file Download diff file Expand all files Collapse all files

27
.travis.yml
View file

@ -1,27 +0,0 @@
os:
- linux
- osx
language: c
env:
- BRANCH=devel
addons:
apt:
packages:
- libssh2-1-dev
before_install:
- if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then HOMEBREW_NO_AUTO_UPDATE=1 brew install libssh2; fi
install:
- export CHOOSENIM_CHOOSE_VERSION=$BRANCH
- |
curl https://nim-lang.org/choosenim/init.sh -sSf > init.sh
sh init.sh -y
- export PATH=$HOME/.nimble/bin:$PATH
script:
- nimble install -y
- nimble test

85
README.md
View file

@ -1,7 +1,3 @@
[![Chat on Gitter](https://badges.gitter.im/gitterHQ/gitter.png)](https://gitter.im/nimgen/Lobby)
[![Build status](https://ci.appveyor.com/api/projects/status/05t5ja88lmv1rt3r/branch/master?svg=true)](https://ci.appveyor.com/project/genotrance/nimgen/branch/master)
[![Build Status](https://travis-ci.org/genotrance/nimgen.svg?branch=master)](https://travis-ci.org/genotrance/nimgen)
Nimgen is a helper for [c2nim](https://github.com/nim-lang/c2nim/) to simplify and automate the wrapping of C libraries. Nimgen is a helper for [c2nim](https://github.com/nim-lang/c2nim/) to simplify and automate the wrapping of C libraries.
Nimgen can be used to automate the process of manipulating C files so that c2nim can be run on them without issues. This includes adding/removing code snippets, removal of complex preprocessor definitions that c2nim doesn't yet comprehend and recursively running on #include files. Nimgen can be used to automate the process of manipulating C files so that c2nim can be run on them without issues. This includes adding/removing code snippets, removal of complex preprocessor definitions that c2nim doesn't yet comprehend and recursively running on #include files.
@ -43,7 +39,7 @@ To see examples of nimgen in action check out the following wrappers:-
* [nim-clblast](https://github.com/numforge/nim-clblast) - OpenCL BLAS wrapper * [nim-clblast](https://github.com/numforge/nim-clblast) - OpenCL BLAS wrapper
* static * static
* Compile C/C++ code into binary * Compile C code into binary
* [nim7z](https://github.com/genotrance/nim7z) - 7z decoder wrapper: [docs](http://nimgen.genotrance.com/nim7z) * [nim7z](https://github.com/genotrance/nim7z) - 7z decoder wrapper: [docs](http://nimgen.genotrance.com/nim7z)
* git sparse checkout * git sparse checkout
* [nimarchive](https://github.com/genotrance/nimarchive) - libarchive wrapper: [docs](http://nimgen.genotrance.com/nimarchive) * [nimarchive](https://github.com/genotrance/nimarchive) - libarchive wrapper: [docs](http://nimgen.genotrance.com/nimarchive)
@ -52,26 +48,18 @@ To see examples of nimgen in action check out the following wrappers:-
* git checkout * git checkout
* [nimclipboard](https://github.com/genotrance/nimclipboard) - libclipboard wrapper: [docs](http://nimgen.genotrance.com/nimclipboard) * [nimclipboard](https://github.com/genotrance/nimclipboard) - libclipboard wrapper: [docs](http://nimgen.genotrance.com/nimclipboard)
* git checkout * git checkout
* [nimfastText](https://github.com/genotrance/nimfastText) - fastText wrapper: [docs](http://nimgen.genotrance.com/nimfastText) * [nimfuzz](https://github.com/genotrance/nimfuzz) - fts_fuzzy_match wrapper: [docs](http://nimgen.genotrance.com/nimfuzz)
* git sparse checkout
* [nimfuzzy](https://github.com/genotrance/nimfuzzy) - fts_fuzzy_match wrapper: [docs](http://nimgen.genotrance.com/nimfuzzy)
* download header file * download header file
* [nimkerberos](https://github.com/genotrance/nimkerberos) - WinKerberos wrapper: [docs](http://nimgen.genotrance.com/nimkerberos) * [nimkerberos](https://github.com/genotrance/nimkerberos) - WinKerberos wrapper: [docs](http://nimgen.genotrance.com/nimkerberos)
* git sparse checkout * git sparse checkout
* [nimmonocypher](https://github.com/genotrance/nimmonocypher) - monocypher wrapper: [docs](http://nimgen.genotrance.com/nimmonocypher)
* git sparse checkout
* [nimnuklear](https://github.com/genotrance/nimnuklear) - nuklear wrapper: [docs](https://nimgen.genotrance.com/nimnuklear)
* git sparse checkout
* [nimpcre](https://github.com/genotrance/nimpcre) - PCRE wrapper: [docs](http://nimgen.genotrance.com/nimpcre) * [nimpcre](https://github.com/genotrance/nimpcre) - PCRE wrapper: [docs](http://nimgen.genotrance.com/nimpcre)
* git checkout * git checkout
* [nimrax](https://github.com/genotrance/nimrax) - Radix tree wrapper: [docs](http://nimgen.genotrance.com/nimrax) * [nimrax](https://github.com/genotrance/nimrax) - Radix tree wrapper: [docs](http://nimgen.genotrance.com/nimrax)
* git checkout * git checkout
* [nimssl](https://github.com/genotrance/nimssl) - OpenSSL wrapper: [docs](http://nimgen.genotrance.com/nimssl) * [nimssl](https://github.com/genotrance/nimssl) - OpenSSL wrapper: [docs](http://nimgen.genotrance.com/nimssl)
* git sparse checkout * git sparse checkout
* [nimtess2](https://github.com/genotrance/nimtess2) - libtess2 wrapper: [docs](http://nimgen.genotrance.com/nimtess2) * [libsvm](https://github.com/genotrance/libsvm) - libsvm wrapper: [docs](http://nimgen.genotrance.com/libsvm)
* git checkout * git sparse checkout
* [duktape-nim](https://github.com/manguluka/duktape-nim) - Duktape wrapper
* static
* Compile in as static binary * Compile in as static binary
* [nimssh2](https://github.com/genotrance/nimssh2) - libssh2 wrapper: [docs](http://nimgen.genotrance.com/nimssh2) * [nimssh2](https://github.com/genotrance/nimssh2) - libssh2 wrapper: [docs](http://nimgen.genotrance.com/nimssh2)
@ -92,8 +80,6 @@ In all sections below, environment variables are supported via Nim's string inte
"${output}/library/include" "${output}/library/include"
"${MY_INCLUDE_PATH}/include" "${MY_INCLUDE_PATH}/include"
Append -win, -lin and -mac/osx for OS specific sections and tasks. E.g. n.global-win, n.post-lin, download-win, execute-lin-mac.unique1. -unix can be used as a shortcut for -lin-mac.
_[n.global]_ _[n.global]_
```output``` = name of the Nimble project once installed, also location to place generated .nim files ```output``` = name of the Nimble project once installed, also location to place generated .nim files
@ -120,21 +106,17 @@ List of all directories or files to exclude from all parsing. If an entry here m
_[n.prepare]_ _[n.prepare]_
The following keys can be used to prepare dependencies such as downloading ZIP files, cloning Git repositories, etc. Multiple entries are possible by appending any .string to the key. E.g. download.file1. The following keys can be used to prepare dependencies such as downloading ZIP files, cloning Git repositories, etc. Multiple entries are possible by appending any .string to the key. E.g. download.file1. -win, -lin and -mac/osx can be used for OS specific tasks. E.g. download-win, execute-lin,mac.unique1
```download``` = url to download to the output directory. ZIP files are automatically extracted. Files are not redownloaded if already present but re-extracted ```download``` = url to download to the output directory. ZIP files are automatically extracted. Files are not redownloaded if already present but re-extracted
```extract``` = ZIP file to extract in case they are local and don't need to be downloaded. Path is relative to output directory. ```extract``` = ZIP file to extract in case they are local and don't need to be downloaded. Path is relative to output directory.
```gitcheckout``` = branch, commit or tag of repository to checkout in following Git command, resets after each use. Use "-b name" for branches ```git``` = url of Git repository to clone. Full repo is pulled so gitremote + gitsparse is preferable. Resets to HEAD if already present
```gitoutput``` = directory for all following Git commands relative to `n.global:output` [default: `n.global:output` directory]
```git``` = url of Git repository to clone. Full repo is pulled so gitremote + gitsparse is preferable. Resets if already present
```gitremote``` = url of Git repository to partially checkout. Use with gitsparse to pull only files and dirs of interest ```gitremote``` = url of Git repository to partially checkout. Use with gitsparse to pull only files and dirs of interest
```gitsparse``` = list of files and/or dirs to include in partial checkout, one per line. Resets if already present ```gitsparse``` = list of files and/or dirs to include in partial checkout, one per line. Resets to HEAD if already present
```execute``` = command to run during preparation ```execute``` = command to run during preparation
@ -144,9 +126,7 @@ _[n.post]_
This section is the same as the prepare section, but for performing actions after the project has been processed. This section is the same as the prepare section, but for performing actions after the project has been processed.
```gitoutput``` = output directory for Git reset [default: `n.global:output` directory] ```reset``` = whether or not to perform a git reset on all files after processing [default: false]
```reset``` = perform a Git reset on all files after processing [default: false]
```execute``` = command to run after processing ```execute``` = command to run after processing
@ -162,7 +142,7 @@ This section allows selection of multiple sourcefiles without requiring a detail
_[sourcefile]_ _[sourcefile]_
The following keys apply to library source code and help with generating the .nim files. -win, -lin and -osx can be used for OS specific tasks. E.g. dynlib-win, pragma-win. The following keys apply to library source code and help with generating the .nim files. -win, -lin and -osx can be used for OS specific tasks. E.g. dynlib-win, pragma-win
```recurse``` = find #include files and process them [default: false] ```recurse``` = find #include files and process them [default: false]
@ -180,13 +160,11 @@ The following keys apply to library source code and help with generating the .ni
```noprocess``` = do not process this source file with c2nim [default: false] - this is useful if a file only needs to be manipulated ```noprocess``` = do not process this source file with c2nim [default: false] - this is useful if a file only needs to be manipulated
```nowildcard``` = ignore any wildcard definitions for this sourcefile
```reset``` = reset the file back to original state after all processing [default: false] ```reset``` = reset the file back to original state after all processing [default: false]
Multiple entries for the all following keys are possible by appending any .string to the key. E.g. dynlib.win, compile.dir Multiple entries for the all following keys are possible by appending any .string to the key. E.g. dynlib.win, compile.dir
```compile``` = file or dir of files of source code to {.compile.} into generated .nim. If directory, picks *.c if C mode and *.cxx, *.cpp, *.cc, *.c++ and *.C for cpp mode. Dir can also include wildcards. e.g. compile = """dir/A*.cxx""" ```compile``` = file or dir of files of source code to {.compile.} into generated .nim
```pragma``` = pragmas to define in generated .nim file. E.g. pragma = "passL: \"-lssl\"" => {.passL: "-lssl".} ```pragma``` = pragmas to define in generated .nim file. E.g. pragma = "passL: \"-lssl\"" => {.passL: "-lssl".}
@ -196,19 +174,15 @@ The following keys apply to library source code (before processing) and generate
```create``` = create a file at exact location with contents specified. File needs to be in the _[n.exclude]_ list in order to be created. ```create``` = create a file at exact location with contents specified. File needs to be in the _[n.exclude]_ list in order to be created.
```pipe``` = execute a command on a file and store the output of the command as the new file contents. E.g. pipe = "cat $file | grep 'static inline'"
```search``` = search string providing context for following prepend/append/replace directives ```search``` = search string providing context for following prepend/append/replace directives
```regex``` = regex search string providing context for the following replace directive. Specify using """ to avoid regex parsing issues ```pipe``` = execute a command on a file and store the output of the command as the new file contents. Ex: pipe = "cat $file | grep 'static inline'"
```prepend``` = string value to prepend into file at beginning or before search ```prepend``` = string value to prepend into file at beginning or before search
```append``` = string value to append into file at the end or after search ```append``` = string value to append into file at the end or after search
```replace``` = string value to replace search string in file. Regex captures can be referred to using $1, $2, etc. ```replace``` = string value to replace search string in file
```move``` = search string providing context for location to move the results of a preceding search or regex match
```comment``` = number of lines to comment from search location ```comment``` = number of lines to comment from search location
@ -222,41 +196,6 @@ The following key only applies before processing and allows renaming the generat
`$replace(srch1=repl1, srch2=reply2)` = rename specific portions in `$nimout` `$replace(srch1=repl1, srch2=reply2)` = rename specific portions in `$nimout`
__Command Line__
A subset of capabilities are available through the command line to enable quick tests using nimgen. Command line flags only apply to source files specified on the command line and do not influence any ```cfg``` files which are expected to be self-sufficient.
```
Usage:
nimgen [options] file.cfg|file.h ...
Params:
-C<compile> add compile entry *
-E<exclude> add n.exclude entry *
-F<flags> set c2nim flags *
-I<include> add n.include dir *
-O<outdir> set output directory
-P<ppflags> set preprocessor flags *
Options:
-c set ctags = true
-d set defines = true
-i set inline = true
-n set noprocess = true
-p set preprocess = true
-r set recurse = true
Editing:
-a<append> append string *
-e<prepend> prepend string *
-l<replace> replace string *
-o#lines comment X lines *
-s<search> search string *
-x<regex> regex search string *
* supports multiple instances
```
__Feedback__ __Feedback__
Nimgen is a work in progress and any feedback or suggestions are welcome. It is hosted on [GitHub](https://github.com/genotrance/nimgen) with an MIT license so issues, forks and PRs are most appreciated. Also join us at https://gitter.im/nimgen/Lobby to chat about nimgen and the future of Nim wrappers. Nimgen is a work in progress and any feedback or suggestions are welcome. It is hosted on [GitHub](https://github.com/genotrance/nimgen) with an MIT license so issues, forks and PRs are most appreciated. Also join us at https://gitter.im/nimgen/Lobby to chat about nimgen and the future of Nim wrappers.

126
appveyor.yml
View file

@ -7,11 +7,6 @@ image:
matrix: matrix:
fast_finish: true fast_finish: true
environment:
matrix:
- NIM_VERSION: 0.20.0
- NIM_VERSION: 0.19.6
for: for:
- -
matrix: matrix:
@ -19,80 +14,93 @@ for:
- image: Visual Studio 2017 - image: Visual Studio 2017
environment: environment:
ARCH: 32 MINGW_DIR: mingw32
MINGW_URL: https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/8.1.0/threads-posix/dwarf MINGW_URL: https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/8.1.0/threads-posix/dwarf/i686-8.1.0-release-posix-dwarf-rt_v6-rev0.7z/download
MINGW_ARCHIVE: i686-8.1.0-release-posix-dwarf-rt_v6-rev0.7z MINGW_ARCHIVE: i686-8.1.0-release-posix-dwarf-rt_v6-rev0.7z
SFNET_URL: https://sourceforge.net/projects/msys2/files/REPOS/MINGW/i686 NIM_URL: https://nim-lang.org/download/nim-0.18.0_x32.zip
LIBSSH2_ARCHIVE: mingw-w64-i686-libssh2-1.8.0-1-any.pkg NIM_ARCHIVE: nim-0.18.0_x32.zip
LIBCRYPTO_ARCHIVE: mingw-w64-i686-openssl-1.0.2.o-1-any.pkg NIM_VERSION: nim-0.18.0
LIBSSH2_ARCHIVE: mingw-w64-i686-libssh2-1.8.0-1-any.pkg.tar.xz
LIBSSH2_ARCHIVE2: mingw-w64-i686-libssh2-1.8.0-1-any.pkg.tar
LIBSSH2_URL: https://sourceforge.net/projects/msys2/files/REPOS/MINGW/i686/mingw-w64-i686-libssh2-1.8.0-1-any.pkg.tar.xz/download
LIBCRYPTO_ARCHIVE: mingw-w64-i686-openssl-1.0.2.o-1-any.pkg.tar.xz
LIBCRYPTO_ARCHIVE2: mingw-w64-i686-openssl-1.0.2.o-1-any.pkg.tar
LIBCRYPTO_URL: https://sourceforge.net/projects/msys2/files/REPOS/MINGW/i686/mingw-w64-i686-openssl-1.0.2.o-1-any.pkg.tar.xz/download
DLLS_URL: http://nim-lang.org/download/dlls.zip
DLLS_ARCHIVE: dlls.zip
BASE_DIR: c:\projects
install: install:
- CD c:\ - CD %BASE_DIR%
- IF not exist "binaries" ( - IF not exist "%MINGW_ARCHIVE%" appveyor DownloadFile "%MINGW_URL%" -FileName "%MINGW_ARCHIVE%"
echo %NIM_VERSION% && - 7z x -y "%MINGW_ARCHIVE%"> nul
MKDIR binaries && - IF not exist "%LIBSSH2_ARCHIVE%" appveyor DownloadFile "%LIBSSH2_URL%" -FileName "%LIBSSH2_ARCHIVE%"
CD binaries && - 7z x -y "%LIBSSH2_ARCHIVE%"> nul
appveyor DownloadFile "%MINGW_URL%/%MINGW_ARCHIVE%/download" -FileName "%MINGW_ARCHIVE%" && - 7z x -y "%LIBSSH2_ARCHIVE2%"> nul
7z x -y "%MINGW_ARCHIVE%"> nul && - IF not exist "%LIBCRYPTO_ARCHIVE%" appveyor DownloadFile "%LIBCRYPTO_URL%" -FileName "%LIBCRYPTO_ARCHIVE%"
del "%MINGW_ARCHIVE%" && - 7z x -y "%LIBCRYPTO_ARCHIVE%"> nul
appveyor DownloadFile "%SFNET_URL%/%LIBSSH2_ARCHIVE%.tar.xz/download" -FileName "%LIBSSH2_ARCHIVE%.tar.xz" && - 7z x -y "%LIBCRYPTO_ARCHIVE2%"> nul
7z x -y "%LIBSSH2_ARCHIVE%.tar.xz"> nul && - IF not exist "%NIM_ARCHIVE%" appveyor DownloadFile "%NIM_URL%" -FileName "%NIM_ARCHIVE%"
del "%LIBSSH2_ARCHIVE%.tar.xz" && - 7z x -y "%NIM_ARCHIVE%"> nul
7z x -y "%LIBSSH2_ARCHIVE%.tar"> nul && - SET PATH=%BASE_DIR%\%MINGW_DIR%\bin;%BASE_DIR%\%NIM_VERSION%\bin;%USERPROFILE%\.nimble\bin;%PATH%
del "%LIBSSH2_ARCHIVE%.tar" && - CD %BASE_DIR%\nimgen
appveyor DownloadFile "%SFNET_URL%/%LIBCRYPTO_ARCHIVE%.tar.xz/download" -FileName "%LIBCRYPTO_ARCHIVE%.tar.xz" && # - git clone --depth 1 https://github.com/nim-lang/nim
7z x -y "%LIBCRYPTO_ARCHIVE%.tar.xz"> nul && # - cd nim
del "%LIBCRYPTO_ARCHIVE%.tar.xz" && # - git clone --depth 1 https://github.com/nim-lang/csources
7z x -y "%LIBCRYPTO_ARCHIVE%.tar"> nul && # - cd csources
del "%LIBCRYPTO_ARCHIVE%.tar" && # - IF "%PLATFORM%" == "x64" ( build64.bat ) else ( build.bat )
appveyor DownloadFile "https://nim-lang.org/download/nim-%NIM_VERSION%_x%ARCH%.zip" -FileName "nim-%NIM_VERSION%_x%ARCH%.zip" && # - cd ..
7z x -y "nim-%NIM_VERSION%_x%ARCH%.zip"> nul && # - IF not exist "%DLLS_ARCHIVE%" appveyor DownloadFile "%DLLS_URL%" -FileName "%DLLS_ARCHIVE%"
del "nim-%NIM_VERSION%_x%ARCH%.zip") # - 7z x -y "%DLLS_ARCHIVE%" -o"%CD%\bin"> nul
- SET PATH=c:\binaries\mingw%ARCH%\bin;c:\binaries\nim-%NIM_VERSION%\bin;%USERPROFILE%\.nimble\bin;%PATH% # - bin\nim c -d:release koch
- CD c:\projects\nimgen # - koch boot -d:release
# - koch nimble -d:release
# - SET PATH=%CD%\bin;%PATH%
# - cd ..
on_finish: on_finish:
- 7z a -r buildlogs-win-pkgs.zip %USERPROFILE%\.nimble\pkgs - 7z a -r buildlogs-win.zip %USERPROFILE%\.nimble\pkgs
- appveyor PushArtifact buildlogs-win-pkgs.zip - appveyor PushArtifact buildlogs-win.zip
- 7z a -r buildlogs-win-projects.zip c:\projects\* - 7z a -r nimgen-docs.zip %BASE_DIR%\nimgen\web
- appveyor PushArtifact buildlogs-win-projects.zip
- 7z a -r nimgen-docs.zip c:\projects\nimgen\web
- appveyor PushArtifact nimgen-docs.zip - appveyor PushArtifact nimgen-docs.zip
cache: cache:
- c:\binaries - c:\projects\i686-8.1.0-release-posix-dwarf-rt_v6-rev0.7z
- c:\projects\mingw-w64-i686-libssh2-1.8.0-1-any.pkg.tar.xz
- c:\projects\mingw-w64-i686-openssl-1.0.2.o-1-any.pkg.tar.xz
- c:\projects\nim-0.18.0_x32.zip
- -
matrix: matrix:
only: only:
- image: Ubuntu - image: Ubuntu
environment:
NIM_URL: https://nim-lang.org/download/nim-0.18.0.tar.xz
NIM_ARCHIVE: nim-0.18.0.tar.xz
NIM_VERSION: nim-0.18.0
BASE_DIR: /home/appveyor/projects
install: install:
- sudo apt -qq update - sudo apt -qq update
- sudo apt -qq install --yes libssh2-1-dev libgcrypt20-dev libgpg-error-dev - sudo apt -qq install --yes python-pygments libssh2-1-dev libgcrypt20-dev libgpg-error-dev
- if [ ! -e /home/appveyor/binaries ]; then - cd $BASE_DIR
echo $NIM_VERSION && - if [ ! -e $NIM_ARCHIVE ]; then curl -s -o $NIM_ARCHIVE $NIM_URL; fi
mkdir /home/appveyor/binaries && - tar xJf $NIM_ARCHIVE
cd /home/appveyor/binaries && - cd $NIM_VERSION
curl -s -o nim-$NIM_VERSION.tar.xz https://nim-lang.org/download/nim-$NIM_VERSION.tar.xz && - sh build.sh
tar xJf nim-$NIM_VERSION.tar.xz && - bin/nim c -d:release koch
cd nim-$NIM_VERSION && - ./koch boot -d:release
sh build.sh && - ./koch nimble -d:release
bin/nim c -d:release koch && - export PATH=$BASE_DIR/$NIM_VERSION/bin:~/.nimble/bin:$PATH
./koch boot -d:release && - cd $BASE_DIR/nimgen
./koch nimble -d:release;
fi
- export PATH=/home/appveyor/binaries/nim-$NIM_VERSION/bin:~/.nimble/bin:$PATH
- cd /home/appveyor/projects/nimgen
on_finish: on_finish:
- zip -r -q buildlogs-lin-pkgs.zip ~/.nimble/pkgs - zip -r -q buildlogs-lin.zip ~/.nimble/pkgs
- appveyor PushArtifact buildlogs-lin-pkgs.zip - appveyor PushArtifact buildlogs-lin.zip
- zip -r -q buildlogs-lin-projects.zip /home/appveyor/projects
- appveyor PushArtifact buildlogs-lin-projects.zip
cache: cache:
- /home/appveyor/binaries - /home/appveyor/projects/nim-0.18.0.tar.xz
build_script: build_script:
- nimble install -y - nimble install -y

6
nimgen.nimble
View file

@ -1,17 +1,17 @@
# Package # Package
version = "0.5.1" version = "0.3.0"
author = "genotrance" author = "genotrance"
description = "c2nim helper to simplify and automate the wrapping of C libraries" description = "c2nim helper to simplify and automate the wrapping of C libraries"
license = "MIT" license = "MIT"
bin = @["nimgen"] bin = @["nimgen"]
srcDir = "src" srcDir = "src"
skipDirs = @["nimgen", "tests", "web"] skipDirs = @["nimgen", "tests"]
# Dependencies # Dependencies
requires "nim >= 0.19.0", "c2nim >= 0.9.14", "regex >= 0.10.0" requires "nim >= 0.17.0", "c2nim >= 0.9.13", "regex >= 0.7.1"
task test, "Test nimgen": task test, "Test nimgen":
exec "nim c -r tests/rununittests.nim" exec "nim c -r tests/rununittests.nim"

4
src/nimgen.nim
View file

@ -2,4 +2,6 @@ import os
import nimgen/runcfg import nimgen/runcfg
runCli() for i in commandLineParams():
if i != "-f":
runCfg(i)

46
src/nimgen/c2nim.nim
View file

@ -1,47 +1,40 @@
import os, regex, strutils import os, ospaths, regex, strutils
when (NimMajor, NimMinor, NimPatch) < (0, 19, 9):
import ospaths
import external, file, fileops, gencore, globals import external, file, fileops, gencore, globals
proc relativePath(path: string): string = template relativePath(path: untyped): untyped =
if gOutput.len() == 0: path.multiReplace([(gOutput, ""), ("\\", "/"), ("//", "/")])
result = path
else:
# multiReplace() bug - #9557
result = path.replace(gOutput, "")
return result.multiReplace([("\\", "/"), ("//", "/")])
proc c2nim*(fl, outfile: string, c2nimConfig: c2nimConfigObj) = proc c2nim*(fl, outfile: string, c2nimConfig: c2nimConfigObj) =
var file = search(fl) var file = search(fl)
if file.len() == 0: if file.len() == 0:
return return
echo "Generating " & outfile echo " Generating " & outfile
# Remove static inline function bodies
removeStatic(file)
fixFuncProtos(file)
var cfile = file var cfile = file
if c2nimConfig.preprocess: if c2nimConfig.preprocess:
cfile = "temp-$#.c" % [outfile.extractFilename()] cfile = "temp-$#.c" % [outfile.extractFilename()]
writeFileFlush(cfile, runPreprocess(file, c2nimConfig.ppflags, c2nimConfig.flags, c2nimConfig.inline)) writeFile(cfile, runPreprocess(file, c2nimConfig.ppflags, c2nimConfig.flags, c2nimConfig.inline))
elif c2nimConfig.ctags: elif c2nimConfig.ctags:
cfile = "temp-$#.c" % [outfile.extractFilename()] cfile = "temp-$#.c" % [outfile.extractFilename()]
writeFileFlush(cfile, runCtags(file)) writeFile(cfile, runCtags(file))
if c2nimConfig.removeBodies:
removeBodies(cfile)
if c2nimConfig.defines and (c2nimConfig.preprocess or c2nimConfig.ctags): if c2nimConfig.defines and (c2nimConfig.preprocess or c2nimConfig.ctags):
prepend(cfile, getDefines(file, c2nimConfig.inline)) prepend(cfile, getDefines(file, c2nimConfig.inline))
removeStatic(cfile)
var var
extflags = "" extflags = ""
passC = "" passC = ""
outlib = "" outlib = ""
outpragma = "" outpragma = ""
passC = "import strutils\n" passC = "import ospaths, strutils\n"
passC &= """const sourcePath = currentSourcePath().split({'\\', '/'})[0..^2].join("/")""" & "\n" passC &= """const sourcePath = currentSourcePath().split({'\\', '/'})[0..^2].join("/")""" & "\n"
@ -105,18 +98,18 @@ proc c2nim*(fl, outfile: string, c2nimConfig: c2nimConfigObj) =
removeFile(cfile) removeFile(cfile)
except: except:
discard discard
else:
if c2nimConfig.removeBodies:
reAddBodies(cfile)
reAddStatic(cfile)
# Nim doesn't like {.cdecl.} for type proc() # Nim doesn't like {.cdecl.} for type proc()
freplace(outfile, re"(?m)(.*? = proc.*?)\{.cdecl.\}", "$1") freplace(outfile, re"(?m)(.*? = proc.*?)\{.cdecl.\}", "$#")
freplace(outfile, " {.cdecl.})", ")") freplace(outfile, " {.cdecl.})", ")")
# Include {.compile.} directives # Include {.compile.} directives
for cpl in c2nimConfig.compile: for cpl in c2nimConfig.compile:
prepend(outfile, compile(cpl, c2nimConfig.flags)) let fcpl = search(cpl)
if getFileInfo(fcpl).kind == pcFile:
prepend(outfile, compile(file=fcpl))
else:
prepend(outfile, compile(dir=fcpl))
# Add any pragmas # Add any pragmas
if outpragma != "": if outpragma != "":
@ -129,3 +122,6 @@ proc c2nim*(fl, outfile: string, c2nimConfig: c2nimConfigObj) =
# Add dynamic library # Add dynamic library
if outlib != "": if outlib != "":
prepend(outfile, outlib) prepend(outfile, outlib)
# Add back static functions for compilation
reAddStatic(file)

100
src/nimgen/external.nim
View file

@ -1,4 +1,4 @@
import os, osproc, regex, streams, strutils import os, osproc, regex, ropes, streams, strutils
import globals import globals
@ -6,25 +6,26 @@ proc sanitizePath*(path: string): string =
path.multiReplace([("\\", "/"), ("//", "/")]) path.multiReplace([("\\", "/"), ("//", "/")])
proc execProc*(cmd: string): string = proc execProc*(cmd: string): string =
var ret: int result = ""
var
p = startProcess(cmd, options = {poStdErrToStdOut, poUsePath, poEvalCommand})
(result, ret) = execCmdEx(cmd) outp = outputStream(p)
if ret != 0: line = newStringOfCap(120).TaintedString
echo "Command failed: " & $ret
while true:
if outp.readLine(line):
result.add(line)
result.add("\n")
elif not running(p): break
var x = p.peekExitCode()
if x != 0:
echo "Command failed: " & $x
echo cmd echo cmd
echo result echo result
quit(1) quit(1)
proc execAction*(cmd: string): string =
var ccmd = ""
when defined(Windows):
ccmd = "cmd /c " & cmd.replace("/", "\\")
when defined(Linux) or defined(MacOSX):
ccmd = "bash -c '" & cmd & "'"
echo "Running '" & ccmd[0..<min(64, len(ccmd))] & "'"
return execProc(ccmd)
proc extractZip*(zipfile: string) = proc extractZip*(zipfile: string) =
var cmd = "unzip -o $#" var cmd = "unzip -o $#"
if defined(Windows): if defined(Windows):
@ -43,10 +44,9 @@ proc downloadUrl*(url: string) =
file = url.extractFilename() file = url.extractFilename()
ext = file.splitFile().ext.toLowerAscii() ext = file.splitFile().ext.toLowerAscii()
var cmd = if defined(Windows): var cmd = "curl $# -o $#"
"powershell [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; wget $# -OutFile $#" if defined(Windows):
else: cmd = "powershell wget $# -OutFile $#"
"curl -L $# -o $#"
if not (ext == ".zip" and fileExists(gOutput/file)): if not (ext == ".zip" and fileExists(gOutput/file)):
echo "Downloading " & file echo "Downloading " & file
@ -55,67 +55,53 @@ proc downloadUrl*(url: string) =
if ext == ".zip": if ext == ".zip":
extractZip(file) extractZip(file)
template setGitDir() = proc gitReset*() =
setCurrentDir(gGitOutput) echo "Resetting Git repo"
setCurrentDir(gOutput)
defer: setCurrentDir(gProjectDir) defer: setCurrentDir(gProjectDir)
proc gitReset*() = discard execProc("git reset --hard HEAD")
echo "Resetting " & gGitOutput
setGitDir()
let cmd = "git reset --hard"
while execCmdEx(cmd)[0].contains("Permission denied"):
sleep(1000)
echo " Retrying ..."
proc gitCheckout*(file: string) = proc gitCheckout*(file: string) =
echo "Resetting " & file echo " Resetting " & file
setGitDir() setCurrentDir(gOutput)
defer: setCurrentDir(gProjectDir)
let cmd = "git checkout $#" % file.replace(gGitOutput & "/", "") discard execProc("git checkout $#" % file.replace(gOutput & "/", ""))
while execCmdEx(cmd)[0].contains("Permission denied"):
sleep(500)
echo " Retrying ..."
proc gitPull() =
if gGitCheckout.len() != 0:
echo "Checking out " & gGitCheckout
discard execProc("git pull --tags origin master")
discard execProc("git checkout " & gGitCheckout)
gGitCheckout = ""
else:
echo "Pulling repository"
discard execProc("git pull --depth=1 origin master")
proc gitRemotePull*(url: string, pull=true) = proc gitRemotePull*(url: string, pull=true) =
if dirExists(gGitOutput/".git"): if dirExists(gOutput/".git"):
if pull: if pull:
gitReset() gitReset()
return return
setGitDir() setCurrentDir(gOutput)
defer: setCurrentDir(gProjectDir)
echo "Setting up Git repo: " & url echo "Setting up Git repo"
discard execProc("git init .") discard execProc("git init .")
discard execProc("git remote add origin " & url) discard execProc("git remote add origin " & url)
if pull: if pull:
gitPull() echo "Checking out artifacts"
discard execProc("git pull --depth=1 origin master")
proc gitSparseCheckout*(plist: string) = proc gitSparseCheckout*(plist: string) =
let sparsefile = ".git/info/sparse-checkout" let sparsefile = ".git/info/sparse-checkout"
if fileExists(gGitOutput/sparsefile): if fileExists(gOutput/sparsefile):
gitReset() gitReset()
return return
setGitDir() setCurrentDir(gOutput)
defer: setCurrentDir(gProjectDir)
discard execProc("git config core.sparsecheckout true") discard execProc("git config core.sparsecheckout true")
writeFile(sparsefile, plist) writeFile(sparsefile, plist)
gitPull() echo "Checking out artifacts"
discard execProc("git pull --depth=1 origin master")
proc runPreprocess*(file, ppflags, flags: string, inline: bool): string = proc runPreprocess*(file, ppflags, flags: string, inline: bool): string =
var var
@ -130,9 +116,9 @@ proc runPreprocess*(file, ppflags, flags: string, inline: bool): string =
# Include content only from file # Include content only from file
var var
rdata: seq[string] = @[] rdata: Rope
start = false start = false
sfile = file.sanitizePath sfile = file.replace("\\", "/")
if inline: if inline:
sfile = sfile.parentDir() sfile = sfile.parentDir()
@ -150,9 +136,9 @@ proc runPreprocess*(file, ppflags, flags: string, inline: bool): string =
line.multiReplace([("_Noreturn", ""), ("(())", ""), ("WINAPI", ""), line.multiReplace([("_Noreturn", ""), ("(())", ""), ("WINAPI", ""),
("__attribute__", ""), ("extern \"C\"", "")]) ("__attribute__", ""), ("extern \"C\"", "")])
.replace(re"\(\([_a-z]+?\)\)", "") .replace(re"\(\([_a-z]+?\)\)", "")
.replace(re"\(\(__format__[\s]*\(__[gnu_]*printf__, [\d]+, [\d]+\)\)\);", ";") .replace(re"\(\(__format__[\s]*\(__[gnu_]*printf__, [\d]+, [\d]+\)\)\);", ";") & "\n"
) )
return rdata.join("\n") return $rdata
proc runCtags*(file: string): string = proc runCtags*(file: string): string =
var var

22
src/nimgen/file.nim
View file

@ -1,9 +1,6 @@
import os, pegs, regex, strutils, tables import os, ospaths, pegs, regex, strutils, tables
when (NimMajor, NimMinor, NimPatch) < (0, 19, 9): import globals
import ospaths
import globals, external
# ### # ###
# File loction # File loction
@ -19,7 +16,8 @@ proc getNimout*(file: string, rename=true): string =
if gRenames.hasKey(file): if gRenames.hasKey(file):
result = gRenames[file] result = gRenames[file]
createDir(parentDir(result)) if not dirExists(parentDir(result)):
createDir(parentDir(result))
proc exclude*(file: string): bool = proc exclude*(file: string): bool =
for excl in gExcludes: for excl in gExcludes:
@ -46,7 +44,7 @@ proc search*(file: string): string =
quit(1) quit(1)
# Only keep relative directory # Only keep relative directory
return result.sanitizePath.replace(gProjectDir & "/", "") return result.multiReplace([("\\", $DirSep), ("//", $DirSep), (gProjectDir & $DirSep, "")])
proc rename*(file: string, renfile: string) = proc rename*(file: string, renfile: string) =
if file.splitFile().ext == ".nim": if file.splitFile().ext == ".nim":
@ -84,12 +82,6 @@ proc openRetry*(file: string, mode: FileMode = fmRead): File =
except IOError: except IOError:
sleep(100) sleep(100)
template writeFileFlush*(file, content: string): untyped =
let f = openRetry(file, fmWrite)
f.write(content)
f.flushFile()
f.close()
template withFile*(file: string, body: untyped): untyped = template withFile*(file: string, body: untyped): untyped =
if fileExists(file): if fileExists(file):
var f = openRetry(file) var f = openRetry(file)
@ -101,7 +93,9 @@ template withFile*(file: string, body: untyped): untyped =
body body
if content != contentOrig: if content != contentOrig:
writeFileFlush(file, content) f = openRetry(file, fmWrite)
write(f, content)
f.close()
else: else:
echo "Missing file " & file echo "Missing file " & file

57
src/nimgen/fileops.nim
View file

@ -31,25 +31,20 @@ proc append*(file: string, data: string, search="") =
if idx != -1: if idx != -1:
content = content[0..<idy] & data & content[idy..<content.len()] content = content[0..<idy] & data & content[idy..<content.len()]
proc freplace*(file: string, pattern: string|Regex, repl="") = proc freplace*(file: string, pattern: string, repl="") =
withFile(file): withFile(file):
content = content.replace(pattern, repl) if pattern in content:
content = content.replace(pattern, repl)
proc move*(file: string, pattern: string|Regex, move: string) = proc freplace*(file: string, pattern: Regex, repl="") =
var tomove: seq[string] = @[]
withFile(file): withFile(file):
when pattern is string: var m: RegexMatch
tomove.add(pattern) if content.find(pattern, m):
if "$#" in repl:
when pattern is Regex: content = content.replace(pattern,
var ms = content.findAll(pattern) proc (m: RegexMatch, s: string): string = repl % s[m.group(0)[0]])
for i, m in ms: else:
tomove.add(content[m.group(0)[0]]) content = content.replace(pattern, repl)
content = content.replace(pattern, "")
for i in tomove:
append(file, i, move)
proc comment*(file: string, pattern: string, numlines: string) = proc comment*(file: string, pattern: string, numlines: string) =
let let
@ -91,36 +86,6 @@ proc removeStatic*(filename: string) =
result.add(body.replace(re"(?m)^(.*\n?)", "//$1")) result.add(body.replace(re"(?m)^(.*\n?)", "//$1"))
) )
proc removeBodies*(filename: string) =
## Replace function bodies with a semicolon and commented
## out body
withFile(filename):
content = content.replace(
re"(?m)(.*?\))(\s*\{(\s*?.*?$)*?[\n\r]\})",
proc (m: RegexMatch, s: string): string =
let funcDecl = s[m.group(0)[0]]
let body = s[m.group(1)[0]].strip()
result = ""
result.add("$#;" % [funcDecl])
result.add(body.replace(re"(?m)^(.*\n?)", "//$1"))
)
proc reAddBodies*(filename: string) =
## Uncomment out the body and remove the semicolon. Undoes
## removeBodies
withFile(filename):
content = content.replace(
re"(?m)(.*?\))(\s*\{(\s*?.*?$)*?[\n\r]\})",
proc (m: RegexMatch, s: string): string =
let funcDecl = s[m.group(0)[0]]
let body = s[m.group(1)[0]].strip()
result = ""
result.add("$# " % [funcDecl])
result.add(body.replace(re"(?m)^\/\/(.*\n?)", "$1"))
)
proc reAddStatic*(filename: string) = proc reAddStatic*(filename: string) =
## Uncomment out the body and remove the semicolon. Undoes ## Uncomment out the body and remove the semicolon. Undoes
## removeStatic ## removeStatic

62
src/nimgen/gencore.nim
View file

@ -4,53 +4,38 @@ import file, globals
proc addEnv*(str: string): string = proc addEnv*(str: string): string =
var newStr = str var newStr = str
if "$output" in newStr or "${output}" in newStr:
newStr = newStr % ["output", gOutput]
for pair in envPairs(): for pair in envPairs():
if pair.key.len() == 0: try:
continue
if ("$" & pair.key) in newStr or ("${" & pair.key & "}") in newStr:
newStr = newStr % [pair.key, pair.value.string] newStr = newStr % [pair.key, pair.value.string]
except ValueError:
# Ignore if there are no values to replace. We
# want to continue anyway
discard
try:
newStr = newStr % ["output", gOutput]
except ValueError:
# Ignore if there are no values to replace. We
# want to continue anyway
discard
# if there are still format args, print a warning
if newStr.contains("$") and not newStr.contains("$replace("):
echo "WARNING: \"", newStr, "\" still contains an uninterpolated value!"
return newStr return newStr
proc compile*(cpl, flags: string): string = proc compile*(dir="", file=""): string =
var data = ""
proc fcompile(file: string): string = proc fcompile(file: string): string =
let fn = file.splitFile().name return "{.compile: \"$#\".}" % file.replace("\\", "/")
var
ufn = fn
uniq = 1
while ufn in gCompile:
ufn = fn & $uniq
uniq += 1
gCompile.add(ufn) var data = ""
if fn == ufn: if dir != "" and dirExists(dir):
return "{.compile: \"$#\".}" % file.replace("\\", "/") for f in walkFiles(dir / "*.c"):
else:
return "{.compile: (\"../$#\", \"$#.o\").}" % [file.replace("\\", "/"), ufn]
proc dcompile(dir: string) =
for f in walkFiles(dir):
data &= fcompile(f) & "\n" data &= fcompile(f) & "\n"
if cpl.contains("*") or cpl.contains("?"): if file != "" and fileExists(file):
dcompile(cpl) data &= fcompile(file) & "\n"
else:
let fcpl = search(cpl)
if getFileInfo(fcpl).kind == pcFile:
data &= fcompile(fcpl) & "\n"
elif getFileInfo(fcpl).kind == pcDir:
if flags.contains("cpp"):
for i in @["*.C", "*.cpp", "*.c++", "*.cc", "*.cxx"]:
dcompile(fcpl / i)
else:
dcompile(fcpl / "*.c")
return data return data
@ -102,6 +87,7 @@ proc getDefines*(file: string, inline=false): string =
for incl in incls: for incl in incls:
let sincl = search(incl) let sincl = search(incl)
if sincl != "": if sincl != "":
echo "Inlining " & sincl
result &= getDefines(sincl) result &= getDefines(sincl)
withFile(file): withFile(file):
for def in content.findAll(re"(?m)^(\s*#\s*define\s+[\w\d_]+\s+[\d\-.xf]+)(?:\r|//|/*).*?$"): for def in content.findAll(re"(?m)^(\s*#\s*define\s+[\w\d_]+\s+[\d\-.xf]+)(?:\r|//|/*).*?$"):

29
src/nimgen/globals.nim
View file

@ -7,32 +7,25 @@ const
defaultCppCompiler* = "g++" defaultCppCompiler* = "g++"
var var
# Config gDoneRecursive*: seq[string] = @[]
gConfig*: Config gDoneInline*: seq[string] = @[]
gExcludes*: seq[string] = @[]
gIncludes*: seq[string] = @[]
gRenames* = initTable[string, string]()
gWildcards* = newConfig()
# n.global gProjectDir* = ""
gOutput* = "." gConfig*: Config
gQuotes* = true
gFilter* = "" gFilter* = ""
gQuotes* = true
gCppCompiler* = getEnv(cppCompilerEnv, defaultCCompiler) gCppCompiler* = getEnv(cppCompilerEnv, defaultCCompiler)
gCCompiler* = getEnv(cCompilerEnv, defaultCppCompiler) gCCompiler* = getEnv(cCompilerEnv, defaultCppCompiler)
gOutput* = ""
# State tracking gIncludes*: seq[string] = @[]
gGitCheckout* = "" gExcludes*: seq[string] = @[]
gGitOutput* = "" gRenames* = initTable[string, string]()
gProjectDir* = "" gWildcards* = newConfig()
gCompile*: seq[string] = @[]
gDoneInline*: seq[string] = @[]
gDoneRecursive*: seq[string] = @[]
type type
c2nimConfigObj* = object c2nimConfigObj* = object
flags*, ppflags*: string flags*, ppflags*: string
recurse*, inline*, preprocess*, removeBodies*, ctags*, defines*: bool recurse*, inline*, preprocess*, ctags*, defines*: bool
dynlib*, compile*, pragma*: seq[string] dynlib*, compile*, pragma*: seq[string]
const gDoc* = """ const gDoc* = """

418
src/nimgen/runcfg.nim
View file

@ -6,19 +6,18 @@ proc `[]`*(table: OrderedTableRef[string, string], key: string): string =
## Gets table values with env vars inserted ## Gets table values with env vars inserted
tables.`[]`(table, key).addEnv tables.`[]`(table, key).addEnv
proc getKey(ukey: string, section = false): tuple[key: string, val: bool] = proc getKey(ukey: string): tuple[key: string, val: bool] =
var kv = if not section: ukey.replace(re"\..*", "").split("-", 1) else: ukey.split("-", 1) var kv = ukey.replace(re"\..*", "").split("-", 1)
if kv.len() == 1: if kv.len() == 1:
kv.add("") kv.add("")
if kv[1] == "": if kv[1] == "":
return (kv[0], true) return (kv[0], true)
for ostyp in kv[1].split("-"): for ostyp in kv[1].split(","):
if (ostyp == "win" and defined(Windows)) or if (ostyp == "win" and defined(Windows)) or
(ostyp == "lin" and defined(Linux)) or (ostyp == "lin" and defined(Linux)) or
((ostyp == "osx" or ostyp == "mac") and defined(MacOSX)) or ((ostyp == "osx" or ostyp == "mac") and defined(MacOSX)):
(ostyp == "unix" and (defined(Linux) or defined(MacOSX))):
return (kv[0], true) return (kv[0], true)
return (kv[0], false) return (kv[0], false)
@ -26,8 +25,7 @@ proc getKey(ukey: string, section = false): tuple[key: string, val: bool] =
proc runFile*(file: string, cfgin: OrderedTableRef = newOrderedTable[string, string]()) = proc runFile*(file: string, cfgin: OrderedTableRef = newOrderedTable[string, string]()) =
var var
cfg = cfgin cfg = cfgin
sfile = search(file) sfile = search(file).sanitizePath
nowildcard = false
if sfile in gDoneRecursive: if sfile in gDoneRecursive:
return return
@ -36,24 +34,16 @@ proc runFile*(file: string, cfgin: OrderedTableRef = newOrderedTable[string, str
echo "Processing " & sfile echo "Processing " & sfile
gDoneRecursive.add(sfile) gDoneRecursive.add(sfile)
for act in cfg.keys(): for pattern in gWildcards.keys():
let (action, val) = getKey(act) var m: RegexMatch
if val == true and action == "nowildcard" and cfg[act] == "true": let pat = pattern.replace(".", "\\.").replace("*", ".*").replace("?", ".?")
nowildcard = true if file.find(toPattern(pat), m):
break echo " Appending keys for wildcard " & pattern
for key in gWildcards[pattern].keys():
if not nowildcard: cfg[key & "." & pattern] = gWildcards[pattern][key]
for pattern in gWildcards.keys():
var m: RegexMatch
let pat = pattern.replace(".", "\\.").replace("*", ".*").replace("?", ".?")
if file.find(toPattern(pat), m):
echo " Appending keys for wildcard " & pattern
for key in gWildcards[pattern].keys():
cfg[key & "." & pattern] = gWildcards[pattern][key]
var var
srch = "" srch = ""
rgx = ""
c2nimConfig = c2nimConfigObj( c2nimConfig = c2nimConfigObj(
flags: "--stdcall", ppflags: "", flags: "--stdcall", ppflags: "",
@ -67,14 +57,14 @@ proc runFile*(file: string, cfgin: OrderedTableRef = newOrderedTable[string, str
if action == "create": if action == "create":
echo "Creating " & file echo "Creating " & file
createDir(file.splitPath().head) createDir(file.splitPath().head)
writeFileFlush(file, cfg[act]) writeFile(file, cfg[act])
if file in gExcludes: if file in gExcludes:
gExcludes.delete(gExcludes.find(file)) gExcludes.delete(gExcludes.find(file))
sfile = file sfile = search(file)
gDoneRecursive.add(sfile) gDoneRecursive.add(sfile)
elif action in @["prepend", "append", "replace", "move", "comment", elif action in @["prepend", "append", "replace", "comment",
"rename", "compile", "dynlib", "pragma", "pipe"] and "rename", "compile", "dynlib", "pragma",
sfile != "": "pipe"] and sfile != "":
if action == "prepend": if action == "prepend":
if srch != "": if srch != "":
prepend(sfile, cfg[act], cfg[srch]) prepend(sfile, cfg[act], cfg[srch])
@ -88,13 +78,6 @@ proc runFile*(file: string, cfgin: OrderedTableRef = newOrderedTable[string, str
elif action == "replace": elif action == "replace":
if srch != "": if srch != "":
freplace(sfile, cfg[srch], cfg[act]) freplace(sfile, cfg[srch], cfg[act])
elif rgx != "":
freplace(sfile, toPattern(cfg[rgx]), cfg[act])
elif action == "move":
if srch != "":
move(sfile, cfg[srch], cfg[act])
elif rgx != "":
move(sfile, toPattern(cfg[rgx]), cfg[act])
elif action == "comment": elif action == "comment":
if srch != "": if srch != "":
comment(sfile, cfg[srch], cfg[act]) comment(sfile, cfg[srch], cfg[act])
@ -109,11 +92,8 @@ proc runFile*(file: string, cfgin: OrderedTableRef = newOrderedTable[string, str
elif action == "pipe": elif action == "pipe":
pipe(sfile, cfg[act]) pipe(sfile, cfg[act])
srch = "" srch = ""
rgx = ""
elif action == "search": elif action == "search":
srch = act srch = act
elif action == "regex":
rgx = act
if file.splitFile().ext != ".nim": if file.splitFile().ext != ".nim":
var var
@ -130,8 +110,6 @@ proc runFile*(file: string, cfgin: OrderedTableRef = newOrderedTable[string, str
c2nimConfig.inline = true c2nimConfig.inline = true
elif action == "preprocess": elif action == "preprocess":
c2nimConfig.preprocess = true c2nimConfig.preprocess = true
elif action == "removeBodies":
c2nimConfig.removeBodies = true
elif action == "ctags": elif action == "ctags":
c2nimConfig.ctags = true c2nimConfig.ctags = true
elif action == "defines": elif action == "defines":
@ -149,263 +127,141 @@ proc runFile*(file: string, cfgin: OrderedTableRef = newOrderedTable[string, str
echo "Cannot use recurse and inline simultaneously" echo "Cannot use recurse and inline simultaneously"
quit(1) quit(1)
removeStatic(sfile)
fixFuncProtos(sfile)
let outfile = getNimout(sfile)
var incout = ""
if c2nimConfig.recurse or c2nimConfig.inline:
var
cfg = newOrderedTable[string, string]()
incls = getIncls(sfile)
for name, value in c2nimConfig.fieldPairs:
when value is string:
cfg[name] = value
when value is bool:
cfg[name] = $value
for i in c2nimConfig.dynlib:
cfg["dynlib." & i] = i
if c2nimConfig.inline:
cfg["noprocess"] = "true"
for inc in incls:
runFile(inc, cfg)
if c2nimConfig.recurse:
incout &= "import $#\n" % inc.search().getNimout()[0 .. ^5]
if not noprocess: if not noprocess:
let outfile = getNimout(sfile)
c2nim(file, outfile, c2nimConfig) c2nim(file, outfile, c2nimConfig)
if c2nimConfig.recurse and incout.len() != 0: if c2nimConfig.recurse:
prepend(outfile, incout) var
cfg = newOrderedTable[string, string]()
incls = getIncls(sfile)
incout = ""
for name, value in c2nimConfig.fieldPairs:
when value is string:
cfg[name] = value
when value is bool:
cfg[name] = $value
for i in c2nimConfig.dynlib:
cfg["dynlib." & i] = i
for inc in incls:
runFile(inc, cfg)
incout &= "import $#\n" % inc.search().getNimout()[0 .. ^5]
if incout.len() != 0:
prepend(outfile, incout)
if reset: if reset:
gitCheckout(sfile) gitCheckout(sfile)
proc setOutputDir(dir: string) =
gOutput = dir.sanitizePath
if dirExists(gOutput):
if "-f" in commandLineParams():
try:
removeDir(gOutput)
except OSError:
echo "Directory in use: " & gOutput
quit(1)
else:
for f in walkFiles(gOutput/"*.nim"):
try:
removeFile(f)
except OSError:
echo "Unable to delete: " & f
quit(1)
createDir(gOutput)
gGitOutput = gOutput
proc runCfg*(cfg: string) = proc runCfg*(cfg: string) =
if not fileExists(cfg): if not fileExists(cfg):
echo "Config doesn't exist: " & cfg echo "Config doesn't exist: " & cfg
quit(1) quit(1)
gProjectDir = parentDir(cfg.expandFilename()).sanitizePath gProjectDir = parentDir(cfg.expandFilename())
gConfig = loadConfig(cfg) gConfig = loadConfig(cfg)
gCppCompiler = getEnv(cppCompilerEnv, defaultCppCompiler).quoteShell
gCCompiler = getEnv(cCompilerEnv, defaultCCompiler).quoteShell
gGitOutput = gOutput
for section in gConfig.keys(): if gConfig.hasKey("n.global"):
let (sname, sval) = getKey(section, true) if gConfig["n.global"].hasKey("output"):
if not sval: gOutput = gConfig["n.global"]["output"]
if dirExists(gOutput):
if "-f" in commandLineParams():
try:
removeDir(gOutput)
except OSError:
echo "Directory in use: " & gOutput
quit(1)
else:
for f in walkFiles(gOutput/"*.nim"):
try:
removeFile(f)
except OSError:
echo "Unable to delete: " & f
quit(1)
createDir(gOutput)
if gConfig["n.global"].hasKey("cpp_compiler"):
gCppCompiler = gConfig["n.global"]["cpp_compiler"]
else:
# Reset on a per project basis
gCppCompiler = getEnv(cppCompilerEnv, defaultCppCompiler)
if gConfig["n.global"].hasKey("c_compiler"):
gCCompiler = gConfig["n.global"]["c_compiler"]
else:
# Reset on a per project basis
gCCompiler = getEnv(cCompilerEnv, defaultCCompiler)
gCppCompiler = gCppCompiler.quoteShell
gCCompiler = gCCompiler.quoteShell
if gConfig["n.global"].hasKey("filter"):
gFilter = gConfig["n.global"]["filter"]
if gConfig["n.global"].hasKey("quotes"):
if gConfig["n.global"]["quotes"] == "false":
gQuotes = false
if gConfig.hasKey("n.include"):
for inc in gConfig["n.include"].keys():
gIncludes.add(inc.addEnv())
if gConfig.hasKey("n.exclude"):
for excl in gConfig["n.exclude"].keys():
gExcludes.add(excl.addEnv())
if gConfig.hasKey("n.prepare"):
for prep in gConfig["n.prepare"].keys():
let (key, val) = getKey(prep)
if val == true:
let prepVal = gConfig["n.prepare"][prep]
if key == "download":
downloadUrl(prepVal)
elif key == "extract":
extractZip(prepVal)
elif key == "git":
gitRemotePull(prepVal)
elif key == "gitremote":
gitRemotePull(prepVal, false)
elif key == "gitsparse":
gitSparseCheckout(prepVal)
elif key == "execute":
discard execProc(prepVal)
elif key == "copy":
doCopy(prepVal)
if gConfig.hasKey("n.wildcard"):
var wildcard = ""
for wild in gConfig["n.wildcard"].keys():
let (key, val) = getKey(wild)
if val == true:
if key == "wildcard":
wildcard = gConfig["n.wildcard"][wild]
else:
gWildcards.setSectionKey(wildcard, wild,
gConfig["n.wildcard"][wild])
for file in gConfig.keys():
if file in @["n.global", "n.include", "n.exclude",
"n.prepare", "n.wildcard", "n.post"]:
continue continue
case sname: if file == "n.sourcefile":
of "n.global": for pattern in gConfig["n.sourcefile"].keys():
for glob in gConfig[section].keys(): for file in walkFiles(pattern.addEnv):
let (key, val) = getKey(glob) runFile(file)
if val == true: else:
let globVal = gConfig[section][glob] runFile(file, gConfig[file])
case key:
of "output":
setOutputDir(globVal)
of "cpp_compiler":
gCppCompiler = globVal.quoteShell
of "c_compiler":
gCCompiler = globVal.quoteShell
of "filter":
gFilter = globVal
of "quotes":
if globVal == "false":
gQuotes = false
of "n.include": if gConfig.hasKey("n.post"):
for inc in gConfig[section].keys(): for post in gConfig["n.post"].keys():
gIncludes.add(inc.addEnv().sanitizePath) let (key, val) = getKey(post)
if val == true:
of "n.exclude": let postVal = gConfig["n.post"][post]
for excl in gConfig[section].keys(): if key == "reset":
gExcludes.add(excl.addEnv().sanitizePath) gitReset()
elif key == "execute":
of "n.prepare": discard execProc(postVal)
for prep in gConfig[section].keys():
let (key, val) = getKey(prep)
if val == true:
let prepVal = gConfig[section][prep]
case key:
of "download":
downloadUrl(prepVal)
of "extract":
extractZip(prepVal)
of "gitcheckout":
gGitCheckout = prepVal
of "gitoutput":
gGitOutput = gOutput/prepVal
createDir(gGitOutput)
of "git":
gitRemotePull(prepVal)
of "gitremote":
gitRemotePull(prepVal, false)
of "gitsparse":
gitSparseCheckout(prepVal)
of "execute":
discard execAction(prepVal)
of "copy":
doCopy(prepVal)
of "n.wildcard":
var wildcard = ""
for wild in gConfig[section].keys():
let (key, val) = getKey(wild)
if val == true:
if key == "wildcard":
wildcard = gConfig[section][wild]
else:
gWildcards.setSectionKey(wildcard, wild,
gConfig[section][wild])
of "n.sourcefile":
for pattern in gConfig[section].keys():
for file in walkFiles(pattern.addEnv):
runFile(file)
of "n.post":
for post in gConfig[section].keys():
let (key, val) = getKey(post)
if val == true:
let postVal = gConfig[section][post]
case key:
of "gitoutput":
gGitOutput = gOutput/postVal
of "reset":
gitReset()
of "execute":
discard execAction(postVal)
else:
runFile(section, gConfig[section])
let gHelp = """
Nimgen is a helper for c2nim to simplify and automate the wrapping of C libraries
Usage:
nimgen [options] file.cfg|file.h ...
Params:
-C<compile> add compile entry *
-E<exclude> add n.exclude entry *
-F<flags> set c2nim flags *
-I<include> add n.include dir *
-O<outdir> set output directory
-P<ppflags> set preprocessor flags *
Options:
-c set ctags = true
-d set defines = true
-i set inline = true
-n set noprocess = true
-p set preprocess = true
-r set recurse = true
Editing:
-a<append> append string *
-e<prepend> prepend string *
-l<replace> replace string *
-o#lines comment X lines *
-s<search> search string *
-x<regex> regex search string *
* supports multiple instances
"""
proc runCli*() =
var
cfg = newOrderedTable[string, string]()
files: seq[string]
uniq = 1
gProjectDir = getCurrentDir().sanitizePath
for param in commandLineParams():
let flag = if param.len() <= 2: param else: param[0..<2]
if fileExists(param):
if param.splitFile().ext.toLowerAscii() == ".cfg":
runCfg(param)
else:
files.add(param)
elif flag == "-C":
cfg["compile." & $uniq] = param[2..^1]
elif flag == "-E":
gExcludes.add(param[2..^1].addEnv().sanitizePath)
elif flag == "-F":
if cfg.hasKey("flags"):
cfg["flags"] = cfg["flags"] & " " & param[2..^1]
else:
cfg["flags"] = param[2..^1]
elif flag == "-I":
gIncludes.add(param[2..^1].addEnv().sanitizePath)
elif flag == "-O":
setOutputDir(param[2..^1])
elif flag == "-P":
if cfg.hasKey("ppflags"):
cfg["ppflags"] = cfg["ppflags"] & " " & param[2..^1]
else:
cfg["ppflags"] = param[2..^1]
elif flag == "-c":
cfg["ctags"] = "true"
elif flag == "-d":
cfg["defines"] = "true"
elif flag == "-i":
cfg["inline"] = "true"
elif flag == "-n":
cfg["noprocess"] = "true"
elif flag == "-p":
cfg["preprocess"] = "true"
elif flag == "-r":
cfg["recurse"] = "true"
elif flag == "-a":
cfg["append." & $uniq] = param[2..^1]
elif flag == "-e":
cfg["prepend." & $uniq] = param[2..^1]
elif flag == "-l":
cfg["replace." & $uniq] = param[2..^1]
elif flag == "-o":
cfg["comment." & $uniq] = param[2..^1]
elif flag == "-s":
cfg["search." & $uniq] = param[2..^1]
elif flag == "-x":
cfg["regex." & $uniq] = param[2..^1]
elif param == "-h" or param == "-?" or param == "--help":
echo gHelp
quit(0)
uniq += 1
for file in files:
runFile(file, cfg)

82
tests/nimgentest.nims
View file

@ -1,78 +1,52 @@
import distros, ospaths, strutils import distros, ospaths, strutils
var var
pygonly = false full = true
comps = @["nim7z", #"nimarchive", comps = @["libsvm", "nim7z", "nimarchive", "nimbass", "nimbigwig", "nimclipboard",
"nimbass", "nimbigwig", "nimfuzz", "nimpcre", "nimrax", "nimssl", "nimssh2"]
"nimclipboard", "nimfuzzy", "nimmonocypher",
#"nimnuklear",
"nimpcre", "nimrax", "nimssl", "nimssh2",
"nimtess2"
]
let
gccver = staticExec("gcc --version").split("\n")[0].split(" ")[^1]
nimver = staticExec("nim -v").split("\n")[0].split(" ")[3]
if nimver >= "0.19.0" and (gccver >= "5.0.0" or detectOs(MacOSX)):
comps.add("nimfastText")
if detectOs(Windows): if detectOs(Windows):
comps.add("nimkerberos") comps.add("nimkerberos")
if not detectOs(MacOSX):
comps.add("nimzbar")
echo "Nim version: " & nimver
echo "GCC version: " & gccver
echo "Testing comps:"
for comp in comps:
echo " " & comp
if paramCount() > 2: if paramCount() > 2:
for i in 3 .. paramCount(): for i in 3 .. paramCount():
if paramStr(i) == "--pygonly": if paramStr(i) == "--full":
pygonly = true full = true
elif paramStr(i).len() > 10 and "--comps=" in paramStr(i)[0 ..< 8]: elif paramStr(i).len() > 10 and "--comps=" in paramStr(i)[0 ..< 8]:
comps = paramStr(i)[8 .. ^1].split(",") comps = paramStr(i)[8 .. ^1].split(",")
for comp in comps: for comp in comps:
if not pygonly: if not dirExists(".."/comp):
if not dirExists(".."/comp): withDir(".."):
withDir(".."): exec "git clone --depth=1 https://github.com/genotrance/" & comp
exec "git clone --depth=1 https://github.com/genotrance/" & comp
exec "nimble uninstall -y " & comp, "", "" exec "nimble uninstall -y " & comp, "", ""
withDir(".."/comp): withDir(".."/comp):
exec "git pull" exec "git pull"
if full:
rmDir(comp) rmDir(comp)
exec "nimble install -y" exec "nimble install -y"
exec "nimble test" exec "nimble test"
exec "nimble install -y" exec "nimble install -y"
exec "nimble test" exec "nimble test"
when defined(windows): if dirExists("web"/comp):
if not pygonly: rmDir("web"/comp)
if dirExists("web"/comp):
rmDir("web"/comp)
mkDir("web"/comp) mkDir("web"/comp)
for file in listFiles(".."/comp/comp) & listFiles(".."/comp): for file in listFiles(".."/comp/comp) & listFiles(".."/comp):
if file.splitFile().ext == ".nim": if file.splitFile().ext == ".nim":
cpFile(file, "web"/comp/extractFilename(file)) cpFile(file, "web"/comp/extractFilename(file))
cpFile("web"/"nimdoc.cfg", "web"/comp/"nimdoc.cfg") cpFile("web"/"nimdoc.cfg", "web"/comp/"nimdoc.cfg")
withDir("web"/comp): withDir("web"/comp):
for file in listFiles("."): for file in listFiles("."):
if file.splitFile().ext == ".nim": if file.splitFile().ext == ".nim":
if not pygonly: exec "nim doc --git.url:. --index:on -o:" & file.changeFileExt("html") & " " & file
exec "nim doc --git.url:. --index:on -o:" & file.changeFileExt("html") & " " & file exec "pygmentize -f html -O full,linenos=1,anchorlinenos=True,lineanchors=L,style=vs -o " & file & ".html " & file
exec "pygmentize -f html -O full,linenos=1,anchorlinenos=True,lineanchors=L,style=vs -o " & file & ".html " & file
if not pygonly: exec "nim buildIndex -o:index.html ."
exec "nim buildIndex -o:index.html ." rmFile("web"/comp/"nimdoc.cfg")
rmFile("web"/comp/"nimdoc.cfg")

4
tests/rununittests.nim
View file

@ -1,10 +1,8 @@
import os, osproc, strutils import os, osproc, strutils
proc main() = proc main() =
var failures = 0
for file in walkFiles(currentSourcePath().splitPath().head / "unittests/*.nim"): for file in walkFiles(currentSourcePath().splitPath().head / "unittests/*.nim"):
let (path, fname, ext) = file.splitFile() let (path, fname, ext) = file.splitFile()
if fname.startswith("test"): if fname.startswith("test"):
failures += execCmd "nim c -r " & file discard execCmd "nim c -r " & file
quit(failures)
main() main()

4
tests/unittests/testfileops.nim
View file

@ -97,6 +97,8 @@ suite "test file ops":
dataDir.createDir() dataDir.createDir()
setup: setup:
if testfilename.existsFile():
removeFile(testfilename)
writeFile(testfilename, testFileContent) writeFile(testfilename, testFileContent)
################### Prepend ####################### ################### Prepend #######################
@ -118,7 +120,7 @@ suite "test file ops":
test "pipe command into file": test "pipe command into file":
when defined(windows): when defined(windows):
pipe(testfilename, "ECHO foo") pipe(testfilename, "(ECHO foo)>>$file")
testfilename.checkFile("foo") testfilename.checkFile("foo")
else: else:
pipe(testfilename, "cat $file | grep 'this is text'") pipe(testfilename, "cat $file | grep 'this is text'")

1
web/CNAME
View file

@ -1 +0,0 @@
nimgen.genotrance.com

1401
web/nimdoc.cfg
View file

File diff suppressed because it is too large Load diff