jyapayne/nimble
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge branch 'nimscript-support'

Browse source
This commit is contained in:
Dominik Picheta 2015-12-24 01:35:32 +00:00
commit d94501234a
10 changed files with 747 additions and 180 deletions
Download patch file Download diff file Expand all files Collapse all files

2
nimble.nimble
View file

@ -1,6 +1,6 @@
[Package] [Package]
name = "nimble" name = "nimble"
version = "0.6.2" version = "0.7.0"
author = "Dominik Picheta" author = "Dominik Picheta"
description = "Nim package manager." description = "Nim package manager."
license = "BSD" license = "BSD"

92
readme.markdown
View file

@ -230,6 +230,28 @@ which can be useful to read the bundled documentation. Example:
The nimble ``init`` command will start a simple wizard which will create The nimble ``init`` command will start a simple wizard which will create
a quick ``.nimble`` file for your project. a quick ``.nimble`` file for your project.
As of version 0.7.0, the ``.nimble`` file that this command creates will
use the new NimScript format.
Check out the [Creating Packages](#creating-packages) section for more info.
### nimble publish
Publishes your Nimble package to the official Nimble package repository.
**Note:** Requires a valid Github account.
### nimble tasks
For a nimble package in the current working directory, list the tasks which that
package defines. This is only supported for packages utilising the new
nimscript .nimble files.
### nimble dump
Outputs information about the package in the current working directory in
an ini-compatible format. Useful for tools wishing to read metadata about
Nimble packages who do not want to use the NimScript evaluator.
## Configuration ## Configuration
At startup Nimble will attempt to read ``~/.config/nimble/nimble.ini`` on Linux At startup Nimble will attempt to read ``~/.config/nimble/nimble.ini`` on Linux
@ -265,7 +287,17 @@ named "foobar" should have a corresponding ``foobar.nimble`` file.
These files specify information about the package including its name, author, These files specify information about the package including its name, author,
license, dependencies and more. Without one Nimble is not able to install license, dependencies and more. Without one Nimble is not able to install
a package. A bare minimum .nimble file follows: a package.
A .nimble file can be created easily using Nimble's ``init`` command. This
command will ask you a bunch of questions about your package, then generate a
.nimble file for you.
**Note:** As of version 0.7.0, the ``init`` command generates .nimble files
using the new NimScript format. Take a look at the next section for more
information.
A bare minimum .nimble file follows:
```ini ```ini
[Package] [Package]
@ -286,6 +318,62 @@ Nimble currently supports installation of packages from a local directory, a
git repository and a mercurial repository. The .nimble file must be present in git repository and a mercurial repository. The .nimble file must be present in
the root of the directory or repository being installed. the root of the directory or repository being installed.
### The new NimScript format
**Warning:** This features is still very experimental. You are encouraged to
try it, but be aware that it may change significantly in the future or
may even be removed completely!
Version 0.7.0 of Nimble introduces support for evaluating .nimble files
as Nim code. This means that you can define metadata about your package
using the Nim programming language.
Because of Nim's flexibility the definitions remain declarative. With the added
ability of using the Nim language to enrich your package specification.
For example, you can define dependencies for specific platforms using Nim's
``when`` statement.
The ini example above looks like this in the NimScript format:
```nim
# Package
version = "0.1.0"
author = "Your Name"
description = "Example .nimble file."
license = "MIT"
# Deps
requires "nim >= 0.10.0"
```
The format is indeed very similar to the ini format. Another great feature
is the ability to define custom Nimble package-specific commands. These are
defined in the .nimble files of course.
```nim
task hello, "This is a hello task":
echo("Hello World!")
```
You can then execute ``nimble hello``, which will result in the following
output:
```
Executing task hello in /Users/user/projects/pkg/pkg.nimble
Hello World!
```
You can place any Nim code inside these tasks. As long as that code does not
access the FFI. The ``nimscript``
[module](http://nim-lang.org/docs/nimscript.html) in Nim's standard library defines
additional functionality such as the ability to execute external processes
which makes this feature very powerful.
You can also check what tasks are supported by the package in the current
directory by using the ``tasks`` command.
### Libraries ### Libraries
Library packages are likely the most popular form of Nimble packages. They are Library packages are likely the most popular form of Nimble packages. They are
@ -431,7 +519,7 @@ their own packages to it! Take a look at
#### Required #### Required
* ``name`` - The name of the package. * ``name`` - The name of the package. *(This is not required in the new NimScript format)*
* ``version`` - The *current* version of this package. This should be incremented * ``version`` - The *current* version of this package. This should be incremented
**after** tagging the current version using ``git tag`` or ``hg tag``. **after** tagging the current version using ``git tag`` or ``hg tag``.
* ``author`` - The name of the author of this package. * ``author`` - The name of the author of this package.

336
src/nimble.nim
View file

@ -10,6 +10,8 @@ import nimblepkg/packageinfo, nimblepkg/version, nimblepkg/tools,
nimblepkg/download, nimblepkg/config, nimblepkg/nimbletypes, nimblepkg/download, nimblepkg/config, nimblepkg/nimbletypes,
nimblepkg/publish nimblepkg/publish
import nimblepkg/nimscriptsupport
when not defined(windows): when not defined(windows):
from posix import getpid from posix import getpid
else: else:
@ -38,13 +40,14 @@ type
nimbleData: JsonNode ## Nimbledata.json nimbleData: JsonNode ## Nimbledata.json
ActionType = enum ActionType = enum
actionNil, actionUpdate, actionInit, actionPublish, actionNil, actionUpdate, actionInit, actionDump, actionPublish,
actionInstall, actionSearch, actionInstall, actionSearch,
actionList, actionBuild, actionPath, actionUninstall, actionCompile actionList, actionBuild, actionPath, actionUninstall, actionCompile,
actionCustom, actionTasks
Action = object Action = object
case typ: ActionType case typ: ActionType
of actionNil, actionList, actionBuild, actionPublish: nil of actionNil, actionList, actionBuild, actionPublish, actionTasks: nil
of actionUpdate: of actionUpdate:
optionalURL: string # Overrides default package list. optionalURL: string # Overrides default package list.
of actionInstall, actionPath, actionUninstall: of actionInstall, actionPath, actionUninstall:
@ -53,7 +56,7 @@ type
packages: seq[PkgTuple] # Optional only for actionInstall. packages: seq[PkgTuple] # Optional only for actionInstall.
of actionSearch: of actionSearch:
search: seq[string] # Search string. search: seq[string] # Search string.
of actionInit: of actionInit, actionDump:
projName: string projName: string
of actionCompile: of actionCompile:
file: string file: string
@ -84,8 +87,12 @@ Commands:
performed by tag and by name. performed by tag and by name.
list [--ver] Lists all packages. list [--ver] Lists all packages.
[-i, --installed] Lists all installed packages. [-i, --installed] Lists all installed packages.
tasks Lists the tasks specified in the Nimble
package's Nimble file.
path pkgname ... Shows absolute path to the installed packages path pkgname ... Shows absolute path to the installed packages
specified. specified.
dump [pkgname] Outputs Nimble package information for
external tools.
Options: Options:
-h, --help Print this help message. -h, --help Print this help message.
@ -99,7 +106,7 @@ Options:
For more information read the Github readme: For more information read the Github readme:
https://github.com/nim-lang/nimble#readme https://github.com/nim-lang/nimble#readme
""" """
nimbleVersion = "0.6.2" nimbleVersion = "0.7.0"
defaultPackageURL = defaultPackageURL =
"https://github.com/nim-lang/packages/raw/master/packages.json" "https://github.com/nim-lang/packages/raw/master/packages.json"
@ -144,6 +151,18 @@ proc prompt(options: Options, question: string): bool =
else: else:
return false return false
proc promptCustom(question, default: string): string =
if default == "":
stdout.write(question, ": ")
let user = stdin.readLine()
if user.len == 0: return promptCustom(question, default)
else: return user
else:
stdout.write(question, " [", default, "]: ")
let user = stdin.readLine()
if user == "": return default
else: return user
proc renameBabelToNimble(options: Options) {.deprecated.} = proc renameBabelToNimble(options: Options) {.deprecated.} =
let babelDir = getHomeDir() / ".babel" let babelDir = getHomeDir() / ".babel"
let nimbleDir = getHomeDir() / ".nimble" let nimbleDir = getHomeDir() / ".nimble"
@ -156,6 +175,64 @@ proc renameBabelToNimble(options: Options) {.deprecated.} =
removeDir(babelDir) removeDir(babelDir)
removeFile(nimbleDir / "babeldata.json") removeFile(nimbleDir / "babeldata.json")
proc parseActionType(action: string): ActionType =
case action.normalize()
of "install", "path":
case action.normalize()
of "install":
result = actionInstall
of "path":
result = actionPath
else:
discard
of "build":
result = actionBuild
of "c", "compile", "js", "cpp", "cc":
result = actionCompile
of "init":
result = actionInit
of "dump":
result = actionDump
of "update", "refresh":
result = actionUpdate
of "search":
result = actionSearch
of "list":
result = actionList
of "uninstall", "remove", "delete", "del", "rm":
result = actionUninstall
of "publish":
result = actionPublish
of "tasks":
result = actionTasks
else:
result = actionCustom
proc initAction(options: var Options, key: string) =
## Intialises `options.actions` fields based on `options.actions.typ` and
## `key`.
let keyNorm = key.normalize()
case options.action.typ
of actionInstall, actionPath:
options.action.packages = @[]
of actionCompile:
options.action.compileOptions = @[]
options.action.file = ""
if keyNorm == "c" or keyNorm == "compile": options.action.backend = ""
else: options.action.backend = keyNorm
of actionInit:
options.action.projName = ""
of actionDump:
options.action.projName = ""
of actionUpdate:
options.action.optionalURL = ""
of actionSearch:
options.action.search = @[]
of actionUninstall:
options.action.packages = @[]
of actionBuild, actionPublish, actionCustom, actionList, actionTasks,
actionNil: discard
proc parseCmdLine(): Options = proc parseCmdLine(): Options =
result.action.typ = actionNil result.action.typ = actionNil
result.config = parseConfig() result.config = parseConfig()
@ -163,41 +240,9 @@ proc parseCmdLine(): Options =
case kind case kind
of cmdArgument: of cmdArgument:
if result.action.typ == actionNil: if result.action.typ == actionNil:
case key setNimScriptCommand(key)
of "install", "path": result.action.typ = parseActionType(key)
case key initAction(result, key)
of "install":
result.action.typ = actionInstall
of "path":
result.action.typ = actionPath
else:
discard
result.action.packages = @[]
of "build":
result.action.typ = actionBuild
of "c", "compile", "js", "cpp", "cc":
result.action.typ = actionCompile
result.action.compileOptions = @[]
result.action.file = ""
if key == "c" or key == "compile": result.action.backend = ""
else: result.action.backend = key
of "init":
result.action.typ = actionInit
result.action.projName = ""
of "update", "refresh":
result.action.typ = actionUpdate
result.action.optionalURL = ""
of "search":
result.action.typ = actionSearch
result.action.search = @[]
of "list":
result.action.typ = actionList
of "uninstall", "remove", "delete", "del", "rm":
result.action.typ = actionUninstall
result.action.packages = @[]
of "publish":
result.action.typ = actionPublish
else: writeHelp()
else: else:
case result.action.typ case result.action.typ
of actionNil: of actionNil:
@ -215,7 +260,7 @@ proc parseCmdLine(): Options =
result.action.optionalURL = key result.action.optionalURL = key
of actionSearch: of actionSearch:
result.action.search.add(key) result.action.search.add(key)
of actionInit: of actionInit, actionDump:
if result.action.projName != "": if result.action.projName != "":
raise newException(NimbleError, raise newException(NimbleError,
"Can only initialize one package at a time.") "Can only initialize one package at a time.")
@ -721,11 +766,13 @@ proc install(packages: seq[PkgTuple],
proc build(options: Options) = proc build(options: Options) =
var pkgInfo = getPkgInfo(getCurrentDir()) var pkgInfo = getPkgInfo(getCurrentDir())
nimScriptHint(pkgInfo)
let paths = processDeps(pkginfo, options) let paths = processDeps(pkginfo, options)
buildFromDir(pkgInfo, paths, false) buildFromDir(pkgInfo, paths, false)
proc compile(options: Options) = proc compile(options: Options) =
var pkgInfo = getPkgInfo(getCurrentDir()) var pkgInfo = getPkgInfo(getCurrentDir())
nimScriptHint(pkgInfo)
let paths = processDeps(pkginfo, options) let paths = processDeps(pkginfo, options)
let realDir = pkgInfo.getRealDir() let realDir = pkgInfo.getRealDir()
@ -827,9 +874,11 @@ proc listPaths(options: Options) =
continue continue
let let
nimScriptFile = path / name.addFileExt("nims")
babelFile = path / name.addFileExt("babel") babelFile = path / name.addFileExt("babel")
nimbleFile = path / name.addFileExt("nimble") nimbleFile = path / name.addFileExt("nimble")
hasSpec = nimbleFile.existsFile or babelFile.existsFile hasSpec = nimScriptFile.existsFile or
nimbleFile.existsFile or babelFile.existsFile
if hasSpec: if hasSpec:
var pkgInfo = getPkgInfo(path) var pkgInfo = getPkgInfo(path)
var v: VersionAndPath var v: VersionAndPath
@ -849,44 +898,105 @@ proc listPaths(options: Options) =
raise newException(NimbleError, raise newException(NimbleError,
"At least one of the specified packages was not found") "At least one of the specified packages was not found")
proc join(x: seq[PkgTuple]; y: string): string =
if x.len == 0: return ""
result = x[0][0] & " " & $x[0][1]
for i in 1 ..< x.len:
result.add y
result.add x[i][0] & " " & $x[i][1]
proc dump(options: Options) =
let proj = addFileExt(options.action.projName, "nimble")
let p = if fileExists(proj): readPackageInfo(proj)
else: getPkgInfo(os.getCurrentDir())
echo "name: ", p.name.escape
echo "version: ", p.version.escape
echo "author: ", p.author.escape
echo "desc: ", p.description.escape
echo "license: ", p.license.escape
echo "skipDirs: ", p.skipDirs.join(", ").escape
echo "skipFiles: ", p.skipFiles.join(", ").escape
echo "skipExt: ", p.skipExt.join(", ").escape
echo "installDirs: ", p.installDirs.join(", ").escape
echo "installFiles: ", p.installFiles.join(", ").escape
echo "installExt: ", p.installExt.join(", ").escape
echo "requires: ", p.requires.join(", ").escape
echo "bin: ", p.bin.join(", ").escape
echo "binDir: ", p.binDir.escape
echo "srcDir: ", p.srcDir.escape
echo "backend: ", p.backend.escape
proc init(options: Options) = proc init(options: Options) =
echo("Initializing new Nimble project!") var nimbleFile: string = ""
var
pkgName, fName: string = ""
outFile: File
if (options.action.projName != ""): echo("In order to initialise a new Nimble package, I will need to ask you\n" &
pkgName = options.action.projName "some questions. Default values are shown in square brackets, press\n" &
fName = pkgName & ".nimble" "enter to use them.")
if (existsFile(os.getCurrentDir() / fName)):
raise newException(NimbleError, "Already have a nimble file.") # Ask for package name.
if options.action.projName != "":
let pkgName = options.action.projName
nimbleFile = pkgName.changeFileExt("nimble")
else: else:
echo("Enter a project name for this (blank to use working directory), " & var pkgName = os.getCurrentDir().splitPath.tail.toValidPackageName()
"Ctrl-C to abort:") pkgName = promptCustom("Enter package name", pkgName)
pkgName = readline(stdin) nimbleFile = pkgName.changeFileExt("nimble")
if (pkgName == ""):
pkgName = os.getCurrentDir().splitPath.tail
if (pkgName == ""):
raise newException(NimbleError, "Could not get default file path.")
fName = pkgName & ".nimble"
# Now need to write out .nimble file with projName and other details validatePackageName(nimbleFile.changeFileExt(""))
if (not existsFile(os.getCurrentDir() / fName) and if existsFile(os.getCurrentDir() / nimbleFile):
open(f=outFile, filename = fName, mode = fmWrite)): raise newException(NimbleError, "Nimble file already exists.")
outFile.writeln("[Package]")
outFile.writeln("name = \"" & pkgName & "\"") # Ask for package version.
outFile.writeln("version = \"0.1.0\"") let pkgVersion = promptCustom("Enter intial version of package", "0.1.0")
outFile.writeln("author = \"Anonymous\"") validateVersion(pkgVersion)
outFile.writeln("description = \"New Nimble project for Nim\"")
outFile.writeln("license = \"MIT\"") # Ask for package author
outFile.writeln("") var defaultAuthor = "Anonymous"
outFile.writeln("[Deps]") if findExe("git") != "":
outFile.writeln("Requires: \"nim >= 0.10.0\"") let (name, exitCode) = doCmdEx("git config --global user.name")
if exitCode == QuitSuccess and name.len > 0:
defaultAuthor = name.strip()
elif defaultAuthor == "Anonymous" and findExe("hg") != "":
let (name, exitCode) = doCmdEx("hg config ui.username")
if exitCode == QuitSuccess and name.len > 0:
defaultAuthor = name.strip()
let pkgAuthor = promptCustom("Enter your name", defaultAuthor)
# Ask for description
let pkgDesc = promptCustom("Enter package description", "")
# Ask for license
# TODO: Provide selection of licenses, or select random default license.
let pkgLicense = promptCustom("Enter package license", "MIT")
# Ask for Nim dependency
let nimDepDef = getNimrodVersion()
let pkgNimDep = promptCustom("Enter lowest supported Nim version", $nimDepDef)
validateVersion(pkgNimDep)
# Now generate the .nimble file.
if existsFile(os.getCurrentDir() / nimbleFile):
raise newException(NimbleError,
"Looks like a Nimble file has already been created.")
var outFile: File
if open(f = outFile, filename = nimbleFile, mode = fmWrite):
outFile.writeLine """# Package
version = $#
author = $#
description = $#
license = $#
# Dependencies
requires "nim >= $#"
""" % [pkgVersion.escape(), pkgAuthor.escape(), pkgDesc.escape(),
pkgLicense.escape(), pkgNimDep]
close(outFile) close(outFile)
else: else:
raise newException(NimbleError, "Unable to open file " & fName & raise newException(NimbleError, "Unable to open file " & nimbleFile &
" for writing: " & osErrorMsg(osLastError())) " for writing: " & osErrorMsg(osLastError()))
proc uninstall(options: Options) = proc uninstall(options: Options) =
@ -946,37 +1056,67 @@ proc uninstall(options: Options) =
removePkgDir(options.getPkgsDir / (pkg.name & '-' & pkg.version), options) removePkgDir(options.getPkgsDir / (pkg.name & '-' & pkg.version), options)
echo("Removed ", pkg.name, " (", $pkg.version, ")") echo("Removed ", pkg.name, " (", $pkg.version, ")")
proc listTasks(options: Options) =
let nimbleFile = findNimbleFile(getCurrentDir(), true)
nimscriptsupport.listTasks(nimbleFile)
proc doAction(options: Options) = proc doAction(options: Options) =
if not existsDir(options.getNimbleDir()): if not existsDir(options.getNimbleDir()):
createDir(options.getNimbleDir()) createDir(options.getNimbleDir())
if not existsDir(options.getPkgsDir): if not existsDir(options.getPkgsDir):
createDir(options.getPkgsDir) createDir(options.getPkgsDir)
case options.action.typ var command = getNimScriptCommand().parseActionType()
of actionUpdate: # The loop is necessary to support tasks using `setCommand`.
update(options) var moreCommands = true
of actionInstall: while moreCommands:
discard install(options.action.packages, options) moreCommands = false
of actionUninstall: case command
uninstall(options) of actionUpdate:
of actionSearch: update(options)
search(options) of actionInstall:
of actionList: let (_, pkgInfo) = install(options.action.packages, options)
if options.queryInstalled: listInstalled(options) if options.action.packages.len == 0:
else: list(options) nimScriptHint(pkgInfo)
of actionPath: of actionUninstall:
listPaths(options) uninstall(options)
of actionBuild: of actionSearch:
build(options) search(options)
of actionCompile: of actionList:
compile(options) if options.queryInstalled: listInstalled(options)
of actionInit: else: list(options)
init(options) of actionPath:
of actionPublish: listPaths(options)
var pkgInfo = getPkgInfo(getCurrentDir()) of actionBuild:
publish(pkgInfo) build(options)
of actionNil: of actionCompile:
assert false compile(options)
of actionInit:
init(options)
of actionPublish:
var pkgInfo = getPkgInfo(getCurrentDir())
publish(pkgInfo)
of actionDump:
dump(options)
of actionTasks:
listTasks(options)
of actionNil:
assert false
of actionCustom:
# Custom command. Attempt to call a NimScript task.
let nimbleFile = findNimbleFile(getCurrentDir(), true)
let oldCmd = getNimScriptCommand()
if not nimbleFile.isNimScript():
writeHelp()
if not execTask(nimbleFile, oldCmd):
echo("FAILURE: Could not find task ", oldCmd, " in ", nimbleFile)
writeHelp()
if getNimScriptCommand().normalize == "nop":
echo("WARNING: Using `setCommand 'nop'` is not necessary.")
break
command = getNimScriptCommand().parseActionType()
moreCommands = hasTaskRequestedCommand()
when isMainModule: when isMainModule:
when defined(release): when defined(release):

3
src/nimble.nimrod.cfg
View file

@ -1 +1,4 @@
-d:ssl -d:ssl
--path:"$lib/packages/docutils"
--noNimblePath
--path:"$nim/"

2
src/nimblepkg/nim.cfg Normal file
View file

@ -0,0 +1,2 @@
--path:"$nim/"
--path:"$lib/packages/docutils"

5
src/nimblepkg/nimbletypes.nim
View file

@ -4,13 +4,15 @@
# recursive imports # recursive imports
import version import version
export version.NimbleError
type type
NimbleError* = object of Exception
BuildFailed* = object of NimbleError BuildFailed* = object of NimbleError
PackageInfo* = object PackageInfo* = object
mypath*: string ## The path of this .nimble file mypath*: string ## The path of this .nimble file
isNimScript*: bool ## Determines if this pkg info was read from a nims file
isMinimal*: bool
name*: string name*: string
version*: string version*: string
author*: string author*: string
@ -27,4 +29,3 @@ type
binDir*: string binDir*: string
srcDir*: string srcDir*: string
backend*: string backend*: string

185
src/nimblepkg/nimscriptsupport.nim Normal file
View file

@ -0,0 +1,185 @@
# Copyright (C) Andreas Rumpf. All rights reserved.
# BSD License. Look at license.txt for more info.
## Implements the new configuration system for Nimble. Uses Nim as a
## scripting language.
import
compiler/ast, compiler/modules, compiler/passes, compiler/passaux,
compiler/condsyms, compiler/options, compiler/sem, compiler/semdata,
compiler/llstream, compiler/vm, compiler/vmdef, compiler/commands,
compiler/msgs, compiler/magicsys, compiler/lists
from compiler/scriptconfig import setupVM
from compiler/idents import getIdent
from compiler/astalgo import strTableGet, `$`
import nimbletypes, version
import os, strutils
const
internalCmd = "NimbleInternal"
proc raiseVariableError(ident, typ: string) {.noinline.} =
raise newException(NimbleError,
"NimScript's variable '" & ident & "' needs a value of type '" & typ & "'.")
proc isStrLit(n: PNode): bool = n.kind in {nkStrLit..nkTripleStrLit}
proc getGlobal(ident: string): string =
let n = vm.globalCtx.getGlobalValue(getSysSym ident)
if n.isStrLit:
result = if n.strVal.isNil: "" else: n.strVal
else:
raiseVariableError(ident, "string")
proc getGlobalAsSeq(ident: string): seq[string] =
let n = vm.globalCtx.getGlobalValue(getSysSym ident)
result = @[]
if n.kind == nkBracket:
for x in n:
if x.isStrLit:
result.add x.strVal
else:
raiseVariableError(ident, "seq[string]")
else:
raiseVariableError(ident, "seq[string]")
proc extractRequires(result: var seq[PkgTuple]) =
let n = vm.globalCtx.getGlobalValue(getSysSym "requiresData")
if n.kind == nkBracket:
for x in n:
if x.kind == nkPar and x.len == 2 and x[0].isStrLit and x[1].isStrLit:
result.add(parseRequires(x[0].strVal & x[1].strVal))
elif x.isStrLit:
result.add(parseRequires(x.strVal))
else:
raiseVariableError("requiresData", "seq[(string, VersionReq)]")
else:
raiseVariableError("requiresData", "seq[(string, VersionReq)]")
proc execScript(scriptName: string) =
## Executes the specified script.
##
## No clean up is performed and must be done manually!
setDefaultLibpath()
passes.gIncludeFile = includeModule
passes.gImportModule = importModule
initDefines()
defineSymbol("nimscript")
defineSymbol("nimconfig")
defineSymbol("nimble")
registerPass(semPass)
registerPass(evalPass)
appendStr(searchPaths, options.libpath)
var m = makeModule(scriptName)
incl(m.flags, sfMainModule)
vm.globalCtx = setupVM(m, scriptName)
compileSystemModule()
processModule(m, llStreamOpen(scriptName, fmRead), nil)
proc cleanup() =
# ensure everything can be called again:
resetAllModulesHard()
clearPasses()
msgs.gErrorMax = 1
msgs.writeLnHook = nil
vm.globalCtx = nil
initDefines()
proc readPackageInfoFromNims*(scriptName: string; result: var PackageInfo) =
## Executes the `scriptName` nimscript file. Reads the package information
## that it populates.
# Setup custom error handling.
msgs.gErrorMax = high(int)
var previousMsg = ""
msgs.writeLnHook =
proc (output: string) =
# The error counter is incremented after the writeLnHook is invoked.
if msgs.gErrorCounter > 0:
raise newException(NimbleError, previousMsg)
previousMsg = output
# Execute the nimscript file.
execScript(scriptName)
# Extract all the necessary fields populated by the nimscript file.
template trivialField(field) =
result.field = getGlobal(astToStr field)
template trivialFieldSeq(field) =
result.field.add getGlobalAsSeq(astToStr field)
# keep reasonable default:
let name = getGlobal"packageName"
if name.len > 0: result.name = name
trivialField version
trivialField author
trivialField description
trivialField license
trivialField srcdir
trivialField bindir
trivialFieldSeq skipDirs
trivialFieldSeq skipFiles
trivialFieldSeq skipExt
trivialFieldSeq installDirs
trivialFieldSeq installFiles
trivialFieldSeq installExt
extractRequires result.requires
let binSeq = getGlobalAsSeq("bin")
for i in binSeq:
result.bin.add(i.addFileExt(ExeExt))
let backend = getGlobal("backend")
if backend.len == 0:
result.backend = "c"
elif cmpIgnoreStyle(backend, "javascript") == 0:
result.backend = "js"
else:
result.backend = backend.toLower()
cleanup()
proc execTask*(scriptName, taskName: string): bool =
## Executes the specified task in the specified script.
##
## `scriptName` should be a filename pointing to the nimscript file.
result = true
options.command = internalCmd
echo("Executing task ", taskName, " in ", scriptName)
execScript(scriptName)
# Explicitly execute the task procedure, instead of relying on hack.
assert vm.globalCtx.module.kind == skModule
let prc = vm.globalCtx.module.tab.strTableGet(getIdent(taskName & "Task"))
if prc.isNil:
# Procedure not defined in the NimScript module.
return false
discard vm.globalCtx.execProc(prc, [])
cleanup()
proc getNimScriptCommand*(): string =
options.command
proc setNimScriptCommand*(command: string) =
options.command = command
proc hasTaskRequestedCommand*(): bool =
## Determines whether the last executed task used ``setCommand``
return getNimScriptCommand() != internalCmd
proc listTasks*(scriptName: string) =
setNimScriptCommand("help")
execScript(scriptName)
# TODO: Make the 'task' template generate explicit data structure containing
# all the task names + descriptions.
cleanup()

259
src/nimblepkg/packageinfo.nim
View file

@ -1,7 +1,7 @@
# Copyright (C) Dominik Picheta. All rights reserved. # Copyright (C) Dominik Picheta. All rights reserved.
# BSD License. Look at license.txt for more info. # BSD License. Look at license.txt for more info.
import parsecfg, json, streams, strutils, parseutils, os import parsecfg, json, streams, strutils, parseutils, os
import version, tools, nimbletypes import version, tools, nimbletypes, nimscriptsupport
when not declared(system.map): when not declared(system.map):
from sequtils import map from sequtils import map
@ -15,7 +15,7 @@ type
downloadMethod*: string downloadMethod*: string
description*: string description*: string
tags*: seq[string] # Even if empty, always a valid non nil seq. \ tags*: seq[string] # Even if empty, always a valid non nil seq. \
# From here on, optional fields set to the emtpy string if not available. # From here on, optional fields set to the empty string if not available.
version*: string version*: string
dvcsTag*: string dvcsTag*: string
web*: string # Info url for humans. web*: string # Info url for humans.
@ -23,9 +23,12 @@ type
MetaData* = object MetaData* = object
url*: string url*: string
proc initPackageInfo(): PackageInfo = NimbleFile* = string
result.mypath = ""
result.name = "" proc initPackageInfo(path: string): PackageInfo =
result.mypath = path
# reasonable default:
result.name = path.splitFile.name
result.version = "" result.version = ""
result.author = "" result.author = ""
result.description = "" result.description = ""
@ -42,6 +45,47 @@ proc initPackageInfo(): PackageInfo =
result.binDir = "" result.binDir = ""
result.backend = "c" result.backend = "c"
proc validatePackageName*(name: string) =
## Raises an error if specified package name contains invalid characters.
##
## A valid package name is one which is a valid nim module name. So only
## underscores, letters and numbers allowed.
if name.len == 0: return
if name[0] in {'0'..'9'}:
raise newException(NimbleError,
"Invalid package name: cannot beging with " & name[0])
var prevWasUnderscore = false
for c in name:
case c
of '_':
if prevWasUnderscore:
raise newException(NimbleError,
"Invalid package name: cannot contain \"__\"")
prevWasUnderscore = true
of AllChars - IdentChars:
raise newException(NimbleError,
"Invalid package name: cannot contain '$1'" % $c)
else:
prevWasUnderscore = false
proc toValidPackageName*(name: string): string =
result = ""
for c in name:
case c
of '_', '-':
if result[^1] != '_': result.add('_')
of AllChars - IdentChars - {'-'}: discard
else: result.add(c)
proc validateVersion*(ver: string) =
for c in ver:
if c notin ({'.'} + Digits):
raise newException(NimbleError,
"Version may only consist of numbers and the '.' character " &
"but found '" & c & "'.")
proc validatePackageInfo(pkgInfo: PackageInfo, path: string) = proc validatePackageInfo(pkgInfo: PackageInfo, path: string) =
if pkgInfo.name == "": if pkgInfo.name == "":
raise newException(NimbleError, "Incorrect .nimble file: " & path & raise newException(NimbleError, "Incorrect .nimble file: " & path &
@ -49,40 +93,29 @@ proc validatePackageInfo(pkgInfo: PackageInfo, path: string) =
if pkgInfo.version == "": if pkgInfo.version == "":
raise newException(NimbleError, "Incorrect .nimble file: " & path & raise newException(NimbleError, "Incorrect .nimble file: " & path &
" does not contain a version field.") " does not contain a version field.")
if pkgInfo.author == "":
raise newException(NimbleError, "Incorrect .nimble file: " & path &
" does not contain an author field.")
if pkgInfo.description == "":
raise newException(NimbleError, "Incorrect .nimble file: " & path &
" does not contain a description field.")
if pkgInfo.license == "":
raise newException(NimbleError, "Incorrect .nimble file: " & path &
" does not contain a license field.")
if pkgInfo.backend notin ["c", "cc", "objc", "cpp", "js"]:
raise newException(NimbleError, "'" & pkgInfo.backend &
"' is an invalid backend.")
for c in pkgInfo.version:
if c notin ({'.'} + Digits):
raise newException(NimbleError,
"Version may only consist of numbers and the '.' character " &
"but found '" & c & "'.")
proc parseRequires(req: string): PkgTuple = if not pkgInfo.isMinimal:
try: if pkgInfo.author == "":
if ' ' in req: raise newException(NimbleError, "Incorrect .nimble file: " & path &
var i = skipUntil(req, Whitespace) " does not contain an author field.")
result.name = req[0 .. i].strip if pkgInfo.description == "":
result.ver = parseVersionRange(req[i .. req.len-1]) raise newException(NimbleError, "Incorrect .nimble file: " & path &
elif '#' in req: " does not contain a description field.")
var i = skipUntil(req, {'#'}) if pkgInfo.license == "":
result.name = req[0 .. i-1] raise newException(NimbleError, "Incorrect .nimble file: " & path &
result.ver = parseVersionRange(req[i .. req.len-1]) " does not contain a license field.")
else: if pkgInfo.backend notin ["c", "cc", "objc", "cpp", "js"]:
result.name = req.strip raise newException(NimbleError, "'" & pkgInfo.backend &
result.ver = VersionRange(kind: verAny) "' is an invalid backend.")
except ParseVersionError:
raise newException(NimbleError, validateVersion(pkgInfo.version)
"Unable to parse dependency version range: " & getCurrentExceptionMsg())
proc nimScriptHint*(pkgInfo: PackageInfo) =
if not pkgInfo.isNimScript:
# TODO: Turn this into a warning.
# TODO: Add a URL explaining more.
echo("NOTE: The .nimble file for this project could make use of " &
"additional features, if converted into the new NimScript format.")
proc multiSplit(s: string): seq[string] = proc multiSplit(s: string): seq[string] =
## Returns ``s`` split by newline and comma characters. ## Returns ``s`` split by newline and comma characters.
@ -100,9 +133,7 @@ proc multiSplit(s: string): seq[string] =
if len(result) < 1: if len(result) < 1:
return @[s] return @[s]
proc readPackageInfo*(path: string): PackageInfo = proc readPackageInfoFromNimble(path: string; result: var PackageInfo) =
result = initPackageInfo()
result.mypath = path
var fs = newFileStream(path, fmRead) var fs = newFileStream(path, fmRead)
if fs != nil: if fs != nil:
var p: CfgParser var p: CfgParser
@ -164,7 +195,72 @@ proc readPackageInfo*(path: string): PackageInfo =
close(p) close(p)
else: else:
raise newException(ValueError, "Cannot open package info: " & path) raise newException(ValueError, "Cannot open package info: " & path)
validatePackageInfo(result, path)
proc getNameVersion*(pkgpath: string): tuple[name, version: string] =
## Splits ``pkgpath`` in the format ``/home/user/.nimble/pkgs/package-0.1``
## into ``(packagea, 0.1)``
##
## Also works for file paths like:
## ``/home/user/.nimble/pkgs/package-0.1/package.nimble``
if pkgPath.splitFile.ext == ".nimble":
return getNameVersion(pkgPath.splitPath.head)
result.name = ""
result.version = ""
let tail = pkgpath.splitPath.tail
if '-' notin tail:
result.name = tail
return
for i in countdown(tail.len-1, 0):
if tail[i] == '-':
result.name = tail[0 .. i-1]
result.version = tail[i+1 .. tail.len-1]
break
proc readPackageInfo*(nf: NimbleFile; onlyMinimalInfo=false): PackageInfo =
## Reads package info from the specifed Nimble file.
##
## Attempts to read it using the "old" Nimble ini format first, if that
## fails attempts to evaluate it as a nimscript file.
##
## If both fail then returns an error.
##
## When ``onlyMinimalInfo`` is true, only the `name` and `version` fields are
## populated. The isNimScript field can also be relied on.
result = initPackageInfo(nf)
var success = false
var iniError: ref NimbleError
# Attempt ini-format first.
try:
readPackageInfoFromNimble(nf, result)
success = true
result.isNimScript = false
except NimbleError:
iniError = (ref NimbleError)(getCurrentException())
if not success:
if onlyMinimalInfo:
let tmp = getNameVersion(nf)
result.name = tmp.name
result.version = tmp.version
result.isNimScript = true
result.isMinimal = true
else:
try:
readPackageInfoFromNims(nf, result)
result.isNimScript = true
except NimbleError:
let msg = "Could not read package info file in " & nf & ";\n" &
" Reading as ini file failed with: \n" &
" " & iniError.msg & ".\n" &
" Evaluating as NimScript file failed with: \n" &
" " & getCurrentExceptionMsg() & "."
raise newException(NimbleError, msg)
validatePackageInfo(result, nf)
proc optionalField(obj: JsonNode, name: string, default = ""): string = proc optionalField(obj: JsonNode, name: string, default = ""): string =
## Queries ``obj`` for the optional ``name`` string. ## Queries ``obj`` for the optional ``name`` string.
@ -241,21 +337,31 @@ proc getPackageList*(packagesPath: string): seq[Package] =
let pkg: Package = p.fromJson() let pkg: Package = p.fromJson()
result.add(pkg) result.add(pkg)
proc findNimbleFile*(dir: string): string = proc findNimbleFile*(dir: string; error: bool): NimbleFile =
result = "" result = ""
var hits = 0
for kind, path in walkDir(dir): for kind, path in walkDir(dir):
if kind == pcFile and path.splitFile.ext in [".babel", ".nimble"]: if kind == pcFile:
if result != "": let ext = path.splitFile.ext
raise newException(NimbleError, case ext
"Only one .nimble file should be present in " & dir) of ".babel", ".nimble":
result = path result = path
inc hits
else: discard
if hits >= 2:
raise newException(NimbleError,
"Only one .nimble file should be present in " & dir)
elif hits == 0:
if error:
raise newException(NimbleError,
"Specified directory does not contain a .nimble file.")
else:
# TODO: Abstract logging.
echo("WARNING: No .nimble file found for ", dir)
proc getPkgInfo*(dir: string): PackageInfo = proc getPkgInfo*(dir: string): PackageInfo =
## Find the .nimble file in ``dir`` and parses it, returning a PackageInfo. ## Find the .nimble file in ``dir`` and parses it, returning a PackageInfo.
let nimbleFile = findNimbleFile(dir) let nimbleFile = findNimbleFile(dir, true)
if nimbleFile == "":
raise newException(NimbleError,
"Specified directory does not contain a .nimble file.")
result = readPackageInfo(nimbleFile) result = readPackageInfo(nimbleFile)
proc getInstalledPkgs*(libsDir: string): proc getInstalledPkgs*(libsDir: string):
@ -266,13 +372,10 @@ proc getInstalledPkgs*(libsDir: string):
result = @[] result = @[]
for kind, path in walkDir(libsDir): for kind, path in walkDir(libsDir):
if kind == pcDir: if kind == pcDir:
let nimbleFile = findNimbleFile(path) let nimbleFile = findNimbleFile(path, false)
if nimbleFile != "": if nimbleFile != "":
let meta = readMetaData(path) let meta = readMetaData(path)
result.add((readPackageInfo(nimbleFile), meta)) result.add((readPackageInfo(nimbleFile, true), meta))
else:
# TODO: Abstract logging.
echo("WARNING: No .nimble file found for ", path)
proc findPkg*(pkglist: seq[tuple[pkginfo: PackageInfo, meta: MetaData]], proc findPkg*(pkglist: seq[tuple[pkginfo: PackageInfo, meta: MetaData]],
dep: PkgTuple, dep: PkgTuple,
@ -284,8 +387,8 @@ proc findPkg*(pkglist: seq[tuple[pkginfo: PackageInfo, meta: MetaData]],
## ##
## **Note**: dep.name here could be a URL, hence the need for pkglist.meta. ## **Note**: dep.name here could be a URL, hence the need for pkglist.meta.
for pkg in pkglist: for pkg in pkglist:
if pkg.pkginfo.name.normalize != dep.name.normalize and if cmpIgnoreStyle(pkg.pkginfo.name, dep.name) != 0 and
pkg.meta.url.normalize != dep.name.normalize: continue cmpIgnoreStyle(pkg.meta.url, dep.name) != 0: continue
if withinRange(newVersion(pkg.pkginfo.version), dep.ver): if withinRange(newVersion(pkg.pkginfo.version), dep.ver):
if not result or newVersion(r.version) < newVersion(pkg.pkginfo.version): if not result or newVersion(r.version) < newVersion(pkg.pkginfo.version):
r = pkg.pkginfo r = pkg.pkginfo
@ -298,8 +401,8 @@ proc findAllPkgs*(pkglist: seq[tuple[pkginfo: PackageInfo, meta: MetaData]],
## packages if multiple are found. ## packages if multiple are found.
result = @[] result = @[]
for pkg in pkglist: for pkg in pkglist:
if pkg.pkginfo.name.normalize != dep.name.normalize and if cmpIgnoreStyle(pkg.pkginfo.name, dep.name) != 0 and
pkg.meta.url.normalize != dep.name.normalize: continue cmpIgnoreStyle(pkg.meta.url, dep.name) != 0: continue
if withinRange(newVersion(pkg.pkginfo.version), dep.ver): if withinRange(newVersion(pkg.pkginfo.version), dep.ver):
result.add pkg.pkginfo result.add pkg.pkginfo
@ -318,22 +421,6 @@ proc getOutputDir*(pkgInfo: PackageInfo, bin: string): string =
else: else:
result = pkgInfo.mypath.splitFile.dir / bin result = pkgInfo.mypath.splitFile.dir / bin
proc getNameVersion*(pkgpath: string): tuple[name, version: string] =
## Splits ``pkgpath`` in the format ``/home/user/.nimble/pkgs/package-0.1``
## into ``(packagea, 0.1)``
result.name = ""
result.version = ""
let tail = pkgpath.splitPath.tail
if '-' notin tail:
result.name = tail
return
for i in countdown(tail.len-1, 0):
if tail[i] == '-':
result.name = tail[0 .. i-1]
result.version = tail[i+1 .. tail.len-1]
break
proc echoPackage*(pkg: Package) = proc echoPackage*(pkg: Package) =
echo(pkg.name & ":") echo(pkg.name & ":")
echo(" url: " & pkg.url & " (" & pkg.downloadMethod & ")") echo(" url: " & pkg.url & " (" & pkg.downloadMethod & ")")
@ -350,8 +437,26 @@ proc getDownloadDirName*(pkg: Package, verRange: VersionRange): string =
result.add "_" result.add "_"
result.add verSimple result.add verSimple
proc isNimScript*(nf: NimbleFile): bool =
readPackageInfo(nf).isNimScript
when isMainModule: when isMainModule:
doAssert getNameVersion("/home/user/.nimble/libs/packagea-0.1") == doAssert getNameVersion("/home/user/.nimble/libs/packagea-0.1") ==
("packagea", "0.1") ("packagea", "0.1")
doAssert getNameVersion("/home/user/.nimble/libs/package-a-0.1") == doAssert getNameVersion("/home/user/.nimble/libs/package-a-0.1") ==
("package-a", "0.1") ("package-a", "0.1")
doAssert getNameVersion("/home/user/.nimble/libs/package-a-0.1/package.nimble") ==
("package-a", "0.1")
validatePackageName("foo_bar")
validatePackageName("f_oo_b_a_r")
try:
validatePackageName("foo__bar")
assert false
except NimbleError:
assert true
doAssert toValidPackageName("foo__bar") == "foo_bar"
doAssert toValidPackageName("jhbasdh!£$@%#^_&*_()qwe") == "jhbasdh_qwe"
echo("All tests passed!")

19
src/nimblepkg/version.nim
View file

@ -33,6 +33,7 @@ type
PkgTuple* = tuple[name: string, ver: VersionRange] PkgTuple* = tuple[name: string, ver: VersionRange]
ParseVersionError* = object of ValueError ParseVersionError* = object of ValueError
NimbleError* = object of Exception
proc newVersion*(ver: string): Version = return Version(ver) proc newVersion*(ver: string): Version = return Version(ver)
proc newSpecial*(spe: string): Special = return Special(spe) proc newSpecial*(spe: string): Special = return Special(spe)
@ -198,6 +199,24 @@ proc parseVersionRange*(s: string): VersionRange =
"Unexpected char in version range: " & s[i]) "Unexpected char in version range: " & s[i])
inc(i) inc(i)
proc parseRequires*(req: string): PkgTuple =
try:
if ' ' in req:
var i = skipUntil(req, Whitespace)
result.name = req[0 .. i].strip
result.ver = parseVersionRange(req[i .. req.len-1])
elif '#' in req:
var i = skipUntil(req, {'#'})
result.name = req[0 .. i-1]
result.ver = parseVersionRange(req[i .. req.len-1])
else:
result.name = req.strip
result.ver = VersionRange(kind: verAny)
except ParseVersionError:
raise newException(NimbleError,
"Unable to parse dependency version range: " & getCurrentExceptionMsg())
proc `$`*(verRange: VersionRange): string = proc `$`*(verRange: VersionRange): string =
case verRange.kind case verRange.kind
of verLater: of verLater:

24
tests/tester.nim
View file

@ -18,6 +18,23 @@ template cd*(dir: string, body: stmt) =
proc processOutput(output: string): seq[string] = proc processOutput(output: string): seq[string] =
output.strip.splitLines().filter((x: string) => (x.len > 0)) output.strip.splitLines().filter((x: string) => (x.len > 0))
test "can install nimscript package":
cd "nimscript":
check execCmdEx("../" & path & " install -y").exitCode == QuitSuccess
test "can execute nimscript tasks":
cd "nimscript":
let (output, exitCode) = execCmdEx("../" & path & " test")
let lines = output.strip.splitLines()
check exitCode == QuitSuccess
check lines[^1] == "10"
test "can list nimscript tasks":
cd "nimscript":
let (output, exitCode) = execCmdEx("../" & path & " tasks")
check output.strip == "test test description"
check exitCode == QuitSuccess
test "can install packagebin2": test "can install packagebin2":
check execCmdEx(path & check execCmdEx(path &
" install -y https://github.com/nimble-test/packagebin2.git").exitCode == " install -y https://github.com/nimble-test/packagebin2.git").exitCode ==
@ -48,6 +65,11 @@ test "issue #27":
cd "issue27": cd "issue27":
check execCmdEx("../" & path & " install -y").exitCode == QuitSuccess check execCmdEx("../" & path & " install -y").exitCode == QuitSuccess
test "can list":
check execCmdEx(path & " list").exitCode == QuitSuccess
check execCmdEx(path & " list -i").exitCode == QuitSuccess
test "can uninstall": test "can uninstall":
block: block:
let (outp, exitCode) = execCmdEx(path & " uninstall -y issue27b") let (outp, exitCode) = execCmdEx(path & " uninstall -y issue27b")
@ -80,3 +102,5 @@ test "can uninstall":
check execCmdEx(path & " uninstall -y PackageA@0.2 issue27b").exitCode == check execCmdEx(path & " uninstall -y PackageA@0.2 issue27b").exitCode ==
QuitSuccess QuitSuccess
check (not dirExists(getHomeDir() / ".nimble" / "pkgs" / "PackageA-0.2.0")) check (not dirExists(getHomeDir() / ".nimble" / "pkgs" / "PackageA-0.2.0"))
check execCmdEx(path & " uninstall -y nimscript").exitCode == QuitSuccess