jyapayne/llvmpy
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Compare commits

base: jyapayne:master
Branches Tags
jyapayne:master
jyapayne:patch-1
jyapayne:llvm-3.4
jyapayne:Cabi_experiment
jyapayne:llpython-dev
jyapayne:llrt
jyapayne:improvefunction
jyapayne:debuginfo
jyapayne:0.12.5
jyapayne:0.12.4
jyapayne:0.12.3
jyapayne:0.12.2
jyapayne:0.12.1
jyapayne:0.12.0
jyapayne:0.11.3
jyapayne:0.11.2
jyapayne:0.11.1
jyapayne:0.11.0
jyapayne:0.10.2
jyapayne:0.10.1
jyapayne:0.10.0
jyapayne:0.9
jyapayne:0.8.3
jyapayne:0.8.2
jyapayne:0.8.1
..
compare: jyapayne:0.8.1
Branches Tags
jyapayne:patch-1
jyapayne:master
jyapayne:llvm-3.4
jyapayne:Cabi_experiment
jyapayne:llpython-dev
jyapayne:llrt
jyapayne:improvefunction
jyapayne:debuginfo
jyapayne:0.12.5
jyapayne:0.12.4
jyapayne:0.12.3
jyapayne:0.12.2
jyapayne:0.12.1
jyapayne:0.12.0
jyapayne:0.11.3
jyapayne:0.11.2
jyapayne:0.11.1
jyapayne:0.11.0
jyapayne:0.10.2
jyapayne:0.10.1
jyapayne:0.10.0
jyapayne:0.9
jyapayne:0.8.3
jyapayne:0.8.2
jyapayne:0.8.1

No commits in common. "master" and "0.8.1" have entirely different histories.
master ... 0.8.1

336 changed files with 12309 additions and 37817 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.gitattributes vendored
View file

@ -1 +0,0 @@
llvm/_version.py export-subst

8
.gitignore vendored
View file

@ -1,8 +0,0 @@
*~
build
_build
*.pyc
*.so
/llvm/_intrinsic_ids.py
llvm_
newbinding/api/*

24
.travis.yml
View file

@ -1,24 +0,0 @@
language: python
python:
- "2.6"
- "2.7"
- "3.2"
- "3.3"
branches:
only:
- master
install:
- wget "http://repo.continuum.io/pkgs/free/linux-64/llvm-3.2-0.tar.bz2"
- tar -xjf llvm-3.2-0.tar.bz2
- PATH+=":`pwd`/bin"
- export LD_LIBRARY_PATH="`pwd`/lib"
- export LLVM_CONFIG_PATH="`pwd`/bin/llvm-config"
- $LLVM_CONFIG_PATH --cflags # test llvm-config
- export LLVMPY_DYNLINK=1
- python setup.py install -q
script: cd ~; python -c "import sys;import llvm;sys.exit(0 if llvm.test() == 0 else 1)"

107
CHANGELOG
View file

@ -1,88 +1,14 @@
2014-04-28 0.12.5:
---------------------
* Fixes memory leaks (#92)
* Fixes tarball (#99)
2014-03-20 0.12.4:
---------------------
* Add dylib_import_library and friends
* Fix BasicBlock downcast
* Module hashing
* Fix test script
0.7, in progress:
2014-02-18 0.12.3:
---------------------
* Fix deprecation message for py2.6
* Fix llvm_cbuilder for using deprecated_alloca
* Merged PR #88 by cantora
* Merged PR #94 by cgohlke
2014-02-04 0.12.2:
---------------------
* enhance wrapper efficiency by moving some capsule code into C++
* fix unclosed file handler in avx_support
* multiple-dimension insert_value, extract_value
* various minor fixes
2013-11-11 0.12.1:
---------------------
* various bug fixes
2013-08-28 0.12.0:
---------------------
* update to LLVM 3.3 and maintain compatibility with LLVM 3.2
* add LLRT for minimal support for 64-bit divmod on 32-bit platform
* start to adopt MCJIT (not quite usable on win32)
* various bug fixes
2013-03-05 0.11.1:
--------------------
* fix test when cc is not available
* fix Python 3 division (Hernan Grecco) (Issue #59)
* add relocation enums and add reloc argument for TargetMachine
2013-03-01 0.11.0:
--------------------
* fix Python 3 support on Windows
* New llvm binding
2013-02-01 0.10.2:
--------------------
* change default to link dynamically to LLVM use:
$ export LLVMPY_DYNLINK=0 # link statically
$ export LLVMPY_DYNLINK=1 # link dynamically
$ unset LLVMPY_DYNLINK # tries to link dynamically if LLVM shared
# objects are found and statically otherwise
* fix llpython for Python 2.6 support
2013-01-25 0.10.1:
--------------------
* fix support for Python 2.6
2013-01-18 0.10.0:
--------------------
* Add LLVM 3.2 support.
* New TargetData class.
* Fixed windows issue (Issue #42).
* Add ExecutionEngine.add_global_mapping and .get_pointer_to_global.
* Improved TargetMachine class and added code-model constants (CM_*).
* Added llvm.passes.build_pass_managers as a simpler way to build PassManagers.
in progress, 0.7:
-----------------
* Add llvm.core.Argument.alignment property.
* Migrate to LLVM 2.8.
* Fix ffi link issue on darwin (Albert Mietus) (Issue #29).
* LLVM tutorial ported (Max Shawabkeh) (Issue #33).
2010-08-31 0.6:
-----------------
0.6, 31-Aug-2010:
* Add and remove function attributes (Krzysztof Goj) (Issue #21).
* Wrap fadd,fsub,fmul (Aaron S Lav) (Issue #31).
* Migrate to LLVM 2.7.
@ -101,16 +27,16 @@ in progress, 0.7:
* Migrate to LLVM 2.5.
2008-11-22 0.5:
-----------------
0.5, 22-Nov-2008:
* Added vicmp, vfcmp instructions and constant expressions.
* Builds on FreeBSD.
* Updated documentation.
* Migrate to LLVM 2.4.
2008-11-21 0.4:
-----------------
0.4, 21-Nov-2008:
* Code cleanup, added license headers.
* Added llvm.core.load_library_permanently() (Issue #12).
* Fix comparison using != (Issue #11).
@ -119,8 +45,8 @@ in progress, 0.7:
* Added viewCFG methods to Function (Paulo Silva).
2008-09-08 0.3:
-----------------
0.3, 8-Sep-2008:
* Passes added.
* Assembly support: create modules from .ll files.
* Various bug fixes.
@ -132,13 +58,13 @@ in progress, 0.7:
* Updated documentation.
2008-06-28 0.2.1:
-------------------
0.2.1, 18-Jun-2008:
* Build cleanly with LLVM 2.3 and 2.3svn.
2008-06-15 0.2:
-----------------
0.2, 15-Jun-2008:
* Independent package, need not be unpacked into llvm/bindings.
* Fixed ownership issues with Module/ModuleProvider.
* Modules, values and types can be stringified, to get their LLVM
@ -153,6 +79,7 @@ in progress, 0.7:
* Lots of cleanup.
2008-05-10 0.1:
-----------------
* Initial release
0.1, 10-May-2008:
* Initial release.

1
LICENSE
View file

@ -1,5 +1,4 @@
Copyright (c) 2008-10, Mahadevan R All rights reserved.
Copyright (c) 2012, Continuum Analytics, Inc. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

4
MANIFEST.in
View file

@ -1,6 +1,4 @@
include CHANGELOG LICENSE README.rst setup.py MANIFEST.in versioneer.py
include CHANGELOG LICENSE README setup.py MANIFEST.in
recursive-include llvm *
recursive-include llvmpy *
recursive-include www *
recursive-include test *
recursive-include tools *

24
README.md Normal file
View file

@ -0,0 +1,24 @@
# llvm-py: Python Bindings for LLVM #
llvm-py provides Python bindings for LLVM.
## Home page ##
https://github.com/AndrewBC/llvm-py
## Versions ##
This package has only been tested with LLVM 2.9, and Python 2.7, (not Python 3.x).
## Quickstart ##
1. Get 2.9 version of LLVM, build it. Make sure '--enable-pic' is passed to LLVM's 'configure'.
2. Get llvm-py and install it:
```
$ git clone git@github.com:AndrewBC/llvm-py.git
$ cd llvm-py
$ python setup.py install
```
3. See documentation at 'www/web/index.html' and examples under 'test'.
## LICENSE ##
llvm-py is distributed under the new BSD license, which is similar to the LLVM license itself.
See the file called LICENSE for the full license text.

67
README.rst
View file

@ -1,67 +0,0 @@
================================
llvmpy: Python bindings for LLVM
================================
Home page
---------
http://www.llvmpy.org
Versions
--------
This package has been tested with LLVM 3.2, Python 2.6, 2.7 and 3.3.
Other Python versions may work.
Quickstart
----------
1. Get and extract LLVM 3.2 source tarball from
`llvm.org <http://llvm.org/releases/download.html#3.2>`_. Then, ``cd`` into
the extracted directory.
2. Run ``./configure --enable-optimized --prefix=LLVM_INSTALL_PATH``.
**Note**: Without the ``--enable-optimized`` flag, debug build will be
selected. Unless you are developing LLVM or llvmpy, it is recommended
that the flag is used to reduce build time and binary size.
**Note**: Use prefix to select the installation path. It is recommended
to separate your custom build from the default system package. Please
replace ``LLVM_INSTALL_PATH`` with your own path.
3. Run ``REQUIRES_RTTI=1 make install`` to build and install.
**Note**: With LLVM 3.2, the default build configuration has C++ RTTI
disabled. However, llvmpy requires RTTI.
**Note**: Use ``make -j2 install`` to enable concurrent build.
Replace ``2`` with the actual number of processor you have.
4. Get llvm-py and install it::
$ git clone git@github.com:llvmpy/llvmpy.git
$ cd llvmpy
$ LLVM_CONFIG_PATH=LLVM_INSTALL_PATH/bin/llvm-config python setup.py install
Run the tests::
$ python -c "import llvm; llvm.test()"
5. See documentation at 'http://www.llvmpy.org' and examples
under 'test'.
Common Build Problems
---------------------
1. If llvmpy cannot be imported due to "undefined symbol:
_ZTIN4llvm24PassRegistrationListenerE", it is because RTTI is not enabled
when building LLVM. "_ZTIN4llvm24PassRegistrationListenerE" is the typeinfo
of PassRegistrationListener class.
LICENSE
-------
llvmpy is distributed under the new BSD license, which is similar to the LLVM
license itself.
See the file called LICENSE for the full license text.

144
README_LLVM_CBUILDER.md
View file

@ -1,144 +0,0 @@
LLVM CBuilder
=============
A few short examples:
(TODO: add more later)
```python
from llvm.core import *
from llvm_cbuilder import *
import llvm_cbuilder.shortnames as C
```
```python
class Square(CDefinition):
_name_ = 'square'
_retty_ = C.double
_argtys_ = [ ('x', C.double) ]
def body(self, x):
y = x * x
self.ret(y)
```
```python
m = Module.new('my_module')
llvm_square = Square()(m)
print(m)
```
```
; ModuleID = 'my_module'
define double @square(double %x) {
decl:
%0 = fmul double %x, %x
ret double %0
}
```
```python
class IsPrime(CDefinition):
_name_ = 'isprime'
_retty_ = C.int
_argtys_ = [('x', C.int)]
def body(self, x):
false = zero = self.constant(C.int, 0)
true = one = self.constant(C.int, 1)
two = self.constant(C.int, 2)
with self.ifelse( x <= two ) as ifelse:
with ifelse.then():
self.ret(true)
with self.ifelse( (x % two) == zero ) as ifelse:
with ifelse.then():
self.ret(false)
idx = self.var(C.int, 3, name='idx')
with self.loop() as loop:
with loop.condition() as setcond:
setcond( idx < x )
with loop.body():
with self.ifelse( (x % idx ) == zero ) as ifelse:
with ifelse.then():
self.ret(false)
idx += two
self.ret(true)
```
```
define i32 @isprime(i32 %x) {
decl:
%0 = icmp sle i32 %x, 2
br i1 %0, label %if.then, label %if.end
if.then: ; preds = %loop.body, %loop.cond, %if.end, %decl
%merge = phi i32 [ 1, %decl ], [ 0, %if.end ], [ 1, %loop.cond ], [ 0, %loop.body ]
ret i32 %merge
if.end: ; preds = %decl
%1 = srem i32 %x, 2
%2 = icmp eq i32 %1, 0
br i1 %2, label %if.then, label %if.end4
if.end4: ; preds = %if.end
br label %loop.cond
loop.cond: ; preds = %if.end7, %if.end4
%idx.0 = phi i32 [ 3, %if.end4 ], [ %6, %if.end7 ]
%3 = icmp slt i32 %idx.0, %x
br i1 %3, label %loop.body, label %if.then
loop.body: ; preds = %loop.cond
%4 = srem i32 %x, %idx.0
%5 = icmp eq i32 %4, 0
br i1 %5, label %if.then, label %if.end7
if.end7: ; preds = %loop.body
%6 = add i32 %idx.0, 2
br label %loop.cond
}
; ModuleID = 'my_module'
define i32 @isprime(i32 %x) {
decl:
%0 = icmp sle i32 %x, 2
br i1 %0, label %if.then, label %if.end
if.then: ; preds = %loop.body, %loop.cond, %if.end, %decl
%merge = phi i32 [ 1, %decl ], [ 0, %if.end ], [ 1, %loop.cond ], [ 0, %loop.body ]
ret i32 %merge
if.end: ; preds = %decl
%1 = srem i32 %x, 2
%2 = icmp eq i32 %1, 0
br i1 %2, label %if.then, label %if.end4
if.end4: ; preds = %if.end
br label %loop.cond
loop.cond: ; preds = %if.end7, %if.end4
%idx.0 = phi i32 [ 3, %if.end4 ], [ %6, %if.end7 ]
%3 = icmp slt i32 %idx.0, %x
br i1 %3, label %loop.body, label %if.then
loop.body: ; preds = %loop.cond
%4 = srem i32 %x, %idx.0
%5 = icmp eq i32 %4, 0
br i1 %5, label %if.then, label %if.end7
if.end7: ; preds = %loop.body
%6 = add i32 %idx.0, 2
br label %loop.cond
}
```

6
buildscripts/condarecipe/bld.bat
View file

@ -1,6 +0,0 @@
set LLVMPY_DYNLINK=0
set INCLUDE=%LIBRARY_INC%
set LIBPATH=%LIBRARY_LIB%
set LIB=%LIBRARY_LIB%
%PYTHON% setup.py install
if errorlevel 1 exit 1

13
buildscripts/condarecipe/build.sh
View file

@ -1,13 +0,0 @@
#!/bin/bash
if [[ (`uname` == Linux) && (`uname -m` != armv6l) ]]
then
export CC=gcc
#gcc44
export CXX=g++
#g++44
fi
export LLVMPY_DYNLINK=$DISTRO_BUILD
$PYTHON setup.py install

28
buildscripts/condarecipe/meta.yaml
View file

@ -1,28 +0,0 @@
package:
name: llvmpy
version: 99.9.9
source:
git_url: git@github.com:llvmpy/llvmpy.git
# git_tag: 0.12.0
requirements:
build:
- llvm
- python
#- chrpath [linux]
run:
- llvm [unix]
- python
test:
imports:
- llvm
- llvmpy
- llvmpy._api
- llvmpy._capsule
- llpython
- llvm_array
- llvm_cbuilder

22
buildscripts/condarecipe/run_test.py
View file

@ -1,22 +0,0 @@
import sys
import platform
import llvm
from llvm.core import Module
from llvm.ee import EngineBuilder
from llvm.utils import check_intrinsics
m = Module.new('fjoidajfa')
eb = EngineBuilder.new(m)
target = eb.select_target()
print('target.triple=%r' % target.triple)
if sys.platform == 'darwin':
s = {'64bit': 'x86_64', '32bit': 'x86'}[platform.architecture()[0]]
assert target.triple.startswith(s + '-apple-darwin')
assert llvm.test(verbosity=2, run_isolated=False) == 0
#check_intrinsics.main()
print('llvm.__version__: %s' % llvm.__version__)
#assert llvm.__version__ == '0.12.0'

157
docs/Makefile
View file

@ -1,157 +0,0 @@
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
SRCDIR = source
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) $(SRCDIR)
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " html to make standalone HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " singlehtml to make a single large HTML file"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " devhelp to make HTML files and a Devhelp project"
@echo " epub to make an epub"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " latexpdf to make LaTeX files and run them through pdflatex"
@echo " text to make text files"
@echo " man to make manual pages"
@echo " texinfo to make Texinfo files"
@echo " info to make Texinfo files and run them through makeinfo"
@echo " gettext to make PO message catalogs"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
clean:
-rm -rf $(BUILDDIR)/*
html:
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
dirhtml:
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
singlehtml:
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
@echo
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
pickle:
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."
json:
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."
htmlhelp:
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file in $(BUILDDIR)/htmlhelp."
qthelp:
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
@echo
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/llvmpy.qhcp"
@echo "To view the help file:"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/llvmpy.qhc"
devhelp:
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
@echo
@echo "Build finished."
@echo "To view the help file:"
@echo "# mkdir -p $$HOME/.local/share/devhelp/llvmpy"
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/llvmpy"
@echo "# devhelp"
epub:
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
@echo
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
latex:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
@echo "Run \`make' in that directory to run these through (pdf)latex" \
"(use \`make latexpdf' here to do that automatically)."
latexpdf:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through pdflatex..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
text:
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
@echo
@echo "Build finished. The text files are in $(BUILDDIR)/text."
man:
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
@echo
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
texinfo:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
@echo "Run \`make' in that directory to run these through makeinfo" \
"(use \`make info' here to do that automatically)."
info:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo "Running Texinfo files through makeinfo..."
make -C $(BUILDDIR)/texinfo info
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
gettext:
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
@echo
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
changes:
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
@echo
@echo "The overview file is in $(BUILDDIR)/changes."
linkcheck:
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in $(BUILDDIR)/linkcheck/output.txt."
doctest:
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."
gh-pages: clean html
python gh-pages.py

138
docs/gh-pages.py
View file

@ -1,138 +0,0 @@
#!/usr/bin/env python
"""Script to commit the doc build outputs into the github-pages repo.
Use:
gh-pages.py [tag]
If no tag is given, the current output of 'git describe' is used. If given,
that is how the resulting directory will be named.
In practice, you should use either actual clean tags from a current build or
something like 'current' as a stable URL for the most current version of the """
#-----------------------------------------------------------------------------
# Imports
#-----------------------------------------------------------------------------
import os
import re
import shutil
import sys
from os import chdir as cd
from os.path import join as pjoin
from subprocess import Popen, PIPE, CalledProcessError, check_call
#-----------------------------------------------------------------------------
# Globals
#-----------------------------------------------------------------------------
pages_dir = 'gh-pages'
html_dir = '_build/html'
pdf_dir = '_build/latex'
pages_repo = 'https://github.com/llvmpy/llvmpy-doc.git'
#-----------------------------------------------------------------------------
# Functions
#-----------------------------------------------------------------------------
def sh(cmd):
"""Execute command in a subshell, return status code."""
return check_call(cmd, shell=True)
def sh2(cmd):
"""Execute command in a subshell, return stdout.
Stderr is unbuffered from the subshell.x"""
p = Popen(cmd, stdout=PIPE, shell=True)
out = p.communicate()[0]
retcode = p.returncode
if retcode:
raise CalledProcessError(retcode, cmd)
else:
return out.rstrip()
def sh3(cmd):
"""Execute command in a subshell, return stdout, stderr
If anything appears in stderr, print it out to sys.stderr"""
p = Popen(cmd, stdout=PIPE, stderr=PIPE, shell=True)
out, err = p.communicate()
retcode = p.returncode
if retcode:
raise CalledProcessError(retcode, cmd)
else:
return out.rstrip(), err.rstrip()
def init_repo(path):
"""clone the gh-pages repo if we haven't already."""
sh("git clone %s %s"%(pages_repo, path))
here = os.getcwdu()
cd(path)
sh('git checkout gh-pages')
cd(here)
#-----------------------------------------------------------------------------
# Script starts
#-----------------------------------------------------------------------------
if __name__ == '__main__':
# The tag can be given as a positional argument
try:
tag = sys.argv[1]
except IndexError:
try:
tag = sh2('git describe --exact-match')
except CalledProcessError:
tag = "dev" # Fallback
startdir = os.getcwdu()
if not os.path.exists(pages_dir):
# init the repo
init_repo(pages_dir)
else:
# ensure up-to-date before operating
cd(pages_dir)
sh('git checkout gh-pages')
sh('git pull')
cd(startdir)
dest = pjoin(pages_dir, tag)
# don't `make html` here, because gh-pages already depends on html in Makefile
# sh('make html')
if tag != 'dev':
# only build pdf for non-dev targets
#sh2('make pdf')
pass
# This is pretty unforgiving: we unconditionally nuke the destination
# directory, and then copy the html tree in there
shutil.rmtree(dest, ignore_errors=True)
shutil.copytree(html_dir, dest)
if tag != 'dev':
#shutil.copy(pjoin(pdf_dir, 'ipython.pdf'), pjoin(dest, 'ipython.pdf'))
pass
try:
cd(pages_dir)
status = sh2('git status | head -1')
branch = re.match('\# On branch (.*)$', status).group(1)
if branch != 'gh-pages':
e = 'On %r, git branch is %r, MUST be "gh-pages"' % (pages_dir,
branch)
raise RuntimeError(e)
sh('git add -A %s' % tag)
sh('git commit -m"Updated doc release: %s"' % tag)
print
print 'Most recent 3 commits:'
sys.stdout.flush()
sh('git --no-pager log --oneline HEAD~3..')
finally:
cd(startdir)
print
print 'Now verify the build in: %r' % dest
print "If everything looks good, 'git push'"

259
docs/source/conf.py
View file

@ -1,259 +0,0 @@
# -*- coding: utf-8 -*-
#
# llvmpy documentation build configuration file, created by
# sphinx-quickstart on Wed Aug 8 17:33:58 2012.
#
# This file is execfile()d with the current directory set to its containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
import sys, os, glob
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#sys.path.insert(0, os.path.abspath('../..'))
# Support sphinx.ext.autodoc to extract docstrings from modules without installing
# complete package.
# The python modules depend on _core, so we must build entire package first though.
built_lib = glob.glob('../../build/lib.*-%d.%d/' % sys.version_info[:2])
if not built_lib:
sys.stderr.write("WARNING: To build complete documentation you must build "
"package first\n")
else:
# lib dir has platform suffix
sys.path.insert(0, os.path.abspath(built_lib[0]))
# -- General configuration -----------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be extensions
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = ['sphinx.ext.mathjax', 'sphinx.ext.autodoc']
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'llvmpy'
copyright = u'2013, Mahadevan R (2008-2010), Continuum Analytics (2012-2013)'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
try:
import llvm
version_strs = llvm.__version__.split('.')
# The short X.Y version.
version = '.'.join(version_strs[:2])
# The full version, including alpha/beta/rc tags.
release = '%s.%s' % (version, '-'.join(version_strs[2].split('-')[:2]))
except ImportError:
version = 'unknown-version'
release = 'unknown-release'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['_build']
# The reST default role (used for this markup: `text`) to use for all documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
#add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
#show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
# -- Options for HTML output ---------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'default'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
#html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
#html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
#html_logo = None
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
#html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
#html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
#html_domain_indices = True
# If false, no index is generated.
#html_use_index = True
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
#html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
#html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = 'llvmpydoc'
# -- Options for LaTeX output --------------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [
('index', 'llvmpy.tex', u'llvmpy Documentation',
u'Mahadevan R (2008-2010), Continuum Analytics (2012)', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# If true, show page references after internal links.
#latex_show_pagerefs = False
# If true, show URL addresses after external links.
#latex_show_urls = False
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_domain_indices = True
# -- Options for manual page output --------------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
('index', 'llvmpy', u'llvmpy Documentation',
[u'Mahadevan R (2008-2010), Continuum Analytics (2012)'], 1)
]
# If true, show URL addresses after external links.
#man_show_urls = False
# -- Options for Texinfo output ------------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
('index', 'llvmpy', u'llvmpy Documentation',
u'Mahadevan R (2008-2010), Continuum Analytics (2012)', 'llvmpy', 'One line description of project.',
'Miscellaneous'),
]
# Documents to append as an appendix to all manuals.
#texinfo_appendices = []
# If false, no module index is generated.
#texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
#texinfo_show_urls = 'footnote'

147
docs/source/doc/comparision.rst
View file

@ -1,147 +0,0 @@
+--------------------------------+
| layout: page |
+--------------------------------+
| title: Comparison Operations |
+--------------------------------+
Integer Comparision # {#icmp}
=============================
Predicates for use with ``icmp`` instruction are listed below. All of
these are integer constants defined in the ``llvm.core`` module.
``ICMP_EQ``
-----------
Equality
``ICMP_NE``
-----------
Inequality
``ICMP_UGT``
------------
Unsigned greater than
``ICMP_UGE``
------------
Unsigned greater than or equal
``ICMP_ULT``
------------
Unsigned less than
``ICMP_ULE``
------------
Unsigned less than or equal
``ICMP_SGT``
------------
Signed greater than
``ICMP_SGE``
------------
Signed greater than or equal
``ICMP_SLT``
------------
Signed less than
``ICMP_SLE``
------------
Signed less than or equal
Float Comparision # {#fcmp}
===========================
Predicates for use with ``fcmp`` instruction are listed below. All of
these are integer constants defined in the ``llvm.core`` module.
``FCMP_FALSE``
--------------
Always false
``FCMP_OEQ``
------------
True if ordered and equal
``FCMP_OGT``
------------
True if ordered and greater than
``FCMP_OGE``
------------
True if ordered and greater than or equal
``FCMP_OLT``
------------
True if ordered and less than
``FCMP_OLE``
------------
True if ordered and less than or equal
``FCMP_ONE``
------------
True if ordered and operands are unequal
``FCMP_ORD``
------------
True if ordered (no NaNs)
``FCMP_UNO``
------------
True if unordered: ``isnan(X) | isnan(Y)``
``FCMP_UEQ``
------------
True if unordered or equal
``FCMP_UGT``
------------
True if unordered or greater than
``FCMP_UGE``
------------
True if unordered, greater than or equal
``FCMP_ULT``
------------
True if unordered, or less than
``FCMP_ULE``
------------
True if unordered, less than or equal
``FCMP_UNE``
------------
True if unordered or not equal
``FCMP_TRUE``
-------------
Always true

9
docs/source/doc/core.rst
View file

@ -1,9 +0,0 @@
********************************
llvm.core
********************************
.. toctree::
:titlesonly:
:glob:
llvm.core.*

12
docs/source/doc/examples.rst
View file

@ -1,12 +0,0 @@
********************************
Examples and LLVM Tutorials
********************************
.. toctree::
:maxdepth: 1
firstexample.rst
examples/index.rst
kaleidoscope/index.rst

36
docs/source/doc/examples/JITTutorial1.rst
View file

@ -1,36 +0,0 @@
A First Function
==================
.. code-block:: python
#!/usr/bin/env python
from llvm.core import *
#create a module
module = Module.new("tut1")
#create a function type taking 3 32-bit integers, return a 32-bit integer
ty_int = Type.int(32)
func_type = Type.function(ty_int, (ty_int,)*3)
#create a function of that type
mul_add = Function.new (module, func_type, "mul_add")
mul_add.calling_convention = CC_C
x = mul_add.args[0]; x.name = "x"
y = mul_add.args[1]; y.name = "y"
z = mul_add.args[2]; z.name = "z"
#implement the function
#new block
blk = mul_add.append_basic_block("entry")
#IR builder
bldr = Builder.new(blk)
tmp_1 = bldr.mul(x, y, "tmp_1")
tmp_2 = bldr.add(tmp_1, z, "tmp_2")
bldr.ret(tmp_2)
print module

55
docs/source/doc/examples/JITTutorial2.rst
View file

@ -1,55 +0,0 @@
A More Complicated Function
===========================
.. code-block:: python
#!/usr/bin/env python
from llvm.core import *
#create a module
module = Module.new("tut2")
#create a function type taking 2 integers, return a 32-bit integer
ty_int = Type.int(32)
func_type = Type.function(ty_int, (ty_int, ty_int))
#create a function of that type
gcd = Function.new(module, func_type, "gcd")
#name function args
x = gcd.args[0]; x.name = "x"
y = gcd.args[1]; y.name = "y"
#implement the function
#blocks...
entry = gcd.append_basic_block("entry")
ret = gcd.append_basic_block("return")
cond_false = gcd.append_basic_block("cond_false")
cond_true = gcd.append_basic_block("cond_true")
cond_false_2 = gcd.append_basic_block("cond_false_2")
#create a llvm::IRBuilder
bldr = Builder.new(entry)
x_eq_y = bldr.icmp(IPRED_EQ, x, y, "tmp")
bldr.cbranch(x_eq_y, ret, cond_false)
bldr.position_at_end (ret)
bldr.ret(x)
bldr.position_at_end(cond_false)
x_lt_y = bldr.icmp(IPRED_ULT, x, y, "tmp")
bldr.cbranch(x_lt_y, cond_true, cond_false_2)
bldr.position_at_end(cond_true)
y_sub_x = bldr.sub(y, x, "tmp")
recur_1 = bldr.call(gcd, (x, y_sub_x,), "tmp")
bldr.ret(recur_1)
bldr.position_at_end(cond_false_2)
x_sub_y = bldr.sub(x, y, "x_sub_y")
recur_2 = bldr.call(gcd, (x_sub_y, y,), "tmp")
bldr.ret(recur_2)
print module

10
docs/source/doc/examples/index.rst
View file

@ -1,10 +0,0 @@
LLVM Tutorials
==============
The following JIT tutorials were contributed by Sebastien Binet.
.. toctree::
:titlesonly:
JITTutorial1.rst
JITTutorial2.rst

125
docs/source/doc/firstexample.rst
View file

@ -1,125 +0,0 @@
Examples
========
A Simple Function
-----------------
Let's create a (LLVM) module containing a single function, corresponding
to the ``C`` function:
.. code-block:: c
int sum(int a, int b)
{
return a + b;
}
Here's how it looks in llvmpy:
.. code-block:: python
#!/usr/bin/env python
# Import the llvmpy modules.
from llvm import *
from llvm.core import *
# Create an (empty) module.
my_module = Module.new('my_module')
# All the types involved here are "int"s. This type is represented
# by an object of the llvm.core.Type class:
ty_int = Type.int() # by default 32 bits
# We need to represent the class of functions that accept two integers
# and return an integer. This is represented by an object of the
# function type (llvm.core.FunctionType):
ty_func = Type.function(ty_int, [ty_int, ty_int])
# Now we need a function named 'sum' of this type. Functions are not
# free-standing (in llvmpy); it needs to be contained in a module.
f_sum = my_module.add_function(ty_func, "sum")
# Let's name the function arguments as 'a' and 'b'.
f_sum.args[0].name = "a"
f_sum.args[1].name = "b"
# Our function needs a "basic block" -- a set of instructions that
# end with a terminator (like return, branch etc.). By convention
# the first block is called "entry".
bb = f_sum.append_basic_block("entry")
# Let's add instructions into the block. For this, we need an
# instruction builder:
builder = Builder.new(bb)
# OK, now for the instructions themselves. We'll create an add
# instruction that returns the sum as a value, which we'll use
# a ret instruction to return.
tmp = builder.add(f_sum.args[0], f_sum.args[1], "tmp")
builder.ret(tmp)
# We've completed the definition now! Let's see the LLVM assembly
# language representation of what we've created:
print my_module
Here is the output:
.. code-block:: llvm
; ModuleID = 'my_module'
define i32 @sum(i32 %a, i32 %b) {
entry:
%tmp = add i32 %a, %b ; <i32> [#uses=1]
ret i32 %tmp
}
Adding JIT Compilation
----------------------
Let's compile this function in-memory and run it.
.. code-block:: python
#!/usr/bin/env python
# Import the llvmpy modules.
from llvm import *
from llvm.core import *
from llvm.ee import * # new import: ee = Execution Engine
#Create a module, as in the previous example.
my_module = Module.new('my_module')
ty_int = Type.int() # by default 32 bits
ty_func = Type.function(ty_int, [ty_int, ty_int])
f_sum = my_module.add_function(ty_func, "sum")
f_sum.args[0].name = "a"
f_sum.args[1].name = "b"
bb = f_sum.append_basic_block("entry")
builder = Builder.new(bb)
tmp = builder.add(f_sum.args[0], f_sum.args[1], "tmp")
builder.ret(tmp)
# Create an execution engine object. This will create a JIT compiler
# on platforms that support it, or an interpreter otherwise.
ee = ExecutionEngine.new(my_module)
# The arguments needs to be passed as "GenericValue" objects.
arg1 = GenericValue.int(ty_int, 100)
arg2 = GenericValue.int(ty_int, 42)
# Now let's compile and run!
retval = ee.run_function(f_sum, [arg1, arg2])
# The return value is also GenericValue. Let's print it.
print "returned", retval.as_int()
And here's the output:
::
returned 142

149
docs/source/doc/functions.rst
View file

@ -1,149 +0,0 @@
+--------------------+
| layout: page |
+--------------------+
| title: Functions |
+--------------------+
Functions are represented by
`llvm.core.Function <llvm.core.Function.html>`_ objects. They are
contained within modules, and can be created either with the method
``module_obj.add_function`` or the static constructor ``Function.new``.
References to functions already present in a module can be retrieved via
``module.get_function_named`` or by the static constructor method
``Function.get``. All functions in a module can be enumerated by
iterating over ``module_obj.functions``.
.. code-block:: python
# create a type, representing functions that take
an integer and return # a floating point value. ft = Type.function(
Type.float(), [ Type.int() ] )
# create a function of this type
f1 = module_obj.add_function(ft, "func1")
# or equivalently, like this:
f2 = Function.new(module_obj, ft, "func2")
# get a reference to an existing function
f3 = module_obj.get_function_named("func3")
# or like this:
f4 = Function.get(module_obj, "func4")
# list all function names in a module
for f in module_obj.functions: print f.name
Intrinsic
=========
References to intrinsic functions can be got via the static constructor
``intrinsic``. This returns a ``Function`` object, calling which is
equivalent to invoking the intrinsic. The ``intrinsic`` method has to be
called with a module object, an intrinsic ID (which is a numeric
constant) and a list of the types of arguments (which LLVM uses to
resolve overloaded intrinsic functions).
.. code-block:: python
# get a reference to the llvm.bswap intrinsic
bswap = Function.intrinsic(mod, INTR_BSWAP, [Type.int()])
# call it
builder.call(bswap, [value])
Here, the constant ``INTR_BSWAP``, available from ``llvm.core``,
represents the LLVM intrinsic
`llvm.bswap <http://www.llvm.org/docs/LangRef.html#int_bswap>`_. The
``[Type.int()]`` selects the version of ``llvm.bswap`` that has a single
32-bit integer argument. The list of intrinsic IDs defined as integer
constants in ``llvm.core``. These are:
{% include intrinsics.csv %}
There are also target-specific intrinsics (which correspond to that
target's CPU instructions) available, but are omitted here for brevity.
Full list can be seen from
[*intrinsic\_ids.py](https://github.com/numba/llvmpy/blob/master/llvm/*\ intrinsic\_ids.py).
See the `LLVM Language
Reference <http://www.llvm.org/docs/LangRef.html>`_ for more information
on the intrinsics, and the
`test <https://github.com/numba/llvmpy/blob/master/test/intrinsic.py>`_
directory in the source distribution for more examples. The intrinsic ID
can be retrieved from a function object with the read-only property
``intrinsic_id``.
**Auto-generation of Intrinsic IDs**
A script (tool/intrgen.py in source tree) generates the intrinsic
IDs automatically. This is necessary when compiling llvmpy with a
different version of LLVM.
Calling Convention # {#callconv}
================================
The function's calling convention can be set using the
``calling_convention`` property. The following (integer) constants
defined in ``llvm.core`` can be used as values:
Value \| Equivalent LLVM Assembly Keyword \|
------\|----------------------------------\| ``CC_C`` \| ``ccc`` \|
``CC_FASTCALL`` \| ``fastcc`` \| ``CC_COLDCALL`` \| ``coldcc`` \|
``CC_X86_STDCALL`` \| ``x86_stdcallcc`` \| ``CC_X86_FASTCALL`` \|
``x86_fastcallcc`` \|
See the `LLVM docs <http://www.llvm.org/docs/LangRef.html#callingconv>`_
for more information on each. Backend-specific numbered conventions can
be directly passed as integers.
An arbitrary string identifying which garbage collector to use can be
set or got with the property ``collector``.
The value objects corresponding to the arguments of a function can be
got using the read-only property ``args``. These can be iterated over,
and also be indexed via integers. An example:
.. code-block:: python
# list all argument names and types for arg in
fn.args: print arg.name, "of type", arg.type
# change the name of the first argument
fn.args[0].name = "objptr"
Basic blocks (see later) are contained within functions. When newly
created, a function has no basic blocks. They have to be added
explicitly, using the ``append_basic_block`` method, which adds a new,
empty basic block as the last one in the function. The first basic block
of the function can be retrieved using the ``get_entry_basic_block``
method. The existing basic blocks can be enumerated by iterating over
using the read-only property ``basic_blocks``. The number of basic
blocks can be got via ``basic_block_count`` method. Note that
``get_entry_basic_block`` is slightly faster than ``basic_blocks[0]``
and so is ``basic_block_count``, over ``len(f.basic_blocks)``.
.. code-block:: python
# add a basic block b1 =
fn.append_basic_block("entry")
# get the first one
b2 = fn.get_entry_basic_block() b2 = fn.basic_mdblocks[0] # slower
than previous method
# print names of all basic blocks
for b in fn.basic_blocks: print b.name
# get number of basic blocks
n = fn.basic_block_count n = len(fn.basic_blocks) # slower than
previous method

115
docs/source/doc/getting_started.rst
View file

@ -1,115 +0,0 @@
Introduction
============
`LLVM <http://www.llvm.org/>`_ (Low-Level Virtual Machine) provides
enough infrastructure to use it as the backend for your compiled, or
JIT-compiled language. It provides extensive optimization support, and
static and dynamic (JIT) backends for many platforms. See the website at
http://www.llvm.org/ to discover more.
Python bindings for LLVM provides a gentler learning curve for working
with the LLVM APIs. It should also be easier to create working
prototypes and experimental languages using this medium.
Together with `clang <http://clang.llvm.org/>`_ or
`llvm-gcc <http://llvm.org/releases/2.7/docs/CommandGuide/html/llvmgcc.html>`_
it also a provides a means to quickly instrument C and C++ sources. For e.g.,
llvm-gcc can be used to generate the LLVM assembly for a given C source file,
which can then be loaded and manipulated (adding profiling code to every function,
say) using a llvmpy based Python script.
License
-------
Both LLVM and llvmpy are distributed under (different) permissive open
source licenses. llvmpy uses the `new BSD
license <http://opensource.org/licenses/bsd-license.php>`_. More
information is available
`here <https://github.com/llvmpy/llvmpy/blob/master/LICENSE>`_.
Platforms
---------
llvmpy has been built/tested/reported to work on various GNU/Linux
flavours, BSD, Mac OS X; on i386 and amd64 architectures. Windows is not
supported, for a variety of reasons.
Versions
--------
llvmpy 0.11.2 uses LLVM 3.2 (or at least 3.1). It may not work with
previous versions.
llvmpy has been built and tested with Python 2.7 and 3.2. It should work with
earlier versions.
Installation
============
The Git repo of llvmpy is at https://github.com/llvmpy/llvmpy.git.
You'll need to build and install it before it can be used. At least the
following will be required for this:
- C and C++ compilers (gcc/g++)
- Python itself
- Python development files (headers and libraries)
- LLVM, either installed or built
On debian-based systems, the first three can be installed with the
command ``sudo apt-get install gcc g++ python python-dev``. Ensure that
your distro's repository has the appropriate version of LLVM!
It does not matter which compiler LLVM itself was built with (``g++``,
``llvm-g++`` or any other); llvmpy can be built with any compiler. It
has been tried only with gcc/g++ though.
llvm-config
-----------
In order to build llvmpy, it's build script needs to know from where it
can invoke the llvm helper program, ``llvm-config``. If you've installed
LLVM, then this will be available in your ``PATH``, and nothing further
needs to be done. If you've built LLVM yourself, or for any reason
``llvm-config`` is not in your ``PATH``, you'll need to pass the full
path of ``llvm-config`` to the build script.
You'll need to be 'root' to install llvmpy. Remember that your ``PATH``
is different from that of 'root', so even if ``llvm-config`` is in your
``PATH``, it may not be available when you do ``sudo``.
Steps
-----
1. Get and extract LLVM 3.2 source tarball from
`llvm.org <http://llvm.org/releases/download.html#3.2>`_. Then, ``cd`` into
the extracted directory.
2. Run ``./configure --enable-optimized --prefix=LLVM_INSTALL_PATH``.
**Note**: Without the ``--enable-optimized`` flag, debug build will be
selected. Unless you are developing LLVM or llvmpy, it is recommended
that the flag is used to reduce build time and binary size.
**Note**: Use prefix to select the installation path. It is recommended
to separate your custom build from the default system package. Please
replace ``LLVM_INSTALL_PATH`` with your own path.
3. Run ``REQUIRES_RTTI=1 make`` to build.
**Note**: With LLVM 3.2, the default build configuration has C++ RTTI
disabled. However, llvmpy requires RTTI.
4. Get llvm-py and install it::
$ git clone git@github.com:llvmpy/llvmpy.git
$ cd llvmpy
$ LLVM_CONFIG_PATH=LLVM_INSTALL_PATH/bin/llvm-config python setup.py install
Run the tests::
$ python -c "import llvm; llvm.test()"
5. See documentation at 'http://www.llvmpy.org' and examples
under 'test'.

312
docs/source/doc/kaleidoscope/PythonLangImpl1.rst
View file

@ -1,312 +0,0 @@
*************************************************
Chapter 1: Tutorial Introduction and the Lexer
*************************************************
Written by `Chris Lattner <mailto:sabre@nondot.org>`_ and `Max
Shawabkeh <http://max99x.com>`_
Introduction
============
Welcome to the "Implementing a language with LLVM" tutorial. This
tutorial runs through the implementation of a simple language, showing
how fun and easy it can be. This tutorial will get you up and started as
well as help to build a framework you can extend to other languages. The
code in this tutorial can also be used as a playground to hack on other
LLVM specific things.
It is useful to point out ahead of time that this tutorial is really
about teaching compiler techniques and LLVM specifically, *not* about
teaching modern and sane software engineering principles. In practice,
this means that we'll take a number of shortcuts to simplify the
exposition. If you dig in and use the code as a basis for future
projects, fixing its deficiencies shouldn't be hard.
We've tried to put this tutorial together in a way that makes chapters
easy to skip over if you are already familiar with or are uninterested
in the various pieces. The structure of the tutorial is:
- :ref:`Chapter 1 <language>`: **Introduction to the Kaleidoscope
language, and the definition of its Lexer** -- This shows where we
are going and the basic functionality that we want it to do. In order
to make this tutorial maximally understandable and hackable, we
choose to implement everything in Python instead of using lexer and
parser generators. LLVM obviously works just fine with such tools,
feel free to use one if you prefer.
- `Chapter 2 <PythonLangImpl2.html>`_: **Implementing a Parser and
AST** -- With the lexer in place, we can talk about parsing
techniques and basic AST construction. This tutorial describes
recursive descent parsing and operator precedence parsing. Nothing in
Chapters 1 or 2 is LLVM-specific, the code doesn't even import the
LLVM modules at this point. :)
- `Chapter 3 <PythonLangImpl3.html>`_: **Code generation to LLVM IR**
-- With the AST ready, we can show off how easy generation of LLVM IR
really is.
- `Chapter 4 <PythonLangImpl4.html>`_: **Adding JIT and Optimizer
support** -- Because a lot of people are interested in using LLVM as
a JIT, we'll dive right into it and show you the 3 lines it takes to
add JIT support. LLVM is also useful in many other ways, but this is
one simple and "sexy" way to shows off its power. :)
- `Chapter 5 <PythonLangImpl5.html>`_: **Extending the Language:
Control Flow** -- With the language up and running, we show how to
extend it with control flow operations (if/then/else and a 'for'
loop). This gives us a chance to talk about simple SSA construction
and control flow.
- `Chapter 6 <PythonLangImpl6.html>`_: **Extending the Language:
User-defined Operators** -- This is a silly but fun chapter that
talks about extending the language to let the user program define
their own arbitrary unary and binary operators (with assignable
precedence!). This lets us build a significant piece of the
"language" as library routines.
- `Chapter 7 <PythonLangImpl7.html>`_: **Extending the Language:
Mutable Variables** -- This chapter talks about adding user-defined
local variables along with an assignment operator. The interesting
part about this is how easy and trivial it is to construct SSA form
in LLVM: no, LLVM does *not* require your front-end to construct SSA
form!
- `Chapter 8 <PythonLangImpl8.html>`_: **Conclusion and other useful
LLVM tidbits** -- This chapter wraps up the series by talking about
potential ways to extend the language, but also includes a bunch of
pointers to info about "special topics" like adding garbage
collection support, exceptions, debugging, support for "spaghetti
stacks", and a bunch of other tips and tricks.
By the end of the tutorial, we'll have written a bit less than 540 lines
of non-comment, non-blank, lines of code. With this small amount of
code, we'll have built up a very reasonable compiler for a non-trivial
language including a hand-written lexer, parser, AST, as well as code
generation support with a JIT compiler. While other systems may have
interesting "hello world" tutorials, I think the breadth of this
tutorial is a great testament to the strengths of LLVM and why you
should consider it if you're interested in language or compiler design.
A note about this tutorial: we expect you to extend the language and
play with it on your own. Take the code and go crazy hacking away at it,
compilers don't need to be scary creatures - it can be a lot of fun to
play with languages!
--------------
.. _language:
The Basic Language
================================
This tutorial will be illustrated with a toy language that we'll call
"`Kaleidoscope <http://en.wikipedia.org/wiki/Kaleidoscope>`_\ " (derived
from "meaning beautiful, form, and view"). Kaleidoscope is a procedural
language that allows you to define functions, use conditionals, math,
etc. Over the course of the tutorial, we'll extend Kaleidoscope to
support the if/then/else construct, a for loop, user defined operators,
JIT compilation with a simple command line interface, etc.
Because we want to keep things simple, the only datatype in Kaleidoscope
is a 64-bit floating point type. As such, all values are implicitly
double precision and the language doesn't require type declarations.
This gives the language a very nice and simple syntax. For example, the
following simple example computes `Fibonacci
numbers <http://en.wikipedia.org/wiki/Fibonacci_number>`_:
.. code-block:: text
# Compute the x'th fibonacci number.
def fib(x)
if x < 3
return 1
else
return fib(x-1)+fib(x-2)
# This expression will compute the 40th number.
fib(40)
We also allow Kaleidoscope to call into standard library functions (the
LLVM JIT makes this completely trivial). This means that you can use the
'extern' keyword to define a function before you use it (this is also
useful for mutually recursive functions). For example:
.. code-block:: python
extern sin(arg);
extern cos(arg);
extern atan2(arg1 arg2);
atan2(sin(0.4), cos(42))
A more interesting example is included in Chapter 6 where we write a
little Kaleidoscope application that
`displays <PythonLangImpl6.html#example>`_ a Mandelbrot Set at various
levels of magnification.
Lets dive into the implementation of this language!
--------------
The Lexer
====================
When it comes to implementing a language, the first thing needed is the
ability to process a text file and recognize what it says. The
traditional way to do this is to use a
`lexer <http://en.wikipedia.org/wiki/Lexical_analysis>`_ (aka
'scanner') to break the input up into "tokens". Each token returned by
the lexer includes a token type and potentially some metadata (e.g. the
numeric value of a number). First, we define the possibilities:
.. code-block:: python
# The lexer yields one of these types for each token.
class EOFToken(object): pass
class DefToken(object): pass
class ExternToken(object): pass
class IdentifierToken(object):
def __init__(self, name):
self.name = name
class NumberToken(object):
def __init__(self, value):
self.value = value
class CharacterToken(object):
def __init__(self, char):
self.char = char
def __eq__(self, other):
return isinstance(other, CharacterToken) and self.char == other.char
def __ne__(self, other):
return not self == other
Each token yielded by our lexer will be of one of the above types. For
simple tokens that are always the same, like the "def" keyword, the
lexer will yield ``DefToken()``. Identifiers, numbers and characters,
on the other hand, have extra data, so when the lexer encounteres the
number 123.45, it will emit it as ``NumberToken(123.45)``. An identifier
``foo`` will be emitted as ``IdentifierToken('foo')``. And finally, an
unknown character like '+' will be returned as ``CharacterToken('+')``.
You may notice that we overload the equality and inequality operators
for the characters; this will later simplify character comparisons in
the parser code.
The actual implementation of the lexer is a single function called
``Tokenize``, which takes a string and
`yields <http://docs.python.org/reference/simple_stmts.html#the-yield-statement>`_
tokens. For simplicity, we will use `regular
expressions <http://docs.python.org/library/re.html>`_ to parse out the
tokens. This is terribly inefficient, but perfectly sufficient for our
needs.
First, we define the regular expressions for our tokens. Numbers and
strings of digits, optionally followed by a period and another string of
digits. Identifiers (and keywords) are alphanumeric string starting with
a letter and comments are anything between a hash (``#``) and the end of
the line.
.. code-block:: python
import re
...
# Regular expressions that tokens and comments of our language.
REGEX_NUMBER = re.compile('[0-9]+(?:.[0-9]+)?')
REGEX_IDENTIFIER = re.compile('[a-zA-Z][a-zA-Z0-9]*')
REGEX_COMMENT = re.compile('#.*')
Next, let's start defining the ``Tokenize`` function itself. The first
thing we need to do is set up a loop that scans the string, while
ignoring whitespace between tokens:
.. code-block:: python
def Tokenize(string):
while string: # Skip whitespace.
if string[0].isspace():
string = string[1:]
continue
Next we want to find out what the next token is. For this we run the
regexes we defined above on the remainder of the string. To simplify the
rest of the code, we run all three regexes each time. As mentioned
above, inefficiencies are ignored for the purpose of this tutorial:
.. code-block:: python
# Run regexes.
comment_match = REGEX_COMMENT.match(string)
number_match = REGEX_NUMBER.match(string)
identifier_match = REGEX_IDENTIFIER.match(string)
Now we check if any of the regexes matched. For comments, we simply
ignore the captured match:
.. code-block:: python
# Check if any of the regexes matched and yield
# the appropriate result.
if comment_match:
comment = comment_match.group(0)
string = string[len(comment):]
# For numbers, we yield the captured match, converted to a float and
# tagged with the appropriate token type:
elif number_match:
number = number_match.group(0)
yield NumberToken(float(number))
string = string[len(number):]
# The identifier case is a little more complex. We have to check for
# keywords to decide whether we have captured an identifier or a keyword:
elif identifier_match:
identifier = identifier_match.group(0)
# Check if we matched a keyword.
if identifier == 'def':
yield DefToken()
elif identifier == 'extern':
yield ExternToken()
else:
yield IdentifierToken(identifier)
string = string[len(identifier):]
# Finally, if we haven't recognized a comment, a number of an identifier,
# we yield the current character as an "unknown character" token. This is
# used, for example, for operators like ``+`` or ``*``:
else: # Yield the unknown character.
yield CharacterToken(string[0])
string = string[1:]
# Once we're done with the loop, we return a final end-of-file token:
yield EOFToken()

1079
docs/source/doc/kaleidoscope/PythonLangImpl2.rst
View file

File diff suppressed because it is too large Load diff

1087
docs/source/doc/kaleidoscope/PythonLangImpl3.rst
View file

File diff suppressed because it is too large Load diff

936
docs/source/doc/kaleidoscope/PythonLangImpl4.rst
View file

@ -1,936 +0,0 @@
*************************************************
Chapter 4: Adding JIT and Optimizer Support
*************************************************
Written by `Chris Lattner <mailto:sabre@nondot.org>`_ and `Max
Shawabkeh <http://max99x.com>`_
Introduction
=======================
Welcome to Chapter 4 of the `Implementing a language with
LLVM <http://www.llvm.org/docs/tutorial/index.html>`_ tutorial. Chapters
1-3 described the implementation of a simple language and added support
for generating LLVM IR. This chapter describes two new techniques:
adding optimizer support to your language, and adding JIT compiler
support. These additions will demonstrate how to get nice, efficient
code for the Kaleidoscope language.
--------------
Trivial Constant Folding
==============================================
Our demonstration for Chapter 3 is elegant and easy to extend.
Unfortunately, it does not produce wonderful code. The LLVM Builder,
however, does give us obvious optimizations when compiling simple code:
.. code-block:: bash
ready> def test(x) 1+2+x
Read function definition:
define double @test(double %x) {
entry:
%addtmp = fadd double 3.000000e+00, %x
ret double %addtmp
}
This code is not a literal transcription of the AST built by parsing the
input. That would be:
.. code-block:: bash
ready> def test(x) 1+2+x
Read function definition:
define double @test(double %x) {
entry:
%addtmp = fadd double 2.000000e+00, 1.000000e+00
%addtmp1 = fadd double %addtmp, %x
ret double %addtmp1
}
Constant folding, as seen above, in particular, is a very common and
very important optimization: so much so that many language implementors
implement constant folding support in their AST representation.
With LLVM, you don't need this support in the AST. Since all calls to
build LLVM IR go through the LLVM IR builder, the builder itself checked
to see if there was a constant folding opportunity when you call it. If
so, it just does the constant fold and return the constant instead of
creating an instruction.
Well, that was easy :). In practice, we recommend always using
``llvm.core.Builder`` when generating code like this. It has no
"syntactic overhead" for its use (you don't have to uglify your compiler
with constant checks everywhere) and it can dramatically reduce the
amount of LLVM IR that is generated in some cases (particular for
languages with a macro preprocessor or that use a lot of constants).
On the other hand, the ``Builder`` is limited by the fact that it does
all of its analysis inline with the code as it is built. If you take a
slightly more complex example:
.. code-block:: bash
ready> def test(x) (1+2+x)*(x+(1+2))
Read a function definition:
define double @test(double %x) {
entry:
%addtmp = fadd double 3.000000e+00, %x ; <double> [#uses=1]
%addtmp1 = fadd double %x, 3.000000e+00 ; <double> [#uses=1]
%multmp = fmul double %addtmp, %addtmp1 ; <double> [#uses=1]
ret double %multmp
}
In this case, the LHS and RHS of the multiplication are the same value.
We'd really like to see this generate"``tmp = x+3; result = tmp*tmp;``
instead of computing ``x+3`` twice.
Unfortunately, no amount of local analysis will be able to detect and
correct this. This requires two transformations: reassociation of
expressions (to make the add's lexically identical) and Common
Subexpression Elimination (CSE) to delete the redundant add instruction.
Fortunately, LLVM provides a broad range of optimizations that you can
use, in the form of "passes".
--------------
LLVM Optimization Passes
=============================================
LLVM provides many optimization passes, which do many different sorts of
things and have different tradeoffs. Unlike other systems, LLVM doesn't
hold to the mistaken notion that one set of optimizations is right for
all languages and for all situations. LLVM allows a compiler implementor
to make complete decisions about what optimizations to use, in which
order, and in what situation.
As a concrete example, LLVM supports both "whole module" passes, which
look across as large of body of code as they can (often a whole file,
but if run at link time, this can be a substantial portion of the whole
program). It also supports and includes "per-function" passes which just
operate on a single function at a time, without looking at other
functions. For more information on passes and how they are run, see the
`How to Write a Pass <http://www.llvm.org/docs/WritingAnLLVMPass.html>`_
document and the `List of LLVM
Passes <http://www.llvm.org/docs/Passes.html>`_.
For Kaleidoscope, we are currently generating functions on the fly, one
at a time, as the user types them in. We aren't shooting for the
ultimate optimization experience in this setting, but we also want to
catch the easy and quick stuff where possible. As such, we will choose
to run a few per-function optimizations as the user types the function
in. If we wanted to make a "static Kaleidoscope compiler", we would use
exactly the code we have now, except that we would defer running the
optimizer until the entire file has been parsed.
In order to get per-function optimizations going, we need to set up a
`FunctionPassManager <http://www.llvm.org/docs/WritingAnLLVMPass.html#passmanager>`_
to hold and organize the LLVM optimizations that we want to run. Once we
have that, we can add a set of optimizations to run. The code looks like
this:
.. code-block:: python
# The function optimization passes manager.
g_llvm_pass_manager = FunctionPassManager.new(g_llvm_module)
# The LLVM execution engine.
g_llvm_executor = ExecutionEngine.new(g_llvm_module)
...
def main():
# Set up the optimizer pipeline. Start with registering info about how the
# target lays out data structures.
g_llvm_pass_manager.add(g_llvm_executor.target_data)
# Do simple "peephole" optimizations and bit-twiddling optzns.
g_llvm_pass_manager.add(PASS_INSTRUCTION_COMBINING)
# Reassociate expressions.
g_llvm_pass_manager.add(PASS_REASSOCIATE)
# Eliminate Common SubExpressions.
g_llvm_pass_manager.add(PASS_GVN)
# Simplify the control flow graph (deleting unreachable blocks, etc).
g_llvm_pass_manager.add(PASS_CFG_SIMPLIFICATION)
g_llvm_pass_manager.initialize()
This code defines a ``FunctionPassManager``, ``g_llvm_pass_manager``.
Once it is set up, we use a series of "add" calls to add a bunch of LLVM
passes. The first pass is basically boilerplate, it adds a pass so that
later optimizations know how the data structures in the program are laid
out. (The "``g_llvm_executor``\ " variable is related to the JIT, which
we will get to in the next section.) In this case, we choose to add 4
optimization passes. The passes we chose here are a pretty standard set
of "cleanup" optimizations that are useful for a wide variety of code. I
won't delve into what they do but, believe me, they are a good starting
place :).
Once the pass manager is set up, we need to make use of it. We do this
by running it after our newly created function is constructed (in
``FunctionNode.CodeGen``), but before it is returned to the client:
.. code-block:: python
return_value = self.body.CodeGen()
g_llvm_builder.ret(return_value)
# Validate the generated code, checking for consistency.
function.verify()
# Optimize the function.
g_llvm_pass_manager.run(function)
As you can see, this is pretty straightforward. The
``FunctionPassManager`` optimizes and updates the LLVM Function in
place, improving (hopefully) its body. With this in place, we can try
our test above again:
.. code-block:: bash
ready> def test(x) (1+2+x)*(x+(1+2))
Read a function definition:
define double @test(double %x) {
entry:
%addtmp = fadd double %x, 3.000000e+00 ; <double> [#uses=2]
%multmp = fmul double %addtmp, %addtmp ; <double> [#uses=1]
ret double %multmp
}
As expected, we now get our nicely optimized code, saving a floating
point add instruction from every execution of this function.
LLVM provides a wide variety of optimizations that can be used in
certain circumstances. Some `documentation about the various
passes <http://www.llvm.org/docs/Passes.html>`_ is available, but it
isn't very complete. Another good source of ideas can come from looking
at the passes that ``llvm-gcc`` or ``llvm-ld`` run to get started. The
``opt`` tool allows you to experiment with passes from the command line,
so you can see if they do anything.
Now that we have reasonable code coming out of our front-end, lets talk
about executing it!
--------------
Adding a JIT Compiler
==============================
Code that is available in LLVM IR can have a wide variety of tools
applied to it. For example, you can run optimizations on it (as we did
above), you can dump it out in textual or binary forms, you can compile
the code to an assembly file (.s) for some target, or you can JIT
compile it. The nice thing about the LLVM IR representation is that it
is the "common currency" between many different parts of the compiler.
In this section, we'll add JIT compiler support to our interpreter. The
basic idea that we want for Kaleidoscope is to have the user enter
function bodies as they do now, but immediately evaluate the top-level
expressions they type in. For example, if they type in "1 + 2", we
should evaluate and print out 3. If they define a function, they should
be able to call it from the command line.
In order to do this, we first declare and initialize the JIT. This is
done by adding and initializing a global variable:
.. code-block:: python
# The LLVM execution engine.
g_llvm_executor = ExecutionEngine.new(g_llvm_module)
This creates an abstract "Execution Engine" which can be either a JIT
compiler or the LLVM interpreter. LLVM will automatically pick a JIT
compiler for you if one is available for your platform, otherwise it
will fall back to the interpreter.
Once the ``ExecutionEngine`` is created, the JIT is ready to be used. We
can use the ``run_function`` method of the execution engine to execute a
compiled function and get its return value. In our case, this means that
we can change the code that parses a top-level expression to look like
this:
.. code-block:: python
def HandleTopLevelExpression(self):
try:
function = self.ParseTopLevelExpr().CodeGen()
result = g_llvm_executor.run_function(function, [])
print 'Evaluated to:', result.as_real(Type.double())
except Exception, e:
print 'Error:', e
try:
self.Next() # Skip for error recovery.
except:
pass
Recall that we compile top-level expressions into a self-contained LLVM
function that takes no arguments and returns the computed double.
With just these two changes, lets see how Kaleidoscope works now!
.. code-block:: bash
ready> 4+5
Read a top level expression:
define double @0() {
entry:
ret double 9.000000e+00
}
Evaluated to: 9.0
Well this looks like it is basically working. The dump of the function
shows the "no argument function that always returns double" that we
synthesize for each top-level expression that is typed in. This
demonstrates very basic functionality, but can we do more?
.. code-block:: bash
ready> def testfunc(x y) x + y*2
Read a function definition:
define double @testfunc(double %x, double %y) {
entry:
%multmp = fmul double %y, 2.000000e+00 ; <double> [#uses=1]
%addtmp = fadd double %multmp, %x ; <double> [#uses=1]
ret double %addtmp
}
ready> testfunc(4, 10)
Read a top level expression:
define double @0() {
entry:
%calltmp = call double @testfunc(double 4.000000e+00, double 1.000000e+01) ; <double> [#uses=1]
ret double %calltmp
}
*Evaluated to: 24.0*
This illustrates that we can now call user code, but there is something
a bit subtle going on here. Note that we only invoke the JIT on the
anonymous functions that *call testfunc*, but we never invoked it on
*testfunc* itself. What actually happened here is that the JIT scanned
for all non-JIT'd functions transitively called from the anonymous
function and compiled all of them before returning from
``run_function()``.
The JIT provides a number of other more advanced interfaces for things
like freeing allocated machine code, rejit'ing functions to update them,
etc. However, even with this simple code, we get some surprisingly
powerful capabilities - check this out (I removed the dump of the
anonymous functions, you should get the idea by now :) :
.. code-block:: bash
ready> extern sin(x)
Read an extern:
declare double @sin(double)
ready> extern cos(x)
Read an extern:
declare double @cos(double)
ready> sin(1.0)
*Evaluated to: 0.841470984808*
ready> def foo(x) sin(x) *sin(x) + cos(x)* cos(x)
Read a function definition:
define double @foo(double %x) {
entry:
%calltmp = call double @sin(double %x) ; <double> [#uses=1]
%calltmp1 = call double @sin(double %x) ; <double> [#uses=1]
%multmp = fmul double %calltmp, %calltmp1 ; <double> [#uses=1]
%calltmp2 = call double @cos(double %x) ; <double> [#uses=1]
%calltmp3 = call double @cos(double %x) ; <double> [#uses=1]
%multmp4 = fmul double %calltmp2, %calltmp3 ; <double> [#uses=1]
%addtmp = fadd double %multmp, %multmp4 ; <double> [#uses=1]
ret double %addtmp
}
ready> foo(4.0)
*Evaluated to: 1.000000*
Whoa, how does the JIT know about sin and cos? The answer is
surprisingly simple: in this example, the JIT started execution of a
function and got to a function call. It realized that the function was
not yet JIT compiled and invoked the standard set of routines to resolve
the function. In this case, there is no body defined for the function,
so the JIT ended up calling ``dlsym("sin")`` on the Python process that
is hosting our Kaleidoscope prompt. Since ``sin`` is defined within the
JIT's address space, it simply patches up calls in the module to call
the libm version of ``sin`` directly.
One interesting application of this is that we can now extend the
language by writing arbitrary C++ code to implement operations. For
example, we can create a C file with the following simple function:
.. code-block:: c
#include <stdio.h>
double putchard(double x) {
putchar((char)x); return 0;
}
We can then compile this into a shared library with GCC::
gcc -shared -fPIC -o putchard.so putchard.c
Now we can load this library into the Python process using
``llvm.core.load_library_permanently`` and access it from Kaleidoscope
to produce simple output to the console::
>>> import llvm.core
>>> llvm.core.load_library_permanently('/home/max/llvmpy-tutorial/putchard.so')
>>> import kaleidoscope
>>> kaleidoscope.main()
ready> extern putchard(x)
Read an extern:
declare double @putchard(double)
ready> putchard(65) + putchard(66) + putchard(67) + putchard(10)
*ABC*
Evaluated to: 0.0
Similar code could be used to implement file I/O, console input, and
many other capabilities in Kaleidoscope.
This completes the JIT and optimizer chapter of the Kaleidoscope
tutorial. At this point, we can compile a non-Turing-complete
programming language, optimize and JIT compile it in a user-driven way.
Next up we'll look into `extending the language with control flow
constructs <PythonLangImpl5.html>`_, tackling some interesting LLVM IR
issues along the way.
--------------
Full Code Listing
===========================
Here is the complete code listing for our running example, enhanced with
the LLVM JIT and optimizer:
.. code-block:: python
#!/usr/bin/env python
import re
from llvm.core import Module, Constant, Type, Function, Builder, FCMP_ULT
from llvm.ee import ExecutionEngine, TargetData
from llvm.passes import FunctionPassManager
from llvm.passes import (PASS_INSTRUCTION_COMBINING,
PASS_REASSOCIATE,
PASS_GVN,
PASS_CFG_SIMPLIFICATION)
Globals
-------
.. code-block:: python
# The LLVM module, which holds all the IR code.
g_llvm_module = Module.new('my cool jit')
# The LLVM instruction builder. Created whenever a new function is entered.
g_llvm_builder = None
# A dictionary that keeps track of which values are defined in the current scope
# and what their LLVM representation is.
g_named_values = {}
# The function optimization passes manager.
g_llvm_pass_manager = FunctionPassManager.new(g_llvm_module)
# The LLVM execution engine.
g_llvm_executor = ExecutionEngine.new(g_llvm_module)
Lexer
-----
.. code-block:: python
# The lexer yields one of these types for each token.
class EOFToken(object):
pass
class DefToken(object):
pass
class ExternToken(object):
pass
class IdentifierToken(object):
def __init__(self, name):
self.name = name
class NumberToken(object):
def __init__(self, value):
self.value = value
class CharacterToken(object):
def __init__(self, char):
self.char = char
def __eq__(self, other):
return isinstance(other, CharacterToken) and self.char == other.char
def __ne__(self, other):
return not self == other
# Regular expressions that tokens and comments of our language.
REGEX_NUMBER = re.compile('[0-9]+(?:\.[0-9]+)?')
REGEX_IDENTIFIER = re.compile('[a-zA-Z][a-zA-Z0-9]*')
REGEX_COMMENT = re.compile('#.*')
def Tokenize(string):
while string:
# Skip whitespace.
if string[0].isspace():
string = string[1:]
continue
# Run regexes.
comment_match = REGEX_COMMENT.match(string)
number_match = REGEX_NUMBER.match(string)
identifier_match = REGEX_IDENTIFIER.match(string)
# Check if any of the regexes matched and yield the appropriate result.
if comment_match:
comment = comment_match.group(0)
string = string[len(comment):]
elif number_match:
number = number_match.group(0)
yield NumberToken(float(number))
string = string[len(number):]
elif identifier_match:
identifier = identifier_match.group(0)
# Check if we matched a keyword.
if identifier == 'def':
yield DefToken()
elif identifier == 'extern':
yield ExternToken()
else:
yield IdentifierToken(identifier)
string = string[len(identifier):]
else:
# Yield the ASCII value of the unknown character.
yield CharacterToken(string[0])
string = string[1:]
yield EOFToken()
Abstract Syntax Tree (aka Parse Tree)
-------------------------------------
.. code-block:: python
# Base class for all expression nodes.
class ExpressionNode(object):
pass
# Expression class for numeric literals like "1.0".
class NumberExpressionNode(ExpressionNode):
def __init__(self, value):
self.value = value
def CodeGen(self):
return Constant.real(Type.double(), self.value)
# Expression class for referencing a variable, like "a".
class VariableExpressionNode(ExpressionNode):
def __init__(self, name):
self.name = name
def CodeGen(self):
if self.name in g_named_values:
return g_named_values[self.name]
else:
raise RuntimeError('Unknown variable name: ' + self.name)
# Expression class for a binary operator.
class BinaryOperatorExpressionNode(ExpressionNode):
def __init__(self, operator, left, right):
self.operator = operator
self.left = left
self.right = right
def CodeGen(self):
left = self.left.CodeGen()
right = self.right.CodeGen()
if self.operator == '+':
return g_llvm_builder.fadd(left, right, 'addtmp')
elif self.operator == '-':
return g_llvm_builder.fsub(left, right, 'subtmp')
elif self.operator == '*':
return g_llvm_builder.fmul(left, right, 'multmp')
elif self.operator == '<':
result = g_llvm_builder.fcmp(FCMP_ULT, left, right, 'cmptmp')
# Convert bool 0 or 1 to double 0.0 or 1.0.
return g_llvm_builder.uitofp(result, Type.double(), 'booltmp')
else:
raise RuntimeError('Unknown binary operator.')
# Expression class for function calls.
class CallExpressionNode(ExpressionNode):
def __init__(self, callee, args):
self.callee = callee
self.args = args
def CodeGen(self):
# Look up the name in the global module table.
callee = g_llvm_module.get_function_named(self.callee)
# Check for argument mismatch error.
if len(callee.args) != len(self.args):
raise RuntimeError('Incorrect number of arguments passed.')
arg_values = [i.CodeGen() for i in self.args]
return g_llvm_builder.call(callee, arg_values, 'calltmp')
# This class represents the "prototype" for a function, which captures its name,
# and its argument names (thus implicitly the number of arguments the function
# takes).
class PrototypeNode(object):
def __init__(self, name, args):
self.name = name
self.args = args
def CodeGen(self):
# Make the function type, eg. double(double,double).
funct_type = Type.function(
Type.double(), [Type.double()] * len(self.args), False)
function = Function.new(g_llvm_module, funct_type, self.name)
# If the name conflicted, there was already something with the same name.
# If it has a body, don't allow redefinition or reextern.
if function.name != self.name:
function.delete()
function = g_llvm_module.get_function_named(self.name)
# If the function already has a body, reject this.
if not function.is_declaration:
raise RuntimeError('Redefinition of function.')
# If F took a different number of args, reject.
if len(callee.args) != len(self.args):
raise RuntimeError('Redeclaration of a function with different number '
'of args.')
# Set names for all arguments and add them to the variables symbol table.
for arg, arg_name in zip(function.args, self.args):
arg.name = arg_name
# Add arguments to variable symbol table.
g_named_values[arg_name] = arg
return function
# This class represents a function definition itself.
class FunctionNode(object):
def __init__(self, prototype, body):
self.prototype = prototype
self.body = body
def CodeGen(self):
# Clear scope.
g_named_values.clear()
# Create a function object.
function = self.prototype.CodeGen()
# Create a new basic block to start insertion into.
block = function.append_basic_block('entry')
global g_llvm_builder
g_llvm_builder = Builder.new(block)
# Finish off the function.
try:
return_value = self.body.CodeGen()
g_llvm_builder.ret(return_value)
# Validate the generated code, checking for consistency.
function.verify()
# Optimize the function.
g_llvm_pass_manager.run(function)
except:
function.delete()
raise
return function
Parser
------
.. code-block:: python
class Parser(object):
def __init__(self, tokens, binop_precedence):
self.tokens = tokens
self.binop_precedence = binop_precedence
self.Next()
# Provide a simple token buffer. Parser.current is the current token the
# parser is looking at. Parser.Next() reads another token from the lexer and
# updates Parser.current with its results.
def Next(self):
self.current = self.tokens.next()
# Gets the precedence of the current token, or -1 if the token is not a
binary # operator.
def GetCurrentTokenPrecedence(self):
if isinstance(self.current, CharacterToken):
return self.binop_precedence.get(self.current.char, -1)
else:
return -1
# identifierexpr ::= identifier | identifier '(' expression* ')'
def ParseIdentifierExpr(self):
identifier_name = self.current.name
self.Next() # eat identifier.
if self.current != CharacterToken('('): # Simple variable reference.
return VariableExpressionNode(identifier_name)
# Call.
self.Next() # eat '('.
args = []
if self.current != CharacterToken(')'):
while True:
args.append(self.ParseExpression())
if self.current == CharacterToken(')'):
break
elif self.current != CharacterToken(','):
raise RuntimeError('Expected ")" or "," in argument list.')
self.Next()
self.Next() # eat ')'.
return CallExpressionNode(identifier_name, args)
# numberexpr ::= number
def ParseNumberExpr(self):
result = NumberExpressionNode(self.current.value)
self.Next() # consume the number.
return result
# parenexpr ::= '(' expression ')'
def ParseParenExpr(self):
self.Next() # eat '('.
contents = self.ParseExpression()
if self.current != CharacterToken(')'):
raise RuntimeError('Expected ")".')
self.Next() # eat ')'.
return contents
# primary ::= identifierexpr | numberexpr | parenexpr
def ParsePrimary(self):
if isinstance(self.current, IdentifierToken):
return self.ParseIdentifierExpr()
elif isinstance(self.current, NumberToken):
return self.ParseNumberExpr()
elif self.current == CharacterToken('('):
return self.ParseParenExpr()
else: raise RuntimeError('Unknown token when expecting an expression.')
# binoprhs ::= (operator primary)*
def ParseBinOpRHS(self, left, left_precedence):
# If this is a binary operator, find its precedence.
while True:
precedence = self.GetCurrentTokenPrecedence()
# If this is a binary operator that binds at least as tightly as the
# current one, consume it; otherwise we are done.
if precedence < left_precedence:
return left
binary_operator = self.current.char
self.Next() # eat the operator.
# Parse the primary expression after the binary operator.
right = self.ParsePrimary()
# If binary_operator binds less tightly with right than the operator after
# right, let the pending operator take right as its left.
next_precedence = self.GetCurrentTokenPrecedence()
if precedence < next_precedence:
right = self.ParseBinOpRHS(right, precedence + 1)
# Merge left/right.
left = BinaryOperatorExpressionNode(binary_operator, left, right)
# expression ::= primary binoprhs
def ParseExpression(self):
left = self.ParsePrimary()
return self.ParseBinOpRHS(left, 0)
# prototype ::= id '(' id* ')'
def ParsePrototype(self):
if not isinstance(self.current, IdentifierToken):
raise RuntimeError('Expected function name in prototype.')
function_name = self.current.name
self.Next() # eat function name.
if self.current != CharacterToken('('):
raise RuntimeError('Expected "(" in prototype.')
self.Next() # eat '('.
arg_names = []
while isinstance(self.current, IdentifierToken):
arg_names.append(self.current.name)
self.Next()
if self.current != CharacterToken(')'):
raise RuntimeError('Expected ")" in prototype.')
# Success.
self.Next() # eat ')'.
return PrototypeNode(function_name, arg_names)
# definition ::= 'def' prototype expression
def ParseDefinition(self):
self.Next() # eat def.
proto = self.ParsePrototype()
body = self.ParseExpression()
return FunctionNode(proto, body)
# toplevelexpr ::= expression
def ParseTopLevelExpr(self):
proto = PrototypeNode('', [])
return FunctionNode(proto, self.ParseExpression())
# external ::= 'extern' prototype
def ParseExtern(self):
self.Next() # eat extern.
return self.ParsePrototype()
# Top-Level parsing
def HandleDefinition(self):
self.Handle(self.ParseDefinition, 'Read a function definition:')
def HandleExtern(self):
self.Handle(self.ParseExtern, 'Read an extern:')
def HandleTopLevelExpression(self):
try:
function = self.ParseTopLevelExpr().CodeGen()
result = g_llvm_executor.run_function(function, [])
print 'Evaluated to:', result.as_real(Type.double())
except Exception, e:
print 'Error:', e
try:
self.Next() # Skip for error recovery.
except:
pass
def Handle(self, function, message):
try:
print message, function().CodeGen()
except Exception, e:
print 'Error:', e
try:
self.Next() # Skip for error recovery.
except:
pass
Main driver code.
-----------------
.. code-block:: python
def main():
# Set up the optimizer pipeline. Start with registering info about how the
# target lays out data structures.
g_llvm_pass_manager.add(g_llvm_executor.target_data)
# Do simple "peephole" optimizations and bit-twiddling optzns.
g_llvm_pass_manager.add(PASS_INSTRUCTION_COMBINING)
# Reassociate expressions.
g_llvm_pass_manager.add(PASS_REASSOCIATE)
# Eliminate Common SubExpressions.
g_llvm_pass_manager.add(PASS_GVN)
# Simplify the control flow graph (deleting unreachable blocks, etc).
g_llvm_pass_manager.add(PASS_CFG_SIMPLIFICATION)
g_llvm_pass_manager.initialize()
# Install standard binary operators.
# 1 is lowest possible precedence. 40 is the highest.
operator_precedence = {
'<': 10,
'+': 20,
'-': 20,
'*': 40
}
# Run the main "interpreter loop".
while True:
print 'ready>',
try:
raw = raw_input()
except KeyboardInterrupt:
break
parser = Parser(Tokenize(raw), operator_precedence)
while True:
# top ::= definition | external | expression | EOF
if isinstance(parser.current, EOFToken):
break
if isinstance(parser.current, DefToken):
parser.HandleDefinition()
elif isinstance(parser.current, ExternToken):
parser.HandleExtern()
else:
parser.HandleTopLevelExpression()
# Print out all of the generated code.
print '', g_llvm_module
if __name__ == '__main__':
main()

1549
docs/source/doc/kaleidoscope/PythonLangImpl5.rst
View file

File diff suppressed because it is too large Load diff

1557
docs/source/doc/kaleidoscope/PythonLangImpl6.rst
View file

File diff suppressed because it is too large Load diff

1835
docs/source/doc/kaleidoscope/PythonLangImpl7.rst
View file

File diff suppressed because it is too large Load diff

275
docs/source/doc/kaleidoscope/PythonLangImpl8.rst
View file

@ -1,275 +0,0 @@
***************************************************
Chapter 8: Conclusion and other useful LLVM tidbits
***************************************************
Written by Chris Lattner
Tutorial Conclusion
===================
Welcome to the the final chapter of the "`Implementing a language with LLVM
<http://www.llvm.org/docs/tutorial/index.html>`_" tutorial.
In the course of this tutorial, we have grown our little Kaleidoscope language
from being a useless toy, to being a semi-interesting (but probably still useless)
toy. :)
It is interesting to see how far we've come, and how little code it has taken.
We built the entire lexer, parser, AST, code generator, and an interactive run-loop
(with a JIT!) by-hand in under 540 lines of (non-comment/non-blank) code.
Our little language supports a couple of interesting features: it supports user
defined binary and unary operators, it uses JIT compilation for immediate evaluation,
and it supports a few control flow constructs with SSA construction.
Part of the idea of this tutorial was to show you how easy and fun it can be to
define, build, and play with languages. Building a compiler need not be a scary
or mystical process! Now that you've seen some of the basics, I strongly encourage
you to take the code and hack on it. For example, try adding:
- ***global variables*** - While global variables have questional value in modern
software engineering, they are often useful when putting together quick
little hacks like the Kaleidoscope compiler itself. Fortunately, our
current setup makes it very easy to add global variables: just have value
lookup check to see if an unresolved variable is in the global variable
symbol table before rejecting it. To create a new global variable, make
an instance of the LLVM GlobalVariable class.
- ***typed variables*** - Kaleidoscope currently only supports variables of type
double. This gives the language a very nice elegance, because only supporting
one type means that you never have to specify types. Different languages have
different ways of handling this. The easiest way is to require the user to
specify types for every variable definition, and record the type of the variable
in the symbol table along with its Value*.
- ***arrays, structs, vectors, etc*** - Once you add types, you can
start extending the type system in all sorts of
interesting ways. Simple arrays are very easy and are quite useful
for many different applications. Adding them is mostly an
exercise in learning how the LLVM `getelementptr
<http://www.llvm.org/docs/LangRef.html#i_getelementptr>`_ instruction works:
it is so nifty/unconventional, it `has its own FAQ!
<http://www.llvm.org/docs/GetElementPtr.html>`_ If you add
support for recursive types (e.g. linked lists), make sure to
read the `section in the LLVM Programmer's Manual
<http://www.llvm.org/docs/ProgrammersManual.html#TypeResolve>`_ that describes
how to construct them.
- ***standard runtime*** - Our current language allows the user to
access arbitrary external functions, and we use it for things like "putchard".
As you extend the language to add higher-level constructs, often these
constructs make the most sense if they are lowered to calls into a
language-supplied runtime. For example, if you add hash tables to the
language, it would probably make sense to add the routines to a runtime,
instead of inlining them all the way.
- ***memory management*** - Currently we can only access the stack in Kaleidoscope.
It would also be useful to be able to allocate heap memory, either with calls
to the standard libc malloc/free interface or with a garbage collector.
If you would like to use garbage collection, note that LLVM fully supports
`Accurate Garbage Collection <http://www.llvm.org/docs/GarbageCollection.html>`_
including algorithms that move objects and need to scan/update the stack.
- ***debugger support*** - LLVM supports generation of `DWARF Debug info
<http://www.llvm.org/docs/SourceLevelDebugging.html>`_ which
is understood by common debuggers like GDB. Adding support for debug info is
fairly straightforward. The best way to understand it is to compile some C/C++
code with ``llvm-gcc -g -O0`` and taking a look at what it produces.
- ***exception handling support*** - LLVM supports generation of `zero cost exceptions
<http://www.llvm.org/docs/ExceptionHandling.html>`_ which interoperate
with code compiled in other languages. You could also generate code by
implicitly making every function return an error value and checking it.
You could also make explicit use of setjmp/longjmp. There are many different
ways to go here.
- ***object orientation, generics, database access, complex numbers, geometric
programming, ...*** - Really, there is no end of crazy features that you can
add to the language.
- ***unusual domains*** - We've been talking about applying LLVM to a domain that
many people are interested in: building a compiler for a specific language.
However, there are many other domains that can use compiler technology that are
not typically considered. For example, LLVM has been used to implement OpenGL
graphics acceleration, translate C++ code to ActionScript, and many other cute
and clever things. Maybe you will be the first to JIT compile a regular expression
interpreter into native code with LLVM?
- ***Have fun*** - try doing something crazy and unusual. Building a language like
everyone else always has, is much less fun than trying something a little crazy or
off the wall and seeing how it turns out. If you get stuck or want to talk about it,
feel free to email the `llvmdev mailing list
<http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev>`_: it has lots of people who are
interested in languages and are often willing to help out.
Before we end this tutorial, I want to talk about some "tips and tricks" for
generating LLVM IR. These are some of the more subtle things that may not be obvious,
but are very useful if you want to take advantage of LLVM's capabilities.
Properties of the LLVM IR
=========================
We have a couple common questions about code in the LLVM IR form - let's
just get these out of the way right now, shall we?
--------------
Target Independence
-------------------
Kaleidoscope is an example of a "portable language": any program
written in Kaleidoscope will work the same way on any target that it
runs on. Many other languages have this property, e.g. LISP, Java, Haskell,
Javascript, Python, etc. (note that while these languages are portable,
not all their libraries are).
One nice aspect of LLVM is that it is often capable of preserving target
independence in the IR: you can take the LLVM IR for a Kaleidoscope-compiled
program and run it on any target that LLVM supports, even emitting C code and
compiling that on targets that LLVM doesn't support natively.
You can trivially tell that the Kaleidoscope compiler generates target-
independent code because it never queries for any target-specific
information when generating code.
The fact that LLVM provides a compact, target-independent,
representation for code gets a lot of people excited. Unfortunately,
these people are usually thinking about C or a language from the
C family when they are asking questions about language portability.
I say "unfortunately", because there is really no way to make (fully general) C
code portable, other than shipping the source code around (and of course, C
source code is not actually portable in general either - ever port a really old
application from 32- to 64-bits?).
The problem with C (again, in its full generality) is that it is heavily
laden with target specific assumptions. As one simple example, the
preprocessor often destructively removes target-independence from the code
when it processes the input text::
#ifdef __i386__
int X = 1;
#else
int X = 42;
#endif
While it is possible to engineer more and more complex solutions to problems like
this, it cannot be solved in full generality in a way that is better than
shipping the actual source code.
That said, there are interesting subsets of C that can be made portable.
If you are willing to fix primitive types to a fixed size (say int = 32-bits, and
long = 64-bits), don't care about ABI compatibility with existing binaries, and
are willing to give up some other minor features, you can have portable code.
This can make sense for specialized domains such as an in-kernel language.
--------------
Safety Guarantees
-----------------
Many of the languages above are also "safe" languages: it is
impossible for a program written in Java to corrupt its address space and
crash the process (assuming the JVM has no bugs). Safety is an
interesting property that requires a combination of language design,
runtime support, and often operating system support.
It is certainly possible to implement a safe language in LLVM, but LLVM
IR does not itself guarantee safety. The LLVM IR allows unsafe pointer casts,
use after free bugs, buffer over-runs, and a variety of other problems. Safety
needs to be implemented as a layer on top of LLVM and, conveniently, several groups
have investigated this. Ask on the `llvmdev mailing list
<http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev>`_ if you are interested
in more details.
--------------
Language-Specific Optimizations
-------------------------------
One thing about LLVM that turns off many people is that it does not solve all
the world's problems in one system (sorry 'world hunger', someone else will
have to solve you some other day). One specific complaint is that people perceive
LLVM as being incapable of performing high-level language-specific optimization:
LLVM "loses too much information".
Unfortunately, this is really not the place to give you a full and unified
version of "Chris Lattner's theory of compiler design". Instead,
I'll make a few observations:
First, you're right that LLVM does lose information.
For example, as of this writing, there is no way to
distinguish in the LLVM IR whether an SSA-value came
from a C "int" or a C "long" on an ILP32 machine
(other than debug info). Both get compiled down to an 'i32'
value and the information about what it came from is lost.
The more general issue here, is that the LLVM type system
uses "structural equivalence" instead of "name equivalence".
Another place this surprises people is if you have two types
in a high-level language that have the same structure (e.g.
two different structs that have a single int field):
these types will compile down into a single LLVM type and it
will be impossible to tell what it came from.
Second, while LLVM does lose information, LLVM is not a
fixed target: we continue to enhance and improve it in many
different ways. In addition to adding new features (LLVM did not
always support exceptions or debug info), we also extend the IR to
capture important information for optimization (e.g. whether an argument
is sign or zero extended, information about pointers aliasing, etc). Many
of the enhancements are user-driven: people want LLVM to include some specific
feature, so they go ahead and extend it.
Third, it is possible and easy to add language-specific optimizations,
and you have a number of choices in how to do it. As one trivial example,
it is easy to add language-specific optimization passes that "know" things
about code compiled for a language. In the case of the C family, there is an
optimization pass that "knows" about the standard C library functions. If you
call "exit(0)" in main(), it knows that it is safe to optimize that into "return
0;" because C specifies what the 'exit' function does.
In addition to simple library knowledge, it is possible to embed a
variety of other language-specific information into the LLVM IR. If
you have a specific need and run into a wall, please bring the topic
up on the llvmdev list. At the very worst, you can always treat LLVM as
if it were a "dumb code generator" and implement the high-level optimizations
you desire in your front-end, on the language-specific AST.
--------------
Tips and Tricks
===============
There is a variety of useful tips and tricks that you come to
know after working on/with LLVM that aren't obvious at first glance.
Instead of letting everyone rediscover them, this section talks about
some of these issues.
--------------
Implementing portable offsetof/sizeof
-------------------------------------
One interesting thing that comes up, if you are trying to keep the
code generated by your compiler "target independent", is that you often
need to know the size of some LLVM type or the offset of some field in an
llvm structure. For example, you might need to pass the size of a type into
a function that allocates memory.
Unfortunately, this can vary widely across targets: for example the width
of a pointer is trivially target-specific. However, there is a `clever way
to use the getelementptr instruction
<http://nondot.org/sabre/LLVMNotes/SizeOf-OffsetOf-VariableSizedStructs.txt>`_ that
allows you to compute this in a portable way.
--------------
Garbage Collected Stack Frames
------------------------------
Some languages want to explicitly manage their stack frames, often
so that they are garbage collected or to allow easy implementation
of closures. There are often better ways to implement these features
than explicit stack frames, but `LLVM does support them
<http://nondot.org/sabre/LLVMNotes/ExplicitlyManagedStackFrames.txt>`_, if you want.
It requires your front-end to convert the code into `Continuation Passing
Style <http://en.wikipedia.org/wiki/Continuation-passing_style>`_
and the use of tail calls (which LLVM also supports).

21
docs/source/doc/kaleidoscope/index.rst
View file

@ -1,21 +0,0 @@
Kaleidoscope
--------------
Implementing a Language with LLVM
The LLVM `Kaleidoscope <http://www.llvm.org/docs/tutorial/>`_ tutorial
has been ported to llvmpy by Max Shawabkeh.
.. toctree::
:titlesonly:
:numbered:
PythonLangImpl1.rst
PythonLangImpl2.rst
PythonLangImpl3.rst
PythonLangImpl4.rst
PythonLangImpl5.rst
PythonLangImpl6.rst
PythonLangImpl7.rst
PythonLangImpl8.rst

10
docs/source/doc/llpython/articles.rst
View file

@ -1,10 +0,0 @@
LLPython Articles
=================
.. toctree::
:titlesonly:
:maxdepth: 1
intro_llpython.rst
nobitey_dev.rst
ctmp_in_llpython.rst

5
docs/source/doc/llpython/ctmp_in_llpython.rst
View file

@ -1,5 +0,0 @@
Compile-time Metaprogramming in LLPython
========================================
In this article, we discuss how LLPython supports compile-time
metaprogramming.

9
docs/source/doc/llpython/index.rst
View file

@ -1,9 +0,0 @@
LLPython Documentation
======================
.. toctree::
:titlesonly:
:maxdepth: 2
articles.rst
reference.rst

46
docs/source/doc/llpython/intro_llpython.rst
View file

@ -1,46 +0,0 @@
====================
Introducing LLPython
====================
In this article, we introduce the llpython package. The primary goal
of the llpython package is to provide a Python dialect/subset that
maps directly to LLVM code. LLPython differs from its originating
LLVM translator, Numba, in the following aspects:
* LLPython code is not intended to work in Python if not translated
and wrapped.
* The LLPython translator only uses LLVM types.
* LLPython is explicitly typed, and does not support type inference.
LLPython does not support implicit casts, all casts must be explicit.
* LLPython supports code that directly calls the C API, the Python C
API, and the llvm.core.Builder methods.
Additionally, we designed the sub-package to have the following
engineering properties:
* Usable from Python 2.7, and 3.X. At the time of writing, we plan
to support Python 2.6.
* Clean from Numba dependencies (other than llvmpy), and can be used
as a standalone code generator without a full Numba installation.
* Provides a series of Python bytecode passes that can be easily
used by other projects.
LLPython Origins
================
We developed LLPython with the initial goal of simplifying writing
LLVM code.
LLPython Internals
==================
In this section, we describe the various passes performed by the
LLPython translator.
Conclusions
===========
LLPython is neat.

6
docs/source/doc/llpython/llpython.byte_control.rst
View file

@ -1,6 +0,0 @@
=====================
llpython.byte_control
=====================
.. automodule:: llpython.byte_control
:members:

6
docs/source/doc/llpython/llpython.byte_flow.rst
View file

@ -1,6 +0,0 @@
==================
llpython.byte_flow
==================
.. automodule:: llpython.byte_flow
:members:

6
docs/source/doc/llpython/llpython.byte_translator.rst
View file

@ -1,6 +0,0 @@
========================
llpython.byte_translator
========================
.. automodule:: llpython.byte_translator
:members:

6
docs/source/doc/llpython/llpython.bytecode_visitor.rst
View file

@ -1,6 +0,0 @@
=========================
llpython.bytecode_visitor
=========================
.. automodule:: llpython.bytecode_visitor
:members:

6
docs/source/doc/llpython/llpython.bytetype.rst
View file

@ -1,6 +0,0 @@
=================
llpython.bytetype
=================
.. automodule:: llpython.bytetype
:members:

6
docs/source/doc/llpython/llpython.control_flow.rst
View file

@ -1,6 +0,0 @@
=====================
llpython.control_flow
=====================
.. automodule:: llpython.control_flow
:members:

6
docs/source/doc/llpython/llpython.gen_bytecode_visitor.rst
View file

@ -1,6 +0,0 @@
=============================
llpython.gen_bytecode_visitor
=============================
.. automodule:: llpython.gen_bytecode_visitor
:members:

6
docs/source/doc/llpython/llpython.nobitey.rst
View file

@ -1,6 +0,0 @@
================
llpython.nobitey
================
.. automodule:: llpython.nobitey
:members:

6
docs/source/doc/llpython/llpython.opcode_util.rst
View file

@ -1,6 +0,0 @@
====================
llpython.opcode_util
====================
.. automodule:: llpython.opcode_util
:members:

6
docs/source/doc/llpython/llpython.phi_injector.rst
View file

@ -1,6 +0,0 @@
=====================
llpython.phi_injector
=====================
.. automodule:: llpython.phi_injector
:members:

6
docs/source/doc/llpython/llpython.pyaddfunc.rst
View file

@ -1,6 +0,0 @@
==================
llpython.pyaddfunc
==================
.. automodule:: llpython.pyaddfunc
:members:

6
docs/source/doc/llpython/llpython.rst
View file

@ -1,6 +0,0 @@
========
llpython
========
.. automodule:: llpython
:members:

5
docs/source/doc/llpython/nobitey_dev.rst
View file

@ -1,5 +0,0 @@
nobitey: Using ctypes and llvmpy to Bypass ctypes
=================================================
In this article, we show how nobitey uses llvmpy to eliminate the
ctypes call overhead.

21
docs/source/doc/llpython/reference.rst
View file

@ -1,21 +0,0 @@
LLPython Module Reference
=========================
Contents:
.. toctree::
:titlesonly:
:maxdepth: 2
llpython.rst
llpython.bytecode_visitor.rst
llpython.byte_control.rst
llpython.byte_flow.rst
llpython.byte_translator.rst
llpython.bytetype.rst
llpython.control_flow.rst
llpython.gen_bytecode_visitor.rst
llpython.nobitey.rst
llpython.opcode_util.rst
llpython.phi_injector.rst
llpython.pyaddfunc.rst

65
docs/source/doc/llvm.core.Argument.rst
View file

@ -1,65 +0,0 @@
+-------------------------------+
| layout: page |
+-------------------------------+
| title: Argument (llvm.core) |
+-------------------------------+
The ``args`` property of ``llvm.core.Function`` objects yields
``llvm.core.Argument`` objects. This allows for setting attributes for
functions arguments. ``Argument`` objects cannot be constructed from
user code, the only way to get a reference to these are from
``Function`` objects.
The method ``add_attribute`` and ``remove_attribute`` can be used to add
or remove the following attributes:
Value\| Equivalent LLVM Assembly Keyword \|
-----\|----------------------------------\| ``ATTR_ZEXT``\ \|
``zeroext`` \| ``ATTR_SEXT``\ \| ``signext`` \| ``ATTR_IN_REG``\ \|
``inreg`` \| ``ATTR_BY_VAL``\ \| ``byval`` \| ``ATTR_STRUCT_RET``\ \|
``sret`` \| ``ATTR_NO_ALIAS``\ \| ``noalias`` \| ``ATTR_NO_CAPTURE``\ \|
``nocapture`` \| ``ATTR_NEST``\ \| ``nest`` \|
These method work exactly like the `corresponding
methods <functions.html#fnattr>`_ of the ``Function`` class above. Refer
`LLVM docs <http://www.llvm.org/docs/LangRef.html#paramattrs>`_ for
information on what each attribute means.
The alignment of any argument can be set via the ``alignment`` property,
to any power of 2.
llvm.core.Argument
==================
Base Class
----------
- `llvm.core.Value <llvm.core.Value.html>`_
Properties
----------
``alignment``
~~~~~~~~~~~~~
The alignment of the argument. Must be a power of 2.
Methods
-------
``add_attribute(attr)``
~~~~~~~~~~~~~~~~~~~~~~~
Add an attribute ``attr`` to the argument, from the set listed above.
``remove_attribute(attr)``
~~~~~~~~~~~~~~~~~~~~~~~~~~
Remove the attribute ``attr`` of the argument.
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.Argument
:members:
:undoc-members:

37
docs/source/doc/llvm.core.ArrayType.rst
View file

@ -1,37 +0,0 @@
+--------------------------------+
| layout: page |
+--------------------------------+
| title: ArrayType (llvm.core) |
+--------------------------------+
llvm.core.ArrayType
===================
Base Class
----------
- `llvm.core.Type <llvm.core.Type.html>`_
Properties
----------
``element``
~~~~~~~~~~~
[read-only]
A ``Type`` object representing the type of the element of the array.
``count``
~~~~~~~~~
[read-only]
The number of elements in the array.
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.ArrayType
:members:
:undoc-members:

51
docs/source/doc/llvm.core.BasicBlock.rst
View file

@ -1,51 +0,0 @@
+---------------------------------+
| layout: page |
+---------------------------------+
| title: BasicBlock (llvm.core) |
+---------------------------------+
A basicblock is a list of instructions. A wellformed basicblock should
end with a terminator. ``Function.verify()`` will verify that. A
terminator is either a branch instruction or return instruction. It is
not possible to have instructions after a branch or return instruction.
llvm.core.BasicBlock
====================
Base Class
----------
- `llvm.core.Value <llvm.core.Value.html>`_
Methods
-------
``delete(self)``
~~~~~~~~~~~~~~~~
Delete this basicblock from the function (``self.function``).
``insert_before(self, name)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TODO
Proporties
----------
``function``
~~~~~~~~~~~~
The parent function of this basicblock.
``instructions``
~~~~~~~~~~~~~~~~
A list of instructions in this basicblock.
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.BasicBlock
:members:
:undoc-members:

418
docs/source/doc/llvm.core.Builder.rst
View file

@ -1,418 +0,0 @@
+------------------------------+
| layout: page |
+------------------------------+
| title: Builder (llvm.core) |
+------------------------------+
The ``Builder`` class corresponds to the
`IRBuilder <http://llvm.org/docs/doxygen/html/classllvm_1_1IRBuilder.html>`_
in C++ llvm. It provides an uniform API to populating
`BasicBlocks <llvm.core.BasicBlock.html>`_. Most of the methods in
``Builder`` correspond to the instructions in the LLVM IR. See `LLVM
documentation <http://llvm.org/docs/LangRef.html>`_ for detail. These
methods have the ``name`` argument for overiding the name of the result
variable. When it is an empty string (default value), LLVM will set a
numeric ID for the result variable.
llvm.core.Builder
=================
- This will become a table of contents (this text will be scraped).
{:toc}
Static Factor Method
--------------------
``new(basic_block)``
~~~~~~~~~~~~~~~~~~~~
Create an instance of ``Builder`` at
`BasicBlock <llvm.core.BasicBlock.html>`_.
Methods
-------
``add(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs+rhs`` for integer values only.
``alloca(self, ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that allocates stack memory for a value of type
``ty``.
``alloca_array(self, ty, size, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that allocates stack memory for a ``size``
elements array of type ``ty``.
``and_(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs & rhs``.
``ashr(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs >> rhs`` using arithmetic
shift.
``bitcast(self, value, dest_ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that cast ``value`` to type ``dest_ty``.
``branch(self, bblk)``
~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that branch to basicblock ``bblk``.
``call(self, fn, args, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that call function ``fn`` with a iterable of
arguments ``args``.
``cbranch(self, if_value, then_blk, else_blk)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that conditionally branch base on the predicate
``if_value``. If ``if_value`` is ``True``, branch to ``then_blk``;
Otherwise, branch to ``else_blk``.
``extract_element(self, vec_val, idx_val, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that extracts an element from a value ``vec_val``
of `llvm.core.VectorType <llvm.core.VectorType.html>`_ at index
``idx_val``.
``extract_value(self, retval, idx, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that extracts an element from an aggregate value
``retval`` at index ``idx``.
``fadd(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs + rhs`` for floating-point
values.
``fcmp(self, rpred, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that compares ``lhs`` and ``rhs`` using the
comparision operation defined by ``rpred``. See
`here <comparision.html#fcmp>`_ for a list of comparators.
``fdiv(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs / rhs`` for floating-point
values.
``fmul(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs * rhs`` for floating-point
values.
``fpext(self, value, dest_ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that extends ``value`` to a float type
``dest_ty``.
``fptosi(self, value, dest_ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that converts a floating-point value ``value`` to
a signed integer type ``dest_ty``.
``fptoui(self, value, dest_ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that converts a floating-point value ``value`` to
an unsigned integer type ``dest_ty``.
``fptrunc(self, value, dest_ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that truncates a floating-point value ``value`` to
a float type ``dest_ty``.
``free(self, ptr)``
~~~~~~~~~~~~~~~~~~~
Insert an instruction that call performs heap deallocation on pointer
``ptr``.
``frem(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs % rhs`` for floating-point
values.
``fsub(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs - rhs`` for floating-point
values.
``gep(self, ptr, indices, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
See `GEP <http://llvm.org/docs/LangRef.html#i_getelementptr>`_.
``getresult(self, retval, idx, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
same as ``extract_value``.
``icmp(self, ipred, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that compares ``lhs`` and ``rhs`` using the
comparision operation defined by ``ipred``. See
`here <comparision.html#icmp>`_ for a list of comparators.
``insert_element(self, vec_val, elt_val, idx_val, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that inserts a value ``elt_val`` into ``vec_val``
of `llvm.core.VectorType <llvm.core.VectorType.html>`_ at index
``idx_val``.
``inttoptr(self, value, dest_ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that converts an integer ``value`` to pointer
``dest_ty``.
``invoke(self, func, args, then_blk, catch_blk, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
See `invoke <http://llvm.org/docs/LangRef.html#i_invoke>`_
``load(self, ptr, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that loads a value at the memory pointed by
``ptr``.
``lshr(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs >> rhs`` using logical shift.
``malloc(self, ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that allocates heap memory of type ``ty``. The
instruction returns a pointer that points to a value of type ``ty``.
``malloc_array(self, ty, size, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Similar to ``malloc`` but allocates an array of ``size`` elements.
``mul(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs * rhs`` for integer types.
``neg(self, val, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``0 - val``.
``not_(self, val, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes an one's complement of ``val``.
``or_(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs | rhs``.
``phi(self, ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~
Create a PHI node of type ``ty``.
``position_at_beginning(self, bblk)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Position the builder at the beginning of the given block. Next
instruction inserted will be first one in the block.
``position_at_end(self, bblk)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Position the builder at the end of the given block. Next instruction
inserted will be last one in the block.
``position_before(self, instr)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Position the builder before the given instruction. The instruction can
belong to a basic block other than the current one.
``ptrtoint(self, value, dest_ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that converts a pointer to an integer ``value`` of
type ``dest_ty``.
``ret(self, value)``
~~~~~~~~~~~~~~~~~~~~
Insert an instruction that returns ``value``.
``ret_many(self, values)``
~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that returns ``values`` which is an iterable of
`llvm.core.Value <llvm.core.Value.html>`_.
``ret_void(self)``
~~~~~~~~~~~~~~~~~~
Insert an instruction that returns nothing (void).
``sdiv(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs / rhs`` for signed integers.
``select(self, cond, then_value, else_value, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``cond ? then_value : else_value``.
``sext(self, value, dest_ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that sign extends an integer ``value`` to type
``dest_ty``.
``shl(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs << rhs``.
``shuffle_vector(self, vecA, vecB, mask, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that performs a vector shuffle base on the two
vectors -- ``vecA`` and ``vecB``, base on a bit mask ``mask``. The mask
must be a constant.
See `LLVM document <http://llvm.org/docs/LangRef.html#i_shufflevector>`_
for detail.
``sitofp(self, value, dest_ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that converts a signed integer ``value`` to a
floating-point type ``dest_ty``.
``srem(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs % rhs`` for signed integers.
``store(self, value, ptr)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that stores ``value`` into the memory pointed by
``ptr``.
``sub(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs - rhs``.
``switch(self, value, else_blk, n=10)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that transfer control flow depending on the
``value``. ``else_blk`` is the default case. ``n`` sets the number of
additional cases.
This method returns an instance of
`SwitchInstruction <llvm.core.Instruction.html#switchinstr>`_ for adding
cases to the switch.
``trunc(self, value, dest_ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that truncates an integer ``value`` to the
destination integer type ``dest_ty``.
``udiv(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs / rhs`` for unsigned integers.
``uitofp(self, value, dest_ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that converts an unsigned integer ``value`` to a
floating-point type ``dest_ty``.
``unreachable(self)``
~~~~~~~~~~~~~~~~~~~~~
Insert an unreachabe instruction, which has no defined semantics. See
`LLVM document <http://llvm.org/docs/LangRef.html#i_unreachable>`_ for
detail.
``urem(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs % rhs`` for unsigned integers.
``vaarg(self, list_val, ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is used to access variable arguments given as ``list_val`` of type
``ty``. see `LLVM
document <http://llvm.org/docs/LangRef.html#int_varargs>`_ about
variable argument intrinsics.
``xor(self, lhs, rhs, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that computes ``lhs xor rhs``.
``zext(self, value, dest_ty, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert an instruction that zero extends ``value`` to type ``dest_ty``.
Properties
----------
``basic_block``
~~~~~~~~~~~~~~~
The `BasicBlock <llvm.core.BasicBlock.html>`_ where the builder is
positioned.
``block``
~~~~~~~~~
Deprecated. Same as ``basic_block``
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.Builder
:members:
:undoc-members:

363
docs/source/doc/llvm.core.Constant.rst
View file

@ -1,363 +0,0 @@
+-------------------------------+
| layout: page |
+-------------------------------+
| title: Constant (llvm.core) |
+-------------------------------+
llvm.core.Constant
==================
``Constant``-s represents constants that appear within the code. The
values of such objects are known at creation time. Constants can be
created from Python constants. A constant expression is also a constant
-- given a ``Constant`` object, an operation (like addition, subtraction
etc) can be specified, to yield a new ``Constant`` object. Let's see
some examples:
.. code-block:: python
#!/usr/bin/env python
ti = Type.int() # a 32-bit int type
k1 = Constant.int(ti, 42) # "int k1 = 42;" k2 = k1.add( Constant.int(
ti, 10 ) ) # "int k2 = k1 + 10;"
tr = Type.float()
r1 = Constant.real(tr, "3.141592") # create from a string
r2 = Constant.real(tr, 1.61803399) # create from a Python float
# llvm.core.Constant
- This will become a table of contents (this text will be scraped).
{:toc}
Static factory methods
----------------------
``null(ty)``
~~~~~~~~~~~~
A null value (all zeros) of type ``ty``
``all_ones(ty)``
~~~~~~~~~~~~~~~~
All 1's value of type ``ty``
``undef(ty)``
~~~~~~~~~~~~~
An undefined value of type ``ty``
``int(ty, value)``
~~~~~~~~~~~~~~~~~~
Integer of type ``ty``, with value ``value`` (a Python int or long)
``int_signextend(ty, value)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Integer of signed type ``ty`` (use for signed types)
``real(ty, value)``
~~~~~~~~~~~~~~~~~~~
Floating point value of type ``ty``, with value ``value`` (a Python
float)
``stringz(value)``
~~~~~~~~~~~~~~~~~~
A null-terminated string. ``value`` is a Python string
``string(value)``
~~~~~~~~~~~~~~~~~
As ``string(ty)``, but not null terminated
``array(ty, consts)``
~~~~~~~~~~~~~~~~~~~~~
Array of type ``ty``, initialized with ``consts`` (an iterable yielding
``Constant`` objects of the appropriate type)
``struct(ty, consts)``
~~~~~~~~~~~~~~~~~~~~~~
Struct (unpacked) of type ``ty``, initialized with ``consts`` (an
iterable yielding ``Constant`` objects of the appropriate type)
``packed_struct(ty, consts)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As ``struct(ty, consts)`` but packed
``vector(consts)``
~~~~~~~~~~~~~~~~~~
Vector, initialized with ``consts`` (an iterable yielding ``Constant``
objects of the appropriate type)
``sizeof(ty)``
~~~~~~~~~~~~~~
Constant value representing the sizeof the type ``ty``
Methods
-------
The following operations on constants are supported. For more details on
any operation, consult the `Constant
Expressions <http://www.llvm.org/docs/LangRef.html#constantexprs>`_
section of the LLVM Language Reference.
``k.neg()``
~~~~~~~~~~~
negation, same as ``0 - k``
``k.not_()``
~~~~~~~~~~~~
1's complement of ``k``. Note trailing underscore.
``k.add(k2)``
~~~~~~~~~~~~~
``k + k2``, where ``k`` and ``k2`` are integers.
``k.fadd(k2)``
~~~~~~~~~~~~~~
``k + k2``, where ``k`` and ``k2`` are floating-point.
``k.sub(k2)``
~~~~~~~~~~~~~
``k - k2``, where ``k`` and ``k2`` are integers.
``k.fsub(k2)``
~~~~~~~~~~~~~~
``k - k2``, where ``k`` and ``k2`` are floating-point.
``k.mul(k2)``
~~~~~~~~~~~~~
``k * k2``, where ``k`` and ``k2`` are integers.
``k.fmul(k2)``
~~~~~~~~~~~~~~
``k * k2``, where ``k`` and ``k2`` are floating-point.
``k.udiv(k2)``
~~~~~~~~~~~~~~
Quotient of unsigned division of ``k`` with ``k2``
``k.sdiv(k2)``
~~~~~~~~~~~~~~
Quotient of signed division of ``k`` with ``k2``
``k.fdiv(k2)``
~~~~~~~~~~~~~~
Quotient of floating point division of ``k`` with ``k2``
``k.urem(k2)``
~~~~~~~~~~~~~~
Reminder of unsigned division of ``k`` with ``k2``
``k.srem(k2)``
~~~~~~~~~~~~~~
Reminder of signed division of ``k`` with ``k2``
``k.frem(k2)``
~~~~~~~~~~~~~~
Reminder of floating point division of ``k`` with ``k2``
``k.and_(k2)``
~~~~~~~~~~~~~~
Bitwise and of ``k`` and ``k2``. Note trailing underscore.
``k.or_(k2)``
~~~~~~~~~~~~~
Bitwise or of ``k`` and ``k2``. Note trailing underscore.
``k.xor(k2)``
~~~~~~~~~~~~~
Bitwise exclusive-or of ``k`` and ``k2``.
``k.icmp(icmp, k2)``
~~~~~~~~~~~~~~~~~~~~
Compare ``k`` with ``k2`` using the predicate ``icmp``. See
`here <comparision.html#icmp>`_ for list of predicates for integer
operands.
``k.fcmp(fcmp, k2)``
~~~~~~~~~~~~~~~~~~~~
Compare ``k`` with ``k2`` using the predicate ``fcmp``. See
`here <comparision.html#fcmp>`_ for list of predicates for real
operands.
``k.shl(k2)``
~~~~~~~~~~~~~
Shift ``k`` left by ``k2`` bits.
``k.lshr(k2)``
~~~~~~~~~~~~~~
Shift ``k`` logically right by ``k2`` bits (new bits are 0s).
``k.ashr(k2)``
~~~~~~~~~~~~~~
Shift ``k`` arithmetically right by ``k2`` bits (new bits are same as
previous sign bit).
``k.gep(indices)``
~~~~~~~~~~~~~~~~~~
GEP, see `LLVM docs <http://www.llvm.org/docs/GetElementPtr.html>`_.
``k.trunc(ty)``
~~~~~~~~~~~~~~~
Truncate ``k`` to a type ``ty`` of lower bitwidth.
``k.sext(ty)``
~~~~~~~~~~~~~~
Sign extend ``k`` to a type ``ty`` of higher bitwidth, while extending
the sign bit.
``k.zext(ty)``
~~~~~~~~~~~~~~
Sign extend ``k`` to a type ``ty`` of higher bitwidth, all new bits are
0s.
``k.fptrunc(ty)``
~~~~~~~~~~~~~~~~~
Truncate floating point constant ``k`` to floating point type ``ty`` of
lower size than k's.
``k.fpext(ty)``
~~~~~~~~~~~~~~~
Extend floating point constant ``k`` to floating point type ``ty`` of
higher size than k's.
``k.uitofp(ty)``
~~~~~~~~~~~~~~~~
Convert an unsigned integer constant ``k`` to floating point constant of
type ``ty``.
``k.sitofp(ty)``
~~~~~~~~~~~~~~~~
Convert a signed integer constant ``k`` to floating point constant of
type ``ty``.
``k.fptoui(ty)``
~~~~~~~~~~~~~~~~
Convert a floating point constant ``k`` to an unsigned integer constant
of type ``ty``.
``k.fptosi(ty)``
~~~~~~~~~~~~~~~~
Convert a floating point constant ``k`` to a signed integer constant of
type ``ty``.
``k.ptrtoint(ty)``
~~~~~~~~~~~~~~~~~~
Convert a pointer constant ``k`` to an integer constant of type ``ty``.
``k.inttoptr(ty)``
~~~~~~~~~~~~~~~~~~
Convert an integer constant ``k`` to a pointer constant of type ``ty``.
``k.bitcast(ty)``
~~~~~~~~~~~~~~~~~
Convert ``k`` to a (equal-width) constant of type ``ty``.
``k.select(cond,k2,k3)``
~~~~~~~~~~~~~~~~~~~~~~~~
Replace value with ``k2`` if the 1-bit integer constant ``cond`` is 1,
else with ``k3``.
``k.extract_element(idx)``
~~~~~~~~~~~~~~~~~~~~~~~~~~
Extract value at ``idx`` (integer constant) from a vector constant
``k``.
``k.insert_element(k2,idx)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Insert value ``k2`` (scalar constant) at index ``idx`` (integer
constant) of vector constant ``k``.
``k.shuffle_vector(k2,mask)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Shuffle vector constant ``k`` based on vector constants ``k2`` and
``mask``.
--------------
# Other Constant Classes
The following subclasses of ``Constant`` do not provide additional
methods, **they serve only to provide richer type information.**
Subclass \| LLVM C++ Class \| Remarks \|
---------\|----------------\|---------\| ``ConstantExpr`` \|
``llvmConstantExpr`` \| A constant expression \|
``ConstantAggregateZero``\ \| ``llvmConstantAggregateZero``\ \| All-zero
constant \| ``ConstantInt``\ \| ``llvmConstantInt``\ \| An integer
constant \| ``ConstantFP``\ \| ``llvmConstantFP``\ \| A floating-point
constant \| ``ConstantArray``\ \| ``llvmConstantArray``\ \| An array
constant \| ``ConstantStruct``\ \| ``llvmConstantStruct``\ \| A
structure constant \| ``ConstantVector``\ \| ``llvmConstantVector``\ \|
A vector constant \| ``ConstantPointerNull``\ \|
``llvmConstantPointerNull``\ \| All-zero pointer constant \|
``UndefValue``\ \| ``llvmUndefValue``\ \| corresponds to ``undef`` of
LLVM IR \|
These types are helpful in ``isinstance`` checks, like so:
{% highlight python %} ti = Type.int(32) k1 = Constant.int(ti, 42) #
int32_t k1 = 42; k2 = Constant.array(ti, [k1, k1]) # int32_t k2[] = {
k1, k1 };
assert isinstance(k1, ConstantInt) assert isinstance(k2, ConstantArray)
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.Constant
:members:
:undoc-members:

158
docs/source/doc/llvm.core.Function.rst
View file

@ -1,158 +0,0 @@
+-------------------------------+
| layout: page |
+-------------------------------+
| title: Function (llvm.core) |
+-------------------------------+
llvm.core.Function
==================
- This will become a table of contents (this text will be scraped).
{:toc}
Base Class
----------
- `llvm.core.GlobalValue <llvm.core.GlobalValue.html>`_
Static Constructors
-------------------
``new(module_obj, func_ty, name)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Create a function named ``name`` of type ``func_ty`` in the module
``module_obj`` and return a ``Function`` object that represents it.
``get(module_obj, name)``
~~~~~~~~~~~~~~~~~~~~~~~~~
Return a ``Function`` object to represent the function named ``name`` in
the module ``module_obj`` or raise ``LLVMException`` if such a function
does not exist.
``get_or_insert(module_obj, func_ty, name)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Similar to ``get``, except that if the function does not exist it is
added first, as though with ``new``.
``intrinsic(module_obj, intrinsic_id, types)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Create and return a ``Function`` object that refers to an intrinsic
function, as described `here <functions.html#intrinsic>`_.
Properties
----------
``calling_convention``
~~~~~~~~~~~~~~~~~~~~~~
The calling convention for the function, as listed
`here <functions.html#callconv>`_.
``collector``
~~~~~~~~~~~~~
A string holding the name of the garbage collection algorithm. See `LLVM
docs <http://www.llvm.org/docs/LangRef.html#gc>`_.
``does_not_throw``
~~~~~~~~~~~~~~~~~~
Setting to True sets the ``ATTR_NO_UNWIND`` attribute, False removes it.
Shortcut to using ``f.add_attribute(ATTR_NO_UNWIND)`` and
``f.remove_attribute(ATTR_NO_UNWIND)``.
``args``
~~~~~~~~
[read-only]
List of `llvm.core.Argument <llvm.core.Argument.html>`_ objects
representing the formal arguments of the function.
``basic_block_count``
~~~~~~~~~~~~~~~~~~~~~
[read-only]
Number of basic blocks belonging to this function. Same as
``len(f.basic_blocks)`` but faster if you just want the count.
``entry_basic_block``
~~~~~~~~~~~~~~~~~~~~~
[read-only]
The `llvm.core.BasicBlock <llvm.core.BasicBlock.html>`_ object
representing the entry basic block for this function, or ``None`` if
there are no basic blocks.
``basic_blocks``
~~~~~~~~~~~~~~~~
[read-only]
List of `llvm.core.BasicBlock <llvm.core.BasicBlock.html>`_ objects
representing the basic blocks belonging to this function.
``intrinsic_id``
~~~~~~~~~~~~~~~~
[read-only]
Returns the ID of the intrinsic if this object represents an intrinsic
instruction. Otherwise 0.
Methods
-------
``delete()``
~~~~~~~~~~~~
Deletes the function from it's module. Do not hold any references to
this object after calling ``delete`` on it.
``append_basic_block(name)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Add a new basic block named ``name``, and return a corresponding
`llvm.core.BasicBlock <llvm.core.BasicBlock.html>`_ object. Note that if
this is not the entry basic block, you'll have to add appropriate branch
instructions from other basic blocks yourself.
``add_attribute(attr)``
~~~~~~~~~~~~~~~~~~~~~~~
Add an attribute ``attr`` to the function, from the set listed above.
``remove_attribute(attr)``
~~~~~~~~~~~~~~~~~~~~~~~~~~
Remove the attribute ``attr`` of the function.
``viewCFG()``
~~~~~~~~~~~~~
Displays the control flow graph using the GraphViz tool.
``viewCFGOnly()``
~~~~~~~~~~~~~~~~~
Displays the control flow graph using the GraphViz tool, but omitting
function bodies.
``verify()``
~~~~~~~~~~~~
Verifies the function. See `LLVM
docs <http://llvm.org/docs/Passes.html#verify>`_.
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.Function
:members:
:undoc-members:

55
docs/source/doc/llvm.core.FunctionType.rst
View file

@ -1,55 +0,0 @@
+-----------------------------------+
| layout: page |
+-----------------------------------+
| title: FunctionType (llvm.core) |
+-----------------------------------+
llvm.core.FunctionType
======================
Base Class
----------
- `llvm.core.Type <llvm.core.Type.html>`_
Properties
----------
``return_type``
~~~~~~~~~~~~~~~
[read-only]
A `Type <llvm.core.Type.html>`_ object, representing the return type of
the function.
``vararg``
~~~~~~~~~~
[read-only]
``True`` if the function is variadic.
``args``
~~~~~~~~
[read-only]
Returns an iterable object that yields `Type <llvm.core.Type.html>`_
objects that represent, in order, the types of the arguments accepted by
the function. Used like this:
.. code-block:: python
func_type = Type.function( Type.int(), [
Type.int(), Type.int() ] ) for arg in func_type.args: assert arg.kind
== TYPE_INTEGER assert arg == Type.int() assert func_type.arg_count
== len(func_type.args)
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.FunctionType
:members:
:undoc-members:

103
docs/source/doc/llvm.core.GlobalValue.rst
View file

@ -1,103 +0,0 @@
+----------------------------------+
| layout: page |
+----------------------------------+
| title: GlobalValue (llvm.core) |
+----------------------------------+
The class ``llvm.core.GlobalValue`` represents module-scope aliases,
variables and functions. Global variables are represented by the
sub-class `llvm.core.GlobalVariable <llvm.core.GlobalVariable.html>`_
and functions by `llvm.core.Function <llvm.core.Function.html>`_.
Global values have the read-write properties ``linkage``, ``section``,
``visibility`` and ``alignment``. Use one of the following constants
(from llvm.core) as values for ``linkage`` (see `LLVM
documentaion <http://www.llvm.org/docs/LangRef.html#linkage>`_ for
details on each):
Value \| Equivalent LLVM Assembly Keyword \|
------\|----------------------------------\| ``LINKAGE_EXTERNAL`` \|
``externally_visible`` \| ``LINKAGE_AVAILABLE_EXTERNALLY`` \|
``available_externally`` \| ``LINKAGE_LINKONCE_ANY`` \| ``linkonce`` \|
``LINKAGE_LINKONCE_ODR`` \| ``linkonce_odr`` \| ``LINKAGE_WEAK_ANY`` \|
``weak`` \| ``LINKAGE_WEAK_ODR`` \| ``weak_odr`` \|
``LINKAGE_APPENDING`` \| ``appending`` \| ``LINKAGE_INTERNAL`` \|
``internal`` \| ``LINKAGE_PRIVATE`` \| ``private`` \|
``LINKAGE_DLLIMPORT`` \| ``dllimport`` \| ``LINKAGE_DLLEXPORT`` \|
``dllexport`` \| ``LINKAGE_EXTERNAL_WEAK`` \| ``extern_weak`` \|
``LINKAGE_GHOST`` \| deprecated -- do not use \| ``LINKAGE_COMMON`` \|
``common`` \| ``LINKAGE_LINKER_PRIVATE`` \| ``linker_private`` \|
The ``section`` property can be assigned strings (like ".rodata"), which
will be used if the target supports it. Visibility property can be set
to one of thse constants (from llvm.core, see also `LLVM
docs <http://www.llvm.org/docs/LangRef.html#visibility>`_):
Value \| Equivalent LLVM Assembly Keyword \|
------\|----------------------------------\| ``VISIBILITY_DEFAULT`` \|
``default`` \| ``VISIBILITY_HIDDEN`` \| ``hidden`` \|
``VISIBILITY_PROTECTED`` \| ``protected`` \|
The ``alignment`` property can be 0 (default), or can be set to a power
of 2. The read-only property ``is_declaration`` can be used to check if
the global is a declaration or not. The module to which the global
belongs to can be retrieved using the ``module`` property (read-only).
llvm.core.GlobalValue
=====================
- This will become a table of contents (this text will be scraped).
{:toc}
Base Class
----------
- `llvm.core.Constant <llvm.core.Constant.html>`_
Properties
----------
``linkage``
~~~~~~~~~~~
The linkage type, takes one of the constants listed above (LINKAGE\_\*).
``section``
~~~~~~~~~~~
A string like ".rodata", indicating the section into which the global is
placed into.
``visibility``
~~~~~~~~~~~~~~
The visibility type, takes one of the constants listed above
(VISIBILITY\_\*).
``alignment``
~~~~~~~~~~~~~
A power-of-2 integer indicating the boundary to align to.
``is_declaration``
~~~~~~~~~~~~~~~~~~
[read-only]
``True`` if the global is a declaration, ``False`` otherwise.
``module``
~~~~~~~~~~
[read-only]
::
The module object to which this global belongs to.
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.GlobalValue
:members:
:undoc-members:

49
docs/source/doc/llvm.core.GlobalVariable.rst
View file

@ -1,49 +0,0 @@
+-------------------------------------+
| layout: page |
+-------------------------------------+
| title: GlobalVariable (llvm.core) |
+-------------------------------------+
llvm.core.GlobalVariable
========================
Global variables (``llvm.core.GlobalVariable``) are subclasses of
`llvm.core.GlobalValue <llvm.core.GlobalValue.html>`_ and represent
module-level variables. These can have optional initializers and can be
marked as constants. Global variables can be created either by using the
``add_global_variable`` method of the `Module <llvm.core.Module.html>`_
class, or by using the static method ``GlobalVariable.new``.
.. code-block:: python
# create a global variable using
add_global_variable method gv1 =
module_obj.add_global_variable(Type.int(), "gv1")
# or equivalently, using a static constructor method
gv2 = GlobalVariable.new(module_obj, Type.int(), "gv2") {% endhighlight
%}
Existing global variables of a module can be accessed by name using
``module_obj.get_global_variable_named(name)`` or
``GlobalVariable.get``. All existing global variables can be enumerated
via iterating over the property ``module_obj.global_variables``.
{% highlight python %} # retrieve a reference to the global variable
gv1, # using the get_global_variable_named method gv1 =
module_obj.get_global_variable_named("gv1")
# or equivalently, using the static ``get`` method:
gv2 = GlobalVariable.get(module_obj, "gv2")
# list all global variables in a module
for gv in module_obj.global_variables: print gv.name, "of type",
gv.type
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.GlobalVariable
:members:
:undoc-members:

249
docs/source/doc/llvm.core.Instruction.rst
View file

@ -1,249 +0,0 @@
+----------------------------------+
| layout: page |
+----------------------------------+
| title: Instruction (llvm.core) |
+----------------------------------+
An ``llvm.core.Instruction`` object represents an LLVM instruction. This
class is the root of a small hierarchy:
::
Instruction
CallOrInvokeInstruction
PHINode
SwitchInstruction
CompareInstruction
Instructions are not created directly, but via a builder. The builder
both creates instructions and adds them to a basic block at the same
time. One way of getting instruction objects are from basic blocks.
Being derived from `llvm.core.User <llvm.core.User.html>`_, the
instruction is-a user, i.e., an instruction in turn uses other values.
The values an instruction uses are its operands. These may be accessed
using ``operands`` property from the
`llvm.core.User <llvm.core.User.html>`_ base.
The name of the instruction (like ``add``, ``mul`` etc) can be got via
the ``opcode_name`` property. The ``basic_block`` property gives the
basic block to which the instruction belongs to. Note that llvmpy does
not allow free-standing instruction objects (i.e., all instructions are
created contained within a basic block).
Classes of instructions can be got via the properties ``is_terminator``,
``is_binary_op``, ``is_shift`` etc. See below for the full list.
- This will become a table of contents (this text will be scraped).
{:toc}
llvm.core.Instruction
=====================
Base Class
----------
- `llvm.core.User <llvm.core.User.html>`_
Properties
----------
``basic_block``
~~~~~~~~~~~~~~~
[read-only] The basic block to which this instruction belongs to.
``is_terminator``
~~~~~~~~~~~~~~~~~
[read-only] True if the instruction is a terminator instruction.
``is_binary_op``
~~~~~~~~~~~~~~~~
[read-only] True if the instruction is a binary operator.
``is_shift``
~~~~~~~~~~~~
[read-only] True if the instruction is a shift instruction.
``is_cast``
~~~~~~~~~~~
[read-only] True if the instruction is a cast instruction.
``is_logical_shift``
~~~~~~~~~~~~~~~~~~~~
[read-only] True if the instruction is a logical shift instruction.
``is_arithmetic_shift``
~~~~~~~~~~~~~~~~~~~~~~~
[read-only] True if the instruction is an arithmetic shift instruction.
``is_associative``
~~~~~~~~~~~~~~~~~~
[read-only] True if the instruction is associative.
``is_commutative``
~~~~~~~~~~~~~~~~~~
[read-only] True if the instruction is commutative.
``is_volatile``
~~~~~~~~~~~~~~~
[read-only] True if the instruction is a volatile load or store.
``opcode``
~~~~~~~~~~
[read-only] The numeric opcode value of the instruction. Do not rely on
the absolute value of this number, it may change with LLVM version.
``opcode_name``
~~~~~~~~~~~~~~~
[read-only] The name of the instruction, like ``add``, ``sub`` etc.
--------------
llvm.core.CallOrInvokeInstruction
=================================
The ``llvm.core.CallOrInvokeInstruction`` is a subclass of
``llvm.core.Instruction``, and represents either a ``call`` or an
``invoke`` instruction.
Base Class
----------
- ``llvm.core.Instruction``
Properties
----------
``calling_convention`` Get or set the calling convention. See
`here <functions.html#callconv>`_ for possible values.
Methods
-------
``add_parameter_attribute(idx, attr)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Add an attribute ``attr`` to the ``idx``-th argument. See
`here <llvm.core.Argument.html>`_ for possible values of ``attr``.
``remove_parameter_attribute(idx, attr)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Remove an attribute ``attr`` from the ``idx``-th argument. See
`here <llvm.core.Argument.html>`_ for possible values of ``attr``.
``set_parameter_alignment(idx, align)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Set the alignment of the ``idx``-th argument to ``align``. ``align``
should be a power of two.
--------------
llvm.core.PHINode
=================
The ``llvm.core.PHINode`` is a subclass of ``llvm.core.Instruction``,
and represents the ``phi`` instruction. When created (using
``Builder.phi``) the phi node contains no incoming blocks (nor their
corresponding values). To add an incoming arc to the phi node, use the
``add_incoming`` method, which takes a source block
(`llvm.core.BasicBlock <llvm.core.BasicBlock.html>`_ object) and a value
(object of `llvm.core.Value <llvm.core.Value.html>`_ or of a class
derived from it) that the phi node will take on if control branches in
from that block.
Base Class
----------
- ``llvm.core.Instruction``
Properties
----------
``incoming_count`` [read-only] The number of incoming arcs for this phi
node.
Methods
-------
``add_incoming(value, block)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Add an incoming arc, from the
`llvm.core.BasicBlock <llvm.core.BasicBlock.html>`_ object ``block``,
with the corresponding value ``value``. ``value`` should be an object of
`llvm.core.Value <llvm.core.Value.html>`_ (or of a descendent class).
``get_incoming_value(idx)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Returns the ``idx``-th incoming arc's value.
``get_incoming_block(idx)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Returns the ``idx``-th incoming arc's block.
llvm.core.SwitchInstruction # {#switchinstr}
============================================
(TODO describe)
Base Class
----------
- ``llvm.core.Instruction``
Methods
-------
``add_case(const, block)``
~~~~~~~~~~~~~~~~~~~~~~~~~~
Add another case to the switch statement. When the expression being
evaluated equals ``const``, then control branches to ``block``. Here
``const`` must be of type
`llvm.core.ConstantInt <llvm.core.Constant.html>`_.
--------------
llvm.core.CompareInstruction
============================
(TODO describe)
Base Class
----------
- ``llvm.core.Instruction``
Properties
----------
``predicate``
~~~~~~~~~~~~~
[read-only]
The predicate of the compare instruction, one of the ``ICMP_*`` or
``FCMP_*`` constants.
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.Instruction
:members:
:undoc-members:

30
docs/source/doc/llvm.core.IntegerType.rst
View file

@ -1,30 +0,0 @@
+----------------------------------+
| layout: page |
+----------------------------------+
| title: IntegerType (llvm.core) |
+----------------------------------+
llvm.core.IntegerType
=====================
Base Class
----------
- `llvm.core.Type <llvm.core.Type.html>`_
Properties
----------
``width``
~~~~~~~~~
[read-only]
The width of the integer type, in number of bits.
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.IntegerType
:members:
:undoc-members:

29
docs/source/doc/llvm.core.Module.rst
View file

@ -1,29 +0,0 @@
+-----------------------------+
| layout: page |
+-----------------------------+
| title: Module (llvm.core) |
+-----------------------------+
llvm.core.Module
================
Modules are top-level container objects. You need to create a module
object first, before you can add global variables, aliases or functions.
Modules are created using the static method ``Module.new``:
.. code-block:: python
#!/usr/bin/env python
from llvm import *
from llvm.core import *
# create a module
my_module = Module.new('my_module')
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.Module
:members:
:undoc-members:

38
docs/source/doc/llvm.core.PointerType.rst
View file

@ -1,38 +0,0 @@
+----------------------------------+
| layout: page |
+----------------------------------+
| title: PointerType (llvm.core) |
+----------------------------------+
llvm.core.PointerType
=====================
Base Class
----------
- `llvm.core.Type <llvm.core.Type.html>`_
Properties
----------
``address_space``
~~~~~~~~~~~~~~~~~
[read-only]
The address space of the pointer.
``pointee``
~~~~~~~~~~~
[read-only]
A `Type <llvm.core.Type.html>`_ object representing the type of the
value pointed to.
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.PointerType
:members:
:undoc-members:

8
docs/source/doc/llvm.core.StructType.rst
View file

@ -1,8 +0,0 @@
llvm.core.StructType
====================
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.StructType
:members:
:undoc-members:

135
docs/source/doc/llvm.core.Type.rst
View file

@ -1,135 +0,0 @@
+---------------------------+
| layout: page |
+---------------------------+
| title: Type (llvm.core) |
+---------------------------+
llvm.core.Type
==============
- This will become a table of contents (this text will be scraped).
{:toc}
Static Constructors
-------------------
``int(n)``
~~~~~~~~~~
Create an integer type of bit width ``n``.
``float()``
~~~~~~~~~~~
Create a 32-bit floating point type.
``double()``
~~~~~~~~~~~~
Create a 64-bit floating point type.
``x86_fp80()``
~~~~~~~~~~~~~~
Create a 80-bit 80x87-style floating point type.
``fp128()``
~~~~~~~~~~~
Create a 128-bit floating point type (112-bit mantissa).
``ppc_fp128()``
~~~~~~~~~~~~~~~
Create a 128-bit float (two 64-bits).
``function(ret, params, vararg=False)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Create a function type, having the return type ``ret`` (must be a
``Type``), accepting the parameters ``params``, where ``params`` is an
iterable, that yields ``Type`` objects representing the type of each
function argument in order. If ``vararg`` is ``True``, function is
variadic.
``struct(eltys, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~
Create an unpacked structure. ``eltys`` is an iterable, that yields
``Type`` objects representing the type of each element in order.
If ``name`` is evaulates ``True`` (not empty), create an *identified
structure*; otherwise, create a *literal structure* by default.
``packed_struct(eltys, name='')``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Like ``struct(eltys)``, but creates a packed struct.
``array(elty, count)``
~~~~~~~~~~~~~~~~~~~~~~
Creates an array type, holding ``count`` elements, each of type ``elty``
(which should be a ``Type``).
``pointer(pty, addrspc=0)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Create a pointer to type ``pty`` (which should be a ``Type``).
``addrspc`` is an integer that represents the address space of the
pointer (see LLVM docs or ask on llvm-dev for more info).
``void()``
~~~~~~~~~~
Creates a void type. Used for function return types.
``label()``
~~~~~~~~~~~
Creates a label type.
``opaque(name)``
~~~~~~~~~~~~~~~~
Opaque `StructType <llvm.core.StructType.html>`_, used for creating
self-referencing types.
Properties
----------
``kind``
~~~~~~~~
[read-only]
A value (enum) representing the "type" of the object. It will be one of
the following constants defined in ``llvm.core``:
.. code-block:: python
# Warning: do not rely on actual numerical
values! TYPE_VOID = 0 TYPE_FLOAT = 1 TYPE_DOUBLE = 2 TYPE_X86_FP80
= 3 TYPE_FP128 = 4 TYPE_PPC_FP128 = 5 TYPE_LABEL = 6 TYPE_INTEGER =
7 TYPE_FUNCTION = 8 TYPE_STRUCT = 9 TYPE_ARRAY = 10 TYPE_POINTER =
11 TYPE_OPAQUE = 12 TYPE_VECTOR = 13 TYPE_METADATA = 14 TYPE_UNION =
15
Example:
^^^^^^^^
.. code-block:: python
assert Type.int().kind == TYPE_INTEGER assert
Type.void().kind == TYPE_VOID
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.Type
:members:
:undoc-members:

46
docs/source/doc/llvm.core.User.rst
View file

@ -1,46 +0,0 @@
+---------------------------+
| layout: page |
+---------------------------+
| title: User (llvm.core) |
+---------------------------+
``User``-s are values that refer to other values. The values so refered
can be retrived by the properties of ``User``. This is the reverse of
the ``Value.uses``. Together these can be used to traverse the use-def
chains of the SSA.
--------------
llvm.core.User # {#user}
========================
Base Class
----------
- `llvm.core.Value <llvm.core.Value.html>`_
Properties
----------
``operands``
~~~~~~~~~~~~
[read-only]
The list of operands (values, of type
`llvm.core.Value <llvm.core.Value.html>`_) that this value refers to.
``operand_count``
~~~~~~~~~~~~~~~~~
[read-only]
The number of operands that this value referes to. Same as
``len(uses.operands)`` but faster if you just want the count.
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.User
:members:
:undoc-members:

72
docs/source/doc/llvm.core.Value.rst
View file

@ -1,72 +0,0 @@
+----------------------------+
| layout: page |
+----------------------------+
| title: Value (llvm.core) |
+----------------------------+
llvm.core.Value
===============
- This will become a table of contents (this text will be scraped).
{:toc}
Properties
----------
``name``
~~~~~~~~
The name of the value.
``type``
~~~~~~~~
[read-only]
An ``llvm.core.Type`` object representing the type of the value.
``uses``
~~~~~~~~
[read-only]
The list of values (``llvm.core.Value``) that use this value.
``use_count``
~~~~~~~~~~~~~
[read-only]
The number of values that use (refer) this value. Same as
``len(val.uses)`` but faster if you just want the count.
``value_id``
~~~~~~~~~~~~
[read-only]
Returns ``llvmValuegetValueID()``. Refer LLVM documentation for more
info.
Special Methods
---------------
``__str__``
~~~~~~~~~~~
``Value`` objects can be stringified into it's LLVM assembly language
representation.
``__eq__``
~~~~~~~~~~
``Value`` objects can be compared for equality. Internally, this
converts both arguments into their LLVM assembly representations and
compares the resultant strings.
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.Value
:members:
:undoc-members:

38
docs/source/doc/llvm.core.VectorType.rst
View file

@ -1,38 +0,0 @@
+---------------------------------+
| layout: page |
+---------------------------------+
| title: VectorType (llvm.core) |
+---------------------------------+
llvm.core.VectorType
====================
Base Class
----------
- `llvm.core.Type <llvm.core.Type.html>`_
Properties
----------
``element``
~~~~~~~~~~~
[read-only]
A `Type <llvm.core.Type.html>`_ object representing the type of the
element of the vector.
``count``
~~~~~~~~~
[read-only]
The number of elements in the vector.
Automatically Generated Documentation
-------------------------------------
.. autoclass:: llvm.core.VectorType
:members:
:undoc-members:

56
docs/source/doc/llvm.ee.EngineBuilder.rst
View file

@ -1,56 +0,0 @@
+----------------------------------+
| layout: page |
+----------------------------------+
| title: EngineBuilder (llvm.ee) |
+----------------------------------+
llvm.ee.EngineBuilder
=====================
A convenient class for building
`llvm.ee.ExecutionEngine <llvm.ee.ExecutionEngine.html>`_. Each
``EngineBuilder`` instance can only create one ``ExecutionEngine``.
Methods
-------
``create(self)``
~~~~~~~~~~~~~~~~
Create and return a new
`ExecutionEngine <llvm.ee.ExecutionEngine.html>`_ instance.
Raise ``llvm.LLVMException`` if the builder cannot create an
``ExecutionEngine`` base on the given configuration.
``force_interpreter(self)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Force the output the output ``ExecutionEngine`` to be an LLVM IR
interpreter.
``force_jit(self)``
~~~~~~~~~~~~~~~~~~~
Force the output the output ``ExecutionEngine`` to be a JIT engine.
``opt(self, level)``
~~~~~~~~~~~~~~~~~~~~
Set the code generation optimization level for a JIT engine. Valid value
of ``level`` is 0-3, inclusive. The default setting is 2. To use vector
instructions, such as SSE on Intel processors, ``level`` must be 3
(aggressive).
Static Factory Methods
----------------------
``new(module)``
~~~~~~~~~~~~~~~
Create a new EngineBuilder. ``module`` must be a
`llvm.core.Module <llvm.core.Module.html>`_ instance. Its ownership is
transferred to the resulting
`ExecutionEngine <llvm.ee.ExecutionEngine.html>`_. Therefore, it is
impossible to create more than one ``ExecutionEngine`` with a single
``EngineBuilder``

62
docs/source/doc/llvm.ee.ExecutionEngine.rst
View file

@ -1,62 +0,0 @@
+------------------------------------+
| layout: page |
+------------------------------------+
| title: ExecutionEngine (llvm.ee) |
+------------------------------------+
llvm.ee.ExecutionEngine
=======================
Methods
-------
``add_module(self, module)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Add a new module to the ExecutionEngine. The ownership is of ``module``
is transferred. When the ``ExecutionEngine`` is destroyed, the module is
destroyed.
``free_machine_code_for(self, fn)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Release memory used for the machine code generated for the function
``fn``.
``get_pointer_to_function(self, fn)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Obtain the pointer to the function ``fn``. This forces the
ExecutionEngine to generate the machine code in lazy mode.
If ``fn`` is not defined, ``ExecutionEngine`` will lookup the symbol
through ``dlsym``.
The returned function pointer can be wrapped as a ``ctypes`` function.
``remove_module(self, module)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Remove the ``module``.
``run_function(self, fn, args)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Execute the function ``fn`` with an iterable of arguments ``args`` which
are of ``GenericValue``. This method returns whatever that is returned
by ``fn`` as a ``GenericValue``.
``run_static_ctors(self)``
~~~~~~~~~~~~~~~~~~~~~~~~~~
``run_static_dtors(self)``
~~~~~~~~~~~~~~~~~~~~~~~~~~
Properties
----------
``target_data``
~~~~~~~~~~~~~~~
Access the `TargetData <llvm.ee.TargetData.html>`_ instance associated
with the ``ExecutionEngine``.

64
docs/source/doc/llvm.ee.GenericValue.rst
View file

@ -1,64 +0,0 @@
+---------------------------------+
| layout: page |
+---------------------------------+
| title: GenericValue (llvm.ee) |
+---------------------------------+
llvm.ee.GenericValue
====================
- This will become a table of contents (this text will be scraped).
{:toc}
Methods
-------
``as_int(self)``
~~~~~~~~~~~~~~~~
Return the value of this ``GenericValue`` instance as an unsigned
integer
``as_int_signed(self)``
~~~~~~~~~~~~~~~~~~~~~~~
Return the value of this ``GenericValue`` instance as a signed integer.
``as_pointer(self)``
~~~~~~~~~~~~~~~~~~~~
Return the value of this ``GenericValue`` instance as a pointer. The
type of the return value is ``int``.
``as_real(self, ty)``
~~~~~~~~~~~~~~~~~~~~~
Return the value of this ``GenericValue`` instance as a real number
which type is specified by ``ty``. ``ty`` must be a
`Type <llvm.core.Type.html>`_ instance of a real number type.
Static Factory Methods
----------------------
``int(ty, intval)``
~~~~~~~~~~~~~~~~~~~
Create a ``GenericValue`` instance with a ``int`` value, which is
zero-extended if necessary. The type of the value is specified by
``ty``, which is a `Type <llvm.core.Type.html>`_ instance.
``int_signed(ty, intval)``
~~~~~~~~~~~~~~~~~~~~~~~~~~
Create a ``GenericValue`` instance with a ``int`` value, which is
sign-extended if necessary. The type of the value is specified by
``ty``, which is a `Type <llvm.core.Type.html>`_ instance.
``pointer(ty, addr)`` or ``pointer(addr)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Create a ``GenericValue`` instance with a ``int`` value, which is
representing a pointer value.
The two argument version is **deprecated**. The old code never used
``ty`` anyway.

70
docs/source/doc/llvm.ee.TargetData.rst
View file

@ -1,70 +0,0 @@
+-------------------------------+
| layout: page |
+-------------------------------+
| title: TargetData (llvm.ee) |
+-------------------------------+
llvm.ee.TargetData
==================
- This will become a table of contents (this text will be scraped).
{:toc}
Methods
-------
``abi_alignment(self, ty)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Returns the minimum ABI-required alignment for the specified type
``ty``.
``abi_size(self, ty)``
~~~~~~~~~~~~~~~~~~~~~~
``callframe_alignment(self, ty)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Returns the minimum ABI-required alignment for the specified type ``ty``
when it is part of a call frame.
``element_at_offset(self, ty, ofs)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``offset_of_element(self, ty, el)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``preferred_alignment(self, ty_or_gv)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``size(self, ty)``
~~~~~~~~~~~~~~~~~~
``store_size(self, ty)``
~~~~~~~~~~~~~~~~~~~~~~~~
``__str__(self)``
~~~~~~~~~~~~~~~~~
Returns the string representation.
Static Factory Methods
----------------------
``new(strrep)``
~~~~~~~~~~~~~~~
Construct a new ``TargetData`` instance from the string representation
Properties
----------
``byte_order``
~~~~~~~~~~~~~~
``pointer_size``
~~~~~~~~~~~~~~~~
``target_integer_type``
~~~~~~~~~~~~~~~~~~~~~~~

42
docs/source/doc/llvm.passes.FunctionPassManager.rst
View file

@ -1,42 +0,0 @@
+--------------------------------------------+
| layout: page |
+--------------------------------------------+
| title: FunctionPassManager (llvm.passes) |
+--------------------------------------------+
llvm.passes.FunctionPassManager
===============================
Base Classes
------------
- `llvm.passes.PassManager <llvm.passes.PassManager.html>`_
Methods
-------
``finalize(self)``
~~~~~~~~~~~~~~~~~~
Finalizes all associated function passes in the LLVM system.
Beware that this destroys all associated passes even if another pass
manager is using those passes. This may result is a segfault.
``initialize(self)``
~~~~~~~~~~~~~~~~~~~~
Initializes all associated function passes in the LLVM system.
``run(self, fn)``
~~~~~~~~~~~~~~~~~
Run all passes on the given function ``fn``.
Static Factory Methods
----------------------
``new(module)``
~~~~~~~~~~~~~~~
Create a ``FunctionPassManager`` instance for a given ``module``.

29
docs/source/doc/llvm.passes.PassManager.rst
View file

@ -1,29 +0,0 @@
+------------------------------------+
| layout: page |
+------------------------------------+
| title: PassManager (llvm.passes) |
+------------------------------------+
llvm.passes.PassManager
=======================
Methods
-------
``add(self, tgt_data_or_pass_id)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Add a pass by its ID. A pass IDs are defined as ``PASS_*``.
``run(self, module)``
~~~~~~~~~~~~~~~~~~~~~
Run all passes on the given ``module``.
Static Factory Methods
----------------------
``new()``
~~~~~~~~~
Creates a new ``PassManager`` instance.

72
docs/source/doc/llvm.passes.PassManagerBuilder.rst
View file

@ -1,72 +0,0 @@
+-------------------------------------------+
| layout: page |
+-------------------------------------------+
| title: PassManagerBuilder (llvm.passes) |
+-------------------------------------------+
llvm.passes.PassManagerBuilder
==============================
Provide a simple API to populate pass managers for language like C/C++.
Refer to `LLVM API
Documentation <http://llvm.org/docs/doxygen/html/classllvm_1_1PassManagerBuilder.html>`_
for detail.
Methods
-------
``populate(self, pm)``
~~~~~~~~~~~~~~~~~~~~~~
Populate a `FunctionPassManager <llvm.passes.FunctionPassManager.html>`_
or `PassManager <llvm.passes.PassManager.html>`_ given as ``pm``.
``use_inliner_with_threshold(self, threshold)``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Use an inliner pass with the given ``threshold``.
Properties
----------
The following properties can be overriden to customize how pass managers
are populated.
``disable_simplify_lib_calls``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Boolean. Default is ``False``.
``disable_unit_at_a_time``
~~~~~~~~~~~~~~~~~~~~~~~~~~
Boolean. Default is ``False``.
``disable_unroll_loops``
~~~~~~~~~~~~~~~~~~~~~~~~
Boolean. Default is ``False``.
``opt_level``
~~~~~~~~~~~~~
Default is ``2``. Valid values are 0-3. Corresponds to O0, O1, O2, O3 as
in C/C++ optimization options.
``size_level``
~~~~~~~~~~~~~~
Default is ``0``.
``vectorize``
~~~~~~~~~~~~~
Default is ``False``.
Static Factory Methods
----------------------
``new()``
~~~~~~~~~
Creates a new ``PassManagerBuilder`` instance.

12
docs/source/doc/llvm_cbuilder.rst
View file

@ -1,12 +0,0 @@
llvm_cbuilder
=============
llvm_cbuilder is a set of Python-contexts you can use to write C-like
constructs in Python which generates llvmpy code directly.
Similar to llpython it allows you to build llvm IR without using the
llvmpy interface directly.
Look in the llvm_cbuilder tests directory for examples of use.

239
docs/source/doc/llvm_concepts.rst
View file

@ -1,239 +0,0 @@
********************
LLVM Concepts
********************
This section explains a few concepts related to LLVM, not specific to
llvmpy.
.. toctree::
:hidden:
Intermediate Representation
===========================
The intermediate representation, or IR for short, is an in-memory data
structure that represents executable code. The IR data structures allow
for creation of types, constants, functions, function arguments,
instructions, global variables and so on. For example, to create a
function *sum* that takes two integers and returns their sum, we need to
follow these steps:
- create an integer type *ti* of required bitwidth
- create a function type *tf* which takes two *ti* -s and returns
another *ti*
- create a function of type *tf* named *sum*
- add a *basic block* to the function
- using a helper object called an *instruction builder*, add two
instructions into the basic block:
- an instruction to add the two
arguments and store the result into a temporary variable
- a return
instruction to return the value of the temporary variable
(A basic block is a block of instructions.)
LLVM has it's own instruction set; the instructions used above (*add*
and *ret*) are from this set. The LLVM instructions are at a higher
level than the usual assembly language; for example there are
instructions related to variable argument handling, exception handling,
and garbage collection. These allow high-level languages to be
represented cleanly in the IR.
SSA Form and PHI Nodes
======================
All LLVM instructions are represented in the *Static Single Assignment*
(SSA) form. Essentially, this means that any variable can be assigned to
only once. Such a representation facilitates better optimization, among
other benefits.
A consequence of single assignment are PHI (Φ) nodes. These are required
when a variable can be assigned a different value based on the path of
control flow. For example, the value of *b* at the end of execution of
the snippet below:
.. code-block:: c
a = 1;
if (v < 10)
a = 2;
b = a;
cannot be determined statically. The value of '2' cannot be assigned to
the 'original' *a*, since *a* can be assigned to only once. There are
two *a* 's in there, and the last assignment has to choose between which
version to pick. This is accomplished by adding a PHI node:
.. code-block:: c
a1 = 1;
if (v < 10)
a2 = 2;
b = PHI(a1, a2);
The PHI node selects *a1* or *a2*, depending on where the control
reached the PHI node. The argument *a1* of the PHI node is associated
with the block *"a1 = 1;"* and *a2* with the block *"a2 = 2;"*.
PHI nodes have to be explicitly created in the LLVM IR. Accordingly the
LLVM instruction set has an instruction called *phi*.
LLVM Assembly Language
======================
The LLVM IR can be represented offline in two formats
- a textual, human-readable form, similar to assembly language, called
the LLVM assembly language (files with .ll extension)
- a binary form, called the LLVM bitcode (files with .bc extension)
All three formats (the in-memory IR, the LLVM assembly language and the
LLVM bitcode) represent the *same* information. Each format can be
converted into the other two formats (using LLVM APIs).
The `LLVM demo page <http://www.llvm.org/demo/>`_ lets you type in C or
C++ code, converts it into LLVM IR and outputs the IR as LLVM assembly
language code.
Just to get a feel of the LLVM assembly language, here's a function in
C, and the corresponding LLVM assembly (as generated by the demo page):
.. code-block:: c
/* compute sum of 1..n */
unsigned sum(unsigned n) {
if (n == 0)
return 0;
else
return n + sum(n-1);
}
The corresponding LLVM assembly:
.. code-block:: llvm
; ModuleID = '/tmp/webcompile/_7149_0.bc'
target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
target triple = "x86_64-linux-gnu"
define i32 @sum(i32 %n) nounwind readnone {
entry:
%0 = icmp eq i32 %n, 0 ; [#uses=1]
br i1 %0, label %bb2, label %bb1
bb1: ; preds = %entry
%1 = add i32 %n, -1 ; [#uses=2]
%2 = icmp eq i32 %1, 0 ; [#uses=1]
br i1 %2, label %sum.exit, label %bb1.i
bb1.i: ; preds = %bb1
%3 = add i32 %n, -2 ; [#uses=1]
%4 = tail call i32 @sum(i32 %3) nounwind ; [#uses=1]
%5 = add i32 %4, %1 ; [#uses=1]
br label %sum.exit
sum.exit: ; preds = %bb1.i, %bb1
%6 = phi i32 [ %5, %bb1.i ], [ 0, %bb1 ] ; [#uses=1]
%7 = add i32 %6, %n ; [#uses=1]
ret i32 %7
bb2: ; preds = %entry
ret i32 0
}
Note the usage of SSA form. The long string called ``target datalayout``
is a specification of the platform ABI (like endianness, sizes of types,
alignment etc.).
The `LLVM Language Reference <http://www.llvm.org/docs/LangRef.html>`_
defines the LLVM assembly language including the entire instruction set.
Modules
=======
`Modules <./llvm.core.Module.html>`_, in the LLVM IR, are similar to a
single *C* language source file (.c file). A module contains:
- functions (declarations and definitions)
- global variables and constants
- global type aliases for structures
Modules are top-level containers; all executable code representation is
contained within modules. Modules may be combined (linked) together to
give a bigger resultant module. During this process LLVM attempts to
reconcile the references between the combined modules.
Optimization and Passes
=======================
LLVM provides quite a few optimization algorithms that work on the IR.
These algorithms are organized as *passes*. Each pass does something
specific, like combining redundant instructions. Passes need not always
optimize the IR, it can also do other operations like inserting
instrumentation code, or analyzing the IR (the result of which can be
used by passes that do optimizations) or even printing call graphs.
This LLVM `documentation page <http://www.llvm.org/docs/Passes.html>`_
describes all the available passes, and what they do.
LLVM does not automatically choose to run any passes, anytime. Passes
have to be explicitly selected and run on each module. This gives you
the flexibility to choose transformations and optimizations that are
most suitable for the code in the module.
There is an LLVM binary called
`opt <http://www.llvm.org/cmds/opt.html>`_, which lets you run passes on
bitcode files from the command line. You can write your own passes (in
C/C++, as a shared library). This can be loaded and executed by +opt+.
(Although llvmpy does not allow you to write your own passes, it does
allow you to navigate the entire IR at any stage, and perform any
transforms on it as you like.)
A "pass manager" is responsible for loading passes, selecting the
correct objects to run them on (for example, a pass may work only on
functions, individually) and actually runs them. ``opt`` is a
command-line wrapper for the pass manager.
LLVM defines two kinds of pass managers:
- The
`FunctionPassManager <http://llvm.org/docs/doxygen/html/classllvm_1_1FunctionPassManager.html>`_
manages function or basic-block passes. These lighter weight passes
can be used immediately after each generated function to reduce
memory footprint.
- The
`PassManager <http://llvm.org/docs/doxygen/html/classllvm_1_1PassManager.html>`_
manages module passes for optimizing the entire module.
Bitcode
=======
LLVM IR can be represented as a bitcode format for disk storage. It is
`suitable for fast loading by JIT
compiler <http://llvm.org/docs/LangRef.html#introduction>`_. See `LLVM
documentation <http://llvm.org/docs/BitCodeFormat.html>`_ for detail
about the bitcode format.
Execution Engine, JIT and Interpreter
=====================================
The *execution engine* implements execution of LLVM IR through an
interpreter or a JIT dynamic compiler. An *execution engine* can contain
multiple modules.
**Note**
Inter-module reference is not possible. That is module ``A`` cannot
call a function in module ``B``, directly.

9
docs/source/doc/llvmcore.rst
View file

@ -1,9 +0,0 @@
********************************
llvm.core
********************************
.. toctree::
:titlesonly:
:glob:
llvm.core.*

9
docs/source/doc/llvmee.rst
View file

@ -1,9 +0,0 @@
********************************
llvm.ee
********************************
.. toctree::
:titlesonly:
:glob:
llvm.ee.*

9
docs/source/doc/llvmpasses.rst
View file

@ -1,9 +0,0 @@
********************************
llvm.passes
********************************
.. toctree::
:titlesonly:
:glob:
llvm.passes.*

89
docs/source/doc/llvmpy_package.rst
View file

@ -1,89 +0,0 @@
***********************
The llvmpy Package
***********************
The llvmpy is a Python package, consisting of 6 modules, that wrap over
enough LLVM APIs to allow the implementation of your own compiler/VM
backend in pure Python. If you're come this far, you probably know why
this is a good idea.
Out of the 6 modules, one is an "extension" module (i.e., it is written
in C), and another one is a small private utility module, which leaves 4
public modules. These are:
- *llvm* -- top-level package, common classes (like exceptions)
- *llvm.core* -- IR-related APIs
- *llvm.ee* -- execution engine related APIs
- *llvm.passes* -- pass manager and passes related APIs
The modules contain only classes and (integer) constants. Mostly simple
Python constructs are used (deliberately) --
`property() <http://docs.python.org/lib/built-in-funcs.html>`_ and
`property
decorators <http://wiki.python.org/moin/PythonDecoratorLibrary>`_ are
probably the most exotic animals around. All classes are "new style"
classes. The APIs are designed to be navigable (and guessable!) once you
know a few conventions. These conventions are highlighted in the
sections below.
Here is a quick overview of the contents of each package:
llvm
----
- LLVMException -- exception class (currently the only one)
llvm.core
---------
- `Module <llvm.core.Module.html>`_ -- represents an LLVM Module
- `Type <types.html>`_ -- represents an LLVM Type
- `Value <values.html>`_ -- represents an LLVM Value, including:
globals, constants, variables, arguments, functions, instructions,
etc..
- `BasicBlock <llvm.core.BasicBlock.html>`_ -- another derived of
Value, represents an LLVM basic block
- `Builder <llvm.core.Builder.html>`_ -- used for creating
instructions, wraps LLVM IRBuilder helper class
- constants *TYPE\_\** that represents various types
- constants *CC\_\** that represent calling conventions
- constants *ICMP\_\** and *FCMP\_\** that represent integer and real
comparison predicates (like less than, greater than etc.)
- constants *LINKAGE\_\** that represent linkage of symbols (external,
internal etc.)
- constants *VISIBILITY\_\** that represents visibility of symbols
(default, hidden, protected)
- constants *ATTR\_\** that represent function parameter attributes
llvm.ee
-------
- `ExecutionEngine <llvm.ee.ExecutionEngine.html>`_ -- represents an
execution engine (which can be an either an interpreter or a JIT)
- `TargetData <llvm.ee.TargetData.html>`_ -- represents the ABI of the
target platform (details like sizes and alignment of primitive types,
endinanness etc)
llvm.passes
-----------
- `PassManager <llvm.passes.PassManager.html>`_ -- represents an LLVM
pass manager
- `FunctionPassManager <llvm.passes.FunctionPassManager.html>`_ --
represents an LLVM function pass manager
- constants *PASS\_\** that represent various passes
A note on the importing of these modules
----------------------------------------
Pythonically, modules are imported with the statement
``import llvm.core``. However, you might find it more convenient to
import llvmpy modules thus:
.. code-block:: python
from llvm import *
from llvm.core import *
from llvm.ee import *
from llvm.passes import *

123
docs/source/doc/types.rst
View file

@ -1,123 +0,0 @@
+----------------+
| layout: page |
+----------------+
| title: Types |
+----------------+
Types are what you think they are. A instance of
`llvm.core.Type <llvm.core.Type.html>`_, or one of its derived classes,
represent a type. llvmpy does not use as many classes to represent
types as does LLVM itself. Some types are represented using
`llvm.core.Type <llvm.core.Type.html>`_ itself and the rest are
represented using derived classes of
`llvm.core.Type <llvm.core.Type.html>`_. As usual, an instance is
created via one of the static methods of `Type <llvm.core.Type.html>`_.
These methods return an instance of either
`llvm.core.Type <llvm.core.Type.html>`_ itself or one of its derived
classes.
The following table lists all the available types along with the static
method which has to be used to construct it and the name of the class
whose object is actually returned by the static method.
Name \| Constructor Method \| Class \|
-----\|:------------------:\|:-----:\| integer of bitwidth *n* \|
Type.int(n) \| `IntegerType <llvm.core.IntegerType.html>`_ \| 32-bit
float \| Type.float() \| `Type <llvm.core.Type.html>`_ \| 64-bit double
\| Type.double() \| `Type <llvm.core.Type.html>`_ \| 80-bit float \|
Type.x86\_fp80() \| `Type <llvm.core.Type.html>`_ \| 128-bit float
(112-bit mantissa) \| Type.fp128() \| `Type <llvm.core.Type.html>`_ \|
128-bit float (two 64-bits) \| Type.ppc\_fp128() \|
`Type <llvm.core.Type.html>`_ \| function \| Type.function(r, p, v) \|
`FunctionType <llvm.core.FunctionType.html>`_ \| unpacked struct \|
Type.struct(eltys, name) \| `StructType <llvm.core.StructType.html>`_ \|
packed struct \| Type.packed\_struct(eltys, name) \|
`StructType <llvm.core.StructType.html>`_ \| opaque struct \|
Type.opaque(name) \| `StructType <llvm.core.StructType.html>`_ \| array
\| Type.array(elty, count) \| `ArrayType <llvm.core.ArrayType.html>`_ \|
pointer to value of type *pty* \| Type.pointer(pty, addrspc) \|
`PointerType <llvm.core.PointerType.html>`_ \| vector \|
Type.vector(elty, count) \| `VectorType <llvm.core.VectorType.html>`_ \|
void \| Type.void() \| `Type <llvm.core.Type.html>`_ \| label \|
Type.label() \| `Type <llvm.core.Type.html>`_ \|
The class hierarchy is:
::
Type
IntegerType
FunctionType
StructType
ArrayType
PointerType
VectorType
--------------
An Example
----------
Here is an example that demonstrates the creation of types:
.. code-block:: python
#!/usr/bin/env python
# integers
int_ty = Type.int() bool_ty = Type.int(1) int_64bit = Type.int(64)
# floats
sprec_real = Type.float() dprec_real = Type.double()
# arrays and vectors
intar_ty = Type.array( int_ty, 10 ) # "typedef int intar_ty[10];"
twodim = Type.array( intar_ty , 10 ) # "typedef int twodim[10][10];"
vec = Type.array( int_ty, 10 )
# structures
s1_ty = Type.struct( [ int_ty, sprec_real ] ) # "struct s1_ty { int
v1; float v2; };"
# pointers
intptr_ty = Type.pointer(int_ty) # "typedef int \*intptr_ty;"
# functions
f1 = Type.function( int_ty, [ int_ty ] ) # functions that take 1
int_ty and return 1 int_ty
f2 = Type.function( Type.void(), [ int_ty, int_ty ] ) # functions that
take 2 int_tys and return nothing
f3 = Type.function( Type.void(), ( int_ty, int_ty ) ) # same as f2;
any iterable can be used
fnargs = [ Type.pointer( Type.int(8) ) ] printf = Type.function(
Type.int(), fnargs, True ) # variadic function
--------------
Another Example: Recursive Type
-------------------------------
The type system was rewritten in LLVM 3.0. The old opaque type was
removed. Instead, identified ``StructType`` can now be defined without a
body. Doing so creates a opaque structure. One can then set the body
after the construction of a structure.
(See `LLVM
Blog <http://blog.llvm.org/2011/11/llvm-30-type-system-rewrite.html>`_
for detail about the new type system.)
The following code defines a opaque structure, named "mystruct". The
body is defined after the construction using ``StructType.set_body``.
The second subtype is a pointer to a "mystruct" type.
.. code-block:: python
ts = Type.opaque('mystruct')
ts.set_body([Type.int(), Type.pointer(ts)])

17
docs/source/doc/userguide.rst
View file

@ -1,17 +0,0 @@
************
User Guide
************
llvmpy provides Python bindings for LLVM. This document explains how
you can setup and use it. A working knowledge of Python and a basic idea
of LLVM is assumed.
.. toctree::
:maxdepth: 1
getting_started.rst
llvm_concepts.rst
llvmpy_package.rst

78
docs/source/doc/values.rst
View file

@ -1,78 +0,0 @@
+-----------------+
| layout: page |
+-----------------+
| title: Values |
+-----------------+
`llvm.core.Value <llvm.core.Value.html>`_ is the base class of all
values computed by a program that may be used as operands to other
values. A value has a type associated with it (an object of
`llvm.core.Type <types.html>`_).
The class hierarchy is:
::
Value
User
Constant
ConstantExpr
ConstantAggregateZero
ConstantInt
ConstantFP
ConstantArray
ConstantStruct
ConstantVector
ConstantPointerNull
UndefValue
GlobalValue
GlobalVariable
Function
Instruction
CallOrInvokeInstruction
PHINode
SwitchInstruction
CompareInstruction
Argument
BasicBlock
The `Value <llvm.core.Value.html>`_ class is abstract, it's not meant to
be instantiated. `User <llvm.core.User.html>`_ is a
`Value <llvm.core.Value.html>`_ that in turn uses (i.e., can refer to)
other values (for e.g., a constant expression 1+2 refers to two constant
values 1 and 2).
`Constant <llvm.core.Constant.html>`_-s represent constants that appear
within code or as initializers of globals. They are constructed using
static methods of `Constant <llvm.core.Constant.html>`_. Various types
of constants are represented by various subclasses of
`Constant <llvm.core.Constant.html>`_. However, most of them are empty
and do not provide any additional attributes or methods over
`Constant <llvm.core.Constant.html>`_.
The `Function <functions.html>`_ object represents an instance of a
function type. Such objects contain
`Argument <llvm.core.Argument.html>`_ objects, which represent the
actual, local-variable-like arguments of the function (not to be
confused with the arguments returned by a function *type* object --
these represent the *type* of the arguments).
The various `Instruction <llvm.core.Instruction.html>`_-s are created by
the `Builder <llvm.core.Builder.html>`_ class. Most instructions are
represented by `Instruction <llvm.core.Instruction.html>`_ itself, but
there are a few subclasses that represent interesting instructions.
`Value <llvm.core.Value.html>`_ objects have a type (read-only), and a
name (read-write).
**Related Links** `functions <functions.html>`_,
`comparision <comparision.html>`_,
`llvm.core.Value <llvm.core.Value.html>`_,
`llvm.core.User <llvm.core.User.html>`_,
`llvm.core.Constant <llvm.core.Constant.html>`_,
`llvm.core.GlobalValue <llvm.core.GlobalValue.html>`_,
`llvm.core.GlobalVariable <llvm.core.GlobalVariable.html>`_,
`llvm.core.Argument <llvm.core.Argument.html>`_,
`llvm.core.Instruction <llvm.core.Instruction.html>`_,
`llvm.core.Builder <llvm.core.Builder.html>`_,
`llvm.core.BasicBlock <llvm.core.BasicBlock.html>`_

30
docs/source/index.rst
View file

@ -1,30 +0,0 @@
.. llvmpy documentation master file, created by
sphinx-quickstart on Wed Aug 8 17:33:58 2012.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
llvmpy
======
Contents:
.. toctree::
:titlesonly:
:maxdepth: 2
doc/userguide.rst
doc/llvm_cbuilder.rst
doc/llpython/index.rst
doc/examples.rst
doc/llvmcore.rst
doc/llvmee.rst
doc/llvmpasses.rst
Indices and tables
------------------
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

152
example/vector_instr.py
View file

@ -1,152 +0,0 @@
'''
This example shows:
1) how to use vector instructions
2) how to take advantage of LLVM loop vectorization to transform scalar
operations to vector operations
'''
from __future__ import print_function
import llvm.core as lc
import llvm.ee as le
import llvm.passes as lp
from ctypes import CFUNCTYPE, POINTER, c_int, c_float
def build_manual_vector():
mod = lc.Module.new('manual.vector')
intty = lc.Type.int(32)
vecty = lc.Type.vector(lc.Type.float(), 4)
aryty = lc.Type.pointer(lc.Type.float())
fnty = lc.Type.function(lc.Type.void(), [aryty, aryty, aryty, intty])
fn = mod.add_function(fnty, name='vector_add')
bbentry = fn.append_basic_block('entry')
bbloopcond = fn.append_basic_block('loop.cond')
bbloopbody = fn.append_basic_block('loop.body')
bbexit = fn.append_basic_block('exit')
builder = lc.Builder.new(bbentry)
# populate function body
in1, in2, out, size = fn.args
ZERO = lc.Constant.null(intty)
loopi_ptr = builder.alloca(intty)
builder.store(ZERO, loopi_ptr)
builder.branch(bbloopcond)
builder.position_at_end(bbloopcond)
loopi = builder.load(loopi_ptr)
loopcond = builder.icmp(lc.ICMP_ULT, loopi, size)
builder.cbranch(loopcond, bbloopbody, bbexit)
builder.position_at_end(bbloopbody)
vecaryty = lc.Type.pointer(vecty)
in1asvec = builder.bitcast(builder.gep(in1, [loopi]), vecaryty)
in2asvec = builder.bitcast(builder.gep(in2, [loopi]), vecaryty)
outasvec = builder.bitcast(builder.gep(out, [loopi]), vecaryty)
vec1 = builder.load(in1asvec)
vec2 = builder.load(in2asvec)
vecout = builder.fadd(vec1, vec2)
builder.store(vecout, outasvec)
next = builder.add(loopi, lc.Constant.int(intty, 4))
builder.store(next, loopi_ptr)
builder.branch(bbloopcond)
builder.position_at_end(bbexit)
builder.ret_void()
return mod, fn
def build_auto_vector():
mod = lc.Module.new('auto.vector')
# Loop vectorize is sensitive to the size of the index size(!?)
intty = lc.Type.int(tuple.__itemsize__ * 8)
aryty = lc.Type.pointer(lc.Type.float())
fnty = lc.Type.function(lc.Type.void(), [aryty, aryty, aryty, intty])
fn = mod.add_function(fnty, name='vector_add')
bbentry = fn.append_basic_block('entry')
bbloopcond = fn.append_basic_block('loop.cond')
bbloopbody = fn.append_basic_block('loop.body')
bbexit = fn.append_basic_block('exit')
builder = lc.Builder.new(bbentry)
# populate function body
in1, in2, out, size = fn.args
in1.add_attribute(lc.ATTR_NO_ALIAS)
in2.add_attribute(lc.ATTR_NO_ALIAS)
out.add_attribute(lc.ATTR_NO_ALIAS)
ZERO = lc.Constant.null(intty)
loopi_ptr = builder.alloca(intty)
builder.store(ZERO, loopi_ptr)
builder.branch(bbloopcond)
builder.position_at_end(bbloopcond)
loopi = builder.load(loopi_ptr)
loopcond = builder.icmp(lc.ICMP_ULT, loopi, size)
builder.cbranch(loopcond, bbloopbody, bbexit)
builder.position_at_end(bbloopbody)
in1elem = builder.load(builder.gep(in1, [loopi]))
in2elem = builder.load(builder.gep(in2, [loopi]))
outelem = builder.fadd(in1elem, in2elem)
builder.store(outelem, builder.gep(out, [loopi]))
next = builder.add(loopi, lc.Constant.int(intty, 1))
builder.store(next, loopi_ptr)
builder.branch(bbloopcond)
builder.position_at_end(bbexit)
builder.ret_void()
return mod, fn
def example(title, module_builder, opt):
print(title.center(80, '='))
mod, fn = module_builder()
eb = le.EngineBuilder.new(mod).opt(3)
if opt:
print('opt')
tm = eb.select_target()
pms = lp.build_pass_managers(mod=mod, tm=tm, opt=3, loop_vectorize=True,
fpm=False)
pms.pm.run(mod)
print(mod)
print(mod.to_native_assembly())
engine = eb.create()
ptr = engine.get_pointer_to_function(fn)
callable = CFUNCTYPE(None, POINTER(c_float), POINTER(c_float),
POINTER(c_float), c_int)(ptr)
N = 20
in1 = (c_float * N)(*range(N))
in2 = (c_float * N)(*range(N))
out = (c_float * N)()
print('in1: ', list(in1))
print('in1: ', list(in2))
callable(in1, in2, out, N)
print('out', list(out))
def main():
example('manual vector function', build_manual_vector, False)
example('auto vector function', build_auto_vector, True)
if __name__ == '__main__':
main()

0
llpython/__init__.py
View file

125
llpython/byte_control.py
View file

@ -1,125 +0,0 @@
# ______________________________________________________________________
from __future__ import absolute_import
import opcode
from . import opcode_util
import pprint
from .bytecode_visitor import BasicBlockVisitor, BenignBytecodeVisitorMixin
from .control_flow import ControlFlowGraph
# ______________________________________________________________________
class ControlFlowBuilder (BenignBytecodeVisitorMixin, BasicBlockVisitor):
'''Visitor responsible for traversing a bytecode basic block map and
building a control flow graph (CFG).
The primary purpose of this transformation is to create a CFG,
which is used by later transformers for dataflow analysis.
'''
def visit (self, flow, nargs = 0, *args, **kws):
'''Given a bytecode flow, and an optional number of arguments,
return a :py:class:`llpython.control_flow.ControlFlowGraph`
instance describing the full control flow of the bytecode
flow.'''
self.nargs = nargs
ret_val = super(ControlFlowBuilder, self).visit(flow, *args, **kws)
del self.nargs
return ret_val
def enter_blocks (self, blocks):
super(ControlFlowBuilder, self).enter_blocks(blocks)
self.blocks = blocks
self.block_list = list(blocks.keys())
self.block_list.sort()
self.cfg = ControlFlowGraph()
self.loop_stack = []
for block in self.block_list:
self.cfg.add_block(block, blocks[block])
def exit_blocks (self, blocks):
super(ControlFlowBuilder, self).exit_blocks(blocks)
assert self.blocks == blocks
self.cfg.compute_dataflow()
self.cfg.update_for_ssa()
ret_val = self.cfg
del self.loop_stack
del self.cfg
del self.block_list
del self.blocks
return ret_val
def enter_block (self, block):
self.block = block
assert block in self.cfg.blocks
if block == 0:
for local_index in range(self.nargs):
self.op_STORE_FAST(0, opcode.opmap['STORE_FAST'], local_index)
return True
def _get_next_block (self, block):
return self.block_list[self.block_list.index(block) + 1]
def exit_block (self, block):
assert block == self.block
del self.block
i, op, arg = self.blocks[block][-1]
opname = opcode.opname[op]
if op in opcode.hasjabs:
self.cfg.add_edge(block, arg)
elif op in opcode.hasjrel:
self.cfg.add_edge(block, i + arg + 3)
elif opname == 'BREAK_LOOP':
loop_i, _, loop_arg = self.loop_stack[-1]
self.cfg.add_edge(block, loop_i + loop_arg + 3)
elif opname != 'RETURN_VALUE':
self.cfg.add_edge(block, self._get_next_block(block))
if op in opcode_util.hascbranch:
self.cfg.add_edge(block, self._get_next_block(block))
def op_LOAD_FAST (self, i, op, arg, *args, **kws):
self.cfg.blocks_reads[self.block].add(arg)
return super(ControlFlowBuilder, self).op_LOAD_FAST(i, op, arg, *args,
**kws)
def op_STORE_FAST (self, i, op, arg, *args, **kws):
self.cfg.writes_local(self.block, i, arg)
return super(ControlFlowBuilder, self).op_STORE_FAST(i, op, arg, *args,
**kws)
def op_SETUP_LOOP (self, i, op, arg, *args, **kws):
self.loop_stack.append((i, op, arg))
return super(ControlFlowBuilder, self).op_SETUP_LOOP(i, op, arg, *args,
**kws)
def op_POP_BLOCK (self, i, op, arg, *args, **kws):
self.loop_stack.pop()
return super(ControlFlowBuilder, self).op_POP_BLOCK(i, op, arg, *args,
**kws)
# ______________________________________________________________________
def build_cfg (func):
'''Given a Python function, create a bytecode flow, visit the flow
object, and return a control flow graph.'''
co_obj = opcode_util.get_code_object(func)
return ControlFlowBuilder().visit(opcode_util.build_basic_blocks(co_obj),
co_obj.co_argcount)
# ______________________________________________________________________
# Main (self-test) routine
def main (*args, **kws):
from tests import llfuncs
if not args:
args = ('doslice',)
for arg in args:
build_cfg(getattr(llfuncs, arg)).pprint()
# ______________________________________________________________________
if __name__ == "__main__":
import sys
main(*sys.argv[1:])
# ______________________________________________________________________
# End of byte_control.py

266
llpython/byte_flow.py
View file

@ -1,266 +0,0 @@
# ______________________________________________________________________
from __future__ import absolute_import
import dis
import opcode
from .bytecode_visitor import BasicBlockVisitor
from . import opcode_util
# ______________________________________________________________________
class BytecodeFlowBuilder (BasicBlockVisitor):
'''Transforms a CFG into a bytecode "flow tree".
The flow tree is a Python dictionary, described loosely by the
following set of productions:
* `flow_tree` ``:=`` ``{`` `blocks` ``*`` ``}``
* `blocks` ``:=`` `block_index` ``:`` ``[`` `bytecode_tree` ``*`` ``]``
* `bytecode_tree` ``:=`` ``(`` `opcode_index` ``,`` `opcode` ``,``
`opname` ``,`` `arg` ``,`` ``[`` `bytecode_tree` ``*`` ``]`` ``)``
The primary purpose of this transformation is to simulate the
value stack, removing it and any stack-specific opcodes.'''
def __init__ (self, *args, **kws):
super(BytecodeFlowBuilder, self).__init__(*args, **kws)
om_items = opcode_util.OPCODE_MAP.items()
self.opmap = dict((opcode.opmap[opname], (opname, pops, pushes, stmt))
for opname, (pops, pushes, stmt) in om_items
if opname in opcode.opmap)
def _visit_op (self, i, op, arg, opname, pops, pushes, appends):
assert pops is not None, ('%s not well defined in opcode_util.'
'OPCODE_MAP' % opname)
if pops:
if pops < 0:
pops = arg - pops - 1
assert pops <= len(self.stack), ("Stack underflow at instruction "
"%d (%s)!" % (i, opname))
stk_args = self.stack[-pops:]
del self.stack[-pops:]
else:
stk_args = []
ret_val = (i, op, opname, arg, stk_args)
if pushes:
self.stack.append(ret_val)
if appends:
self.block.append(ret_val)
return ret_val
def _op (self, i, op, arg):
opname, pops, pushes, appends = self.opmap[op]
return self._visit_op(i, op, arg, opname, pops, pushes, appends)
def visit_cfg (self, cfg):
self.cfg = cfg
ret_val = self.visit(cfg.blocks)
del self.cfg
return ret_val
def enter_blocks (self, blocks):
labels = list(blocks.keys())
labels.sort()
self.blocks = dict((index, [])
for index in labels)
self.loop_stack = []
self.stacks = {}
def exit_blocks (self, blocks):
ret_val = self.blocks
del self.stacks
del self.loop_stack
del self.blocks
return ret_val
def enter_block (self, block):
self.block_no = block
self.block = self.blocks[block]
in_blocks = self.cfg.blocks_in[block]
if len(in_blocks) == 0:
self.stack = []
else:
pred_stack = None
for pred in in_blocks:
if pred in self.stacks:
pred_stack = self.stacks[pred]
break
if pred_stack is not None:
self.stack = pred_stack[:]
else:
raise NotImplementedError()
def exit_block (self, block):
assert self.block_no == block
self.stacks[block] = self.stack
del self.stack
del self.block
del self.block_no
op_BINARY_ADD = _op
op_BINARY_AND = _op
op_BINARY_DIVIDE = _op
op_BINARY_FLOOR_DIVIDE = _op
op_BINARY_LSHIFT = _op
op_BINARY_MODULO = _op
op_BINARY_MULTIPLY = _op
op_BINARY_OR = _op
op_BINARY_POWER = _op
op_BINARY_RSHIFT = _op
op_BINARY_SUBSCR = _op
op_BINARY_SUBTRACT = _op
op_BINARY_TRUE_DIVIDE = _op
op_BINARY_XOR = _op
def op_BREAK_LOOP (self, i, op, arg):
loop_i, _, loop_arg = self.loop_stack[-1]
assert arg is None
return self._op(i, op, loop_i + loop_arg + 3)
#op_BUILD_CLASS = _op
op_BUILD_LIST = _op
op_BUILD_MAP = _op
op_BUILD_SLICE = _op
op_BUILD_TUPLE = _op
op_CALL_FUNCTION = _op
op_CALL_FUNCTION_KW = _op
op_CALL_FUNCTION_VAR = _op
op_CALL_FUNCTION_VAR_KW = _op
op_COMPARE_OP = _op
#op_CONTINUE_LOOP = _op
op_DELETE_ATTR = _op
op_DELETE_FAST = _op
op_DELETE_GLOBAL = _op
op_DELETE_NAME = _op
op_DELETE_SLICE = _op
op_DELETE_SUBSCR = _op
def op_DUP_TOP (self, i, op, arg):
self.stack.append(self.stack[-1])
def op_DUP_TOPX (self, i, op, arg):
self.stack += self.stack[-arg:]
#op_END_FINALLY = _op
op_EXEC_STMT = _op
#op_EXTENDED_ARG = _op
op_FOR_ITER = _op
op_GET_ITER = _op
op_IMPORT_FROM = _op
op_IMPORT_NAME = _op
op_IMPORT_STAR = _op
op_INPLACE_ADD = _op
op_INPLACE_AND = _op
op_INPLACE_DIVIDE = _op
op_INPLACE_FLOOR_DIVIDE = _op
op_INPLACE_LSHIFT = _op
op_INPLACE_MODULO = _op
op_INPLACE_MULTIPLY = _op
op_INPLACE_OR = _op
op_INPLACE_POWER = _op
op_INPLACE_RSHIFT = _op
op_INPLACE_SUBTRACT = _op
op_INPLACE_TRUE_DIVIDE = _op
op_INPLACE_XOR = _op
op_JUMP_ABSOLUTE = _op
op_JUMP_FORWARD = _op
def op_JUMP_IF_FALSE (self, i, op, arg):
opname, _, _, _ = self.opmap[op]
ret_val = (i, op, opname, arg, [self.stack[-1]])
self.block.append(ret_val)
return ret_val
op_JUMP_IF_TRUE = op_JUMP_IF_FALSE
op_LIST_APPEND = _op
op_LOAD_ATTR = _op
op_LOAD_CLOSURE = _op
op_LOAD_CONST = _op
op_LOAD_DEREF = _op
op_LOAD_FAST = _op
op_LOAD_GLOBAL = _op
op_LOAD_LOCALS = _op
op_LOAD_NAME = _op
op_MAKE_CLOSURE = _op
op_MAKE_FUNCTION = _op
op_NOP = _op
def op_POP_BLOCK (self, i, op, arg):
self.loop_stack.pop()
return self._op(i, op, arg)
op_POP_JUMP_IF_FALSE = _op
op_POP_JUMP_IF_TRUE = _op
op_POP_TOP = _op
op_PRINT_EXPR = _op
op_PRINT_ITEM = _op
op_PRINT_ITEM_TO = _op
op_PRINT_NEWLINE = _op
op_PRINT_NEWLINE_TO = _op
op_RAISE_VARARGS = _op
op_RETURN_VALUE = _op
def op_ROT_FOUR (self, i, op, arg):
self.stack[-4:] = (self.stack[-1], self.stack[-4], self.stack[-3],
self.stack[-2])
def op_ROT_THREE (self, i, op, arg):
self.stack[-3:] = (self.stack[-1], self.stack[-3], self.stack[-2])
def op_ROT_TWO (self, i, op, arg):
self.stack[-2:] = (self.stack[-1], self.stack[-2])
#op_SETUP_EXCEPT = _op
#op_SETUP_FINALLY = _op
def op_SETUP_LOOP (self, i, op, arg):
self.loop_stack.append((i, op, arg))
self.block.append((i, op, self.opnames[op], arg, []))
op_SLICE = _op
#op_STOP_CODE = _op
op_STORE_ATTR = _op
op_STORE_DEREF = _op
op_STORE_FAST = _op
op_STORE_GLOBAL = _op
op_STORE_MAP = _op
op_STORE_NAME = _op
op_STORE_SLICE = _op
op_STORE_SUBSCR = _op
op_UNARY_CONVERT = _op
op_UNARY_INVERT = _op
op_UNARY_NEGATIVE = _op
op_UNARY_NOT = _op
op_UNARY_POSITIVE = _op
op_UNPACK_SEQUENCE = _op
#op_WITH_CLEANUP = _op
op_YIELD_VALUE = _op
# ______________________________________________________________________
def build_flow (func):
'''Given a Python function, return a bytecode flow tree for that
function.'''
import byte_control
cfg = byte_control.build_cfg(func)
return BytecodeFlowBuilder().visit_cfg(cfg)
# ______________________________________________________________________
# Main (self-test) routine
def main (*args):
import pprint
from tests import llfuncs
if not args:
args = ('doslice',)
for arg in args:
pprint.pprint(build_flow(getattr(llfuncs, arg)))
# ______________________________________________________________________
if __name__ == '__main__':
import sys
main(*sys.argv[1:])
# ______________________________________________________________________
# End of byte_flow.py

612
llpython/byte_translator.py
View file

@ -1,612 +0,0 @@
# ______________________________________________________________________
'''Defines a bytecode based LLVM translator for llpython code.
'''
# ______________________________________________________________________
# Module imports
from __future__ import absolute_import
import opcode
import types
import logging
import llvm.core as lc
from . import opcode_util
from . import bytetype
from .bytecode_visitor import BytecodeFlowVisitor
from .byte_flow import BytecodeFlowBuilder
from .byte_control import ControlFlowBuilder
from .phi_injector import PhiInjector, synthetic_opname
# ______________________________________________________________________
# Module data
logger = logging.getLogger(__name__)
# XXX Stolen from numba.translate:
_compare_mapping_float = {'>':lc.FCMP_OGT,
'<':lc.FCMP_OLT,
'==':lc.FCMP_OEQ,
'>=':lc.FCMP_OGE,
'<=':lc.FCMP_OLE,
'!=':lc.FCMP_ONE}
_compare_mapping_sint = {'>':lc.ICMP_SGT,
'<':lc.ICMP_SLT,
'==':lc.ICMP_EQ,
'>=':lc.ICMP_SGE,
'<=':lc.ICMP_SLE,
'!=':lc.ICMP_NE}
# XXX Stolen from numba.llvm_types:
class LLVMCaster (object):
def build_pointer_cast(_, builder, lval1, lty2):
return builder.bitcast(lval1, lty2)
def build_int_cast(_, builder, lval1, lty2, unsigned = False):
width1 = lval1.type.width
width2 = lty2.width
ret_val = lval1
if width2 > width1:
if unsigned:
ret_val = builder.zext(lval1, lty2)
else:
ret_val = builder.sext(lval1, lty2)
elif width2 < width1:
ret_val = builder.trunc(lval1, lty2)
return ret_val
def build_float_ext(_, builder, lval1, lty2):
return builder.fpext(lval1, lty2)
def build_float_trunc(_, builder, lval1, lty2):
return builder.fptrunc(lval1, lty2)
def build_int_to_float_cast(_, builder, lval1, lty2, unsigned = False):
ret_val = None
if unsigned:
ret_val = builder.uitofp(lval1, lty2)
else:
ret_val = builder.sitofp(lval1, lty2)
return ret_val
def build_int_to_ptr_cast(_, builder, lval1, lty2):
return builder.inttoptr(lval1, lty2)
def build_float_to_int_cast(_, builder, lval1, lty2, unsigned = False):
ret_val = None
if unsigned:
ret_val = builder.fptoui(lval1, lty2)
else:
ret_val = builder.fptosi(lval1, lty2)
return ret_val
CAST_MAP = {
lc.TYPE_POINTER : build_pointer_cast,
lc.TYPE_INTEGER: build_int_cast,
(lc.TYPE_FLOAT, lc.TYPE_DOUBLE) : build_float_ext,
(lc.TYPE_DOUBLE, lc.TYPE_FLOAT) : build_float_trunc,
(lc.TYPE_INTEGER, lc.TYPE_FLOAT) : build_int_to_float_cast,
(lc.TYPE_INTEGER, lc.TYPE_DOUBLE) : build_int_to_float_cast,
(lc.TYPE_INTEGER, lc.TYPE_POINTER) : build_int_to_ptr_cast,
(lc.TYPE_FLOAT, lc.TYPE_INTEGER) : build_float_to_int_cast,
(lc.TYPE_DOUBLE, lc.TYPE_INTEGER) : build_float_to_int_cast,
}
@classmethod
def build_cast(cls, builder, lval1, lty2, *args, **kws):
ret_val = lval1
lty1 = lval1.type
lkind1 = lty1.kind
lkind2 = lty2.kind
if lkind1 == lkind2:
if lkind1 in cls.CAST_MAP:
ret_val = cls.CAST_MAP[lkind1](cls, builder, lval1, lty2,
*args, **kws)
else:
raise NotImplementedError(lkind1)
else:
map_index = (lkind1, lkind2)
if map_index in cls.CAST_MAP:
ret_val = cls.CAST_MAP[map_index](cls, builder, lval1, lty2,
*args, **kws)
else:
raise NotImplementedError(lkind1, lkind2)
return ret_val
# ______________________________________________________________________
# Class definitions
class LLVMTranslator (BytecodeFlowVisitor):
'''Transformer responsible for visiting a set of bytecode flow
trees, emitting LLVM code.
Unlike other translators in :py:mod:`llpython`, this
incorporates the full transformation chain, starting with
:py:class:`llpython.byte_flow.BytecodeFlowBuilder`, then
:py:class:`llpython.byte_control.ControlFlowBuilder`, and
then :py:class:`llpython.phi_injector.PhiInjector`.'''
def __init__ (self, llvm_module = None, *args, **kws):
'''Constructor for LLVMTranslator.'''
super(LLVMTranslator, self).__init__(*args, **kws)
if llvm_module is None:
llvm_module = lc.Module.new('Translated_Module_%d' % (id(self),))
self.llvm_module = llvm_module
self.bytecode_flow_builder = BytecodeFlowBuilder()
self.control_flow_builder = ControlFlowBuilder()
self.phi_injector = PhiInjector()
def translate (self, function, llvm_type = None, llvm_function = None,
env = None):
'''Translate a function to the given LLVM function type.
If no type is given, then assume the function is of LLVM type
"void ()".
The optional env parameter allows extension of the global
environment.'''
if llvm_type is None:
if llvm_function is None:
llvm_type = lc.Type.function(bytetype.lvoid, ())
else:
llvm_type = llvm_function.type.pointee
if env is None:
env = {}
else:
env = env.copy()
env.update((name, method)
for name, method in lc.Builder.__dict__.items()
if not name.startswith('_'))
env.update((name, value)
for name, value in bytetype.__dict__.items()
if not name.startswith('_'))
self.loop_stack = []
self.llvm_type = llvm_type
self.target_function_name = env.get('target_function_name',
function.__name__)
self.function = function
self.code_obj = opcode_util.get_code_object(function)
func_globals = getattr(function, 'func_globals',
getattr(function, '__globals__', {})).copy()
func_globals.update(env)
self.globals = func_globals
nargs = self.code_obj.co_argcount
self.cfg = self.control_flow_builder.visit(
opcode_util.build_basic_blocks(self.code_obj), nargs)
self.cfg.blocks = self.bytecode_flow_builder.visit_cfg(self.cfg)
self.llvm_function = llvm_function
flow = self.phi_injector.visit_cfg(self.cfg, nargs)
ret_val = self.visit(flow)
del self.cfg
del self.globals
del self.code_obj
del self.target_function_name
del self.function
del self.llvm_type
del self.loop_stack
return ret_val
def enter_flow_object (self, flow):
super(LLVMTranslator, self).enter_flow_object(flow)
if self.llvm_function is None:
self.llvm_function = self.llvm_module.add_function(
self.llvm_type, self.target_function_name)
self.llvm_blocks = {}
self.llvm_definitions = {}
self.pending_phis = {}
for block in self.block_list:
if 0 in self.cfg.blocks_reaching[block]:
bb = self.llvm_function.append_basic_block(
'BLOCK_%d' % (block,))
self.llvm_blocks[block] = bb
def exit_flow_object (self, flow):
super(LLVMTranslator, self).exit_flow_object(flow)
ret_val = self.llvm_function
del self.pending_phis
del self.llvm_definitions
del self.llvm_blocks
if __debug__ and logger.getEffectiveLevel() < logging.DEBUG:
logger.debug(str(ret_val))
return ret_val
def enter_block (self, block):
ret_val = False
if block in self.llvm_blocks:
self.llvm_block = self.llvm_blocks[block]
self.builder = lc.Builder.new(self.llvm_block)
ret_val = True
return ret_val
def exit_block (self, block):
bb_instrs = self.llvm_block.instructions
if ((len(bb_instrs) == 0) or
(not bb_instrs[-1].is_terminator)):
out_blocks = list(self.cfg.blocks_out[block])
assert len(out_blocks) == 1
self.builder.branch(self.llvm_blocks[out_blocks[0]])
del self.llvm_block
del self.builder
def visit_synthetic_op (self, i, op, arg, *args, **kws):
method = getattr(self, 'op_%s' % (synthetic_opname[op],))
return method(i, op, arg, *args, **kws)
def op_REF_ARG (self, i, op, arg, *args, **kws):
return [self.llvm_function.args[arg]]
def op_BUILD_PHI (self, i, op, arg, *args, **kws):
phi_type = None
incoming = []
pending = []
for child_arg in arg:
child_block, _, child_opname, child_arg, _ = child_arg
assert child_opname == 'REF_DEF'
if child_arg in self.llvm_definitions:
child_def = self.llvm_definitions[child_arg]
if phi_type is None:
phi_type = child_def.type
incoming.append((child_block, child_def))
else:
pending.append((child_arg, child_block))
phi = self.builder.phi(phi_type)
for block_index, defn in incoming:
phi.add_incoming(defn, self.llvm_blocks[block_index])
for defn_index, block_index in pending:
if defn_index not in self.pending_phis:
self.pending_phis[defn_index] = []
self.pending_phis[defn_index].append((phi, block_index))
return [phi]
def op_DEFINITION (self, i, op, def_index, *args, **kws):
assert len(args) == 1
arg = args[0]
if def_index in self.pending_phis:
for phi, block_index in self.pending_phis[def_index]:
phi.add_incoming(arg, self.llvm_blocks[block_index])
self.llvm_definitions[def_index] = arg
return args
def op_REF_DEF (self, i, op, arg, *args, **kws):
return [self.llvm_definitions[arg]]
def op_BINARY_ADD (self, i, op, arg, *args, **kws):
arg1, arg2 = args
if arg1.type.kind == lc.TYPE_INTEGER:
ret_val = [self.builder.add(arg1, arg2)]
elif arg1.type.kind in (lc.TYPE_FLOAT, lc.TYPE_DOUBLE):
ret_val = [self.builder.fadd(arg1, arg2)]
elif arg1.type.kind == lc.TYPE_POINTER:
ret_val = [self.builder.gep(arg1, [arg2])]
else:
raise NotImplementedError("LLVMTranslator.op_BINARY_ADD for %r" %
(args,))
return ret_val
def op_BINARY_AND (self, i, op, arg, *args, **kws):
return [self.builder.and_(args[0], args[1])]
def op_BINARY_DIVIDE (self, i, op, arg, *args, **kws):
arg1, arg2 = args
if arg1.type.kind == lc.TYPE_INTEGER:
ret_val = [self.builder.sdiv(arg1, arg2)]
elif arg1.type.kind in (lc.TYPE_FLOAT, lc.TYPE_DOUBLE):
ret_val = [self.builder.fdiv(arg1, arg2)]
else:
raise NotImplementedError("LLVMTranslator.op_BINARY_DIVIDE for %r"
% (args,))
return ret_val
def op_BINARY_FLOOR_DIVIDE (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_BINARY_FLOOR_DIVIDE")
def op_BINARY_LSHIFT (self, i, op, arg, *args, **kws):
return [self.builder.shl(args[0], args[1])]
def op_BINARY_MODULO (self, i, op, arg, *args, **kws):
arg1, arg2 = args
if arg1.type.kind == lc.TYPE_INTEGER:
ret_val = [self.builder.srem(arg1, arg2)]
elif arg1.type.kind in (lc.TYPE_FLOAT, lc.TYPE_DOUBLE):
ret_val = [self.builder.frem(arg1, arg2)]
else:
raise NotImplementedError("LLVMTranslator.op_BINARY_MODULO for %r"
% (args,))
return ret_val
def op_BINARY_MULTIPLY (self, i, op, arg, *args, **kws):
arg1, arg2 = args
if arg1.type.kind == lc.TYPE_INTEGER:
ret_val = [self.builder.mul(arg1, arg2)]
elif arg1.type.kind in (lc.TYPE_FLOAT, lc.TYPE_DOUBLE):
ret_val = [self.builder.fmul(arg1, arg2)]
else:
raise NotImplementedError("LLVMTranslator.op_BINARY_MULTIPLY for "
"%r" % (args,))
return ret_val
def op_BINARY_OR (self, i, op, arg, *args, **kws):
return [self.builder.or_(args[0], args[1])]
def op_BINARY_POWER (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_BINARY_POWER")
def op_BINARY_RSHIFT (self, i, op, arg, *args, **kws):
return [self.builder.lshr(args[0], args[1])]
def op_BINARY_SUBSCR (self, i, op, arg, *args, **kws):
arr_val = args[0]
index_vals = args[1:]
ret_val = gep_result = self.builder.gep(arr_val, index_vals)
if (gep_result.type.kind == lc.TYPE_POINTER and
gep_result.type.pointee.kind != lc.TYPE_POINTER):
ret_val = self.builder.load(gep_result)
return [ret_val]
def op_BINARY_SUBTRACT (self, i, op, arg, *args, **kws):
arg1, arg2 = args
if arg1.type.kind == lc.TYPE_INTEGER:
ret_val = [self.builder.sub(arg1, arg2)]
elif arg1.type.kind in (lc.TYPE_FLOAT, lc.TYPE_DOUBLE):
ret_val = [self.builder.fsub(arg1, arg2)]
else:
raise NotImplementedError("LLVMTranslator.op_BINARY_SUBTRACT for "
"%r" % (args,))
return ret_val
op_BINARY_TRUE_DIVIDE = op_BINARY_DIVIDE
def op_BINARY_XOR (self, i, op, arg, *args, **kws):
return [self.builder.xor(args[0], args[1])]
def op_BREAK_LOOP (self, i, op, arg, *args, **kws):
return [self.builder.branch(self.llvm_blocks[arg])]
def op_BUILD_SLICE (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_BUILD_SLICE")
def op_BUILD_TUPLE (self, i, op, arg, *args, **kws):
return args
def op_CALL_FUNCTION (self, i, op, arg, *args, **kws):
fn = args[0]
args = args[1:]
fn_name = getattr(fn, '__name__', None)
if isinstance(fn, (types.FunctionType, types.MethodType)):
ret_val = [fn(self.builder, *args)]
elif isinstance(fn, lc.Value):
ret_val = [self.builder.call(fn, args)]
elif isinstance(fn, lc.Type):
if isinstance(fn, lc.FunctionType):
ret_val = [self.builder.call(
self.llvm_module.get_or_insert_function(fn, fn_name),
args)]
else:
assert len(args) == 1
ret_val = [LLVMCaster.build_cast(self.builder, args[0], fn)]
else:
raise NotImplementedError("Don't know how to call %s() (%r @ %d)!"
% (fn_name, fn, i))
return ret_val
def op_CALL_FUNCTION_KW (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_CALL_FUNCTION_KW")
def op_CALL_FUNCTION_VAR (self, i, op, arg, *args, **kws):
args = list(args)
var_args = list(args.pop())
args.extend(var_args)
return self.op_CALL_FUNCTION(i, op, arg, *args, **kws)
def op_CALL_FUNCTION_VAR_KW (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_CALL_FUNCTION_VAR_KW")
def op_COMPARE_OP (self, i, op, arg, *args, **kws):
arg1, arg2 = args
cmp_kind = opcode.cmp_op[arg]
if isinstance(arg1.type, lc.IntegerType):
ret_val = [self.builder.icmp(_compare_mapping_sint[cmp_kind],
arg1, arg2)]
elif arg1.type.kind in (lc.TYPE_FLOAT, lc.TYPE_DOUBLE):
ret_val = [self.builder.fcmp(_compare_mapping_float[cmp_kind],
arg1, arg2)]
else:
raise NotImplementedError('Comparison of type %r' % (arg1.type,))
return ret_val
def op_CONTINUE_LOOP (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_CONTINUE_LOOP")
def op_DELETE_ATTR (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_DELETE_ATTR")
def op_DELETE_SLICE (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_DELETE_SLICE")
def op_FOR_ITER (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_FOR_ITER")
def op_GET_ITER (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_GET_ITER")
op_INPLACE_ADD = op_BINARY_ADD
op_INPLACE_AND = op_BINARY_AND
op_INPLACE_DIVIDE = op_BINARY_DIVIDE
op_INPLACE_FLOOR_DIVIDE = op_BINARY_FLOOR_DIVIDE
op_INPLACE_LSHIFT = op_BINARY_LSHIFT
op_INPLACE_MODULO = op_BINARY_MODULO
op_INPLACE_MULTIPLY = op_BINARY_MULTIPLY
op_INPLACE_OR = op_BINARY_OR
op_INPLACE_POWER = op_BINARY_POWER
op_INPLACE_RSHIFT = op_BINARY_RSHIFT
op_INPLACE_SUBTRACT = op_BINARY_SUBTRACT
op_INPLACE_TRUE_DIVIDE = op_BINARY_TRUE_DIVIDE
op_INPLACE_XOR = op_BINARY_XOR
def op_JUMP_ABSOLUTE (self, i, op, arg, *args, **kws):
return [self.builder.branch(self.llvm_blocks[arg])]
def op_JUMP_FORWARD (self, i, op, arg, *args, **kws):
return [self.builder.branch(self.llvm_blocks[i + arg + 3])]
def op_JUMP_IF_FALSE (self, i, op, arg, *args, **kws):
cond = args[0]
block_false = self.llvm_blocks[i + 3 + arg]
block_true = self.llvm_blocks[i + 3]
return [self.builder.cbranch(cond, block_true, block_false)]
# raise NotImplementedError("LLVMTranslator.op_JUMP_IF_FALSE")
def op_JUMP_IF_FALSE_OR_POP (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_JUMP_IF_FALSE_OR_POP")
def op_JUMP_IF_TRUE (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_JUMP_IF_TRUE")
def op_JUMP_IF_TRUE_OR_POP (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_JUMP_IF_TRUE_OR_POP")
def op_LOAD_ATTR (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_LOAD_ATTR")
def op_LOAD_CONST (self, i, op, arg, *args, **kws):
py_val = self.code_obj.co_consts[arg]
if isinstance(py_val, int):
ret_val = [lc.Constant.int(bytetype.lc_int, py_val)]
elif isinstance(py_val, float):
ret_val = [lc.Constant.double(py_val)]
elif py_val == None:
ret_val = [None]
else:
raise NotImplementedError('Constant converstion for %r' %
(py_val,))
return ret_val
def op_LOAD_DEREF (self, i, op, arg, *args, **kws):
name = self.code_obj.co_freevars[arg]
ret_val = self.globals[name]
if isinstance(ret_val, lc.Type) and not hasattr(ret_val, '__name__'):
ret_val.__name__ = name
return [ret_val]
def op_LOAD_GLOBAL (self, i, op, arg, *args, **kws):
name = self.code_obj.co_names[arg]
ret_val = self.globals[name]
if isinstance(ret_val, lc.Type) and not hasattr(ret_val, '__name__'):
ret_val.__name__ = name
return [ret_val]
def op_POP_BLOCK (self, i, op, arg, *args, **kws):
self.loop_stack.pop()
return [self.builder.branch(self.llvm_blocks[i + 1])]
def op_POP_JUMP_IF_FALSE (self, i, op, arg, *args, **kws):
return [self.builder.cbranch(args[0], self.llvm_blocks[i + 3],
self.llvm_blocks[arg])]
def op_POP_JUMP_IF_TRUE (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_POP_JUMP_IF_TRUE")
def op_POP_TOP (self, i, op, arg, *args, **kws):
return args
def op_RETURN_VALUE (self, i, op, arg, *args, **kws):
if args[0] is None:
ret_val = [self.builder.ret_void()]
else:
ret_val = [self.builder.ret(args[0])]
return ret_val
def op_SETUP_LOOP (self, i, op, arg, *args, **kws):
self.loop_stack.append((i, arg))
return [self.builder.branch(self.llvm_blocks[i + 3])]
def op_SLICE (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_SLICE")
def op_STORE_ATTR (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_STORE_ATTR")
def op_STORE_SLICE (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_STORE_SLICE")
def op_STORE_SUBSCR (self, i, op, arg, *args, **kws):
store_val, arr_val, index_val = args
dest_addr = self.builder.gep(arr_val, [index_val])
return [self.builder.store(store_val, dest_addr)]
def op_UNARY_CONVERT (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_UNARY_CONVERT")
def op_UNARY_INVERT (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_UNARY_INVERT")
def op_UNARY_NEGATIVE (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_UNARY_NEGATIVE")
def op_UNARY_NOT (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_UNARY_NOT")
def op_UNARY_POSITIVE (self, i, op, arg, *args, **kws):
raise NotImplementedError("LLVMTranslator.op_UNARY_POSITIVE")
# ______________________________________________________________________
def translate_function (func, lltype, llvm_module = None, **kws):
'''Given a function and an LLVM function type, emit LLVM code for
that function using a new LLVMTranslator instance.'''
translator = LLVMTranslator(llvm_module)
ret_val = translator.translate(func, lltype, env = kws)
return ret_val
# ______________________________________________________________________
def translate_into_function (py_function, llvm_function, **kws):
translator = LLVMTranslator(llvm_function.module)
ret_val = translator.translate(py_function, llvm_function = llvm_function,
env = kws)
return ret_val
# ______________________________________________________________________
def llpython (lltype, llvm_module = None, **kws):
'''Decorator version of translate_function().'''
def _llpython (func):
return translate_function(func, lltype, llvm_module, **kws)
return _llpython
# ______________________________________________________________________
def llpython_into (llvm_function, **kws):
def _llpython_into (func):
return translate_into_function(llvm_function, func, **kws)
return _llpython_into
# ______________________________________________________________________
# Main (self-test) routine
def main (*args):
from tests import llfuncs, llfunctys
if not args:
args = ('doslice',)
elif 'all' in args:
args = [llfunc
for llfunc in dir(llfuncs) if not llfunc.startswith('_')]
llvm_module = lc.Module.new('test_module')
for arg in args:
translate_function(getattr(llfuncs, arg), getattr(llfunctys, arg),
llvm_module)
print(llvm_module)
# ______________________________________________________________________
if __name__ == '__main__':
import sys
main(*sys.argv[1:])
# ______________________________________________________________________
# End of byte_translator.py

360
llpython/bytecode_visitor.py
View file

@ -1,360 +0,0 @@
# ______________________________________________________________________
from __future__ import absolute_import
import itertools
import opcode
from .opcode_util import itercode
# ______________________________________________________________________
class BytecodeVisitor (object):
opnames = [name.split('+')[0] for name in opcode.opname]
def visit_op (self, i, op, arg, *args, **kws):
if op < 0:
ret_val = self.visit_synthetic_op(i, op, arg, *args, **kws)
else:
method = getattr(self, 'op_' + self.opnames[op])
ret_val = method(i, op, arg, *args, **kws)
return ret_val
def visit_synthetic_op (self, i, op, arg, *args, **kws):
raise NotImplementedError(
'BytecodeVisitor.visit_synthetic_op() must be overloaded if using '
'synthetic opcodes.')
def _not_implemented (self, i, op, arg, *args, **kws):
raise NotImplementedError("BytecodeVisitor.op_%s (@ bytecode index %d)"
% (self.opnames[op], i))
op_BINARY_ADD = _not_implemented
op_BINARY_AND = _not_implemented
op_BINARY_DIVIDE = _not_implemented
op_BINARY_FLOOR_DIVIDE = _not_implemented
op_BINARY_LSHIFT = _not_implemented
op_BINARY_MODULO = _not_implemented
op_BINARY_MULTIPLY = _not_implemented
op_BINARY_OR = _not_implemented
op_BINARY_POWER = _not_implemented
op_BINARY_RSHIFT = _not_implemented
op_BINARY_SUBSCR = _not_implemented
op_BINARY_SUBTRACT = _not_implemented
op_BINARY_TRUE_DIVIDE = _not_implemented
op_BINARY_XOR = _not_implemented
op_BREAK_LOOP = _not_implemented
op_BUILD_CLASS = _not_implemented
op_BUILD_LIST = _not_implemented
op_BUILD_MAP = _not_implemented
op_BUILD_SET = _not_implemented
op_BUILD_SLICE = _not_implemented
op_BUILD_TUPLE = _not_implemented
op_CALL_FUNCTION = _not_implemented
op_CALL_FUNCTION_KW = _not_implemented
op_CALL_FUNCTION_VAR = _not_implemented
op_CALL_FUNCTION_VAR_KW = _not_implemented
op_COMPARE_OP = _not_implemented
op_CONTINUE_LOOP = _not_implemented
op_DELETE_ATTR = _not_implemented
op_DELETE_DEREF = _not_implemented
op_DELETE_FAST = _not_implemented
op_DELETE_GLOBAL = _not_implemented
op_DELETE_NAME = _not_implemented
op_DELETE_SLICE = _not_implemented
op_DELETE_SUBSCR = _not_implemented
op_DUP_TOP = _not_implemented
op_DUP_TOPX = _not_implemented
op_DUP_TOP_TWO = _not_implemented
op_END_FINALLY = _not_implemented
op_EXEC_STMT = _not_implemented
op_EXTENDED_ARG = _not_implemented
op_FOR_ITER = _not_implemented
op_GET_ITER = _not_implemented
op_IMPORT_FROM = _not_implemented
op_IMPORT_NAME = _not_implemented
op_IMPORT_STAR = _not_implemented
op_INPLACE_ADD = _not_implemented
op_INPLACE_AND = _not_implemented
op_INPLACE_DIVIDE = _not_implemented
op_INPLACE_FLOOR_DIVIDE = _not_implemented
op_INPLACE_LSHIFT = _not_implemented
op_INPLACE_MODULO = _not_implemented
op_INPLACE_MULTIPLY = _not_implemented
op_INPLACE_OR = _not_implemented
op_INPLACE_POWER = _not_implemented
op_INPLACE_RSHIFT = _not_implemented
op_INPLACE_SUBTRACT = _not_implemented
op_INPLACE_TRUE_DIVIDE = _not_implemented
op_INPLACE_XOR = _not_implemented
op_JUMP_ABSOLUTE = _not_implemented
op_JUMP_FORWARD = _not_implemented
op_JUMP_IF_FALSE = _not_implemented
op_JUMP_IF_FALSE_OR_POP = _not_implemented
op_JUMP_IF_TRUE = _not_implemented
op_JUMP_IF_TRUE_OR_POP = _not_implemented
op_LIST_APPEND = _not_implemented
op_LOAD_ATTR = _not_implemented
op_LOAD_BUILD_CLASS = _not_implemented
op_LOAD_CLOSURE = _not_implemented
op_LOAD_CONST = _not_implemented
op_LOAD_DEREF = _not_implemented
op_LOAD_FAST = _not_implemented
op_LOAD_GLOBAL = _not_implemented
op_LOAD_LOCALS = _not_implemented
op_LOAD_NAME = _not_implemented
op_MAKE_CLOSURE = _not_implemented
op_MAKE_FUNCTION = _not_implemented
op_MAP_ADD = _not_implemented
op_NOP = _not_implemented
op_POP_BLOCK = _not_implemented
op_POP_EXCEPT = _not_implemented
op_POP_JUMP_IF_FALSE = _not_implemented
op_POP_JUMP_IF_TRUE = _not_implemented
op_POP_TOP = _not_implemented
op_PRINT_EXPR = _not_implemented
op_PRINT_ITEM = _not_implemented
op_PRINT_ITEM_TO = _not_implemented
op_PRINT_NEWLINE = _not_implemented
op_PRINT_NEWLINE_TO = _not_implemented
op_RAISE_VARARGS = _not_implemented
op_RETURN_VALUE = _not_implemented
op_ROT_FOUR = _not_implemented
op_ROT_THREE = _not_implemented
op_ROT_TWO = _not_implemented
op_SETUP_EXCEPT = _not_implemented
op_SETUP_FINALLY = _not_implemented
op_SETUP_LOOP = _not_implemented
op_SETUP_WITH = _not_implemented
op_SET_ADD = _not_implemented
op_SLICE = _not_implemented
op_STOP_CODE = _not_implemented
op_STORE_ATTR = _not_implemented
op_STORE_DEREF = _not_implemented
op_STORE_FAST = _not_implemented
op_STORE_GLOBAL = _not_implemented
op_STORE_LOCALS = _not_implemented
op_STORE_MAP = _not_implemented
op_STORE_NAME = _not_implemented
op_STORE_SLICE = _not_implemented
op_STORE_SUBSCR = _not_implemented
op_UNARY_CONVERT = _not_implemented
op_UNARY_INVERT = _not_implemented
op_UNARY_NEGATIVE = _not_implemented
op_UNARY_NOT = _not_implemented
op_UNARY_POSITIVE = _not_implemented
op_UNPACK_EX = _not_implemented
op_UNPACK_SEQUENCE = _not_implemented
op_WITH_CLEANUP = _not_implemented
op_YIELD_VALUE = _not_implemented
# ______________________________________________________________________
class BytecodeIterVisitor (BytecodeVisitor):
def visit (self, co_obj):
self.enter_code_object(co_obj)
for i, op, arg in itercode(co_obj.co_code):
self.visit_op(i, op, arg)
return self.exit_code_object(co_obj)
def enter_code_object (self, co_obj):
pass
def exit_code_object (self, co_obj):
pass
# ______________________________________________________________________
class BasicBlockVisitor (BytecodeVisitor):
def visit (self, blocks):
self.enter_blocks(blocks)
block_indices = list(blocks.keys())
block_indices.sort()
for block_index in block_indices:
self.enter_block(block_index)
for i, op, arg in blocks[block_index]:
self.visit_op(i, op, arg)
self.exit_block(block_index)
return self.exit_blocks(blocks)
def enter_blocks (self, blocks):
pass
def exit_blocks (self, blocks):
pass
def enter_block (self, block_index):
pass
def exit_block (self, block_index):
pass
# ______________________________________________________________________
class BytecodeFlowVisitor (BytecodeVisitor):
def visit (self, flow):
self.block_list = list(flow.keys())
self.block_list.sort()
self.enter_flow_object(flow)
for block in self.block_list:
prelude = self.enter_block(block)
prelude_isa_list = isinstance(prelude, list)
if prelude or prelude_isa_list:
if not prelude_isa_list:
prelude = []
new_stmts = list(self.visit_op(i, op, arg, *args)
for i, op, _, arg, args in flow[block])
self.new_flow[block] = list(itertools.chain(
prelude, *new_stmts))
self.exit_block(block)
del self.block_list
return self.exit_flow_object(flow)
def visit_op (self, i, op, arg, *args, **kws):
new_args = []
for child_i, child_op, _, child_arg, child_args in args:
new_args.extend(self.visit_op(child_i, child_op, child_arg,
*child_args))
ret_val = super(BytecodeFlowVisitor, self).visit_op(i, op, arg,
*new_args)
return ret_val
def enter_flow_object (self, flow):
self.new_flow = {}
def exit_flow_object (self, flow):
ret_val = self.new_flow
del self.new_flow
return ret_val
def enter_block (self, block):
pass
def exit_block (self, block):
pass
# ______________________________________________________________________
class BenignBytecodeVisitorMixin (object):
def _do_nothing (self, i, op, arg, *args, **kws):
return [(i, op, self.opnames[op], arg, args)]
op_BINARY_ADD = _do_nothing
op_BINARY_AND = _do_nothing
op_BINARY_DIVIDE = _do_nothing
op_BINARY_FLOOR_DIVIDE = _do_nothing
op_BINARY_LSHIFT = _do_nothing
op_BINARY_MODULO = _do_nothing
op_BINARY_MULTIPLY = _do_nothing
op_BINARY_OR = _do_nothing
op_BINARY_POWER = _do_nothing
op_BINARY_RSHIFT = _do_nothing
op_BINARY_SUBSCR = _do_nothing
op_BINARY_SUBTRACT = _do_nothing
op_BINARY_TRUE_DIVIDE = _do_nothing
op_BINARY_XOR = _do_nothing
op_BREAK_LOOP = _do_nothing
op_BUILD_CLASS = _do_nothing
op_BUILD_LIST = _do_nothing
op_BUILD_MAP = _do_nothing
op_BUILD_SET = _do_nothing
op_BUILD_SLICE = _do_nothing
op_BUILD_TUPLE = _do_nothing
op_CALL_FUNCTION = _do_nothing
op_CALL_FUNCTION_KW = _do_nothing
op_CALL_FUNCTION_VAR = _do_nothing
op_CALL_FUNCTION_VAR_KW = _do_nothing
op_COMPARE_OP = _do_nothing
op_CONTINUE_LOOP = _do_nothing
op_DELETE_ATTR = _do_nothing
op_DELETE_DEREF = _do_nothing
op_DELETE_FAST = _do_nothing
op_DELETE_GLOBAL = _do_nothing
op_DELETE_NAME = _do_nothing
op_DELETE_SLICE = _do_nothing
op_DELETE_SUBSCR = _do_nothing
op_DUP_TOP = _do_nothing
op_DUP_TOPX = _do_nothing
op_DUP_TOP_TWO = _do_nothing
op_END_FINALLY = _do_nothing
op_EXEC_STMT = _do_nothing
op_EXTENDED_ARG = _do_nothing
op_FOR_ITER = _do_nothing
op_GET_ITER = _do_nothing
op_IMPORT_FROM = _do_nothing
op_IMPORT_NAME = _do_nothing
op_IMPORT_STAR = _do_nothing
op_INPLACE_ADD = _do_nothing
op_INPLACE_AND = _do_nothing
op_INPLACE_DIVIDE = _do_nothing
op_INPLACE_FLOOR_DIVIDE = _do_nothing
op_INPLACE_LSHIFT = _do_nothing
op_INPLACE_MODULO = _do_nothing
op_INPLACE_MULTIPLY = _do_nothing
op_INPLACE_OR = _do_nothing
op_INPLACE_POWER = _do_nothing
op_INPLACE_RSHIFT = _do_nothing
op_INPLACE_SUBTRACT = _do_nothing
op_INPLACE_TRUE_DIVIDE = _do_nothing
op_INPLACE_XOR = _do_nothing
op_JUMP_ABSOLUTE = _do_nothing
op_JUMP_FORWARD = _do_nothing
op_JUMP_IF_FALSE = _do_nothing
op_JUMP_IF_FALSE_OR_POP = _do_nothing
op_JUMP_IF_TRUE = _do_nothing
op_JUMP_IF_TRUE_OR_POP = _do_nothing
op_LIST_APPEND = _do_nothing
op_LOAD_ATTR = _do_nothing
op_LOAD_BUILD_CLASS = _do_nothing
op_LOAD_CLOSURE = _do_nothing
op_LOAD_CONST = _do_nothing
op_LOAD_DEREF = _do_nothing
op_LOAD_FAST = _do_nothing
op_LOAD_GLOBAL = _do_nothing
op_LOAD_LOCALS = _do_nothing
op_LOAD_NAME = _do_nothing
op_MAKE_CLOSURE = _do_nothing
op_MAKE_FUNCTION = _do_nothing
op_MAP_ADD = _do_nothing
op_NOP = _do_nothing
op_POP_BLOCK = _do_nothing
op_POP_EXCEPT = _do_nothing
op_POP_JUMP_IF_FALSE = _do_nothing
op_POP_JUMP_IF_TRUE = _do_nothing
op_POP_TOP = _do_nothing
op_PRINT_EXPR = _do_nothing
op_PRINT_ITEM = _do_nothing
op_PRINT_ITEM_TO = _do_nothing
op_PRINT_NEWLINE = _do_nothing
op_PRINT_NEWLINE_TO = _do_nothing
op_RAISE_VARARGS = _do_nothing
op_RETURN_VALUE = _do_nothing
op_ROT_FOUR = _do_nothing
op_ROT_THREE = _do_nothing
op_ROT_TWO = _do_nothing
op_SETUP_EXCEPT = _do_nothing
op_SETUP_FINALLY = _do_nothing
op_SETUP_LOOP = _do_nothing
op_SETUP_WITH = _do_nothing
op_SET_ADD = _do_nothing
op_SLICE = _do_nothing
op_STOP_CODE = _do_nothing
op_STORE_ATTR = _do_nothing
op_STORE_DEREF = _do_nothing
op_STORE_FAST = _do_nothing
op_STORE_GLOBAL = _do_nothing
op_STORE_LOCALS = _do_nothing
op_STORE_MAP = _do_nothing
op_STORE_NAME = _do_nothing
op_STORE_SLICE = _do_nothing
op_STORE_SUBSCR = _do_nothing
op_UNARY_CONVERT = _do_nothing
op_UNARY_INVERT = _do_nothing
op_UNARY_NEGATIVE = _do_nothing
op_UNARY_NOT = _do_nothing
op_UNARY_POSITIVE = _do_nothing
op_UNPACK_EX = _do_nothing
op_UNPACK_SEQUENCE = _do_nothing
op_WITH_CLEANUP = _do_nothing
op_YIELD_VALUE = _do_nothing
# ______________________________________________________________________
# End of bytecode_visitor.py

42
llpython/bytetype.py
View file

@ -1,42 +0,0 @@
# ______________________________________________________________________
import ctypes
import llvm.core as lc
# ______________________________________________________________________
lvoid = lc.Type.void()
li1 = lc.Type.int(1)
li8 = lc.Type.int(8)
li16 = lc.Type.int(16)
li32 = lc.Type.int(32)
li64 = lc.Type.int(64)
liptr = lc.Type.int(ctypes.sizeof(ctypes.c_void_p) * 8)
lc_size_t = lc.Type.int(ctypes.sizeof(
getattr(ctypes, 'c_ssize_t', getattr(ctypes, 'c_size_t'))) * 8)
lfloat = lc.Type.float()
ldouble = lc.Type.double()
li8_ptr = lc.Type.pointer(li8)
lc_int = lc.Type.int(ctypes.sizeof(ctypes.c_int) * 8)
lc_long = lc.Type.int(ctypes.sizeof(ctypes.c_long) * 8)
l_pyobject_head = [lc_size_t, lc.Type.pointer(li32)]
l_pyobject_head_struct = lc.Type.struct(l_pyobject_head)
l_pyobj_p = l_pyobject_head_struct_p = lc.Type.pointer(l_pyobject_head_struct)
l_pyfunc = lc.Type.function(l_pyobj_p, (l_pyobj_p, l_pyobj_p))
strlen = lc.Type.function(lc_size_t, (li8_ptr,))
strncpy = lc.Type.function(li8_ptr, (li8_ptr, li8_ptr, lc_size_t))
strndup = lc.Type.function(li8_ptr, (li8_ptr, lc_size_t))
malloc = lc.Type.function(li8_ptr, (lc_size_t,))
free = lc.Type.function(lvoid, (li8_ptr,))
Py_BuildValue = lc.Type.function(l_pyobj_p, [li8_ptr], True)
PyArg_ParseTuple = lc.Type.function(lc_int, [l_pyobj_p, li8_ptr], True)
PyEval_SaveThread = lc.Type.function(li8_ptr, [])
PyEval_RestoreThread = lc.Type.function(lc.Type.void(), [li8_ptr])
# ______________________________________________________________________
# End of bytetype.py

217
llpython/control_flow.py
View file

@ -1,217 +0,0 @@
# ______________________________________________________________________
import pprint
# ______________________________________________________________________
class ControlFlowGraph (object):
def __init__ (self):
self.blocks = {}
self.blocks_in = {}
self.blocks_out = {}
self.blocks_reads = {}
self.blocks_writes = {}
self.blocks_writer = {}
self.blocks_dom = {}
self.blocks_reaching = {}
def add_block (self, key, value = None):
self.blocks[key] = value
if key not in self.blocks_in:
self.blocks_in[key] = set()
self.blocks_out[key] = set()
self.blocks_reads[key] = set()
self.blocks_writes[key] = set()
self.blocks_writer[key] = {}
def add_edge (self, from_block, to_block):
self.blocks_out[from_block].add(to_block)
self.blocks_in[to_block].add(from_block)
def unlink_unreachables (self):
changed = True
next_blocks = self.blocks.keys()
next_blocks.remove(0)
while changed:
changed = False
blocks = next_blocks
next_blocks = blocks[:]
for block in blocks:
if len(self.blocks_in[block]) == 0:
blocks_out = self.blocks_out[block]
for out_edge in blocks_out:
self.blocks_in[out_edge].discard(block)
blocks_out.clear()
next_blocks.remove(block)
changed = True
def compute_dataflow (self):
'''Compute the dominator and reaching dataflow relationships
in the CFG.'''
blocks = set(self.blocks.keys())
nonentry_blocks = blocks.copy()
for block in blocks:
self.blocks_dom[block] = blocks
self.blocks_reaching[block] = set((block,))
if len(self.blocks_in[block]) == 0:
self.blocks_dom[block] = set((block,))
nonentry_blocks.remove(block)
changed = True
while changed:
changed = False
for block in nonentry_blocks:
olddom = self.blocks_dom[block]
newdom = set.intersection(*[self.blocks_dom[pred]
for pred in self.blocks_in[block]])
newdom.add(block)
if newdom != olddom:
changed = True
self.blocks_dom[block] = newdom
oldreaching = self.blocks_reaching[block]
newreaching = set.union(
*[self.blocks_reaching[pred]
for pred in self.blocks_in[block]])
newreaching.add(block)
if newreaching != oldreaching:
changed = True
self.blocks_reaching[block] = newreaching
return self.blocks_dom, self.blocks_reaching
def update_for_ssa (self):
'''Modify the blocks_writes map to reflect phi nodes inserted
for static single assignment representations.'''
joins = [block for block in self.blocks.keys()
if len(self.blocks_in[block]) > 1]
changed = True
while changed:
changed = False
for block in joins:
phis_needed = self.phi_needed(block)
for affected_local in phis_needed:
if affected_local not in self.blocks_writes[block]:
changed = True
# NOTE: For this to work, we assume that basic
# blocks are indexed by their instruction
# index in the VM bytecode.
self.writes_local(block, block, affected_local)
if changed:
# Any modifications have invalidated the reaching
# definitions, so delete any memoized results.
if hasattr(self, 'reaching_definitions'):
del self.reaching_definitions
def idom (self, block):
'''Compute the immediate dominator (idom) of the given block
key. Returns None if the block has no in edges.
Note that in the case where there are multiple immediate
dominators (a join after a non-loop branch), this returns one
of the predecessors, but is not guaranteed to reliably select
one over the others (depends on the ordering of the set type
iterator).'''
preds = self.blocks_in[block]
npreds = len(preds)
if npreds == 0:
ret_val = None
elif npreds == 1:
ret_val = tuple(preds)[0]
else:
ret_val = [pred for pred in preds
if block not in self.blocks_dom[pred]][0]
return ret_val
def block_writes_to_writer_map (self, block):
ret_val = {}
for local in self.blocks_writes[block]:
ret_val[local] = block
return ret_val
def get_reaching_definitions (self, block):
'''Return a nested map for the given block
s.t. ret_val[pred][local] equals the block key for the
definition of local that reaches the argument block via that
predecessor.
Useful for actually populating phi nodes, once you know you
need them.'''
has_memoized = hasattr(self, 'reaching_definitions')
if has_memoized and block in self.reaching_definitions:
ret_val = self.reaching_definitions[block]
else:
preds = self.blocks_in[block]
ret_val = {}
for pred in preds:
ret_val[pred] = self.block_writes_to_writer_map(pred)
crnt = self.idom(pred)
while crnt != None:
crnt_writer_map = self.block_writes_to_writer_map(crnt)
# This order of update favors the first definitions
# encountered in the traversal since the traversal
# visits blocks in reverse execution order.
crnt_writer_map.update(ret_val[pred])
ret_val[pred] = crnt_writer_map
crnt = self.idom(crnt)
if not has_memoized:
self.reaching_definitions = {}
self.reaching_definitions[block] = ret_val
return ret_val
def nreaches (self, block):
'''For each local, find the number of unique reaching
definitions the current block has.'''
reaching_definitions = self.get_reaching_definitions(block)
definition_map = {}
for pred in self.blocks_in[block]:
reaching_from_pred = reaching_definitions[pred]
for local in reaching_from_pred.keys():
if local not in definition_map:
definition_map[local] = set()
definition_map[local].add(reaching_from_pred[local])
ret_val = {}
for local in definition_map.keys():
ret_val[local] = len(definition_map[local])
return ret_val
def writes_local (self, block, write_instr_index, local_index):
self.blocks_writes[block].add(local_index)
block_writers = self.blocks_writer[block]
old_index = block_writers.get(local_index, -1)
# This checks for a corner case that would impact
# numba.translate.Translate.build_phi_nodes().
assert old_index != write_instr_index, (
"Found corner case for STORE_FAST at a CFG join!")
block_writers[local_index] = max(write_instr_index, old_index)
def phi_needed (self, join):
'''Return the set of locals that will require a phi node to be
generated at the given join.'''
nreaches = self.nreaches(join)
return set([local for local in nreaches.keys()
if nreaches[local] > 1])
def pprint (self, *args, **kws):
pprint.pprint(self.__dict__, *args, **kws)
def pformat (self, *args, **kws):
return pprint.pformat(self.__dict__, *args, **kws)
def to_dot (self, graph_name = None):
'''Return a dot (digraph visualizer in Graphviz) graph
description as a string.'''
if graph_name is None:
graph_name = 'CFG_%d' % id(self)
lines_out = []
for block_index in self.blocks:
lines_out.append(
'BLOCK_%r [shape=box, label="BLOCK_%r\\nr: %r, w: %r"];' %
(block_index, block_index,
tuple(self.blocks_reads[block_index]),
tuple(self.blocks_writes[block_index])))
for block_index in self.blocks:
for out_edge in self.blocks_out[block_index]:
lines_out.append('BLOCK_%r -> BLOCK_%r;' %
(block_index, out_edge))
return 'digraph %s {\n%s\n}\n' % (graph_name, '\n'.join(lines_out))
# ______________________________________________________________________
# End of control_flow.py

26
llpython/gen_bytecode_visitor.py
View file

@ -1,26 +0,0 @@
# ______________________________________________________________________
from __future__ import absolute_import
from . import opcode_util
# ______________________________________________________________________
def generate_bytecode_visitor (classname = 'BytecodeVisitor',
baseclass = 'object'):
opnames = list(set((opname.split('+')[0]
for opname in opcode_util.OPCODE_MAP.keys())))
opnames.sort()
return 'class %s (%s):\n%s\n' % (
classname, baseclass,
'\n\n'.join((' def op_%s (self, i, op, arg):\n'
' raise NotImplementedError("%s.op_%s")' %
(opname, classname, opname)
for opname in opnames)))
# ______________________________________________________________________
if __name__ == "__main__":
import sys
print(generate_bytecode_visitor(*sys.argv[1:]))
# ______________________________________________________________________
# End of gen_bytecode_visitor.py

367
llpython/nobitey.py
View file

@ -1,367 +0,0 @@
# ______________________________________________________________________
from __future__ import absolute_import
import sys
import os.path
import imp
import io
import types
import llvm.core as lc
import llvm.ee as le
from . import bytetype, byte_translator
from .pyaddfunc import pyaddfunc
LLVM_TO_INT_PARSE_STR_MAP = {
8 : 'b',
16 : 'h',
32 : 'i', # Note that on 32-bit systems sizeof(int) == sizeof(long)
64 : 'L', # Seeing sizeof(long long) == 8 on both 32 and 64-bit platforms
}
LLVM_TO_PARSE_STR_MAP = {
lc.TYPE_FLOAT : 'f',
lc.TYPE_DOUBLE : 'd',
}
# ______________________________________________________________________
# XXX Stolen from numba.translate
def get_string_constant (module, const_str):
const_name = "__STR_%x" % (hash(const_str),)
try:
ret_val = module.get_global_variable_named(const_name)
except:
lconst_str = lc.Constant.stringz(const_str)
ret_val = module.add_global_variable(lconst_str.type, const_name)
ret_val.initializer = lconst_str
ret_val.linkage = lc.LINKAGE_INTERNAL
return ret_val
# ______________________________________________________________________
class NoBitey (object):
def __init__ (self, target_module = None, type_annotations = None):
if target_module is None:
target_module = lc.Module.new('NoBitey_%d' % id(self))
if type_annotations is None:
type_annotations = {}
self.target_module = target_module
self.type_aliases = type_annotations # Reserved for future use.
def _build_parse_string (self, llvm_type):
kind = llvm_type.kind
if kind == lc.TYPE_INTEGER:
ret_val = LLVM_TO_INT_PARSE_STR_MAP[llvm_type.width]
elif kind in LLVM_TO_PARSE_STR_MAP:
ret_val = LLVM_TO_PARSE_STR_MAP[kind]
else:
raise TypeError('Unsupported LLVM type: %s' % str(llvm_type))
return ret_val
def build_parse_string (self, llvm_tys):
"""Given a set of LLVM types, return a string for parsing
them via PyArg_ParseTuple."""
return ''.join((self._build_parse_string(ty)
for ty in llvm_tys))
def handle_abi_casts (self, builder, result):
if result.type.kind == lc.TYPE_FLOAT:
# NOTE: The C ABI apparently casts floats to doubles when
# an argument must be pushed on the stack, as is the case
# when calling a variable argument function.
# XXX Is there documentation on this where I can find all
# coercion rules? Do we still need some libffi
# integration?
result = builder.fpext(result, bytetype.ldouble)
return result
def build_wrapper_function (self, llvm_function, engine = None):
arg_types = llvm_function.type.pointee.args
return_type = llvm_function.type.pointee.return_type
li32_0 = lc.Constant.int(bytetype.li32, 0)
def get_llvm_function (builder):
if self.target_module != llvm_function.module:
llvm_function_ptr = self.target_module.add_global_variable(
llvm_function.type, llvm_function.name)
llvm_function_ptr.initializer = lc.Constant.inttoptr(
lc.Constant.int(
bytetype.liptr,
engine.get_pointer_to_function(llvm_function)),
llvm_function.type)
llvm_function_ptr.linkage = lc.LINKAGE_INTERNAL
ret_val = builder.load(llvm_function_ptr)
else:
ret_val = llvm_function
return ret_val
def build_parse_args (builder):
return [builder.alloca(arg_type) for arg_type in arg_types]
def build_parse_string (builder):
parse_str = get_string_constant(
self.target_module, self.build_parse_string(arg_types))
return builder.gep(parse_str, (li32_0, li32_0))
def load_target_args (builder, args):
return [builder.load(arg) for arg in args]
def build_build_string (builder):
build_str = get_string_constant(
self.target_module, self._build_parse_string(return_type))
return builder.gep(build_str, (li32_0, li32_0))
handle_abi_casts = self.handle_abi_casts
target_function_name = llvm_function.name + "_wrapper"
# __________________________________________________
@byte_translator.llpython(bytetype.l_pyfunc, self.target_module,
**locals())
def _wrapper (self, args):
ret_val = l_pyobj_p(0)
parse_args = build_parse_args()
parse_result = PyArg_ParseTuple(args, build_parse_string(),
*parse_args)
if parse_result != li32(0):
thread_state = PyEval_SaveThread()
target_args = load_target_args(parse_args)
llresult = handle_abi_casts(get_llvm_function()(*target_args))
PyEval_RestoreThread(thread_state)
ret_val = Py_BuildValue(build_build_string(), llresult)
return ret_val
# __________________________________________________
return _wrapper
def wrap_llvm_module (self, llvm_module, engine = None, py_module = None):
'''
Shamefully adapted from bitey.bind.wrap_llvm_module().
'''
functions = [func for func in llvm_module.functions
if not func.name.startswith("_")
and not func.is_declaration
and func.linkage == lc.LINKAGE_EXTERNAL]
if engine is None:
engine = le.ExecutionEngine.new(llvm_module)
wrappers = [self.build_wrapper_function(func, engine)
for func in functions]
if __debug__: print(self.target_module)
if self.target_module != llvm_module:
engine.add_module(self.target_module)
py_wrappers = [pyaddfunc(wrapper.name,
engine.get_pointer_to_function(wrapper))
for wrapper in wrappers]
if py_module:
for py_wrapper in py_wrappers:
setattr(py_module, py_wrapper.__name__[:-8], py_wrapper)
setattr(py_module, '_llvm_module', llvm_module)
setattr(py_module, '_llvm_engine', engine)
if self.target_module != llvm_module:
setattr(py_module, '_llvm_wrappers', self.target_module)
return engine, py_wrappers
def wrap_llvm_module_in_python (self, llvm_module, py_module = None):
'''
Mildly reworked and abstracted bitey.bind.wrap_llvm_bitcode().
Abstracted to accept any existing LLVM Module object, and
return a Python wrapper module (even if one wasn't originally
specified).
'''
if py_module is None:
py_module = types.ModuleType(str(llvm_module.id))
engine = le.ExecutionEngine.new(llvm_module)
self.wrap_llvm_module(llvm_module, engine, py_module)
return py_module
def wrap_llvm_bitcode (self, bitcode, py_module = None):
'''
Intended to be drop-in replacement of
bitey.bind.wrap_llvm_bitcode().
'''
return self.wrap_llvm_module_in_python(
lc.Module.from_bitcode(io.BytesIO(bitcode)), py_module)
def wrap_llvm_assembly (self, llvm_asm, py_module = None):
return self.wrap_llvm_module_in_python(
lc.Module.from_assembly(io.BytesIO(llvm_asm)), py_module)
# ______________________________________________________________________
class NoBiteyLoader(object):
"""
Load LLVM compiled bitcode and autogenerate a ctypes binding.
Initially copied and adapted from bitey.loader module.
"""
def __init__(self, pkg, name, source, preload, postload):
self.package = pkg
self.name = name
self.fullname = '.'.join((pkg,name))
self.source = source
self.preload = preload
self.postload = postload
@classmethod
def _check_magic(cls, filename):
if os.path.exists(filename):
magic = open(filename,"rb").read(4)
if magic == b'\xde\xc0\x17\x0b':
return True
elif magic[:2] == b'\x42\x43':
return True
else:
return False
else:
return False
@classmethod
def build_module(cls, fullname, source_path, source_data, preload=None,
postload=None):
name = fullname.split(".")[-1]
mod = imp.new_module(name)
if preload:
exec(preload, mod.__dict__, mod.__dict__)
type_annotations = getattr(mod, '_type_annotations', None)
nb = NoBitey(type_annotations = type_annotations)
if source_path.endswith(('.o', '.bc')):
nb.wrap_llvm_bitcode(source_data, mod)
elif source_path.endswith('.s'):
nb.wrap_llvm_assembly(source_data, mod)
if postload:
exec(postload, mod.__dict__, mod.__dict__)
return mod
@classmethod
def find_module(cls, fullname, paths = None):
if paths is None:
paths = sys.path
names = fullname.split('.')
modname = names[-1]
source_paths = None
for f in paths:
path = os.path.join(os.path.realpath(f), modname)
source = path + '.o'
if cls._check_magic(source):
source_paths = path, source
break
source = path + '.bc'
if os.path.exists(source):
source_paths = path, source
break
source = path + '.s'
if os.path.exists(source):
source_paths = path, source
break
if source_paths:
path, source = source_paths
return cls('.'.join(names[:-1]), modname, source,
path + ".pre.py", path + ".post.py")
def get_code(self, module):
pass
def get_data(self, module):
pass
def get_filename(self, name):
return self.source
def get_source(self, name):
with open(self.source, 'rb') as f:
return f.read()
def is_package(self, *args, **kw):
return False
def load_module(self, fullname):
if fullname in sys.modules:
return sys.modules[fullname]
preload = None
postload = None
# Get the preload file (if any)
if os.path.exists(self.preload):
with open(self.preload) as f:
preload = f.read()
# Get the source
with open(self.source, 'rb') as f:
source_data = f.read()
# Get the postload file (if any)
if os.path.exists(self.postload):
with open(self.postload) as f:
postload = f.read()
mod = self.build_module(fullname, self.get_filename(None), source_data,
preload, postload)
sys.modules[fullname] = mod
mod.__loader__ = self
mod.__file__ = self.source
return mod
@classmethod
def install(cls):
if cls not in sys.meta_path:
sys.meta_path.append(cls)
@classmethod
def remove(cls):
sys.meta_path.remove(cls)
# ______________________________________________________________________
def _mk_add_42 (llvm_module, at_type = bytetype.lc_long):
f = llvm_module.add_function(
lc.Type.function(at_type, [at_type]), 'add_42_%s' % str(at_type))
block = f.append_basic_block('entry')
builder = lc.Builder.new(block)
if at_type.kind == lc.TYPE_INTEGER:
const_42 = lc.Constant.int(at_type, 42)
add = builder.add
elif at_type.kind in (lc.TYPE_FLOAT, lc.TYPE_DOUBLE):
const_42 = lc.Constant.real(at_type, 42.)
add = builder.fadd
else:
raise TypeError('Unsupported type: %s' % str(at_type))
builder.ret(add(f.args[0], const_42))
return f
# ______________________________________________________________________
def build_test_module ():
llvm_module = lc.Module.new('nobitey_test')
for ty in (bytetype.li32, bytetype.li64, bytetype.lfloat,
bytetype.ldouble):
fn = _mk_add_42(llvm_module, ty)
return llvm_module
# ______________________________________________________________________
def test_wrap_module (arg = None):
# Build up a module.
m = build_test_module()
if arg and arg.lower() == 'separated':
wrap_module = NoBitey().wrap_llvm_module_in_python(m)
else:
wrap_module = NoBitey(m).wrap_llvm_module_in_python(m)
# Now try running the generated wrappers.
for py_wf_name in ('add_42_i32', 'add_42_i64', 'add_42_float',
'add_42_double'):
py_wf = getattr(wrap_module, py_wf_name)
for i in range(42):
result = py_wf(i)
expected = i + 42
assert result == expected, "%r != %r in %r" % (
result, expected, py_wf)
return wrap_module
# ______________________________________________________________________
def main (*args):
if args:
for arg in args:
test_wrap_module(arg)
else:
test_wrap_module()
if __name__ == "__main__":
main(*sys.argv[1:])
# ______________________________________________________________________
# End of nobitey.py

225
llpython/opcode_util.py
View file

@ -1,225 +0,0 @@
# ______________________________________________________________________
import dis
import opcode
# ______________________________________________________________________
# Module data
hasjump = opcode.hasjrel + opcode.hasjabs
hascbranch = [op for op in hasjump
if 'IF' in opcode.opname[op]
or opcode.opname[op] in ('FOR_ITER', 'SETUP_LOOP')]
# Since the actual opcode value may change, manage opcode abstraction
# data by opcode name.
OPCODE_MAP = {
'BINARY_ADD': (2, 1, None),
'BINARY_AND': (2, 1, None),
'BINARY_DIVIDE': (2, 1, None),
'BINARY_FLOOR_DIVIDE': (2, 1, None),
'BINARY_LSHIFT': (2, 1, None),
'BINARY_MODULO': (2, 1, None),
'BINARY_MULTIPLY': (2, 1, None),
'BINARY_OR': (2, 1, None),
'BINARY_POWER': (2, 1, None),
'BINARY_RSHIFT': (2, 1, None),
'BINARY_SUBSCR': (2, 1, None),
'BINARY_SUBTRACT': (2, 1, None),
'BINARY_TRUE_DIVIDE': (2, 1, None),
'BINARY_XOR': (2, 1, None),
'BREAK_LOOP': (0, None, 1),
'BUILD_CLASS': (None, None, None),
'BUILD_LIST': (-1, 1, None),
'BUILD_MAP': (None, None, None),
'BUILD_SET': (None, None, None),
'BUILD_SLICE': (None, None, None),
'BUILD_TUPLE': (-1, 1, None),
'CALL_FUNCTION': (-2, 1, None),
'CALL_FUNCTION_KW': (-3, 1, None),
'CALL_FUNCTION_VAR': (-3, 1, None),
'CALL_FUNCTION_VAR_KW': (-4, 1, None),
'COMPARE_OP': (2, 1, None),
'CONTINUE_LOOP': (None, None, None),
'DELETE_ATTR': (1, None, 1),
'DELETE_DEREF': (None, None, None),
'DELETE_FAST': (0, None, 1),
'DELETE_GLOBAL': (0, None, 1),
'DELETE_NAME': (0, None, 1),
'DELETE_SLICE+0': (1, None, 1),
'DELETE_SLICE+1': (2, None, 1),
'DELETE_SLICE+2': (2, None, 1),
'DELETE_SLICE+3': (3, None, 1),
'DELETE_SUBSCR': (2, None, 1),
'DUP_TOP': (None, None, None),
'DUP_TOPX': (None, None, None),
'DUP_TOP_TWO': (None, None, None),
'END_FINALLY': (None, None, None),
'EXEC_STMT': (None, None, None),
'EXTENDED_ARG': (None, None, None),
'FOR_ITER': (1, 1, 1),
'GET_ITER': (1, 1, None),
'IMPORT_FROM': (None, None, None),
'IMPORT_NAME': (None, None, None),
'IMPORT_STAR': (1, None, 1),
'INPLACE_ADD': (2, 1, None),
'INPLACE_AND': (2, 1, None),
'INPLACE_DIVIDE': (2, 1, None),
'INPLACE_FLOOR_DIVIDE': (2, 1, None),
'INPLACE_LSHIFT': (2, 1, None),
'INPLACE_MODULO': (2, 1, None),
'INPLACE_MULTIPLY': (2, 1, None),
'INPLACE_OR': (2, 1, None),
'INPLACE_POWER': (2, 1, None),
'INPLACE_RSHIFT': (2, 1, None),
'INPLACE_SUBTRACT': (2, 1, None),
'INPLACE_TRUE_DIVIDE': (2, 1, None),
'INPLACE_XOR': (2, 1, None),
'JUMP_ABSOLUTE': (0, None, 1),
'JUMP_FORWARD': (0, None, 1),
'JUMP_IF_FALSE': (1, 1, 1),
'JUMP_IF_FALSE_OR_POP': (None, None, None),
'JUMP_IF_TRUE': (1, 1, 1),
'JUMP_IF_TRUE_OR_POP': (None, None, None),
'LIST_APPEND': (2, 0, 1),
'LOAD_ATTR': (1, 1, None),
'LOAD_BUILD_CLASS': (None, None, None),
'LOAD_CLOSURE': (None, None, None),
'LOAD_CONST': (0, 1, None),
'LOAD_DEREF': (0, 1, None),
'LOAD_FAST': (0, 1, None),
'LOAD_GLOBAL': (0, 1, None),
'LOAD_LOCALS': (None, None, None),
'LOAD_NAME': (0, 1, None),
'MAKE_CLOSURE': (None, None, None),
'MAKE_FUNCTION': (-2, 1, None),
'MAP_ADD': (None, None, None),
'NOP': (0, None, None),
'POP_BLOCK': (0, None, 1),
'POP_EXCEPT': (None, None, None),
'POP_JUMP_IF_FALSE': (1, None, 1),
'POP_JUMP_IF_TRUE': (1, None, 1),
'POP_TOP': (1, None, 1),
'PRINT_EXPR': (1, None, 1),
'PRINT_ITEM': (1, None, 1),
'PRINT_ITEM_TO': (2, None, 1),
'PRINT_NEWLINE': (0, None, 1),
'PRINT_NEWLINE_TO': (1, None, 1),
'RAISE_VARARGS': (None, None, None),
'RETURN_VALUE': (1, None, 1),
'ROT_FOUR': (None, None, None),
'ROT_THREE': (None, None, None),
'ROT_TWO': (None, None, None),
'SETUP_EXCEPT': (None, None, None),
'SETUP_FINALLY': (None, None, None),
'SETUP_LOOP': (None, None, None),
'SETUP_WITH': (None, None, None),
'SET_ADD': (None, None, None),
'SLICE+0': (1, 1, None),
'SLICE+1': (2, 1, None),
'SLICE+2': (2, 1, None),
'SLICE+3': (3, 1, None),
'STOP_CODE': (None, None, None),
'STORE_ATTR': (2, None, 1),
'STORE_DEREF': (1, 0, 1),
'STORE_FAST': (1, None, 1),
'STORE_GLOBAL': (1, None, 1),
'STORE_LOCALS': (None, None, None),
'STORE_MAP': (1, None, 1),
'STORE_NAME': (1, None, 1),
'STORE_SLICE+0': (1, None, 1),
'STORE_SLICE+1': (2, None, 1),
'STORE_SLICE+2': (2, None, 1),
'STORE_SLICE+3': (3, None, 1),
'STORE_SUBSCR': (3, None, 1),
'UNARY_CONVERT': (1, 1, None),
'UNARY_INVERT': (1, 1, None),
'UNARY_NEGATIVE': (1, 1, None),
'UNARY_NOT': (1, 1, None),
'UNARY_POSITIVE': (1, 1, None),
'UNPACK_EX': (None, None, None),
'UNPACK_SEQUENCE': (None, None, None),
'WITH_CLEANUP': (None, None, None),
'YIELD_VALUE': (1, None, 1),
}
# ______________________________________________________________________
# Module functions
def itercode(code, start = 0):
"""Return a generator of byte-offset, opcode, and argument
from a byte-code-string
"""
i = 0
extended_arg = 0
if isinstance(code[0], str):
code = [ord(c) for c in code]
n = len(code)
while i < n:
op = code[i]
num = i + start
i = i + 1
oparg = None
if op >= opcode.HAVE_ARGUMENT:
oparg = code[i] + (code[i + 1] * 256) + extended_arg
extended_arg = 0
i = i + 2
if op == opcode.EXTENDED_ARG:
extended_arg = oparg * 65536
delta = yield num, op, oparg
if delta is not None:
abs_rel, dst = delta
assert abs_rel == 'abs' or abs_rel == 'rel'
i = dst if abs_rel == 'abs' else i + dst
# ______________________________________________________________________
def extendlabels(code, labels = None):
"""Extend the set of jump target labels to account for the
passthrough targets of conditional branches.
This allows us to create a control flow graph where there is at
most one branch per basic block.
"""
if labels is None:
labels = []
if isinstance(code[0], str):
code = [ord(c) for c in code]
n = len(code)
i = 0
while i < n:
op = code[i]
i += 1
if op >= dis.HAVE_ARGUMENT:
i += 2
label = -1
if op in hasjump:
label = i
if label >= 0:
if label not in labels:
labels.append(label)
elif op == opcode.opmap['BREAK_LOOP']:
if i not in labels:
labels.append(i)
return labels
# ______________________________________________________________________
def get_code_object (func):
return getattr(func, '__code__', getattr(func, 'func_code', None))
# ______________________________________________________________________
def build_basic_blocks (co_obj):
co_code = co_obj.co_code
labels = extendlabels(co_code, dis.findlabels(co_code))
labels.sort()
blocks = dict((index, list(itercode(co_code[index:next_index], index)))
for index, next_index in zip([0] + labels,
labels + [len(co_code)]))
return blocks
# ______________________________________________________________________
# End of opcode_util.py

153
llpython/phi_injector.py
View file

@ -1,153 +0,0 @@
# ______________________________________________________________________
from .bytecode_visitor import BytecodeFlowVisitor, BenignBytecodeVisitorMixin
# ______________________________________________________________________
synthetic_opname = []
synthetic_opmap = {}
def def_synth_op (opname):
global synthetic_opname, synthetic_opmap
ret_val = -(len(synthetic_opname) + 1)
synthetic_opname.insert(0, opname)
synthetic_opmap[opname] = ret_val
return ret_val
REF_ARG = def_synth_op('REF_ARG')
BUILD_PHI = def_synth_op('BUILD_PHI')
DEFINITION = def_synth_op('DEFINITION')
REF_DEF = def_synth_op('REF_DEF')
# ______________________________________________________________________
class PhiInjector (BenignBytecodeVisitorMixin, BytecodeFlowVisitor):
'''Transformer responsible for modifying a bytecode flow, removing
LOAD_FAST and STORE_FAST opcodes, and replacing them with a static
single assignment (SSA) representation.
In order to support SSA, PhiInjector adds the following synthetic
opcodes to transformed flows:
* REF_ARG: Specifically reference an incomming argument value.
* BUILD_PHI: Build a phi node to disambiguate between several
possible definitions at a control flow join.
* DEFINITION: Unique value definition indexed by the "arg" field
in the tuple.
* REF_DEF: Reference a specific value definition.'''
def visit_cfg (self, cfg, nargs = 0, *args, **kws):
self.cfg = cfg
ret_val = self.visit(cfg.blocks, nargs)
del self.cfg
return ret_val
def visit (self, flow, nargs = 0, *args, **kws):
self.nargs = nargs
self.definitions = []
self.phis = []
self.prev_blocks = []
self.blocks_locals = dict((block, {})
for block in self.cfg.blocks.keys())
ret_val = super(PhiInjector, self).visit(flow, *args, **kws)
for block, _, _, args, _ in self.phis:
local = args.pop()
reaching_definitions = self.cfg.reaching_definitions[block]
for prev in reaching_definitions.keys():
if 0 in self.cfg.blocks_reaching[prev]:
args.append((prev, REF_DEF, 'REF_DEF',
self.blocks_locals[prev][local], ()))
args.sort()
del self.blocks_locals
del self.prev_blocks
del self.phis
del self.definitions
del self.nargs
return ret_val
def add_definition (self, index, local, arg):
definition_index = len(self.definitions)
definition = (index, DEFINITION, 'DEFINITION', definition_index,
(arg,))
self.definitions.append(definition)
self.blocks_locals[self.block][local] = definition_index
return definition
def add_phi (self, index, local):
ret_val = (index, BUILD_PHI, 'BUILD_PHI', [local], ())
self.phis.append(ret_val)
return ret_val
def enter_block (self, block):
ret_val = False
self.block = block
if block == 0:
if self.nargs > 0:
ret_val = [self.add_definition(-1, arg,
(-1, REF_ARG, 'REF_ARG', arg,
()))
for arg in range(self.nargs)]
else:
ret_val = True
elif 0 in self.cfg.blocks_reaching[block]:
ret_val = True
prev_block_locals = None
for pred_block in self.cfg.blocks_in[block]:
if pred_block in self.prev_blocks:
prev_block_locals = self.blocks_locals[pred_block]
break
assert prev_block_locals is not None, "Internal translation error"
self.blocks_locals[block] = prev_block_locals.copy()
phis_needed = self.cfg.phi_needed(block)
if phis_needed:
ret_val = [self.add_definition(block, local,
self.add_phi(block, local))
for local in phis_needed]
return ret_val
def exit_block (self, block):
if 0 in self.cfg.blocks_reaching[block]:
self.prev_blocks.append(block)
del self.block
def op_STORE_FAST (self, i, op, arg, *args, **kws):
assert len(args) == 1
return [self.add_definition(i, arg, args[0])]
def op_LOAD_FAST (self, i, op, arg, *args, **kws):
return [(i, REF_DEF, 'REF_DEF', self.blocks_locals[self.block][arg],
args)]
# ______________________________________________________________________
def inject_phis (func):
'''Given a Python function, return a bytecode flow object that has
been transformed by a fresh PhiInjector instance.'''
import byte_control, byte_flow
argcount = byte_control.opcode_util.get_code_object(func).co_argcount
cfg = byte_control.build_cfg(func)
cfg.blocks = byte_flow.BytecodeFlowBuilder().visit_cfg(cfg)
return PhiInjector().visit_cfg(cfg, argcount)
# ______________________________________________________________________
# Main (self-test) routine
def main (*args):
import pprint
from tests import llfuncs
if not args:
args = ('doslice',)
for arg in args:
pprint.pprint(inject_phis(getattr(llfuncs, arg)))
# ______________________________________________________________________
if __name__ == "__main__":
import sys
main(*sys.argv[1:])
# ______________________________________________________________________
# End of phi_injector.py

40
llpython/pyaddfunc.py
View file

@ -1,40 +0,0 @@
# ______________________________________________________________________
import ctypes
# ______________________________________________________________________
class PyMethodDef (ctypes.Structure):
_fields_ = [
('ml_name', ctypes.c_char_p),
('ml_meth', ctypes.c_void_p),
('ml_flags', ctypes.c_int),
('ml_doc', ctypes.c_char_p),
]
PyCFunction_NewEx = ctypes.pythonapi.PyCFunction_NewEx
PyCFunction_NewEx.argtypes = (ctypes.POINTER(PyMethodDef),
ctypes.c_void_p,
ctypes.c_void_p)
PyCFunction_NewEx.restype = ctypes.py_object
cache = {} # Unsure if this is necessary to keep the PyMethodDef
# structures from being garbage collected. Assuming so...
def pyaddfunc (func_name, func_ptr, func_doc = None):
global cache
if bytes != str:
func_name = bytes(ord(ch) for ch in func_name)
key = (func_name, func_ptr)
if key in cache:
_, ret_val = cache[key]
else:
mdef = PyMethodDef(bytes(func_name),
func_ptr,
1, # == METH_VARARGS (hopefully remains so...)
func_doc)
ret_val = PyCFunction_NewEx(ctypes.byref(mdef), 0, 0)
cache[key] = (mdef, ret_val)
return ret_val
# ______________________________________________________________________
# End of pyaddfunc.py

Some files were not shown because too many files have changed in this diff Show more