Compare commits

..

2 commits

Author SHA1 Message Date
Ben Firshman
fa815962cf Merge pull request #1025 from aanand/add-deprecation-notice-to-website
Add deprecation banner to fig.sh
2015-02-25 19:22:01 +00:00
Aanand Prasad
d98ba29b62 Add deprecation banner to fig.sh
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
2015-02-25 16:40:18 +00:00
295 changed files with 4664 additions and 26117 deletions

View file

@ -1,9 +1 @@
*.egg-info
.coverage
.git .git
.tox
build
coverage-html
docs/_site
venv
.tox

7
.gitignore vendored
View file

@ -1,11 +1,8 @@
*.egg-info *.egg-info
*.pyc *.pyc
.coverage* .tox
/.tox
/build /build
/coverage-html
/dist /dist
/docs/_site /docs/_site
/venv /venv
README.rst fig.spec
compose/GITSHA

View file

@ -1,25 +0,0 @@
- repo: git://github.com/pre-commit/pre-commit-hooks
sha: 'v0.4.2'
hooks:
- id: check-added-large-files
- id: check-docstring-first
- id: check-merge-conflict
- id: check-yaml
- id: check-json
- id: debug-statements
- id: end-of-file-fixer
- id: flake8
- id: name-tests-test
exclude: 'tests/(integration/testcases\.py|helpers\.py)'
- id: requirements-txt-fixer
- id: trailing-whitespace
- repo: git://github.com/asottile/reorder_python_imports
sha: v0.1.0
hooks:
- id: reorder-python-imports
language_version: 'python2.7'
args:
- --add-import
- from __future__ import absolute_import
- --add-import
- from __future__ import unicode_literals

View file

@ -1,29 +0,0 @@
sudo: required
language: python
matrix:
include:
- os: linux
services:
- docker
- os: osx
language: generic
install: ./script/travis/install
script:
- ./script/travis/ci
- ./script/travis/build-binary
before_deploy:
- "./script/travis/render-bintray-config.py < ./script/travis/bintray.json.tmpl > ./bintray.json"
deploy:
provider: bintray
user: docker-compose-roleuser
key: '$BINTRAY_API_KEY'
file: ./bintray.json
skip_cleanup: true
on:
all_branches: true

File diff suppressed because it is too large Load diff

View file

@ -1 +0,0 @@
CHANGELOG.md

242
CHANGES.md Normal file
View file

@ -0,0 +1,242 @@
Change log
==========
1.0.1 (2014-11-04)
------------------
- Added an `--allow-insecure-ssl` option to allow `fig up`, `fig run` and `fig pull` to pull from insecure registries.
- Fixed `fig run` not showing output in Jenkins.
- Fixed a bug where Fig couldn't build Dockerfiles with ADD statements pointing at URLs.
1.0.0 (2014-10-16)
------------------
The highlights:
- [Fig has joined Docker.](https://www.orchardup.com/blog/orchard-is-joining-docker) Fig will continue to be maintained, but we'll also be incorporating the best bits of Fig into Docker itself.
This means the GitHub repository has moved to [https://github.com/docker/fig](https://github.com/docker/fig) and our IRC channel is now #docker-fig on Freenode.
- Fig can be used with the [official Docker OS X installer](https://docs.docker.com/installation/mac/). Boot2Docker will mount the home directory from your host machine so volumes work as expected.
- Fig supports Docker 1.3.
- It is now possible to connect to the Docker daemon using TLS by using the `DOCKER_CERT_PATH` and `DOCKER_TLS_VERIFY` environment variables.
- There is a new `fig port` command which outputs the host port binding of a service, in a similar way to `docker port`.
- There is a new `fig pull` command which pulls the latest images for a service.
- There is a new `fig restart` command which restarts a service's containers.
- Fig creates multiple containers in service by appending a number to the service name (e.g. `db_1`, `db_2`, etc). As a convenience, Fig will now give the first container an alias of the service name (e.g. `db`).
This link alias is also a valid hostname and added to `/etc/hosts` so you can connect to linked services using their hostname. For example, instead of resolving the environment variables `DB_PORT_5432_TCP_ADDR` and `DB_PORT_5432_TCP_PORT`, you could just use the hostname `db` and port `5432` directly.
- Volume definitions now support `ro` mode, expanding `~` and expanding environment variables.
- `.dockerignore` is supported when building.
- The project name can be set with the `FIG_PROJECT_NAME` environment variable.
- The `--env` and `--entrypoint` options have been added to `fig run`.
- The Fig binary for Linux is now linked against an older version of glibc so it works on CentOS 6 and Debian Wheezy.
Other things:
- `fig ps` now works on Jenkins and makes fewer API calls to the Docker daemon.
- `--verbose` displays more useful debugging output.
- When starting a service where `volumes_from` points to a service without any containers running, that service will now be started.
- Lots of docs improvements. Notably, environment variables are documented and official repositories are used throughout.
Thanks @dnephin, @d11wtq, @marksteve, @rubbish, @jbalonso, @timfreund, @alunduil, @mieciu, @shuron, @moss, @suzaku and @chmouel! Whew.
0.5.2 (2014-07-28)
------------------
- Added a `--no-cache` option to `fig build`, which bypasses the cache just like `docker build --no-cache`.
- Fixed the `dns:` fig.yml option, which was causing fig to error out.
- Fixed a bug where fig couldn't start under Python 2.6.
- Fixed a log-streaming bug that occasionally caused fig to exit.
Thanks @dnephin and @marksteve!
0.5.1 (2014-07-11)
------------------
- If a service has a command defined, `fig run [service]` with no further arguments will run it.
- The project name now defaults to the directory containing fig.yml, not the current working directory (if they're different)
- `volumes_from` now works properly with containers as well as services
- Fixed a race condition when recreating containers in `fig up`
Thanks @ryanbrainard and @d11wtq!
0.5.0 (2014-07-11)
------------------
- Fig now starts links when you run `fig run` or `fig up`.
For example, if you have a `web` service which depends on a `db` service, `fig run web ...` will start the `db` service.
- Environment variables can now be resolved from the environment that Fig is running in. Just specify it as a blank variable in your `fig.yml` and, if set, it'll be resolved:
```
environment:
RACK_ENV: development
SESSION_SECRET:
```
- `volumes_from` is now supported in `fig.yml`. All of the volumes from the specified services and containers will be mounted:
```
volumes_from:
- service_name
- container_name
```
- A host address can now be specified in `ports`:
```
ports:
- "0.0.0.0:8000:8000"
- "127.0.0.1:8001:8001"
```
- The `net` and `workdir` options are now supported in `fig.yml`.
- The `hostname` option now works in the same way as the Docker CLI, splitting out into a `domainname` option.
- TTY behaviour is far more robust, and resizes are supported correctly.
- Load YAML files safely.
Thanks to @d11wtq, @ryanbrainard, @rail44, @j0hnsmith, @binarin, @Elemecca, @mozz100 and @marksteve for their help with this release!
0.4.2 (2014-06-18)
------------------
- Fix various encoding errors when using `fig run`, `fig up` and `fig build`.
0.4.1 (2014-05-08)
------------------
- Add support for Docker 0.11.0. (Thanks @marksteve!)
- Make project name configurable. (Thanks @jefmathiot!)
- Return correct exit code from `fig run`.
0.4.0 (2014-04-29)
------------------
- Support Docker 0.9 and 0.10
- Display progress bars correctly when pulling images (no more ski slopes)
- `fig up` now stops all services when any container exits
- Added support for the `privileged` config option in fig.yml (thanks @kvz!)
- Shortened and aligned log prefixes in `fig up` output
- Only containers started with `fig run` link back to their own service
- Handle UTF-8 correctly when streaming `fig build/run/up` output (thanks @mauvm and @shanejonas!)
- Error message improvements
0.3.2 (2014-03-05)
------------------
- Added an `--rm` option to `fig run`. (Thanks @marksteve!)
- Added an `expose` option to `fig.yml`.
0.3.1 (2014-03-04)
------------------
- Added contribution instructions. (Thanks @kvz!)
- Fixed `fig rm` throwing an error.
- Fixed a bug in `fig ps` on Docker 0.8.1 when there is a container with no command.
0.3.0 (2014-03-03)
------------------
- We now ship binaries for OS X and Linux. No more having to install with Pip!
- Add `-f` flag to specify alternate `fig.yml` files
- Add support for custom link names
- Fix a bug where recreating would sometimes hang
- Update docker-py to support Docker 0.8.0.
- Various documentation improvements
- Various error message improvements
Thanks @marksteve, @Gazler and @teozkr!
0.2.2 (2014-02-17)
------------------
- Resolve dependencies using Cormen/Tarjan topological sort
- Fix `fig up` not printing log output
- Stop containers in reverse order to starting
- Fix scale command not binding ports
Thanks to @barnybug and @dustinlacewell for their work on this release.
0.2.1 (2014-02-04)
------------------
- General improvements to error reporting (#77, #79)
0.2.0 (2014-01-31)
------------------
- Link services to themselves so run commands can access the running service. (#67)
- Much better documentation.
- Make service dependency resolution more reliable. (#48)
- Load Fig configurations with a `.yaml` extension. (#58)
Big thanks to @cameronmaske, @mrchrisadams and @damianmoore for their help with this release.
0.1.4 (2014-01-27)
------------------
- Add a link alias without the project name. This makes the environment variables a little shorter: `REDIS_1_PORT_6379_TCP_ADDR`. (#54)
0.1.3 (2014-01-23)
------------------
- Fix ports sometimes being configured incorrectly. (#46)
- Fix log output sometimes not displaying. (#47)
0.1.2 (2014-01-22)
------------------
- Add `-T` option to `fig run` to disable pseudo-TTY. (#34)
- Fix `fig up` requiring the ubuntu image to be pulled to recreate containers. (#33) Thanks @cameronmaske!
- Improve reliability, fix arrow keys and fix a race condition in `fig run`. (#34, #39, #40)
0.1.1 (2014-01-17)
------------------
- Fix bug where ports were not exposed correctly (#29). Thanks @dustinlacewell!
0.1.0 (2014-01-16)
------------------
- Containers are recreated on each `fig up`, ensuring config is up-to-date with `fig.yml` (#2)
- Add `fig scale` command (#9)
- Use `DOCKER_HOST` environment variable to find Docker daemon, for consistency with the official Docker client (was previously `DOCKER_URL`) (#19)
- Truncate long commands in `fig ps` (#18)
- Fill out CLI help banners for commands (#15, #16)
- Show a friendlier error when `fig.yml` is missing (#4)
- Fix bug with `fig build` logging (#3)
- Fix bug where builds would time out if a step took a long time without generating output (#6)
- Fix bug where streaming container output over the Unix socket raised an error (#7)
Big thanks to @tomstuart, @EnTeQuAk, @schickling, @aronasorman and @GeoffreyPlitt.
0.0.2 (2014-01-02)
------------------
- Improve documentation
- Try to connect to Docker on `tcp://localdocker:4243` and a UNIX socket in addition to `localhost`.
- Improve `fig up` behaviour
- Add confirmation prompt to `fig rm`
- Add `fig build` command
0.0.1 (2013-12-20)
------------------
Initial release.

View file

@ -1,8 +1,4 @@
# Contributing to Compose # Contributing to Fig
Compose is a part of the Docker project, and follows the same rules and
principles. Take a read of [Docker's contributing guidelines](https://github.com/docker/docker/blob/master/CONTRIBUTING.md)
to get an overview.
## TL;DR ## TL;DR
@ -10,65 +6,99 @@ Pull requests will need:
- Tests - Tests
- Documentation - Documentation
- [To be signed off](https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work) - [To be signed off](#sign-your-work)
- A logical series of [well written commits](https://github.com/alphagov/styleguides/blob/master/git.md) - A logical series of [well written commits](https://github.com/alphagov/styleguides/blob/master/git.md)
## Development environment ## Development environment
If you're looking contribute to Compose If you're looking contribute to [Fig](http://www.fig.sh/)
but you're new to the project or maybe even to Python, here are the steps but you're new to the project or maybe even to Python, here are the steps
that should get you started. that should get you started.
1. Fork [https://github.com/docker/compose](https://github.com/docker/compose) 1. Fork [https://github.com/docker/fig](https://github.com/docker/fig) to your username.
to your username. 1. Clone your forked repository locally `git clone git@github.com:yourusername/fig.git`.
2. Clone your forked repository locally `git clone git@github.com:yourusername/compose.git`. 1. Enter the local directory `cd fig`.
3. You must [configure a remote](https://help.github.com/articles/configuring-a-remote-for-a-fork/) for your fork so that you can [sync changes you make](https://help.github.com/articles/syncing-a-fork/) with the original repository. 1. Set up a development environment by running `python setup.py develop`. This will install the dependencies and set up a symlink from your `fig` executable to the checkout of the repository. When you now run `fig` from anywhere on your machine, it will run your development version of Fig.
4. Enter the local directory `cd compose`.
5. Set up a development environment by running `python setup.py develop`. This
will install the dependencies and set up a symlink from your `docker-compose`
executable to the checkout of the repository. When you now run
`docker-compose` from anywhere on your machine, it will run your development
version of Compose.
## Install pre-commit hooks
This step is optional, but recommended. Pre-commit hooks will run style checks
and in some cases fix style issues for you, when you commit code.
Install the git pre-commit hooks using [tox](https://tox.readthedocs.io) by
running `tox -e pre-commit` or by following the
[pre-commit install guide](http://pre-commit.com/#install).
To run the style checks at any time run `tox -e pre-commit`.
## Submitting a pull request
See Docker's [basic contribution workflow](https://docs.docker.com/opensource/workflow/make-a-contribution/#the-basic-contribution-workflow) for a guide on how to submit a pull request for code or documentation.
## Running the test suite ## Running the test suite
Use the test script to run linting checks and then the full test suite against $ script/test
different Python interpreters:
$ script/test/default ## Sign your work
Tests are run against a Docker daemon inside a container, so that we can test The sign-off is a simple line at the end of the explanation for the
against multiple Docker versions. By default they'll run against only the latest patch, which certifies that you wrote it or otherwise have the right to
Docker version - set the `DOCKER_VERSIONS` environment variable to "all" to run pass it on as an open-source patch. The rules are pretty simple: if you
against all supported versions: can certify the below (from [developercertificate.org](http://developercertificate.org/)):
$ DOCKER_VERSIONS=all script/test/default Developer's Certificate of Origin 1.1
Arguments to `script/test/default` are passed through to the `tox` executable, so By making a contribution to this project, I certify that:
you can specify a test directory, file, module, class or method:
$ script/test/default tests/unit (a) The contribution was created in whole or in part by me and I
$ script/test/default tests/unit/cli_test.py have the right to submit it under the open source license
$ script/test/default tests/unit/config_test.py::ConfigTest indicated in the file; or
$ script/test/default tests/unit/config_test.py::ConfigTest::test_load
## Finding things to work on (b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
We use a [ZenHub board](https://www.zenhub.io/) to keep track of specific things we are working on and planning to work on. If you're looking for things to work on, stuff in the backlog is a great place to start. (c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
For more information about our project planning, take a look at our [GitHub wiki](https://github.com/docker/compose/wiki). (d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
then you just add a line saying
Signed-off-by: Random J Developer <random@developer.example.org>
using your real name (sorry, no pseudonyms or anonymous contributions.)
The easiest way to do this is to use the `--signoff` flag when committing. E.g.:
$ git commit --signoff
## Building binaries
Linux:
$ script/build-linux
OS X:
$ script/build-osx
Note that this only works on Mountain Lion, not Mavericks, due to a [bug in PyInstaller](http://www.pyinstaller.org/ticket/807).
## Release process
1. Open pull request that:
- Updates version in `fig/__init__.py`
- Updates version in `docs/install.md`
- Adds release notes to `CHANGES.md`
2. Create unpublished GitHub release with release notes
3. Build Linux version on any Docker host with `script/build-linux` and attach to release
4. Build OS X version on Mountain Lion with `script/build-osx` and attach to release as `fig-Darwin-x86_64` and `fig-Linux-x86_64`.
5. Publish GitHub release, creating tag
6. Update website with `script/deploy-docs`
7. Upload PyPi package
$ git checkout $VERSION
$ python setup.py sdist upload

View file

@ -1,71 +1,15 @@
FROM debian:wheezy FROM debian:wheezy
RUN apt-get update -qq && apt-get install -qy python python-pip python-dev git && apt-get clean
RUN set -ex; \
apt-get update -qq; \
apt-get install -y \
locales \
gcc \
make \
zlib1g \
zlib1g-dev \
libssl-dev \
git \
ca-certificates \
curl \
libsqlite3-dev \
libbz2-dev \
; \
rm -rf /var/lib/apt/lists/*
RUN curl https://get.docker.com/builds/Linux/x86_64/docker-1.8.3 \
-o /usr/local/bin/docker && \
chmod +x /usr/local/bin/docker
# Build Python 2.7.13 from source
RUN set -ex; \
curl -L https://www.python.org/ftp/python/2.7.13/Python-2.7.13.tgz | tar -xz; \
cd Python-2.7.13; \
./configure --enable-shared; \
make; \
make install; \
cd ..; \
rm -rf /Python-2.7.13
# Build python 3.4 from source
RUN set -ex; \
curl -L https://www.python.org/ftp/python/3.4.6/Python-3.4.6.tgz | tar -xz; \
cd Python-3.4.6; \
./configure --enable-shared; \
make; \
make install; \
cd ..; \
rm -rf /Python-3.4.6
# Make libpython findable
ENV LD_LIBRARY_PATH /usr/local/lib
# Install pip
RUN set -ex; \
curl -L https://bootstrap.pypa.io/get-pip.py | python
# Python3 requires a valid locale
RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen && locale-gen
ENV LANG en_US.UTF-8
RUN useradd -d /home/user -m -s /bin/bash user RUN useradd -d /home/user -m -s /bin/bash user
WORKDIR /code/ WORKDIR /code/
RUN pip install tox==2.1.1
ADD requirements.txt /code/ ADD requirements.txt /code/
RUN pip install -r requirements.txt
ADD requirements-dev.txt /code/ ADD requirements-dev.txt /code/
ADD .pre-commit-config.yaml /code/ RUN pip install -r requirements-dev.txt
ADD setup.py /code/
ADD tox.ini /code/
ADD compose /code/compose/
RUN tox --notest
ADD . /code/ ADD . /code/
RUN chown -R user /code/ RUN python setup.py install
ENTRYPOINT ["/code/.tox/py27/bin/docker-compose"] RUN chown -R user /code/

View file

@ -1,14 +0,0 @@
FROM alpine:3.4
ARG version
RUN apk -U add \
python \
py-pip
COPY requirements.txt /code/requirements.txt
RUN pip install -r /code/requirements.txt
COPY dist/docker_compose-${version}-py2.py3-none-any.whl /code/
RUN pip install --no-deps /code/docker_compose-${version}-py2.py3-none-any.whl
ENTRYPOINT ["/usr/bin/docker-compose"]

64
Jenkinsfile vendored
View file

@ -1,64 +0,0 @@
#!groovy
def image
def buildImage = { ->
wrappedNode(label: "ubuntu && !zfs", cleanWorkspace: true) {
stage("build image") {
checkout(scm)
def imageName = "dockerbuildbot/compose:${gitCommit()}"
image = docker.image(imageName)
try {
image.pull()
} catch (Exception exc) {
image = docker.build(imageName, ".")
image.push()
}
}
}
}
def runTests = { Map settings ->
def dockerVersions = settings.get("dockerVersions", null)
def pythonVersions = settings.get("pythonVersions", null)
if (!pythonVersions) {
throw new Exception("Need Python versions to test. e.g.: `runTests(pythonVersions: 'py27,py34')`")
}
if (!dockerVersions) {
throw new Exception("Need Docker versions to test. e.g.: `runTests(dockerVersions: 'all')`")
}
{ ->
wrappedNode(label: "ubuntu && !zfs", cleanWorkspace: true) {
stage("test python=${pythonVersions} / docker=${dockerVersions}") {
checkout(scm)
def storageDriver = sh(script: 'docker info | awk -F \': \' \'$1 == "Storage Driver" { print $2; exit }\'', returnStdout: true).trim()
echo "Using local system's storage driver: ${storageDriver}"
sh """docker run \\
-t \\
--rm \\
--privileged \\
--volume="\$(pwd)/.git:/code/.git" \\
--volume="/var/run/docker.sock:/var/run/docker.sock" \\
-e "TAG=${image.id}" \\
-e "STORAGE_DRIVER=${storageDriver}" \\
-e "DOCKER_VERSIONS=${dockerVersions}" \\
-e "BUILD_NUMBER=\$BUILD_TAG" \\
-e "PY_TEST_VERSIONS=${pythonVersions}" \\
--entrypoint="script/ci" \\
${image.id} \\
--verbose
"""
}
}
}
}
buildImage()
// TODO: break this out into meaningful "DOCKER_VERSIONS" values instead of all
parallel(
failFast: true,
all_py27: runTests(pythonVersions: "py27", dockerVersions: "all"),
all_py34: runTests(pythonVersions: "py34", dockerVersions: "all"),
)

View file

@ -1,46 +1,5 @@
# Compose maintainers file Aanand Prasad <aanand.prasad@gmail.com> (@aanand)
# Ben Firshman <ben@firshman.co.uk> (@bfirsh)
# This file describes who runs the docker/compose project and how. Chris Corbyn <chris@w3style.co.uk> (@d11wtq)
# This is a living document - if you see something out of date or missing, speak up! Daniel Nephin <dnephin@gmail.com> (@dnephin)
# Nathan LeClaire <nathan.leclaire@gmail.com> (@nathanleclaire)
# It is structured to be consumable by both humans and programs.
# To extract its contents programmatically, use any TOML-compliant parser.
#
# This file is compiled into the MAINTAINERS file in docker/opensource.
#
[Org]
[Org."Core maintainers"]
people = [
"aanand",
"bfirsh",
"dnephin",
"mnowster",
]
[people]
# A reference list of all people associated with the project.
# All other sections should refer to people by their canonical key
# in the people section.
# ADD YOURSELF HERE IN ALPHABETICAL ORDER
[people.aanand]
Name = "Aanand Prasad"
Email = "aanand.prasad@gmail.com"
GitHub = "aanand"
[people.bfirsh]
Name = "Ben Firshman"
Email = "ben@firshman.co.uk"
GitHub = "bfirsh"
[people.dnephin]
Name = "Daniel Nephin"
Email = "dnephin@gmail.com"
GitHub = "dnephin"
[people.mnowster]
Name = "Mazz Mosley"
Email = "mazz@houseofmnowster.com"
GitHub = "mnowster"

View file

@ -4,12 +4,7 @@ include requirements.txt
include requirements-dev.txt include requirements-dev.txt
include tox.ini include tox.ini
include *.md include *.md
exclude README.md recursive-exclude tests *
include README.rst
include compose/config/*.json
include compose/GITSHA
recursive-include contrib/completion *
recursive-include tests *
global-exclude *.pyc global-exclude *.pyc
global-exclude *.pyo global-exclude *.pyo
global-exclude *.un~ global-exclude *.un~

View file

@ -1,65 +1,46 @@
Docker Compose Fig
============== ===
![Docker Compose](logo.png?raw=true "Docker Compose Logo")
Compose is a tool for defining and running multi-container Docker applications. [![wercker status](https://app.wercker.com/status/d5dbac3907301c3d5ce735e2d5e95a5b/s/master "wercker status")](https://app.wercker.com/project/bykey/d5dbac3907301c3d5ce735e2d5e95a5b)
With Compose, you use a Compose file to configure your application's services.
Then, using a single command, you create and start all the services
from your configuration. To learn more about all the features of Compose
see [the list of features](https://github.com/docker/docker.github.io/blob/master/compose/overview.md#features).
Compose is great for development, testing, and staging environments, as well as Fast, isolated development environments using Docker.
CI workflows. You can learn more about each case in
[Common Use Cases](https://github.com/docker/docker.github.io/blob/master/compose/overview.md#common-use-cases).
Using Compose is basically a three-step process. Define your app's environment with Docker so it can be reproduced anywhere:
1. Define your app's environment with a `Dockerfile` so it can be FROM python:2.7
reproduced anywhere. ADD . /code
2. Define the services that make up your app in `docker-compose.yml` so WORKDIR /code
they can be run together in an isolated environment: RUN pip install -r requirements.txt
3. Lastly, run `docker-compose up` and Compose will start and run your entire app. CMD python app.py
A `docker-compose.yml` looks like this: Define the services that make up your app so they can be run together in an isolated environment:
version: '2' ```yaml
web:
build: .
links:
- db
ports:
- "8000:8000"
- "49100:22"
db:
image: postgres
```
services: (No more installing Postgres on your laptop!)
web:
build: .
ports:
- "5000:5000"
volumes:
- .:/code
redis:
image: redis
For more information about the Compose file, see the Then type `fig up`, and Fig will start and run your entire app:
[Compose file reference](https://github.com/docker/docker.github.io/blob/master/compose/compose-file/compose-versioning.md)
Compose has commands for managing the whole lifecycle of your application: ![example fig run](https://orchardup.com/static/images/fig-example-large.gif)
* Start, stop and rebuild services There are commands to:
* View the status of running services
* Stream the log output of running services - start, stop and rebuild services
* Run a one-off command on a service - view the status of running services
- tail running services' log output
- run a one-off command on a service
Installation and documentation Installation and documentation
------------------------------ ------------------------------
- Full documentation is available on [Docker's website](https://docs.docker.com/compose/). Full documentation is available on [Fig's website](http://www.fig.sh/).
- If you have any questions, you can talk in real-time with other developers in the #docker-compose IRC channel on Freenode. [Click here to join using IRCCloud.](https://www.irccloud.com/invite?hostname=irc.freenode.net&channel=%23docker-compose)
- Code repository for Compose is on [Github](https://github.com/docker/compose)
- If you find any problems please fill out an [issue](https://github.com/docker/compose/issues/new)
Contributing
------------
[![Build Status](http://jenkins.dockerproject.org/buildStatus/icon?job=Compose%20Master)](http://jenkins.dockerproject.org/job/Compose%20Master/)
Want to help build Compose? Check out our [contributing documentation](https://github.com/docker/compose/blob/master/CONTRIBUTING.md).
Releasing
---------
Releases are built by maintainers, following an outline of the [release process](https://github.com/docker/compose/blob/master/project/RELEASE-PROCESS.md).

View file

@ -1,32 +0,0 @@
# Roadmap
## An even better tool for development environments
Compose is a great tool for development environments, but it could be even better. For example:
- It should be possible to define hostnames for containers which work from the host machine, e.g. “mywebcontainer.local”. This is needed by apps comprising multiple web services which generate links to one another (e.g. a frontend website and a separate admin webapp)
## More than just development environments
Compose currently works really well in development, but we want to make the Compose file format better for test, staging, and production environments. To support these use cases, there will need to be improvements to the file format, improvements to the command-line tool, integrations with other tools, and perhaps new tools altogether.
Some specific things we are considering:
- Compose currently will attempt to get your application into the correct state when running `up`, but it has a number of shortcomings:
- It should roll back to a known good state if it fails.
- It should allow a user to check the actions it is about to perform before running them.
- It should be possible to partially modify the config file for different environments (dev/test/staging/prod), passing in e.g. custom ports, volume mount paths, or volume drivers. ([#1377](https://github.com/docker/compose/issues/1377))
- Compose should recommend a technique for zero-downtime deploys.
- It should be possible to continuously attempt to keep an application in the correct state, instead of just performing `up` a single time.
## Integration with Swarm
Compose should integrate really well with Swarm so you can take an application you've developed on your laptop and run it on a Swarm cluster.
The current state of integration is documented in [SWARM.md](SWARM.md).
## Applications spanning multiple teams
Compose works well for applications that are in a single repository and depend on services that are hosted on Docker Hub. If your application depends on another application within your organisation, Compose doesn't work as well.
There are several ideas about how this could work, such as [including external files](https://github.com/docker/fig/issues/318).

View file

@ -1 +0,0 @@
This file has moved to: https://docs.docker.com/compose/swarm/

View file

@ -1,24 +0,0 @@
version: '{branch}-{build}'
install:
- "SET PATH=C:\\Python27-x64;C:\\Python27-x64\\Scripts;%PATH%"
- "python --version"
- "pip install tox==2.1.1 virtualenv==13.1.2"
# Build the binary after tests
build: false
test_script:
- "tox -e py27,py34 -- tests/unit"
- ps: ".\\script\\build\\windows.ps1"
artifacts:
- path: .\dist\docker-compose-Windows-x86_64.exe
name: "Compose Windows binary"
deploy:
- provider: Environment
name: master-builds
on:
branch: master

View file

@ -1,3 +0,0 @@
#!/usr/bin/env python
from compose.cli.main import main
main()

3
bin/fig Executable file
View file

@ -0,0 +1,3 @@
#!/usr/bin/env python
from fig.cli.main import main
main()

View file

@ -1,4 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
__version__ = '1.12.0dev'

View file

@ -1,6 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
from compose.cli.main import main
main()

View file

@ -1,258 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import json
import logging
import six
from docker.utils import split_command
from docker.utils.ports import split_port
from .cli.errors import UserError
from .config.serialize import denormalize_config
from .network import get_network_defs_for_service
from .service import format_environment
from .service import NoSuchImageError
from .service import parse_repository_tag
log = logging.getLogger(__name__)
SERVICE_KEYS = {
'working_dir': 'WorkingDir',
'user': 'User',
'labels': 'Labels',
}
IGNORED_KEYS = {'build'}
SUPPORTED_KEYS = {
'image',
'ports',
'expose',
'networks',
'command',
'environment',
'entrypoint',
} | set(SERVICE_KEYS)
VERSION = '0.1'
class NeedsPush(Exception):
def __init__(self, image_name):
self.image_name = image_name
class NeedsPull(Exception):
def __init__(self, image_name, service_name):
self.image_name = image_name
self.service_name = service_name
class MissingDigests(Exception):
def __init__(self, needs_push, needs_pull):
self.needs_push = needs_push
self.needs_pull = needs_pull
def serialize_bundle(config, image_digests):
return json.dumps(to_bundle(config, image_digests), indent=2, sort_keys=True)
def get_image_digests(project, allow_push=False):
digests = {}
needs_push = set()
needs_pull = set()
for service in project.services:
try:
digests[service.name] = get_image_digest(
service,
allow_push=allow_push,
)
except NeedsPush as e:
needs_push.add(e.image_name)
except NeedsPull as e:
needs_pull.add(e.service_name)
if needs_push or needs_pull:
raise MissingDigests(needs_push, needs_pull)
return digests
def get_image_digest(service, allow_push=False):
if 'image' not in service.options:
raise UserError(
"Service '{s.name}' doesn't define an image tag. An image name is "
"required to generate a proper image digest for the bundle. Specify "
"an image repo and tag with the 'image' option.".format(s=service))
_, _, separator = parse_repository_tag(service.options['image'])
# Compose file already uses a digest, no lookup required
if separator == '@':
return service.options['image']
try:
image = service.image()
except NoSuchImageError:
action = 'build' if 'build' in service.options else 'pull'
raise UserError(
"Image not found for service '{service}'. "
"You might need to run `docker-compose {action} {service}`."
.format(service=service.name, action=action))
if image['RepoDigests']:
# TODO: pick a digest based on the image tag if there are multiple
# digests
return image['RepoDigests'][0]
if 'build' not in service.options:
raise NeedsPull(service.image_name, service.name)
if not allow_push:
raise NeedsPush(service.image_name)
return push_image(service)
def push_image(service):
try:
digest = service.push()
except:
log.error(
"Failed to push image for service '{s.name}'. Please use an "
"image tag that can be pushed to a Docker "
"registry.".format(s=service))
raise
if not digest:
raise ValueError("Failed to get digest for %s" % service.name)
repo, _, _ = parse_repository_tag(service.options['image'])
identifier = '{repo}@{digest}'.format(repo=repo, digest=digest)
# only do this if RepoDigests isn't already populated
image = service.image()
if not image['RepoDigests']:
# Pull by digest so that image['RepoDigests'] is populated for next time
# and we don't have to pull/push again
service.client.pull(identifier)
log.info("Stored digest for {}".format(service.image_name))
return identifier
def to_bundle(config, image_digests):
if config.networks:
log.warn("Unsupported top level key 'networks' - ignoring")
if config.volumes:
log.warn("Unsupported top level key 'volumes' - ignoring")
config = denormalize_config(config)
return {
'Version': VERSION,
'Services': {
name: convert_service_to_bundle(
name,
service_dict,
image_digests[name],
)
for name, service_dict in config['services'].items()
},
}
def convert_service_to_bundle(name, service_dict, image_digest):
container_config = {'Image': image_digest}
for key, value in service_dict.items():
if key in IGNORED_KEYS:
continue
if key not in SUPPORTED_KEYS:
log.warn("Unsupported key '{}' in services.{} - ignoring".format(key, name))
continue
if key == 'environment':
container_config['Env'] = format_environment({
envkey: envvalue for envkey, envvalue in value.items()
if envvalue
})
continue
if key in SERVICE_KEYS:
container_config[SERVICE_KEYS[key]] = value
continue
set_command_and_args(
container_config,
service_dict.get('entrypoint', []),
service_dict.get('command', []))
container_config['Networks'] = make_service_networks(name, service_dict)
ports = make_port_specs(service_dict)
if ports:
container_config['Ports'] = ports
return container_config
# See https://github.com/docker/swarmkit/blob/agent/exec/container/container.go#L95
def set_command_and_args(config, entrypoint, command):
if isinstance(entrypoint, six.string_types):
entrypoint = split_command(entrypoint)
if isinstance(command, six.string_types):
command = split_command(command)
if entrypoint:
config['Command'] = entrypoint + command
return
if command:
config['Args'] = command
def make_service_networks(name, service_dict):
networks = []
for network_name, network_def in get_network_defs_for_service(service_dict).items():
for key in network_def.keys():
log.warn(
"Unsupported key '{}' in services.{}.networks.{} - ignoring"
.format(key, name, network_name))
networks.append(network_name)
return networks
def make_port_specs(service_dict):
ports = []
internal_ports = [
internal_port
for port_def in service_dict.get('ports', [])
for internal_port in split_port(port_def)[0]
]
internal_ports += service_dict.get('expose', [])
for internal_port in internal_ports:
spec = make_port_spec(internal_port)
if spec not in ports:
ports.append(spec)
return ports
def make_port_spec(value):
components = six.text_type(value).partition('/')
return {
'Protocol': components[2] or 'tcp',
'Port': int(components[0]),
}

View file

@ -1,37 +0,0 @@
from __future__ import absolute_import
from __future__ import print_function
from __future__ import unicode_literals
import subprocess
import sys
# Attempt to detect https://github.com/docker/compose/issues/4344
try:
# We don't try importing pip because it messes with package imports
# on some Linux distros (Ubuntu, Fedora)
# https://github.com/docker/compose/issues/4425
# https://github.com/docker/compose/issues/4481
# https://github.com/pypa/pip/blob/master/pip/_vendor/__init__.py
s_cmd = subprocess.Popen(
['pip', 'freeze'], stderr=subprocess.PIPE, stdout=subprocess.PIPE
)
packages = s_cmd.communicate()[0].splitlines()
dockerpy_installed = len(
list(filter(lambda p: p.startswith(b'docker-py=='), packages))
) > 0
if dockerpy_installed:
from .colors import red
print(
red('ERROR:'),
"Dependency conflict: an older version of the 'docker-py' package "
"is polluting the namespace. "
"Run the following command to remedy the issue:\n"
"pip uninstall docker docker-py; pip install docker",
file=sys.stderr
)
sys.exit(1)
except OSError:
# pip command is not available, which indicates it's probably the binary
# distribution of Compose which is not affected
pass

View file

@ -1,132 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import logging
import os
import re
import ssl
import six
from . import errors
from . import verbose_proxy
from .. import config
from ..config.environment import Environment
from ..const import API_VERSIONS
from ..project import Project
from .docker_client import docker_client
from .docker_client import tls_config_from_options
from .utils import get_version_info
log = logging.getLogger(__name__)
def project_from_options(project_dir, options):
environment = Environment.from_env_file(project_dir)
host = options.get('--host')
if host is not None:
host = host.lstrip('=')
return get_project(
project_dir,
get_config_path_from_options(project_dir, options, environment),
project_name=options.get('--project-name'),
verbose=options.get('--verbose'),
host=host,
tls_config=tls_config_from_options(options),
environment=environment
)
def get_config_from_options(base_dir, options):
environment = Environment.from_env_file(base_dir)
config_path = get_config_path_from_options(
base_dir, options, environment
)
return config.load(
config.find(base_dir, config_path, environment)
)
def get_config_path_from_options(base_dir, options, environment):
file_option = options.get('--file')
if file_option:
return file_option
config_files = environment.get('COMPOSE_FILE')
if config_files:
return config_files.split(os.pathsep)
return None
def get_tls_version(environment):
compose_tls_version = environment.get('COMPOSE_TLS_VERSION', None)
if not compose_tls_version:
return None
tls_attr_name = "PROTOCOL_{}".format(compose_tls_version)
if not hasattr(ssl, tls_attr_name):
log.warn(
'The "{}" protocol is unavailable. You may need to update your '
'version of Python or OpenSSL. Falling back to TLSv1 (default).'
.format(compose_tls_version)
)
return None
return getattr(ssl, tls_attr_name)
def get_client(environment, verbose=False, version=None, tls_config=None, host=None,
tls_version=None):
client = docker_client(
version=version, tls_config=tls_config, host=host,
environment=environment, tls_version=get_tls_version(environment)
)
if verbose:
version_info = six.iteritems(client.version())
log.info(get_version_info('full'))
log.info("Docker base_url: %s", client.base_url)
log.info("Docker version: %s",
", ".join("%s=%s" % item for item in version_info))
return verbose_proxy.VerboseProxy('docker', client)
return client
def get_project(project_dir, config_path=None, project_name=None, verbose=False,
host=None, tls_config=None, environment=None):
if not environment:
environment = Environment.from_env_file(project_dir)
config_details = config.find(project_dir, config_path, environment)
project_name = get_project_name(
config_details.working_dir, project_name, environment
)
config_data = config.load(config_details)
api_version = environment.get(
'COMPOSE_API_VERSION',
API_VERSIONS[config_data.version])
client = get_client(
verbose=verbose, version=api_version, tls_config=tls_config,
host=host, environment=environment
)
with errors.handle_connection_errors(client):
return Project.from_config(project_name, config_data, client)
def get_project_name(working_dir, project_name=None, environment=None):
def normalize_name(name):
return re.sub(r'[^a-z0-9]', '', name.lower())
if not environment:
environment = Environment.from_env_file(working_dir)
project_name = project_name or environment.get('COMPOSE_PROJECT_NAME')
if project_name:
return normalize_name(project_name)
project = os.path.basename(os.path.abspath(working_dir))
if project:
return normalize_name(project)
return 'default'

View file

@ -1,74 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import logging
from docker import APIClient
from docker.errors import TLSParameterError
from docker.tls import TLSConfig
from docker.utils import kwargs_from_env
from ..const import HTTP_TIMEOUT
from .errors import UserError
from .utils import generate_user_agent
from .utils import unquote_path
log = logging.getLogger(__name__)
def tls_config_from_options(options):
tls = options.get('--tls', False)
ca_cert = unquote_path(options.get('--tlscacert'))
cert = unquote_path(options.get('--tlscert'))
key = unquote_path(options.get('--tlskey'))
verify = options.get('--tlsverify')
skip_hostname_check = options.get('--skip-hostname-check', False)
advanced_opts = any([ca_cert, cert, key, verify])
if tls is True and not advanced_opts:
return True
elif advanced_opts: # --tls is a noop
client_cert = None
if cert or key:
client_cert = (cert, key)
return TLSConfig(
client_cert=client_cert, verify=verify, ca_cert=ca_cert,
assert_hostname=False if skip_hostname_check else None
)
return None
def docker_client(environment, version=None, tls_config=None, host=None,
tls_version=None):
"""
Returns a docker-py client configured using environment variables
according to the same logic as the official Docker client.
"""
try:
kwargs = kwargs_from_env(environment=environment, ssl_version=tls_version)
except TLSParameterError:
raise UserError(
"TLS configuration is invalid - make sure your DOCKER_TLS_VERIFY "
"and DOCKER_CERT_PATH are set correctly.\n"
"You might need to run `eval \"$(docker-machine env default)\"`")
if host:
kwargs['base_url'] = host
if tls_config:
kwargs['tls'] = tls_config
if version:
kwargs['version'] = version
timeout = environment.get('COMPOSE_HTTP_TIMEOUT')
if timeout:
kwargs['timeout'] = int(timeout)
else:
kwargs['timeout'] = HTTP_TIMEOUT
kwargs['user_agent'] = generate_user_agent()
return APIClient(**kwargs)

View file

@ -1,59 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
from inspect import getdoc
from docopt import docopt
from docopt import DocoptExit
def docopt_full_help(docstring, *args, **kwargs):
try:
return docopt(docstring, *args, **kwargs)
except DocoptExit:
raise SystemExit(docstring)
class DocoptDispatcher(object):
def __init__(self, command_class, options):
self.command_class = command_class
self.options = options
def parse(self, argv):
command_help = getdoc(self.command_class)
options = docopt_full_help(command_help, argv, **self.options)
command = options['COMMAND']
if command is None:
raise SystemExit(command_help)
handler = get_handler(self.command_class, command)
docstring = getdoc(handler)
if docstring is None:
raise NoSuchCommand(command, self)
command_options = docopt_full_help(docstring, options['ARGS'], options_first=True)
return options, handler, command_options
def get_handler(command_class, command):
command = command.replace('-', '_')
# we certainly want to have "exec" command, since that's what docker client has
# but in python exec is a keyword
if command == "exec":
command = "exec_command"
if not hasattr(command_class, command):
raise NoSuchCommand(command, command_class)
return getattr(command_class, command)
class NoSuchCommand(Exception):
def __init__(self, command, supercommand):
super(NoSuchCommand, self).__init__("No such command: %s" % command)
self.command = command
self.supercommand = supercommand

View file

@ -1,137 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import contextlib
import logging
import socket
from distutils.spawn import find_executable
from textwrap import dedent
from docker.errors import APIError
from requests.exceptions import ConnectionError as RequestsConnectionError
from requests.exceptions import ReadTimeout
from requests.exceptions import SSLError
from requests.packages.urllib3.exceptions import ReadTimeoutError
from ..const import API_VERSION_TO_ENGINE_VERSION
from .utils import is_docker_for_mac_installed
from .utils import is_mac
from .utils import is_ubuntu
from .utils import is_windows
log = logging.getLogger(__name__)
class UserError(Exception):
def __init__(self, msg):
self.msg = dedent(msg).strip()
def __unicode__(self):
return self.msg
__str__ = __unicode__
class ConnectionError(Exception):
pass
@contextlib.contextmanager
def handle_connection_errors(client):
try:
yield
except SSLError as e:
log.error('SSL error: %s' % e)
raise ConnectionError()
except RequestsConnectionError as e:
if e.args and isinstance(e.args[0], ReadTimeoutError):
log_timeout_error(client.timeout)
raise ConnectionError()
exit_with_error(get_conn_error_message(client.base_url))
except APIError as e:
log_api_error(e, client.api_version)
raise ConnectionError()
except (ReadTimeout, socket.timeout) as e:
log_timeout_error(client.timeout)
raise ConnectionError()
def log_timeout_error(timeout):
log.error(
"An HTTP request took too long to complete. Retry with --verbose to "
"obtain debug information.\n"
"If you encounter this issue regularly because of slow network "
"conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher "
"value (current value: %s)." % timeout)
def log_api_error(e, client_version):
if b'client is newer than server' not in e.explanation:
log.error(e.explanation)
return
version = API_VERSION_TO_ENGINE_VERSION.get(client_version)
if not version:
# They've set a custom API version
log.error(e.explanation)
return
log.error(
"The Docker Engine version is less than the minimum required by "
"Compose. Your current project requires a Docker Engine of "
"version {version} or greater.".format(version=version))
def exit_with_error(msg):
log.error(dedent(msg).strip())
raise ConnectionError()
def get_conn_error_message(url):
if find_executable('docker') is None:
return docker_not_found_msg("Couldn't connect to Docker daemon.")
if is_docker_for_mac_installed():
return conn_error_docker_for_mac
if find_executable('docker-machine') is not None:
return conn_error_docker_machine
return conn_error_generic.format(url=url)
def docker_not_found_msg(problem):
return "{} You might need to install Docker:\n\n{}".format(
problem, docker_install_url())
def docker_install_url():
if is_mac():
return docker_install_url_mac
elif is_ubuntu():
return docker_install_url_ubuntu
elif is_windows():
return docker_install_url_windows
else:
return docker_install_url_generic
docker_install_url_mac = "https://docs.docker.com/engine/installation/mac/"
docker_install_url_ubuntu = "https://docs.docker.com/engine/installation/ubuntulinux/"
docker_install_url_windows = "https://docs.docker.com/engine/installation/windows/"
docker_install_url_generic = "https://docs.docker.com/engine/installation/"
conn_error_docker_machine = """
Couldn't connect to Docker daemon - you might need to run `docker-machine start default`.
"""
conn_error_docker_for_mac = """
Couldn't connect to Docker daemon. You might need to start Docker for Mac.
"""
conn_error_generic = """
Couldn't connect to Docker daemon at {url} - is it running?
If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
"""

View file

@ -1,51 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import logging
import os
import six
import texttable
from compose.cli import colors
def get_tty_width():
tty_size = os.popen('stty size', 'r').read().split()
if len(tty_size) != 2:
return 0
_, width = tty_size
return int(width)
class Formatter(object):
"""Format tabular data for printing."""
def table(self, headers, rows):
table = texttable.Texttable(max_width=get_tty_width())
table.set_cols_dtype(['t' for h in headers])
table.add_rows([headers] + rows)
table.set_deco(table.HEADER)
table.set_chars(['-', '|', '+', '-'])
return table.draw()
class ConsoleWarningFormatter(logging.Formatter):
"""A logging.Formatter which prints WARNING and ERROR messages with
a prefix of the log level colored appropriate for the log level.
"""
def get_level_message(self, record):
separator = ': '
if record.levelno == logging.WARNING:
return colors.yellow(record.levelname) + separator
if record.levelno == logging.ERROR:
return colors.red(record.levelname) + separator
return ''
def format(self, record):
if isinstance(record.msg, six.binary_type):
record.msg = record.msg.decode('utf-8')
message = super(ConsoleWarningFormatter, self).format(record)
return '{0}{1}'.format(self.get_level_message(record), message)

View file

@ -1,237 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import sys
from collections import namedtuple
from itertools import cycle
from threading import Thread
from docker.errors import APIError
from six.moves import _thread as thread
from six.moves.queue import Empty
from six.moves.queue import Queue
from . import colors
from compose import utils
from compose.cli.signals import ShutdownException
from compose.utils import split_buffer
class LogPresenter(object):
def __init__(self, prefix_width, color_func):
self.prefix_width = prefix_width
self.color_func = color_func
def present(self, container, line):
prefix = container.name_without_project.ljust(self.prefix_width)
return '{prefix} {line}'.format(
prefix=self.color_func(prefix + ' |'),
line=line)
def build_log_presenters(service_names, monochrome):
"""Return an iterable of functions.
Each function can be used to format the logs output of a container.
"""
prefix_width = max_name_width(service_names)
def no_color(text):
return text
for color_func in cycle([no_color] if monochrome else colors.rainbow()):
yield LogPresenter(prefix_width, color_func)
def max_name_width(service_names, max_index_width=3):
"""Calculate the maximum width of container names so we can make the log
prefixes line up like so:
db_1 | Listening
web_1 | Listening
"""
return max(len(name) for name in service_names) + max_index_width
class LogPrinter(object):
"""Print logs from many containers to a single output stream."""
def __init__(self,
containers,
presenters,
event_stream,
output=sys.stdout,
cascade_stop=False,
log_args=None):
self.containers = containers
self.presenters = presenters
self.event_stream = event_stream
self.output = utils.get_output_stream(output)
self.cascade_stop = cascade_stop
self.log_args = log_args or {}
def run(self):
if not self.containers:
return
queue = Queue()
thread_args = queue, self.log_args
thread_map = build_thread_map(self.containers, self.presenters, thread_args)
start_producer_thread((
thread_map,
self.event_stream,
self.presenters,
thread_args))
for line in consume_queue(queue, self.cascade_stop):
remove_stopped_threads(thread_map)
if not line:
if not thread_map:
# There are no running containers left to tail, so exit
return
# We got an empty line because of a timeout, but there are still
# active containers to tail, so continue
continue
self.output.write(line)
self.output.flush()
def remove_stopped_threads(thread_map):
for container_id, tailer_thread in list(thread_map.items()):
if not tailer_thread.is_alive():
thread_map.pop(container_id, None)
def build_thread(container, presenter, queue, log_args):
tailer = Thread(
target=tail_container_logs,
args=(container, presenter, queue, log_args))
tailer.daemon = True
tailer.start()
return tailer
def build_thread_map(initial_containers, presenters, thread_args):
return {
container.id: build_thread(container, next(presenters), *thread_args)
for container in initial_containers
}
class QueueItem(namedtuple('_QueueItem', 'item is_stop exc')):
@classmethod
def new(cls, item):
return cls(item, None, None)
@classmethod
def exception(cls, exc):
return cls(None, None, exc)
@classmethod
def stop(cls):
return cls(None, True, None)
def tail_container_logs(container, presenter, queue, log_args):
generator = get_log_generator(container)
try:
for item in generator(container, log_args):
queue.put(QueueItem.new(presenter.present(container, item)))
except Exception as e:
queue.put(QueueItem.exception(e))
return
if log_args.get('follow'):
queue.put(QueueItem.new(presenter.color_func(wait_on_exit(container))))
queue.put(QueueItem.stop())
def get_log_generator(container):
if container.has_api_logs:
return build_log_generator
return build_no_log_generator
def build_no_log_generator(container, log_args):
"""Return a generator that prints a warning about logs and waits for
container to exit.
"""
yield "WARNING: no logs are available with the '{}' log driver\n".format(
container.log_driver)
def build_log_generator(container, log_args):
# if the container doesn't have a log_stream we need to attach to container
# before log printer starts running
if container.log_stream is None:
stream = container.logs(stdout=True, stderr=True, stream=True, **log_args)
else:
stream = container.log_stream
return split_buffer(stream)
def wait_on_exit(container):
try:
exit_code = container.wait()
return "%s exited with code %s\n" % (container.name, exit_code)
except APIError as e:
return "Unexpected API error for %s (HTTP code %s)\nResponse body:\n%s\n" % (
container.name, e.response.status_code,
e.response.text or '[empty]'
)
def start_producer_thread(thread_args):
producer = Thread(target=watch_events, args=thread_args)
producer.daemon = True
producer.start()
def watch_events(thread_map, event_stream, presenters, thread_args):
for event in event_stream:
if event['action'] == 'stop':
thread_map.pop(event['id'], None)
if event['action'] != 'start':
continue
if event['id'] in thread_map:
if thread_map[event['id']].is_alive():
continue
# Container was stopped and started, we need a new thread
thread_map.pop(event['id'], None)
thread_map[event['id']] = build_thread(
event['container'],
next(presenters),
*thread_args)
def consume_queue(queue, cascade_stop):
"""Consume the queue by reading lines off of it and yielding them."""
while True:
try:
item = queue.get(timeout=0.1)
except Empty:
yield None
continue
# See https://github.com/docker/compose/issues/189
except thread.error:
raise ShutdownException()
if item.exc:
raise item.exc
if item.is_stop:
if cascade_stop:
raise StopIteration
else:
continue
yield item.item

File diff suppressed because it is too large Load diff

View file

@ -1,21 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import signal
class ShutdownException(Exception):
pass
def shutdown(signal, frame):
raise ShutdownException()
def set_signal_handler(handler):
signal.signal(signal.SIGINT, handler)
signal.signal(signal.SIGTERM, handler)
def set_signal_handler_to_shutdown():
set_signal_handler(shutdown)

View file

@ -1,137 +0,0 @@
from __future__ import absolute_import
from __future__ import division
from __future__ import unicode_literals
import os
import platform
import ssl
import subprocess
import sys
import docker
import compose
from ..const import IS_WINDOWS_PLATFORM
# WindowsError is not defined on non-win32 platforms. Avoid runtime errors by
# defining it as OSError (its parent class) if missing.
try:
WindowsError
except NameError:
WindowsError = OSError
def yesno(prompt, default=None):
"""
Prompt the user for a yes or no.
Can optionally specify a default value, which will only be
used if they enter a blank line.
Unrecognised input (anything other than "y", "n", "yes",
"no" or "") will return None.
"""
answer = input(prompt).strip().lower()
if answer == "y" or answer == "yes":
return True
elif answer == "n" or answer == "no":
return False
elif answer == "":
return default
else:
return None
def input(prompt):
"""
Version of input (raw_input in Python 2) which forces a flush of sys.stdout
to avoid problems where the prompt fails to appear due to line buffering
"""
sys.stdout.write(prompt)
sys.stdout.flush()
return sys.stdin.readline().rstrip('\n')
def call_silently(*args, **kwargs):
"""
Like subprocess.call(), but redirects stdout and stderr to /dev/null.
"""
with open(os.devnull, 'w') as shutup:
try:
return subprocess.call(*args, stdout=shutup, stderr=shutup, **kwargs)
except WindowsError:
# On Windows, subprocess.call() can still raise exceptions. Normalize
# to POSIXy behaviour by returning a nonzero exit code.
return 1
def is_mac():
return platform.system() == 'Darwin'
def is_ubuntu():
return platform.system() == 'Linux' and platform.linux_distribution()[0] == 'Ubuntu'
def is_windows():
return IS_WINDOWS_PLATFORM
def get_version_info(scope):
versioninfo = 'docker-compose version {}, build {}'.format(
compose.__version__,
get_build_version())
if scope == 'compose':
return versioninfo
if scope == 'full':
return (
"{}\n"
"docker-py version: {}\n"
"{} version: {}\n"
"OpenSSL version: {}"
).format(
versioninfo,
docker.version,
platform.python_implementation(),
platform.python_version(),
ssl.OPENSSL_VERSION)
raise ValueError("{} is not a valid version scope".format(scope))
def get_build_version():
filename = os.path.join(os.path.dirname(compose.__file__), 'GITSHA')
if not os.path.exists(filename):
return 'unknown'
with open(filename) as fh:
return fh.read().strip()
def is_docker_for_mac_installed():
return is_mac() and os.path.isdir('/Applications/Docker.app')
def generate_user_agent():
parts = [
"docker-compose/{}".format(compose.__version__),
"docker-py/{}".format(docker.__version__),
]
try:
p_system = platform.system()
p_release = platform.release()
except IOError:
pass
else:
parts.append("{}/{}".format(p_system, p_release))
return " ".join(parts)
def unquote_path(s):
if not s:
return s
if s[0] == '"' and s[-1] == '"':
return s[1:-1]
return s

View file

@ -1,11 +0,0 @@
# flake8: noqa
from __future__ import absolute_import
from __future__ import unicode_literals
from . import environment
from .config import ConfigurationError
from .config import DOCKER_CONFIG_KEYS
from .config import find
from .config import load
from .config import merge_environment
from .config import parse_environment

File diff suppressed because it is too large Load diff

View file

@ -1,188 +0,0 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"id": "config_schema_v1.json",
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"$ref": "#/definitions/service"
}
},
"additionalProperties": false,
"definitions": {
"service": {
"id": "#/definitions/service",
"type": "object",
"properties": {
"build": {"type": "string"},
"cap_add": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"cap_drop": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"cgroup_parent": {"type": "string"},
"command": {
"oneOf": [
{"type": "string"},
{"type": "array", "items": {"type": "string"}}
]
},
"container_name": {"type": "string"},
"cpu_shares": {"type": ["number", "string"]},
"cpu_quota": {"type": ["number", "string"]},
"cpuset": {"type": "string"},
"devices": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"dns": {"$ref": "#/definitions/string_or_list"},
"dns_search": {"$ref": "#/definitions/string_or_list"},
"dockerfile": {"type": "string"},
"domainname": {"type": "string"},
"entrypoint": {
"oneOf": [
{"type": "string"},
{"type": "array", "items": {"type": "string"}}
]
},
"env_file": {"$ref": "#/definitions/string_or_list"},
"environment": {"$ref": "#/definitions/list_or_dict"},
"expose": {
"type": "array",
"items": {
"type": ["string", "number"],
"format": "expose"
},
"uniqueItems": true
},
"extends": {
"oneOf": [
{
"type": "string"
},
{
"type": "object",
"properties": {
"service": {"type": "string"},
"file": {"type": "string"}
},
"required": ["service"],
"additionalProperties": false
}
]
},
"extra_hosts": {"$ref": "#/definitions/list_or_dict"},
"external_links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"hostname": {"type": "string"},
"image": {"type": "string"},
"ipc": {"type": "string"},
"labels": {"$ref": "#/definitions/list_or_dict"},
"links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"log_driver": {"type": "string"},
"log_opt": {"type": "object"},
"mac_address": {"type": "string"},
"mem_limit": {"type": ["number", "string"]},
"memswap_limit": {"type": ["number", "string"]},
"mem_swappiness": {"type": "integer"},
"net": {"type": "string"},
"pid": {"type": ["string", "null"]},
"ports": {
"type": "array",
"items": {
"type": ["string", "number"],
"format": "ports"
},
"uniqueItems": true
},
"privileged": {"type": "boolean"},
"read_only": {"type": "boolean"},
"restart": {"type": "string"},
"security_opt": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"shm_size": {"type": ["number", "string"]},
"stdin_open": {"type": "boolean"},
"stop_signal": {"type": "string"},
"tty": {"type": "boolean"},
"ulimits": {
"type": "object",
"patternProperties": {
"^[a-z]+$": {
"oneOf": [
{"type": "integer"},
{
"type":"object",
"properties": {
"hard": {"type": "integer"},
"soft": {"type": "integer"}
},
"required": ["soft", "hard"],
"additionalProperties": false
}
]
}
}
},
"user": {"type": "string"},
"volumes": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"volume_driver": {"type": "string"},
"volumes_from": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"working_dir": {"type": "string"}
},
"dependencies": {
"memswap_limit": ["mem_limit"]
},
"additionalProperties": false
},
"string_or_list": {
"oneOf": [
{"type": "string"},
{"$ref": "#/definitions/list_of_strings"}
]
},
"list_of_strings": {
"type": "array",
"items": {"type": "string"},
"uniqueItems": true
},
"list_or_dict": {
"oneOf": [
{
"type": "object",
"patternProperties": {
".+": {
"type": ["string", "number", "null"]
}
},
"additionalProperties": false
},
{"type": "array", "items": {"type": "string"}, "uniqueItems": true}
]
},
"constraints": {
"service": {
"id": "#/definitions/constraints/service",
"anyOf": [
{
"required": ["build"],
"not": {"required": ["image"]}
},
{
"required": ["image"],
"not": {"anyOf": [
{"required": ["build"]},
{"required": ["dockerfile"]}
]}
}
]
}
}
}
}

View file

@ -1,329 +0,0 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"id": "config_schema_v2.0.json",
"type": "object",
"properties": {
"version": {
"type": "string"
},
"services": {
"id": "#/properties/services",
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"$ref": "#/definitions/service"
}
},
"additionalProperties": false
},
"networks": {
"id": "#/properties/networks",
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"$ref": "#/definitions/network"
}
}
},
"volumes": {
"id": "#/properties/volumes",
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"$ref": "#/definitions/volume"
}
},
"additionalProperties": false
}
},
"additionalProperties": false,
"definitions": {
"service": {
"id": "#/definitions/service",
"type": "object",
"properties": {
"build": {
"oneOf": [
{"type": "string"},
{
"type": "object",
"properties": {
"context": {"type": "string"},
"dockerfile": {"type": "string"},
"args": {"$ref": "#/definitions/list_or_dict"}
},
"additionalProperties": false
}
]
},
"cap_add": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"cap_drop": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"cgroup_parent": {"type": "string"},
"command": {
"oneOf": [
{"type": "string"},
{"type": "array", "items": {"type": "string"}}
]
},
"container_name": {"type": "string"},
"cpu_shares": {"type": ["number", "string"]},
"cpu_quota": {"type": ["number", "string"]},
"cpuset": {"type": "string"},
"depends_on": {"$ref": "#/definitions/list_of_strings"},
"devices": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"dns": {"$ref": "#/definitions/string_or_list"},
"dns_search": {"$ref": "#/definitions/string_or_list"},
"domainname": {"type": "string"},
"entrypoint": {
"oneOf": [
{"type": "string"},
{"type": "array", "items": {"type": "string"}}
]
},
"env_file": {"$ref": "#/definitions/string_or_list"},
"environment": {"$ref": "#/definitions/list_or_dict"},
"expose": {
"type": "array",
"items": {
"type": ["string", "number"],
"format": "expose"
},
"uniqueItems": true
},
"extends": {
"oneOf": [
{
"type": "string"
},
{
"type": "object",
"properties": {
"service": {"type": "string"},
"file": {"type": "string"}
},
"required": ["service"],
"additionalProperties": false
}
]
},
"external_links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"extra_hosts": {"$ref": "#/definitions/list_or_dict"},
"hostname": {"type": "string"},
"image": {"type": "string"},
"ipc": {"type": "string"},
"labels": {"$ref": "#/definitions/list_or_dict"},
"links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"logging": {
"type": "object",
"properties": {
"driver": {"type": "string"},
"options": {"type": "object"}
},
"additionalProperties": false
},
"mac_address": {"type": "string"},
"mem_limit": {"type": ["number", "string"]},
"memswap_limit": {"type": ["number", "string"]},
"mem_swappiness": {"type": "integer"},
"network_mode": {"type": "string"},
"networks": {
"oneOf": [
{"$ref": "#/definitions/list_of_strings"},
{
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"oneOf": [
{
"type": "object",
"properties": {
"aliases": {"$ref": "#/definitions/list_of_strings"},
"ipv4_address": {"type": "string"},
"ipv6_address": {"type": "string"}
},
"additionalProperties": false
},
{"type": "null"}
]
}
},
"additionalProperties": false
}
]
},
"oom_score_adj": {"type": "integer", "minimum": -1000, "maximum": 1000},
"group_add": {
"type": "array",
"items": {
"type": ["string", "number"]
},
"uniqueItems": true
},
"pid": {"type": ["string", "null"]},
"ports": {
"type": "array",
"items": {
"type": ["string", "number"],
"format": "ports"
},
"uniqueItems": true
},
"privileged": {"type": "boolean"},
"read_only": {"type": "boolean"},
"restart": {"type": "string"},
"security_opt": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"shm_size": {"type": ["number", "string"]},
"stdin_open": {"type": "boolean"},
"stop_grace_period": {"type": "string", "format": "duration"},
"stop_signal": {"type": "string"},
"tmpfs": {"$ref": "#/definitions/string_or_list"},
"tty": {"type": "boolean"},
"ulimits": {
"type": "object",
"patternProperties": {
"^[a-z]+$": {
"oneOf": [
{"type": "integer"},
{
"type":"object",
"properties": {
"hard": {"type": "integer"},
"soft": {"type": "integer"}
},
"required": ["soft", "hard"],
"additionalProperties": false
}
]
}
}
},
"user": {"type": "string"},
"volumes": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"volume_driver": {"type": "string"},
"volumes_from": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"working_dir": {"type": "string"}
},
"dependencies": {
"memswap_limit": ["mem_limit"]
},
"additionalProperties": false
},
"network": {
"id": "#/definitions/network",
"type": "object",
"properties": {
"driver": {"type": "string"},
"driver_opts": {
"type": "object",
"patternProperties": {
"^.+$": {"type": ["string", "number"]}
}
},
"ipam": {
"type": "object",
"properties": {
"driver": {"type": "string"},
"config": {
"type": "array"
}
},
"additionalProperties": false
},
"external": {
"type": ["boolean", "object"],
"properties": {
"name": {"type": "string"}
},
"additionalProperties": false
},
"internal": {"type": "boolean"}
},
"additionalProperties": false
},
"volume": {
"id": "#/definitions/volume",
"type": ["object", "null"],
"properties": {
"driver": {"type": "string"},
"driver_opts": {
"type": "object",
"patternProperties": {
"^.+$": {"type": ["string", "number"]}
}
},
"external": {
"type": ["boolean", "object"],
"properties": {
"name": {"type": "string"}
},
"additionalProperties": false
}
},
"additionalProperties": false
},
"string_or_list": {
"oneOf": [
{"type": "string"},
{"$ref": "#/definitions/list_of_strings"}
]
},
"list_of_strings": {
"type": "array",
"items": {"type": "string"},
"uniqueItems": true
},
"list_or_dict": {
"oneOf": [
{
"type": "object",
"patternProperties": {
".+": {
"type": ["string", "number", "null"]
}
},
"additionalProperties": false
},
{"type": "array", "items": {"type": "string"}, "uniqueItems": true}
]
},
"constraints": {
"service": {
"id": "#/definitions/constraints/service",
"anyOf": [
{"required": ["build"]},
{"required": ["image"]}
],
"properties": {
"build": {
"required": ["context"]
}
}
}
}
}
}

View file

@ -1,376 +0,0 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"id": "config_schema_v2.1.json",
"type": "object",
"properties": {
"version": {
"type": "string"
},
"services": {
"id": "#/properties/services",
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"$ref": "#/definitions/service"
}
},
"additionalProperties": false
},
"networks": {
"id": "#/properties/networks",
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"$ref": "#/definitions/network"
}
}
},
"volumes": {
"id": "#/properties/volumes",
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"$ref": "#/definitions/volume"
}
},
"additionalProperties": false
}
},
"additionalProperties": false,
"definitions": {
"service": {
"id": "#/definitions/service",
"type": "object",
"properties": {
"build": {
"oneOf": [
{"type": "string"},
{
"type": "object",
"properties": {
"context": {"type": "string"},
"dockerfile": {"type": "string"},
"args": {"$ref": "#/definitions/list_or_dict"}
},
"additionalProperties": false
}
]
},
"cap_add": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"cap_drop": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"cgroup_parent": {"type": "string"},
"command": {
"oneOf": [
{"type": "string"},
{"type": "array", "items": {"type": "string"}}
]
},
"container_name": {"type": "string"},
"cpu_shares": {"type": ["number", "string"]},
"cpu_quota": {"type": ["number", "string"]},
"cpuset": {"type": "string"},
"depends_on": {
"oneOf": [
{"$ref": "#/definitions/list_of_strings"},
{
"type": "object",
"additionalProperties": false,
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"type": "object",
"additionalProperties": false,
"properties": {
"condition": {
"type": "string",
"enum": ["service_started", "service_healthy"]
}
},
"required": ["condition"]
}
}
}
]
},
"devices": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"dns": {"$ref": "#/definitions/string_or_list"},
"dns_search": {"$ref": "#/definitions/string_or_list"},
"domainname": {"type": "string"},
"entrypoint": {
"oneOf": [
{"type": "string"},
{"type": "array", "items": {"type": "string"}}
]
},
"env_file": {"$ref": "#/definitions/string_or_list"},
"environment": {"$ref": "#/definitions/list_or_dict"},
"expose": {
"type": "array",
"items": {
"type": ["string", "number"],
"format": "expose"
},
"uniqueItems": true
},
"extends": {
"oneOf": [
{
"type": "string"
},
{
"type": "object",
"properties": {
"service": {"type": "string"},
"file": {"type": "string"}
},
"required": ["service"],
"additionalProperties": false
}
]
},
"external_links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"extra_hosts": {"$ref": "#/definitions/list_or_dict"},
"healthcheck": {"$ref": "#/definitions/healthcheck"},
"hostname": {"type": "string"},
"image": {"type": "string"},
"ipc": {"type": "string"},
"isolation": {"type": "string"},
"labels": {"$ref": "#/definitions/list_or_dict"},
"links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"logging": {
"type": "object",
"properties": {
"driver": {"type": "string"},
"options": {"type": "object"}
},
"additionalProperties": false
},
"mac_address": {"type": "string"},
"mem_limit": {"type": ["number", "string"]},
"memswap_limit": {"type": ["number", "string"]},
"mem_swappiness": {"type": "integer"},
"network_mode": {"type": "string"},
"networks": {
"oneOf": [
{"$ref": "#/definitions/list_of_strings"},
{
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"oneOf": [
{
"type": "object",
"properties": {
"aliases": {"$ref": "#/definitions/list_of_strings"},
"ipv4_address": {"type": "string"},
"ipv6_address": {"type": "string"},
"link_local_ips": {"$ref": "#/definitions/list_of_strings"}
},
"additionalProperties": false
},
{"type": "null"}
]
}
},
"additionalProperties": false
}
]
},
"oom_score_adj": {"type": "integer", "minimum": -1000, "maximum": 1000},
"group_add": {
"type": "array",
"items": {
"type": ["string", "number"]
},
"uniqueItems": true
},
"pid": {"type": ["string", "null"]},
"ports": {
"type": "array",
"items": {
"type": ["string", "number"],
"format": "ports"
},
"uniqueItems": true
},
"privileged": {"type": "boolean"},
"read_only": {"type": "boolean"},
"restart": {"type": "string"},
"security_opt": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"shm_size": {"type": ["number", "string"]},
"sysctls": {"$ref": "#/definitions/list_or_dict"},
"stdin_open": {"type": "boolean"},
"stop_grace_period": {"type": "string", "format": "duration"},
"stop_signal": {"type": "string"},
"tmpfs": {"$ref": "#/definitions/string_or_list"},
"tty": {"type": "boolean"},
"ulimits": {
"type": "object",
"patternProperties": {
"^[a-z]+$": {
"oneOf": [
{"type": "integer"},
{
"type":"object",
"properties": {
"hard": {"type": "integer"},
"soft": {"type": "integer"}
},
"required": ["soft", "hard"],
"additionalProperties": false
}
]
}
}
},
"user": {"type": "string"},
"userns_mode": {"type": "string"},
"volumes": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"volume_driver": {"type": "string"},
"volumes_from": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"working_dir": {"type": "string"}
},
"dependencies": {
"memswap_limit": ["mem_limit"]
},
"additionalProperties": false
},
"healthcheck": {
"id": "#/definitions/healthcheck",
"type": "object",
"additionalProperties": false,
"properties": {
"disable": {"type": "boolean"},
"interval": {"type": "string"},
"retries": {"type": "number"},
"test": {
"oneOf": [
{"type": "string"},
{"type": "array", "items": {"type": "string"}}
]
},
"timeout": {"type": "string"}
}
},
"network": {
"id": "#/definitions/network",
"type": "object",
"properties": {
"driver": {"type": "string"},
"driver_opts": {
"type": "object",
"patternProperties": {
"^.+$": {"type": ["string", "number"]}
}
},
"ipam": {
"type": "object",
"properties": {
"driver": {"type": "string"},
"config": {
"type": "array"
}
},
"additionalProperties": false
},
"external": {
"type": ["boolean", "object"],
"properties": {
"name": {"type": "string"}
},
"additionalProperties": false
},
"internal": {"type": "boolean"},
"enable_ipv6": {"type": "boolean"},
"labels": {"$ref": "#/definitions/list_or_dict"}
},
"additionalProperties": false
},
"volume": {
"id": "#/definitions/volume",
"type": ["object", "null"],
"properties": {
"driver": {"type": "string"},
"driver_opts": {
"type": "object",
"patternProperties": {
"^.+$": {"type": ["string", "number"]}
}
},
"external": {
"type": ["boolean", "object"],
"properties": {
"name": {"type": "string"}
},
"additionalProperties": false
},
"labels": {"$ref": "#/definitions/list_or_dict"}
},
"additionalProperties": false
},
"string_or_list": {
"oneOf": [
{"type": "string"},
{"$ref": "#/definitions/list_of_strings"}
]
},
"list_of_strings": {
"type": "array",
"items": {"type": "string"},
"uniqueItems": true
},
"list_or_dict": {
"oneOf": [
{
"type": "object",
"patternProperties": {
".+": {
"type": ["string", "number", "null"]
}
},
"additionalProperties": false
},
{"type": "array", "items": {"type": "string"}, "uniqueItems": true}
]
},
"constraints": {
"service": {
"id": "#/definitions/constraints/service",
"anyOf": [
{"required": ["build"]},
{"required": ["image"]}
],
"properties": {
"build": {
"required": ["context"]
}
}
}
}
}
}

View file

@ -1,383 +0,0 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"id": "config_schema_v3.0.json",
"type": "object",
"required": ["version"],
"properties": {
"version": {
"type": "string"
},
"services": {
"id": "#/properties/services",
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"$ref": "#/definitions/service"
}
},
"additionalProperties": false
},
"networks": {
"id": "#/properties/networks",
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"$ref": "#/definitions/network"
}
}
},
"volumes": {
"id": "#/properties/volumes",
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"$ref": "#/definitions/volume"
}
},
"additionalProperties": false
}
},
"additionalProperties": false,
"definitions": {
"service": {
"id": "#/definitions/service",
"type": "object",
"properties": {
"deploy": {"$ref": "#/definitions/deployment"},
"build": {
"oneOf": [
{"type": "string"},
{
"type": "object",
"properties": {
"context": {"type": "string"},
"dockerfile": {"type": "string"},
"args": {"$ref": "#/definitions/list_or_dict"}
},
"additionalProperties": false
}
]
},
"cap_add": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"cap_drop": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"cgroup_parent": {"type": "string"},
"command": {
"oneOf": [
{"type": "string"},
{"type": "array", "items": {"type": "string"}}
]
},
"container_name": {"type": "string"},
"depends_on": {"$ref": "#/definitions/list_of_strings"},
"devices": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"dns": {"$ref": "#/definitions/string_or_list"},
"dns_search": {"$ref": "#/definitions/string_or_list"},
"domainname": {"type": "string"},
"entrypoint": {
"oneOf": [
{"type": "string"},
{"type": "array", "items": {"type": "string"}}
]
},
"env_file": {"$ref": "#/definitions/string_or_list"},
"environment": {"$ref": "#/definitions/list_or_dict"},
"expose": {
"type": "array",
"items": {
"type": ["string", "number"],
"format": "expose"
},
"uniqueItems": true
},
"external_links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"extra_hosts": {"$ref": "#/definitions/list_or_dict"},
"healthcheck": {"$ref": "#/definitions/healthcheck"},
"hostname": {"type": "string"},
"image": {"type": "string"},
"ipc": {"type": "string"},
"labels": {"$ref": "#/definitions/list_or_dict"},
"links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"logging": {
"type": "object",
"properties": {
"driver": {"type": "string"},
"options": {
"type": "object",
"patternProperties": {
"^.+$": {"type": ["string", "number", "null"]}
}
}
},
"additionalProperties": false
},
"mac_address": {"type": "string"},
"network_mode": {"type": "string"},
"networks": {
"oneOf": [
{"$ref": "#/definitions/list_of_strings"},
{
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"oneOf": [
{
"type": "object",
"properties": {
"aliases": {"$ref": "#/definitions/list_of_strings"},
"ipv4_address": {"type": "string"},
"ipv6_address": {"type": "string"}
},
"additionalProperties": false
},
{"type": "null"}
]
}
},
"additionalProperties": false
}
]
},
"pid": {"type": ["string", "null"]},
"ports": {
"type": "array",
"items": {
"type": ["string", "number"],
"format": "ports"
},
"uniqueItems": true
},
"privileged": {"type": "boolean"},
"read_only": {"type": "boolean"},
"restart": {"type": "string"},
"security_opt": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"shm_size": {"type": ["number", "string"]},
"sysctls": {"$ref": "#/definitions/list_or_dict"},
"stdin_open": {"type": "boolean"},
"stop_grace_period": {"type": "string", "format": "duration"},
"stop_signal": {"type": "string"},
"tmpfs": {"$ref": "#/definitions/string_or_list"},
"tty": {"type": "boolean"},
"ulimits": {
"type": "object",
"patternProperties": {
"^[a-z]+$": {
"oneOf": [
{"type": "integer"},
{
"type":"object",
"properties": {
"hard": {"type": "integer"},
"soft": {"type": "integer"}
},
"required": ["soft", "hard"],
"additionalProperties": false
}
]
}
}
},
"user": {"type": "string"},
"userns_mode": {"type": "string"},
"volumes": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"working_dir": {"type": "string"}
},
"additionalProperties": false
},
"healthcheck": {
"id": "#/definitions/healthcheck",
"type": "object",
"additionalProperties": false,
"properties": {
"disable": {"type": "boolean"},
"interval": {"type": "string"},
"retries": {"type": "number"},
"test": {
"oneOf": [
{"type": "string"},
{"type": "array", "items": {"type": "string"}}
]
},
"timeout": {"type": "string"}
}
},
"deployment": {
"id": "#/definitions/deployment",
"type": ["object", "null"],
"properties": {
"mode": {"type": "string"},
"replicas": {"type": "integer"},
"labels": {"$ref": "#/definitions/list_or_dict"},
"update_config": {
"type": "object",
"properties": {
"parallelism": {"type": "integer"},
"delay": {"type": "string", "format": "duration"},
"failure_action": {"type": "string"},
"monitor": {"type": "string", "format": "duration"},
"max_failure_ratio": {"type": "number"}
},
"additionalProperties": false
},
"resources": {
"type": "object",
"properties": {
"limits": {"$ref": "#/definitions/resource"},
"reservations": {"$ref": "#/definitions/resource"}
}
},
"restart_policy": {
"type": "object",
"properties": {
"condition": {"type": "string"},
"delay": {"type": "string", "format": "duration"},
"max_attempts": {"type": "integer"},
"window": {"type": "string", "format": "duration"}
},
"additionalProperties": false
},
"placement": {
"type": "object",
"properties": {
"constraints": {"type": "array", "items": {"type": "string"}}
},
"additionalProperties": false
}
},
"additionalProperties": false
},
"resource": {
"id": "#/definitions/resource",
"type": "object",
"properties": {
"cpus": {"type": "string"},
"memory": {"type": "string"}
},
"additionalProperties": false
},
"network": {
"id": "#/definitions/network",
"type": ["object", "null"],
"properties": {
"driver": {"type": "string"},
"driver_opts": {
"type": "object",
"patternProperties": {
"^.+$": {"type": ["string", "number"]}
}
},
"ipam": {
"type": "object",
"properties": {
"driver": {"type": "string"},
"config": {
"type": "array",
"items": {
"type": "object",
"properties": {
"subnet": {"type": "string"}
},
"additionalProperties": false
}
}
},
"additionalProperties": false
},
"external": {
"type": ["boolean", "object"],
"properties": {
"name": {"type": "string"}
},
"additionalProperties": false
},
"internal": {"type": "boolean"},
"labels": {"$ref": "#/definitions/list_or_dict"}
},
"additionalProperties": false
},
"volume": {
"id": "#/definitions/volume",
"type": ["object", "null"],
"properties": {
"driver": {"type": "string"},
"driver_opts": {
"type": "object",
"patternProperties": {
"^.+$": {"type": ["string", "number"]}
}
},
"external": {
"type": ["boolean", "object"],
"properties": {
"name": {"type": "string"}
},
"additionalProperties": false
},
"labels": {"$ref": "#/definitions/list_or_dict"}
},
"additionalProperties": false
},
"string_or_list": {
"oneOf": [
{"type": "string"},
{"$ref": "#/definitions/list_of_strings"}
]
},
"list_of_strings": {
"type": "array",
"items": {"type": "string"},
"uniqueItems": true
},
"list_or_dict": {
"oneOf": [
{
"type": "object",
"patternProperties": {
".+": {
"type": ["string", "number", "null"]
}
},
"additionalProperties": false
},
{"type": "array", "items": {"type": "string"}, "uniqueItems": true}
]
},
"constraints": {
"service": {
"id": "#/definitions/constraints/service",
"anyOf": [
{"required": ["build"]},
{"required": ["image"]}
],
"properties": {
"build": {
"required": ["context"]
}
}
}
}
}
}

View file

@ -1,428 +0,0 @@
{
"$schema": "http://json-schema.org/draft-04/schema#",
"id": "config_schema_v3.1.json",
"type": "object",
"required": ["version"],
"properties": {
"version": {
"type": "string"
},
"services": {
"id": "#/properties/services",
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"$ref": "#/definitions/service"
}
},
"additionalProperties": false
},
"networks": {
"id": "#/properties/networks",
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"$ref": "#/definitions/network"
}
}
},
"volumes": {
"id": "#/properties/volumes",
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"$ref": "#/definitions/volume"
}
},
"additionalProperties": false
},
"secrets": {
"id": "#/properties/secrets",
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"$ref": "#/definitions/secret"
}
},
"additionalProperties": false
}
},
"additionalProperties": false,
"definitions": {
"service": {
"id": "#/definitions/service",
"type": "object",
"properties": {
"deploy": {"$ref": "#/definitions/deployment"},
"build": {
"oneOf": [
{"type": "string"},
{
"type": "object",
"properties": {
"context": {"type": "string"},
"dockerfile": {"type": "string"},
"args": {"$ref": "#/definitions/list_or_dict"}
},
"additionalProperties": false
}
]
},
"cap_add": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"cap_drop": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"cgroup_parent": {"type": "string"},
"command": {
"oneOf": [
{"type": "string"},
{"type": "array", "items": {"type": "string"}}
]
},
"container_name": {"type": "string"},
"depends_on": {"$ref": "#/definitions/list_of_strings"},
"devices": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"dns": {"$ref": "#/definitions/string_or_list"},
"dns_search": {"$ref": "#/definitions/string_or_list"},
"domainname": {"type": "string"},
"entrypoint": {
"oneOf": [
{"type": "string"},
{"type": "array", "items": {"type": "string"}}
]
},
"env_file": {"$ref": "#/definitions/string_or_list"},
"environment": {"$ref": "#/definitions/list_or_dict"},
"expose": {
"type": "array",
"items": {
"type": ["string", "number"],
"format": "expose"
},
"uniqueItems": true
},
"external_links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"extra_hosts": {"$ref": "#/definitions/list_or_dict"},
"healthcheck": {"$ref": "#/definitions/healthcheck"},
"hostname": {"type": "string"},
"image": {"type": "string"},
"ipc": {"type": "string"},
"labels": {"$ref": "#/definitions/list_or_dict"},
"links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"logging": {
"type": "object",
"properties": {
"driver": {"type": "string"},
"options": {
"type": "object",
"patternProperties": {
"^.+$": {"type": ["string", "number", "null"]}
}
}
},
"additionalProperties": false
},
"mac_address": {"type": "string"},
"network_mode": {"type": "string"},
"networks": {
"oneOf": [
{"$ref": "#/definitions/list_of_strings"},
{
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9._-]+$": {
"oneOf": [
{
"type": "object",
"properties": {
"aliases": {"$ref": "#/definitions/list_of_strings"},
"ipv4_address": {"type": "string"},
"ipv6_address": {"type": "string"}
},
"additionalProperties": false
},
{"type": "null"}
]
}
},
"additionalProperties": false
}
]
},
"pid": {"type": ["string", "null"]},
"ports": {
"type": "array",
"items": {
"type": ["string", "number"],
"format": "ports"
},
"uniqueItems": true
},
"privileged": {"type": "boolean"},
"read_only": {"type": "boolean"},
"restart": {"type": "string"},
"security_opt": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"shm_size": {"type": ["number", "string"]},
"secrets": {
"type": "array",
"items": {
"oneOf": [
{"type": "string"},
{
"type": "object",
"properties": {
"source": {"type": "string"},
"target": {"type": "string"},
"uid": {"type": "string"},
"gid": {"type": "string"},
"mode": {"type": "number"}
}
}
]
}
},
"sysctls": {"$ref": "#/definitions/list_or_dict"},
"stdin_open": {"type": "boolean"},
"stop_grace_period": {"type": "string", "format": "duration"},
"stop_signal": {"type": "string"},
"tmpfs": {"$ref": "#/definitions/string_or_list"},
"tty": {"type": "boolean"},
"ulimits": {
"type": "object",
"patternProperties": {
"^[a-z]+$": {
"oneOf": [
{"type": "integer"},
{
"type":"object",
"properties": {
"hard": {"type": "integer"},
"soft": {"type": "integer"}
},
"required": ["soft", "hard"],
"additionalProperties": false
}
]
}
}
},
"user": {"type": "string"},
"userns_mode": {"type": "string"},
"volumes": {"type": "array", "items": {"type": "string"}, "uniqueItems": true},
"working_dir": {"type": "string"}
},
"additionalProperties": false
},
"healthcheck": {
"id": "#/definitions/healthcheck",
"type": "object",
"additionalProperties": false,
"properties": {
"disable": {"type": "boolean"},
"interval": {"type": "string"},
"retries": {"type": "number"},
"test": {
"oneOf": [
{"type": "string"},
{"type": "array", "items": {"type": "string"}}
]
},
"timeout": {"type": "string"}
}
},
"deployment": {
"id": "#/definitions/deployment",
"type": ["object", "null"],
"properties": {
"mode": {"type": "string"},
"replicas": {"type": "integer"},
"labels": {"$ref": "#/definitions/list_or_dict"},
"update_config": {
"type": "object",
"properties": {
"parallelism": {"type": "integer"},
"delay": {"type": "string", "format": "duration"},
"failure_action": {"type": "string"},
"monitor": {"type": "string", "format": "duration"},
"max_failure_ratio": {"type": "number"}
},
"additionalProperties": false
},
"resources": {
"type": "object",
"properties": {
"limits": {"$ref": "#/definitions/resource"},
"reservations": {"$ref": "#/definitions/resource"}
}
},
"restart_policy": {
"type": "object",
"properties": {
"condition": {"type": "string"},
"delay": {"type": "string", "format": "duration"},
"max_attempts": {"type": "integer"},
"window": {"type": "string", "format": "duration"}
},
"additionalProperties": false
},
"placement": {
"type": "object",
"properties": {
"constraints": {"type": "array", "items": {"type": "string"}}
},
"additionalProperties": false
}
},
"additionalProperties": false
},
"resource": {
"id": "#/definitions/resource",
"type": "object",
"properties": {
"cpus": {"type": "string"},
"memory": {"type": "string"}
},
"additionalProperties": false
},
"network": {
"id": "#/definitions/network",
"type": ["object", "null"],
"properties": {
"driver": {"type": "string"},
"driver_opts": {
"type": "object",
"patternProperties": {
"^.+$": {"type": ["string", "number"]}
}
},
"ipam": {
"type": "object",
"properties": {
"driver": {"type": "string"},
"config": {
"type": "array",
"items": {
"type": "object",
"properties": {
"subnet": {"type": "string"}
},
"additionalProperties": false
}
}
},
"additionalProperties": false
},
"external": {
"type": ["boolean", "object"],
"properties": {
"name": {"type": "string"}
},
"additionalProperties": false
},
"internal": {"type": "boolean"},
"labels": {"$ref": "#/definitions/list_or_dict"}
},
"additionalProperties": false
},
"volume": {
"id": "#/definitions/volume",
"type": ["object", "null"],
"properties": {
"driver": {"type": "string"},
"driver_opts": {
"type": "object",
"patternProperties": {
"^.+$": {"type": ["string", "number"]}
}
},
"external": {
"type": ["boolean", "object"],
"properties": {
"name": {"type": "string"}
},
"additionalProperties": false
},
"labels": {"$ref": "#/definitions/list_or_dict"}
},
"additionalProperties": false
},
"secret": {
"id": "#/definitions/secret",
"type": "object",
"properties": {
"file": {"type": "string"},
"external": {
"type": ["boolean", "object"],
"properties": {
"name": {"type": "string"}
}
},
"labels": {"$ref": "#/definitions/list_or_dict"}
},
"additionalProperties": false
},
"string_or_list": {
"oneOf": [
{"type": "string"},
{"$ref": "#/definitions/list_of_strings"}
]
},
"list_of_strings": {
"type": "array",
"items": {"type": "string"},
"uniqueItems": true
},
"list_or_dict": {
"oneOf": [
{
"type": "object",
"patternProperties": {
".+": {
"type": ["string", "number", "null"]
}
},
"additionalProperties": false
},
{"type": "array", "items": {"type": "string"}, "uniqueItems": true}
]
},
"constraints": {
"service": {
"id": "#/definitions/constraints/service",
"anyOf": [
{"required": ["build"]},
{"required": ["image"]}
],
"properties": {
"build": {
"required": ["context"]
}
}
}
}
}
}

View file

@ -1,120 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import codecs
import contextlib
import logging
import os
import six
from ..const import IS_WINDOWS_PLATFORM
from .errors import ConfigurationError
log = logging.getLogger(__name__)
def split_env(env):
if isinstance(env, six.binary_type):
env = env.decode('utf-8', 'replace')
if '=' in env:
return env.split('=', 1)
else:
return env, None
def env_vars_from_file(filename):
"""
Read in a line delimited file of environment variables.
"""
if not os.path.exists(filename):
raise ConfigurationError("Couldn't find env file: %s" % filename)
elif not os.path.isfile(filename):
raise ConfigurationError("%s is not a file." % (filename))
env = {}
with contextlib.closing(codecs.open(filename, 'r', 'utf-8')) as fileobj:
for line in fileobj:
line = line.strip()
if line and not line.startswith('#'):
k, v = split_env(line)
env[k] = v
return env
class Environment(dict):
def __init__(self, *args, **kwargs):
super(Environment, self).__init__(*args, **kwargs)
self.missing_keys = []
@classmethod
def from_env_file(cls, base_dir):
def _initialize():
result = cls()
if base_dir is None:
return result
env_file_path = os.path.join(base_dir, '.env')
try:
return cls(env_vars_from_file(env_file_path))
except ConfigurationError:
pass
return result
instance = _initialize()
instance.update(os.environ)
return instance
@classmethod
def from_command_line(cls, parsed_env_opts):
result = cls()
for k, v in parsed_env_opts.items():
# Values from the command line take priority, unless they're unset
# in which case they take the value from the system's environment
if v is None and k in os.environ:
result[k] = os.environ[k]
else:
result[k] = v
return result
def __getitem__(self, key):
try:
return super(Environment, self).__getitem__(key)
except KeyError:
if IS_WINDOWS_PLATFORM:
try:
return super(Environment, self).__getitem__(key.upper())
except KeyError:
pass
if key not in self.missing_keys:
log.warn(
"The {} variable is not set. Defaulting to a blank string."
.format(key)
)
self.missing_keys.append(key)
return ""
def __contains__(self, key):
result = super(Environment, self).__contains__(key)
if IS_WINDOWS_PLATFORM:
return (
result or super(Environment, self).__contains__(key.upper())
)
return result
def get(self, key, *args, **kwargs):
if IS_WINDOWS_PLATFORM:
return super(Environment, self).get(
key,
super(Environment, self).get(key.upper(), *args, **kwargs)
)
return super(Environment, self).get(key, *args, **kwargs)
def get_boolean(self, key):
# Convert a value to a boolean using "common sense" rules.
# Unset, empty, "0" and "false" (i-case) yield False.
# All other values yield True.
value = self.get(key)
if not value:
return False
if value.lower() in ['0', 'false']:
return False
return True

View file

@ -1,46 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
VERSION_EXPLANATION = (
'You might be seeing this error because you\'re using the wrong Compose file version. '
'Either specify a supported version ("2.0", "2.1", "3.0") and place your '
'service definitions under the `services` key, or omit the `version` key '
'and place your service definitions at the root of the file to use '
'version 1.\nFor more on the Compose file format versions, see '
'https://docs.docker.com/compose/compose-file/')
class ConfigurationError(Exception):
def __init__(self, msg):
self.msg = msg
def __str__(self):
return self.msg
class DependencyError(ConfigurationError):
pass
class CircularReference(ConfigurationError):
def __init__(self, trail):
self.trail = trail
@property
def msg(self):
lines = [
"{} in {}".format(service_name, filename)
for (filename, service_name) in self.trail
]
return "Circular reference:\n {}".format("\n extends ".join(lines))
class ComposeFileNotFound(ConfigurationError):
def __init__(self, supported_filenames):
super(ComposeFileNotFound, self).__init__("""
Can't find a suitable configuration file in this directory or any
parent. Are you in the right directory?
Supported filenames: %s
""" % ", ".join(supported_filenames))

View file

@ -1,103 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import logging
from string import Template
import six
from .errors import ConfigurationError
from compose.const import COMPOSEFILE_V1 as V1
from compose.const import COMPOSEFILE_V2_0 as V2_0
log = logging.getLogger(__name__)
class Interpolator(object):
def __init__(self, templater, mapping):
self.templater = templater
self.mapping = mapping
def interpolate(self, string):
try:
return self.templater(string).substitute(self.mapping)
except ValueError:
raise InvalidInterpolation(string)
def interpolate_environment_variables(version, config, section, environment):
if version in (V2_0, V1):
interpolator = Interpolator(Template, environment)
else:
interpolator = Interpolator(TemplateWithDefaults, environment)
def process_item(name, config_dict):
return dict(
(key, interpolate_value(name, key, val, section, interpolator))
for key, val in (config_dict or {}).items()
)
return dict(
(name, process_item(name, config_dict or {}))
for name, config_dict in config.items()
)
def interpolate_value(name, config_key, value, section, interpolator):
try:
return recursive_interpolate(value, interpolator)
except InvalidInterpolation as e:
raise ConfigurationError(
'Invalid interpolation format for "{config_key}" option '
'in {section} "{name}": "{string}"'.format(
config_key=config_key,
name=name,
section=section,
string=e.string))
def recursive_interpolate(obj, interpolator):
if isinstance(obj, six.string_types):
return interpolator.interpolate(obj)
if isinstance(obj, dict):
return dict(
(key, recursive_interpolate(val, interpolator))
for (key, val) in obj.items()
)
if isinstance(obj, list):
return [recursive_interpolate(val, interpolator) for val in obj]
return obj
class TemplateWithDefaults(Template):
idpattern = r'[_a-z][_a-z0-9]*(?::?-[^}]+)?'
# Modified from python2.7/string.py
def substitute(self, mapping):
# Helper function for .sub()
def convert(mo):
# Check the most common path first.
named = mo.group('named') or mo.group('braced')
if named is not None:
if ':-' in named:
var, _, default = named.partition(':-')
return mapping.get(var) or default
if '-' in named:
var, _, default = named.partition('-')
return mapping.get(var, default)
val = mapping[named]
return '%s' % (val,)
if mo.group('escaped') is not None:
return self.delimiter
if mo.group('invalid') is not None:
self._invalid(mo)
raise ValueError('Unrecognized named group in pattern',
self.pattern)
return self.pattern.sub(convert, self.template)
class InvalidInterpolation(Exception):
def __init__(self, string):
self.string = string

View file

@ -1,108 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import six
import yaml
from compose.config import types
from compose.config.config import V1
from compose.config.config import V2_1
def serialize_config_type(dumper, data):
representer = dumper.represent_str if six.PY3 else dumper.represent_unicode
return representer(data.repr())
yaml.SafeDumper.add_representer(types.VolumeFromSpec, serialize_config_type)
yaml.SafeDumper.add_representer(types.VolumeSpec, serialize_config_type)
def denormalize_config(config):
denormalized_services = [
denormalize_service_dict(service_dict, config.version)
for service_dict in config.services
]
services = {
service_dict.pop('name'): service_dict
for service_dict in denormalized_services
}
networks = config.networks.copy()
for net_name, net_conf in networks.items():
if 'external_name' in net_conf:
del net_conf['external_name']
volumes = config.volumes.copy()
for vol_name, vol_conf in volumes.items():
if 'external_name' in vol_conf:
del vol_conf['external_name']
version = config.version
if version == V1:
version = V2_1
return {
'version': version,
'services': services,
'networks': networks,
'volumes': volumes,
}
def serialize_config(config):
return yaml.safe_dump(
denormalize_config(config),
default_flow_style=False,
indent=2,
width=80)
def serialize_ns_time_value(value):
result = (value, 'ns')
table = [
(1000., 'us'),
(1000., 'ms'),
(1000., 's'),
(60., 'm'),
(60., 'h')
]
for stage in table:
tmp = value / stage[0]
if tmp == int(value / stage[0]):
value = tmp
result = (int(value), stage[1])
else:
break
return '{0}{1}'.format(*result)
def denormalize_service_dict(service_dict, version):
service_dict = service_dict.copy()
if 'restart' in service_dict:
service_dict['restart'] = types.serialize_restart_spec(
service_dict['restart']
)
if version == V1 and 'network_mode' not in service_dict:
service_dict['network_mode'] = 'bridge'
if 'depends_on' in service_dict and version != V2_1:
service_dict['depends_on'] = sorted([
svc for svc in service_dict['depends_on'].keys()
])
if 'healthcheck' in service_dict:
if 'interval' in service_dict['healthcheck']:
service_dict['healthcheck']['interval'] = serialize_ns_time_value(
service_dict['healthcheck']['interval']
)
if 'timeout' in service_dict['healthcheck']:
service_dict['healthcheck']['timeout'] = serialize_ns_time_value(
service_dict['healthcheck']['timeout']
)
if 'secrets' in service_dict:
service_dict['secrets'] = map(lambda s: s.repr(), service_dict['secrets'])
return service_dict

View file

@ -1,72 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
from compose.config.errors import DependencyError
def get_service_name_from_network_mode(network_mode):
return get_source_name_from_network_mode(network_mode, 'service')
def get_container_name_from_network_mode(network_mode):
return get_source_name_from_network_mode(network_mode, 'container')
def get_source_name_from_network_mode(network_mode, source_type):
if not network_mode:
return
if not network_mode.startswith(source_type+':'):
return
_, net_name = network_mode.split(':', 1)
return net_name
def get_service_names(links):
return [link.split(':')[0] for link in links]
def get_service_names_from_volumes_from(volumes_from):
return [volume_from.source for volume_from in volumes_from]
def get_service_dependents(service_dict, services):
name = service_dict['name']
return [
service for service in services
if (name in get_service_names(service.get('links', [])) or
name in get_service_names_from_volumes_from(service.get('volumes_from', [])) or
name == get_service_name_from_network_mode(service.get('network_mode')) or
name in service.get('depends_on', []))
]
def sort_service_dicts(services):
# Topological sort (Cormen/Tarjan algorithm).
unmarked = services[:]
temporary_marked = set()
sorted_services = []
def visit(n):
if n['name'] in temporary_marked:
if n['name'] in get_service_names(n.get('links', [])):
raise DependencyError('A service can not link to itself: %s' % n['name'])
if n['name'] in n.get('volumes_from', []):
raise DependencyError('A service can not mount itself as volume: %s' % n['name'])
if n['name'] in n.get('depends_on', []):
raise DependencyError('A service can not depend on itself: %s' % n['name'])
raise DependencyError('Circular dependency between %s' % ' and '.join(temporary_marked))
if n in unmarked:
temporary_marked.add(n['name'])
for m in get_service_dependents(n, services):
visit(m)
temporary_marked.remove(n['name'])
unmarked.remove(n)
sorted_services.insert(0, n)
while unmarked:
visit(unmarked[-1])
return sorted_services

View file

@ -1,260 +0,0 @@
"""
Types for objects parsed from the configuration.
"""
from __future__ import absolute_import
from __future__ import unicode_literals
import os
import re
from collections import namedtuple
import six
from ..const import COMPOSEFILE_V1 as V1
from .errors import ConfigurationError
from compose.const import IS_WINDOWS_PLATFORM
from compose.utils import splitdrive
win32_root_path_pattern = re.compile(r'^[A-Za-z]\:\\.*')
class VolumeFromSpec(namedtuple('_VolumeFromSpec', 'source mode type')):
# TODO: drop service_names arg when v1 is removed
@classmethod
def parse(cls, volume_from_config, service_names, version):
func = cls.parse_v1 if version == V1 else cls.parse_v2
return func(service_names, volume_from_config)
@classmethod
def parse_v1(cls, service_names, volume_from_config):
parts = volume_from_config.split(':')
if len(parts) > 2:
raise ConfigurationError(
"volume_from {} has incorrect format, should be "
"service[:mode]".format(volume_from_config))
if len(parts) == 1:
source = parts[0]
mode = 'rw'
else:
source, mode = parts
type = 'service' if source in service_names else 'container'
return cls(source, mode, type)
@classmethod
def parse_v2(cls, service_names, volume_from_config):
parts = volume_from_config.split(':')
if len(parts) > 3:
raise ConfigurationError(
"volume_from {} has incorrect format, should be one of "
"'<service name>[:<mode>]' or "
"'container:<container name>[:<mode>]'".format(volume_from_config))
if len(parts) == 1:
source = parts[0]
return cls(source, 'rw', 'service')
if len(parts) == 2:
if parts[0] == 'container':
type, source = parts
return cls(source, 'rw', type)
source, mode = parts
return cls(source, mode, 'service')
if len(parts) == 3:
type, source, mode = parts
if type not in ('service', 'container'):
raise ConfigurationError(
"Unknown volumes_from type '{}' in '{}'".format(
type,
volume_from_config))
return cls(source, mode, type)
def repr(self):
return '{v.type}:{v.source}:{v.mode}'.format(v=self)
def parse_restart_spec(restart_config):
if not restart_config:
return None
parts = restart_config.split(':')
if len(parts) > 2:
raise ConfigurationError(
"Restart %s has incorrect format, should be "
"mode[:max_retry]" % restart_config)
if len(parts) == 2:
name, max_retry_count = parts
else:
name, = parts
max_retry_count = 0
return {'Name': name, 'MaximumRetryCount': int(max_retry_count)}
def serialize_restart_spec(restart_spec):
if not restart_spec:
return ''
parts = [restart_spec['Name']]
if restart_spec['MaximumRetryCount']:
parts.append(six.text_type(restart_spec['MaximumRetryCount']))
return ':'.join(parts)
def parse_extra_hosts(extra_hosts_config):
if not extra_hosts_config:
return {}
if isinstance(extra_hosts_config, dict):
return dict(extra_hosts_config)
if isinstance(extra_hosts_config, list):
extra_hosts_dict = {}
for extra_hosts_line in extra_hosts_config:
# TODO: validate string contains ':' ?
host, ip = extra_hosts_line.split(':', 1)
extra_hosts_dict[host.strip()] = ip.strip()
return extra_hosts_dict
def normalize_path_for_engine(path):
"""Windows paths, c:\my\path\shiny, need to be changed to be compatible with
the Engine. Volume paths are expected to be linux style /c/my/path/shiny/
"""
drive, tail = splitdrive(path)
if drive:
path = '/' + drive.lower().rstrip(':') + tail
return path.replace('\\', '/')
class VolumeSpec(namedtuple('_VolumeSpec', 'external internal mode')):
@classmethod
def _parse_unix(cls, volume_config):
parts = volume_config.split(':')
if len(parts) > 3:
raise ConfigurationError(
"Volume %s has incorrect format, should be "
"external:internal[:mode]" % volume_config)
if len(parts) == 1:
external = None
internal = os.path.normpath(parts[0])
else:
external = os.path.normpath(parts[0])
internal = os.path.normpath(parts[1])
mode = 'rw'
if len(parts) == 3:
mode = parts[2]
return cls(external, internal, mode)
@classmethod
def _parse_win32(cls, volume_config, normalize):
# relative paths in windows expand to include the drive, eg C:\
# so we join the first 2 parts back together to count as one
mode = 'rw'
def separate_next_section(volume_config):
drive, tail = splitdrive(volume_config)
parts = tail.split(':', 1)
if drive:
parts[0] = drive + parts[0]
return parts
parts = separate_next_section(volume_config)
if len(parts) == 1:
internal = parts[0]
external = None
else:
external = parts[0]
parts = separate_next_section(parts[1])
external = os.path.normpath(external)
internal = parts[0]
if len(parts) > 1:
if ':' in parts[1]:
raise ConfigurationError(
"Volume %s has incorrect format, should be "
"external:internal[:mode]" % volume_config
)
mode = parts[1]
if normalize:
external = normalize_path_for_engine(external) if external else None
return cls(external, internal, mode)
@classmethod
def parse(cls, volume_config, normalize=False):
"""Parse a volume_config path and split it into external:internal[:mode]
parts to be returned as a valid VolumeSpec.
"""
if IS_WINDOWS_PLATFORM:
return cls._parse_win32(volume_config, normalize)
else:
return cls._parse_unix(volume_config)
def repr(self):
external = self.external + ':' if self.external else ''
return '{ext}{v.internal}:{v.mode}'.format(ext=external, v=self)
@property
def is_named_volume(self):
res = self.external and not self.external.startswith(('.', '/', '~'))
if not IS_WINDOWS_PLATFORM:
return res
return (
res and not self.external.startswith('\\') and
not win32_root_path_pattern.match(self.external)
)
class ServiceLink(namedtuple('_ServiceLink', 'target alias')):
@classmethod
def parse(cls, link_spec):
target, _, alias = link_spec.partition(':')
if not alias:
alias = target
return cls(target, alias)
def repr(self):
if self.target == self.alias:
return self.target
return '{s.target}:{s.alias}'.format(s=self)
@property
def merge_field(self):
return self.alias
class ServiceSecret(namedtuple('_ServiceSecret', 'source target uid gid mode')):
@classmethod
def parse(cls, spec):
if isinstance(spec, six.string_types):
return cls(spec, None, None, None, None)
return cls(
spec.get('source'),
spec.get('target'),
spec.get('uid'),
spec.get('gid'),
spec.get('mode'),
)
@property
def merge_field(self):
return self.source
def repr(self):
return dict(
[(k, v) for k, v in self._asdict().items() if v is not None]
)

View file

@ -1,423 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import json
import logging
import os
import re
import sys
import six
from docker.utils.ports import split_port
from jsonschema import Draft4Validator
from jsonschema import FormatChecker
from jsonschema import RefResolver
from jsonschema import ValidationError
from ..const import COMPOSEFILE_V1 as V1
from .errors import ConfigurationError
from .errors import VERSION_EXPLANATION
from .sort_services import get_service_name_from_network_mode
log = logging.getLogger(__name__)
DOCKER_CONFIG_HINTS = {
'cpu_share': 'cpu_shares',
'add_host': 'extra_hosts',
'hosts': 'extra_hosts',
'extra_host': 'extra_hosts',
'device': 'devices',
'link': 'links',
'memory_swap': 'memswap_limit',
'port': 'ports',
'privilege': 'privileged',
'priviliged': 'privileged',
'privilige': 'privileged',
'volume': 'volumes',
'workdir': 'working_dir',
}
VALID_NAME_CHARS = '[a-zA-Z0-9\._\-]'
VALID_EXPOSE_FORMAT = r'^\d+(\-\d+)?(\/[a-zA-Z]+)?$'
@FormatChecker.cls_checks(format="ports", raises=ValidationError)
def format_ports(instance):
try:
split_port(instance)
except ValueError as e:
raise ValidationError(six.text_type(e))
return True
@FormatChecker.cls_checks(format="expose", raises=ValidationError)
def format_expose(instance):
if isinstance(instance, six.string_types):
if not re.match(VALID_EXPOSE_FORMAT, instance):
raise ValidationError(
"should be of the format 'PORT[/PROTOCOL]'")
return True
def match_named_volumes(service_dict, project_volumes):
service_volumes = service_dict.get('volumes', [])
for volume_spec in service_volumes:
if volume_spec.is_named_volume and volume_spec.external not in project_volumes:
raise ConfigurationError(
'Named volume "{0}" is used in service "{1}" but no'
' declaration was found in the volumes section.'.format(
volume_spec.repr(), service_dict.get('name')
)
)
def python_type_to_yaml_type(type_):
type_name = type(type_).__name__
return {
'dict': 'mapping',
'list': 'array',
'int': 'number',
'float': 'number',
'bool': 'boolean',
'unicode': 'string',
'str': 'string',
'bytes': 'string',
}.get(type_name, type_name)
def validate_config_section(filename, config, section):
"""Validate the structure of a configuration section. This must be done
before interpolation so it's separate from schema validation.
"""
if not isinstance(config, dict):
raise ConfigurationError(
"In file '{filename}', {section} must be a mapping, not "
"{type}.".format(
filename=filename,
section=section,
type=anglicize_json_type(python_type_to_yaml_type(config))))
for key, value in config.items():
if not isinstance(key, six.string_types):
raise ConfigurationError(
"In file '{filename}', the {section} name {name} must be a "
"quoted string, i.e. '{name}'.".format(
filename=filename,
section=section,
name=key))
if not isinstance(value, (dict, type(None))):
raise ConfigurationError(
"In file '{filename}', {section} '{name}' must be a mapping not "
"{type}.".format(
filename=filename,
section=section,
name=key,
type=anglicize_json_type(python_type_to_yaml_type(value))))
def validate_top_level_object(config_file):
if not isinstance(config_file.config, dict):
raise ConfigurationError(
"Top level object in '{}' needs to be an object not '{}'.".format(
config_file.filename,
type(config_file.config)))
def validate_ulimits(service_config):
ulimit_config = service_config.config.get('ulimits', {})
for limit_name, soft_hard_values in six.iteritems(ulimit_config):
if isinstance(soft_hard_values, dict):
if not soft_hard_values['soft'] <= soft_hard_values['hard']:
raise ConfigurationError(
"Service '{s.name}' has invalid ulimit '{ulimit}'. "
"'soft' value can not be greater than 'hard' value ".format(
s=service_config,
ulimit=ulimit_config))
def validate_extends_file_path(service_name, extends_options, filename):
"""
The service to be extended must either be defined in the config key 'file',
or within 'filename'.
"""
error_prefix = "Invalid 'extends' configuration for %s:" % service_name
if 'file' not in extends_options and filename is None:
raise ConfigurationError(
"%s you need to specify a 'file', e.g. 'file: something.yml'" % error_prefix
)
def validate_network_mode(service_config, service_names):
network_mode = service_config.config.get('network_mode')
if not network_mode:
return
if 'networks' in service_config.config:
raise ConfigurationError("'network_mode' and 'networks' cannot be combined")
dependency = get_service_name_from_network_mode(network_mode)
if not dependency:
return
if dependency not in service_names:
raise ConfigurationError(
"Service '{s.name}' uses the network stack of service '{dep}' which "
"is undefined.".format(s=service_config, dep=dependency))
def validate_links(service_config, service_names):
for link in service_config.config.get('links', []):
if link.split(':')[0] not in service_names:
raise ConfigurationError(
"Service '{s.name}' has a link to service '{link}' which is "
"undefined.".format(s=service_config, link=link))
def validate_depends_on(service_config, service_names):
deps = service_config.config.get('depends_on', {})
for dependency in deps.keys():
if dependency not in service_names:
raise ConfigurationError(
"Service '{s.name}' depends on service '{dep}' which is "
"undefined.".format(s=service_config, dep=dependency)
)
def get_unsupported_config_msg(path, error_key):
msg = "Unsupported config option for {}: '{}'".format(path_string(path), error_key)
if error_key in DOCKER_CONFIG_HINTS:
msg += " (did you mean '{}'?)".format(DOCKER_CONFIG_HINTS[error_key])
return msg
def anglicize_json_type(json_type):
if json_type.startswith(('a', 'e', 'i', 'o', 'u')):
return 'an ' + json_type
return 'a ' + json_type
def is_service_dict_schema(schema_id):
return schema_id in ('config_schema_v1.json', '#/properties/services')
def handle_error_for_schema_with_id(error, path):
schema_id = error.schema['id']
if is_service_dict_schema(schema_id) and error.validator == 'additionalProperties':
return "Invalid service name '{}' - only {} characters are allowed".format(
# The service_name is the key to the json object
list(error.instance)[0],
VALID_NAME_CHARS)
if error.validator == 'additionalProperties':
if schema_id == '#/definitions/service':
invalid_config_key = parse_key_from_error_msg(error)
return get_unsupported_config_msg(path, invalid_config_key)
if not error.path:
return '{}\n\n{}'.format(error.message, VERSION_EXPLANATION)
def handle_generic_error(error, path):
msg_format = None
error_msg = error.message
if error.validator == 'oneOf':
msg_format = "{path} {msg}"
config_key, error_msg = _parse_oneof_validator(error)
if config_key:
path.append(config_key)
elif error.validator == 'type':
msg_format = "{path} contains an invalid type, it should be {msg}"
error_msg = _parse_valid_types_from_validator(error.validator_value)
elif error.validator == 'required':
error_msg = ", ".join(error.validator_value)
msg_format = "{path} is invalid, {msg} is required."
elif error.validator == 'dependencies':
config_key = list(error.validator_value.keys())[0]
required_keys = ",".join(error.validator_value[config_key])
msg_format = "{path} is invalid: {msg}"
path.append(config_key)
error_msg = "when defining '{}' you must set '{}' as well".format(
config_key,
required_keys)
elif error.cause:
error_msg = six.text_type(error.cause)
msg_format = "{path} is invalid: {msg}"
elif error.path:
msg_format = "{path} value {msg}"
if msg_format:
return msg_format.format(path=path_string(path), msg=error_msg)
return error.message
def parse_key_from_error_msg(error):
return error.message.split("'")[1]
def path_string(path):
return ".".join(c for c in path if isinstance(c, six.string_types))
def _parse_valid_types_from_validator(validator):
"""A validator value can be either an array of valid types or a string of
a valid type. Parse the valid types and prefix with the correct article.
"""
if not isinstance(validator, list):
return anglicize_json_type(validator)
if len(validator) == 1:
return anglicize_json_type(validator[0])
return "{}, or {}".format(
", ".join([anglicize_json_type(validator[0])] + validator[1:-1]),
anglicize_json_type(validator[-1]))
def _parse_oneof_validator(error):
"""oneOf has multiple schemas, so we need to reason about which schema, sub
schema or constraint the validation is failing on.
Inspecting the context value of a ValidationError gives us information about
which sub schema failed and which kind of error it is.
"""
types = []
for context in error.context:
if context.validator == 'oneOf':
_, error_msg = _parse_oneof_validator(context)
return path_string(context.path), error_msg
if context.validator == 'required':
return (None, context.message)
if context.validator == 'additionalProperties':
invalid_config_key = parse_key_from_error_msg(context)
return (None, "contains unsupported option: '{}'".format(invalid_config_key))
if context.path:
return (
path_string(context.path),
"contains {}, which is an invalid type, it should be {}".format(
json.dumps(context.instance),
_parse_valid_types_from_validator(context.validator_value)),
)
if context.validator == 'uniqueItems':
return (
None,
"contains non unique items, please remove duplicates from {}".format(
context.instance),
)
if context.validator == 'type':
types.append(context.validator_value)
valid_types = _parse_valid_types_from_validator(types)
return (None, "contains an invalid type, it should be {}".format(valid_types))
def process_service_constraint_errors(error, service_name, version):
if version == V1:
if 'image' in error.instance and 'build' in error.instance:
return (
"Service {} has both an image and build path specified. "
"A service can either be built to image or use an existing "
"image, not both.".format(service_name))
if 'image' in error.instance and 'dockerfile' in error.instance:
return (
"Service {} has both an image and alternate Dockerfile. "
"A service can either be built to image or use an existing "
"image, not both.".format(service_name))
if 'image' not in error.instance and 'build' not in error.instance:
return (
"Service {} has neither an image nor a build context specified. "
"At least one must be provided.".format(service_name))
def process_config_schema_errors(error):
path = list(error.path)
if 'id' in error.schema:
error_msg = handle_error_for_schema_with_id(error, path)
if error_msg:
return error_msg
return handle_generic_error(error, path)
def validate_against_config_schema(config_file):
schema = load_jsonschema(config_file.version)
format_checker = FormatChecker(["ports", "expose"])
validator = Draft4Validator(
schema,
resolver=RefResolver(get_resolver_path(), schema),
format_checker=format_checker)
handle_errors(
validator.iter_errors(config_file.config),
process_config_schema_errors,
config_file.filename)
def validate_service_constraints(config, service_name, version):
def handler(errors):
return process_service_constraint_errors(errors, service_name, version)
schema = load_jsonschema(version)
validator = Draft4Validator(schema['definitions']['constraints']['service'])
handle_errors(validator.iter_errors(config), handler, None)
def get_schema_path():
return os.path.dirname(os.path.abspath(__file__))
def load_jsonschema(version):
filename = os.path.join(
get_schema_path(),
"config_schema_v{0}.json".format(version))
with open(filename, "r") as fh:
return json.load(fh)
def get_resolver_path():
schema_path = get_schema_path()
if sys.platform == "win32":
scheme = "///"
# TODO: why is this necessary?
schema_path = schema_path.replace('\\', '/')
else:
scheme = "//"
return "file:{}{}/".format(scheme, schema_path)
def handle_errors(errors, format_error_func, filename):
"""jsonschema returns an error tree full of information to explain what has
gone wrong. Process each error and pull out relevant information and re-write
helpful error messages that are relevant.
"""
errors = list(sorted(errors, key=str))
if not errors:
return
error_msg = '\n'.join(format_error_func(error) for error in errors)
raise ConfigurationError(
"The Compose file{file_msg} is invalid because:\n{error_msg}".format(
file_msg=" '{}'".format(filename) if filename else "",
error_msg=error_msg))

View file

@ -1,41 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import sys
DEFAULT_TIMEOUT = 10
HTTP_TIMEOUT = 60
IMAGE_EVENTS = ['delete', 'import', 'load', 'pull', 'push', 'save', 'tag', 'untag']
IS_WINDOWS_PLATFORM = (sys.platform == "win32")
LABEL_CONTAINER_NUMBER = 'com.docker.compose.container-number'
LABEL_ONE_OFF = 'com.docker.compose.oneoff'
LABEL_PROJECT = 'com.docker.compose.project'
LABEL_SERVICE = 'com.docker.compose.service'
LABEL_NETWORK = 'com.docker.compose.network'
LABEL_VERSION = 'com.docker.compose.version'
LABEL_VOLUME = 'com.docker.compose.volume'
LABEL_CONFIG_HASH = 'com.docker.compose.config-hash'
SECRETS_PATH = '/run/secrets'
COMPOSEFILE_V1 = '1'
COMPOSEFILE_V2_0 = '2.0'
COMPOSEFILE_V2_1 = '2.1'
COMPOSEFILE_V3_0 = '3.0'
COMPOSEFILE_V3_1 = '3.1'
API_VERSIONS = {
COMPOSEFILE_V1: '1.21',
COMPOSEFILE_V2_0: '1.22',
COMPOSEFILE_V2_1: '1.24',
COMPOSEFILE_V3_0: '1.25',
COMPOSEFILE_V3_1: '1.25',
}
API_VERSION_TO_ENGINE_VERSION = {
API_VERSIONS[COMPOSEFILE_V1]: '1.9.0',
API_VERSIONS[COMPOSEFILE_V2_0]: '1.10.0',
API_VERSIONS[COMPOSEFILE_V2_1]: '1.12.0',
API_VERSIONS[COMPOSEFILE_V3_0]: '1.13.0',
API_VERSIONS[COMPOSEFILE_V3_1]: '1.13.0',
}

View file

@ -1,33 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
class OperationFailedError(Exception):
def __init__(self, reason):
self.msg = reason
class StreamParseError(RuntimeError):
def __init__(self, reason):
self.msg = reason
class HealthCheckException(Exception):
def __init__(self, reason):
self.msg = reason
class HealthCheckFailed(HealthCheckException):
def __init__(self, container_id):
super(HealthCheckFailed, self).__init__(
'Container "{}" is unhealthy.'.format(container_id)
)
class NoHealthCheckConfigured(HealthCheckException):
def __init__(self, service_name):
super(NoHealthCheckConfigured, self).__init__(
'Service "{}" is missing a healthcheck configuration'.format(
service_name
)
)

View file

@ -1,231 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import logging
from docker.errors import NotFound
from docker.types import IPAMConfig
from docker.types import IPAMPool
from docker.utils import version_gte
from docker.utils import version_lt
from .config import ConfigurationError
from .const import LABEL_NETWORK
from .const import LABEL_PROJECT
log = logging.getLogger(__name__)
OPTS_EXCEPTIONS = [
'com.docker.network.driver.overlay.vxlanid_list',
]
class Network(object):
def __init__(self, client, project, name, driver=None, driver_opts=None,
ipam=None, external_name=None, internal=False, enable_ipv6=False,
labels=None):
self.client = client
self.project = project
self.name = name
self.driver = driver
self.driver_opts = driver_opts
self.ipam = create_ipam_config_from_dict(ipam)
self.external_name = external_name
self.internal = internal
self.enable_ipv6 = enable_ipv6
self.labels = labels
def ensure(self):
if self.external_name:
try:
self.inspect()
log.debug(
'Network {0} declared as external. No new '
'network will be created.'.format(self.name)
)
except NotFound:
raise ConfigurationError(
'Network {name} declared as external, but could'
' not be found. Please create the network manually'
' using `{command} {name}` and try again.'.format(
name=self.external_name,
command='docker network create'
)
)
return
try:
data = self.inspect()
check_remote_network_config(data, self)
except NotFound:
driver_name = 'the default driver'
if self.driver:
driver_name = 'driver "{}"'.format(self.driver)
log.info(
'Creating network "{}" with {}'
.format(self.full_name, driver_name)
)
self.client.create_network(
name=self.full_name,
driver=self.driver,
options=self.driver_opts,
ipam=self.ipam,
internal=self.internal,
enable_ipv6=self.enable_ipv6,
labels=self._labels,
attachable=version_gte(self.client._version, '1.24') or None,
)
def remove(self):
if self.external_name:
log.info("Network %s is external, skipping", self.full_name)
return
log.info("Removing network {}".format(self.full_name))
self.client.remove_network(self.full_name)
def inspect(self):
return self.client.inspect_network(self.full_name)
@property
def full_name(self):
if self.external_name:
return self.external_name
return '{0}_{1}'.format(self.project, self.name)
@property
def _labels(self):
if version_lt(self.client._version, '1.23'):
return None
labels = self.labels.copy() if self.labels else {}
labels.update({
LABEL_PROJECT: self.project,
LABEL_NETWORK: self.name,
})
return labels
def create_ipam_config_from_dict(ipam_dict):
if not ipam_dict:
return None
return IPAMConfig(
driver=ipam_dict.get('driver'),
pool_configs=[
IPAMPool(
subnet=config.get('subnet'),
iprange=config.get('ip_range'),
gateway=config.get('gateway'),
aux_addresses=config.get('aux_addresses'),
)
for config in ipam_dict.get('config', [])
],
)
def check_remote_network_config(remote, local):
if local.driver and remote.get('Driver') != local.driver:
raise ConfigurationError(
'Network "{}" needs to be recreated - driver has changed'
.format(local.full_name)
)
local_opts = local.driver_opts or {}
remote_opts = remote.get('Options') or {}
for k in set.union(set(remote_opts.keys()), set(local_opts.keys())):
if k in OPTS_EXCEPTIONS:
continue
if remote_opts.get(k) != local_opts.get(k):
raise ConfigurationError(
'Network "{}" needs to be recreated - options have changed'
.format(local.full_name)
)
def build_networks(name, config_data, client):
network_config = config_data.networks or {}
networks = {
network_name: Network(
client=client, project=name, name=network_name,
driver=data.get('driver'),
driver_opts=data.get('driver_opts'),
ipam=data.get('ipam'),
external_name=data.get('external_name'),
internal=data.get('internal'),
enable_ipv6=data.get('enable_ipv6'),
labels=data.get('labels'),
)
for network_name, data in network_config.items()
}
if 'default' not in networks:
networks['default'] = Network(client, name, 'default')
return networks
class ProjectNetworks(object):
def __init__(self, networks, use_networking):
self.networks = networks or {}
self.use_networking = use_networking
@classmethod
def from_services(cls, services, networks, use_networking):
service_networks = {
network: networks.get(network)
for service in services
for network in get_network_names_for_service(service)
}
unused = set(networks) - set(service_networks) - {'default'}
if unused:
log.warn(
"Some networks were defined but are not used by any service: "
"{}".format(", ".join(unused)))
return cls(service_networks, use_networking)
def remove(self):
if not self.use_networking:
return
for network in self.networks.values():
try:
network.remove()
except NotFound:
log.warn("Network %s not found.", network.full_name)
def initialize(self):
if not self.use_networking:
return
for network in self.networks.values():
network.ensure()
def get_network_defs_for_service(service_dict):
if 'network_mode' in service_dict:
return {}
networks = service_dict.get('networks', {'default': None})
return dict(
(net, (config or {}))
for net, config in networks.items()
)
def get_network_names_for_service(service_dict):
return get_network_defs_for_service(service_dict).keys()
def get_networks(service_dict, network_definitions):
networks = {}
for name, netdef in get_network_defs_for_service(service_dict).items():
network = network_definitions.get(name)
if network:
networks[network.full_name] = netdef
else:
raise ConfigurationError(
'Service "{}" uses an undefined network "{}"'
.format(service_dict['name'], name))
return networks

View file

@ -1,259 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import logging
import operator
import sys
from threading import Thread
from docker.errors import APIError
from six.moves import _thread as thread
from six.moves.queue import Empty
from six.moves.queue import Queue
from compose.cli.signals import ShutdownException
from compose.errors import HealthCheckFailed
from compose.errors import NoHealthCheckConfigured
from compose.errors import OperationFailedError
from compose.utils import get_output_stream
log = logging.getLogger(__name__)
STOP = object()
def parallel_execute(objects, func, get_name, msg, get_deps=None):
"""Runs func on objects in parallel while ensuring that func is
ran on object only after it is ran on all its dependencies.
get_deps called on object must return a collection with its dependencies.
get_name called on object must return its name.
"""
objects = list(objects)
stream = get_output_stream(sys.stderr)
writer = ParallelStreamWriter(stream, msg)
for obj in objects:
writer.initialize(get_name(obj))
events = parallel_execute_iter(objects, func, get_deps)
errors = {}
results = []
error_to_reraise = None
for obj, result, exception in events:
if exception is None:
writer.write(get_name(obj), 'done')
results.append(result)
elif isinstance(exception, APIError):
errors[get_name(obj)] = exception.explanation
writer.write(get_name(obj), 'error')
elif isinstance(exception, (OperationFailedError, HealthCheckFailed, NoHealthCheckConfigured)):
errors[get_name(obj)] = exception.msg
writer.write(get_name(obj), 'error')
elif isinstance(exception, UpstreamError):
writer.write(get_name(obj), 'error')
else:
errors[get_name(obj)] = exception
error_to_reraise = exception
for obj_name, error in errors.items():
stream.write("\nERROR: for {} {}\n".format(obj_name, error))
if error_to_reraise:
raise error_to_reraise
return results, errors
def _no_deps(x):
return []
class State(object):
"""
Holds the state of a partially-complete parallel operation.
state.started: objects being processed
state.finished: objects which have been processed
state.failed: objects which either failed or whose dependencies failed
"""
def __init__(self, objects):
self.objects = objects
self.started = set()
self.finished = set()
self.failed = set()
def is_done(self):
return len(self.finished) + len(self.failed) >= len(self.objects)
def pending(self):
return set(self.objects) - self.started - self.finished - self.failed
def parallel_execute_iter(objects, func, get_deps):
"""
Runs func on objects in parallel while ensuring that func is
ran on object only after it is ran on all its dependencies.
Returns an iterator of tuples which look like:
# if func returned normally when run on object
(object, result, None)
# if func raised an exception when run on object
(object, None, exception)
# if func raised an exception when run on one of object's dependencies
(object, None, UpstreamError())
"""
if get_deps is None:
get_deps = _no_deps
results = Queue()
state = State(objects)
while True:
feed_queue(objects, func, get_deps, results, state)
try:
event = results.get(timeout=0.1)
except Empty:
continue
# See https://github.com/docker/compose/issues/189
except thread.error:
raise ShutdownException()
if event is STOP:
break
obj, _, exception = event
if exception is None:
log.debug('Finished processing: {}'.format(obj))
state.finished.add(obj)
else:
log.debug('Failed: {}'.format(obj))
state.failed.add(obj)
yield event
def producer(obj, func, results):
"""
The entry point for a producer thread which runs func on a single object.
Places a tuple on the results queue once func has either returned or raised.
"""
try:
result = func(obj)
results.put((obj, result, None))
except Exception as e:
results.put((obj, None, e))
def feed_queue(objects, func, get_deps, results, state):
"""
Starts producer threads for any objects which are ready to be processed
(i.e. they have no dependencies which haven't been successfully processed).
Shortcuts any objects whose dependencies have failed and places an
(object, None, UpstreamError()) tuple on the results queue.
"""
pending = state.pending()
log.debug('Pending: {}'.format(pending))
for obj in pending:
deps = get_deps(obj)
try:
if any(dep[0] in state.failed for dep in deps):
log.debug('{} has upstream errors - not processing'.format(obj))
results.put((obj, None, UpstreamError()))
state.failed.add(obj)
elif all(
dep not in objects or (
dep in state.finished and (not ready_check or ready_check(dep))
) for dep, ready_check in deps
):
log.debug('Starting producer thread for {}'.format(obj))
t = Thread(target=producer, args=(obj, func, results))
t.daemon = True
t.start()
state.started.add(obj)
except (HealthCheckFailed, NoHealthCheckConfigured) as e:
log.debug(
'Healthcheck for service(s) upstream of {} failed - '
'not processing'.format(obj)
)
results.put((obj, None, e))
if state.is_done():
results.put(STOP)
class UpstreamError(Exception):
pass
class ParallelStreamWriter(object):
"""Write out messages for operations happening in parallel.
Each operation has it's own line, and ANSI code characters are used
to jump to the correct line, and write over the line.
"""
def __init__(self, stream, msg):
self.stream = stream
self.msg = msg
self.lines = []
def initialize(self, obj_index):
if self.msg is None:
return
self.lines.append(obj_index)
self.stream.write("{} {} ... \r\n".format(self.msg, obj_index))
self.stream.flush()
def write(self, obj_index, status):
if self.msg is None:
return
position = self.lines.index(obj_index)
diff = len(self.lines) - position
# move up
self.stream.write("%c[%dA" % (27, diff))
# erase
self.stream.write("%c[2K\r" % 27)
self.stream.write("{} {} ... {}\r".format(self.msg, obj_index, status))
# move back down
self.stream.write("%c[%dB" % (27, diff))
self.stream.flush()
def parallel_operation(containers, operation, options, message):
parallel_execute(
containers,
operator.methodcaller(operation, **options),
operator.attrgetter('name'),
message)
def parallel_remove(containers, options):
stopped_containers = [c for c in containers if not c.is_running]
parallel_operation(stopped_containers, 'remove', options, 'Removing')
def parallel_start(containers, options):
parallel_operation(containers, 'start', options, 'Starting')
def parallel_pause(containers, options):
parallel_operation(containers, 'pause', options, 'Pausing')
def parallel_unpause(containers, options):
parallel_operation(containers, 'unpause', options, 'Unpausing')
def parallel_kill(containers, options):
parallel_operation(containers, 'kill', options, 'Killing')

View file

@ -1,111 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
from compose import utils
class StreamOutputError(Exception):
pass
def stream_output(output, stream):
is_terminal = hasattr(stream, 'isatty') and stream.isatty()
stream = utils.get_output_stream(stream)
all_events = []
lines = {}
diff = 0
for event in utils.json_stream(output):
all_events.append(event)
is_progress_event = 'progress' in event or 'progressDetail' in event
if not is_progress_event:
print_output_event(event, stream, is_terminal)
stream.flush()
continue
if not is_terminal:
continue
# if it's a progress event and we have a terminal, then display the progress bars
image_id = event.get('id')
if not image_id:
continue
if image_id not in lines:
lines[image_id] = len(lines)
stream.write("\n")
diff = len(lines) - lines[image_id]
# move cursor up `diff` rows
stream.write("%c[%dA" % (27, diff))
print_output_event(event, stream, is_terminal)
if 'id' in event:
# move cursor back down
stream.write("%c[%dB" % (27, diff))
stream.flush()
return all_events
def print_output_event(event, stream, is_terminal):
if 'errorDetail' in event:
raise StreamOutputError(event['errorDetail']['message'])
terminator = ''
if is_terminal and 'stream' not in event:
# erase current line
stream.write("%c[2K\r" % 27)
terminator = "\r"
elif 'progressDetail' in event:
return
if 'time' in event:
stream.write("[%s] " % event['time'])
if 'id' in event:
stream.write("%s: " % event['id'])
if 'from' in event:
stream.write("(from %s) " % event['from'])
status = event.get('status', '')
if 'progress' in event:
stream.write("%s %s%s" % (status, event['progress'], terminator))
elif 'progressDetail' in event:
detail = event['progressDetail']
total = detail.get('total')
if 'current' in detail and total:
percentage = float(detail['current']) / float(total) * 100
stream.write('%s (%.1f%%)%s' % (status, percentage, terminator))
else:
stream.write('%s%s' % (status, terminator))
elif 'stream' in event:
stream.write("%s%s" % (event['stream'], terminator))
else:
stream.write("%s%s\n" % (status, terminator))
def get_digest_from_pull(events):
for event in events:
status = event.get('status')
if not status or 'Digest' not in status:
continue
_, digest = status.split(':', 1)
return digest.strip()
return None
def get_digest_from_push(events):
for event in events:
digest = event.get('aux', {}).get('Digest')
if digest:
return digest
return None

View file

@ -1,616 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import datetime
import logging
import operator
from functools import reduce
import enum
from docker.errors import APIError
from . import parallel
from .config import ConfigurationError
from .config.config import V1
from .config.sort_services import get_container_name_from_network_mode
from .config.sort_services import get_service_name_from_network_mode
from .const import IMAGE_EVENTS
from .const import LABEL_ONE_OFF
from .const import LABEL_PROJECT
from .const import LABEL_SERVICE
from .container import Container
from .network import build_networks
from .network import get_networks
from .network import ProjectNetworks
from .service import BuildAction
from .service import ContainerNetworkMode
from .service import ConvergenceStrategy
from .service import NetworkMode
from .service import Service
from .service import ServiceNetworkMode
from .utils import microseconds_from_time_nano
from .volume import ProjectVolumes
log = logging.getLogger(__name__)
@enum.unique
class OneOffFilter(enum.Enum):
include = 0
exclude = 1
only = 2
@classmethod
def update_labels(cls, value, labels):
if value == cls.only:
labels.append('{0}={1}'.format(LABEL_ONE_OFF, "True"))
elif value == cls.exclude:
labels.append('{0}={1}'.format(LABEL_ONE_OFF, "False"))
elif value == cls.include:
pass
else:
raise ValueError("Invalid value for one_off: {}".format(repr(value)))
class Project(object):
"""
A collection of services.
"""
def __init__(self, name, services, client, networks=None, volumes=None):
self.name = name
self.services = services
self.client = client
self.volumes = volumes or ProjectVolumes({})
self.networks = networks or ProjectNetworks({}, False)
def labels(self, one_off=OneOffFilter.exclude):
labels = ['{0}={1}'.format(LABEL_PROJECT, self.name)]
OneOffFilter.update_labels(one_off, labels)
return labels
@classmethod
def from_config(cls, name, config_data, client):
"""
Construct a Project from a config.Config object.
"""
use_networking = (config_data.version and config_data.version != V1)
networks = build_networks(name, config_data, client)
project_networks = ProjectNetworks.from_services(
config_data.services,
networks,
use_networking)
volumes = ProjectVolumes.from_config(name, config_data, client)
project = cls(name, [], client, project_networks, volumes)
for service_dict in config_data.services:
service_dict = dict(service_dict)
if use_networking:
service_networks = get_networks(service_dict, networks)
else:
service_networks = {}
service_dict.pop('networks', None)
links = project.get_links(service_dict)
network_mode = project.get_network_mode(
service_dict, list(service_networks.keys())
)
volumes_from = get_volumes_from(project, service_dict)
if config_data.version != V1:
service_dict['volumes'] = [
volumes.namespace_spec(volume_spec)
for volume_spec in service_dict.get('volumes', [])
]
secrets = get_secrets(
service_dict['name'],
service_dict.pop('secrets', None) or [],
config_data.secrets)
project.services.append(
Service(
service_dict.pop('name'),
client=client,
project=name,
use_networking=use_networking,
networks=service_networks,
links=links,
network_mode=network_mode,
volumes_from=volumes_from,
secrets=secrets,
**service_dict)
)
return project
@property
def service_names(self):
return [service.name for service in self.services]
def get_service(self, name):
"""
Retrieve a service by name. Raises NoSuchService
if the named service does not exist.
"""
for service in self.services:
if service.name == name:
return service
raise NoSuchService(name)
def validate_service_names(self, service_names):
"""
Validate that the given list of service names only contains valid
services. Raises NoSuchService if one of the names is invalid.
"""
valid_names = self.service_names
for name in service_names:
if name not in valid_names:
raise NoSuchService(name)
def get_services(self, service_names=None, include_deps=False):
"""
Returns a list of this project's services filtered
by the provided list of names, or all services if service_names is None
or [].
If include_deps is specified, returns a list including the dependencies for
service_names, in order of dependency.
Preserves the original order of self.services where possible,
reordering as needed to resolve dependencies.
Raises NoSuchService if any of the named services do not exist.
"""
if service_names is None or len(service_names) == 0:
service_names = self.service_names
unsorted = [self.get_service(name) for name in service_names]
services = [s for s in self.services if s in unsorted]
if include_deps:
services = reduce(self._inject_deps, services, [])
uniques = []
[uniques.append(s) for s in services if s not in uniques]
return uniques
def get_services_without_duplicate(self, service_names=None, include_deps=False):
services = self.get_services(service_names, include_deps)
for service in services:
service.remove_duplicate_containers()
return services
def get_links(self, service_dict):
links = []
if 'links' in service_dict:
for link in service_dict.get('links', []):
if ':' in link:
service_name, link_name = link.split(':', 1)
else:
service_name, link_name = link, None
try:
links.append((self.get_service(service_name), link_name))
except NoSuchService:
raise ConfigurationError(
'Service "%s" has a link to service "%s" which does not '
'exist.' % (service_dict['name'], service_name))
del service_dict['links']
return links
def get_network_mode(self, service_dict, networks):
network_mode = service_dict.pop('network_mode', None)
if not network_mode:
if self.networks.use_networking:
return NetworkMode(networks[0]) if networks else NetworkMode('none')
return NetworkMode(None)
service_name = get_service_name_from_network_mode(network_mode)
if service_name:
return ServiceNetworkMode(self.get_service(service_name))
container_name = get_container_name_from_network_mode(network_mode)
if container_name:
try:
return ContainerNetworkMode(Container.from_id(self.client, container_name))
except APIError:
raise ConfigurationError(
"Service '{name}' uses the network stack of container '{dep}' which "
"does not exist.".format(name=service_dict['name'], dep=container_name))
return NetworkMode(network_mode)
def start(self, service_names=None, **options):
containers = []
def start_service(service):
service_containers = service.start(quiet=True, **options)
containers.extend(service_containers)
services = self.get_services(service_names)
def get_deps(service):
return {
(self.get_service(dep), config)
for dep, config in service.get_dependency_configs().items()
}
parallel.parallel_execute(
services,
start_service,
operator.attrgetter('name'),
'Starting',
get_deps)
return containers
def stop(self, service_names=None, one_off=OneOffFilter.exclude, **options):
containers = self.containers(service_names, one_off=one_off)
def get_deps(container):
# actually returning inversed dependencies
return {(other, None) for other in containers
if container.service in
self.get_service(other.service).get_dependency_names()}
parallel.parallel_execute(
containers,
self.build_container_operation_with_timeout_func('stop', options),
operator.attrgetter('name'),
'Stopping',
get_deps)
def pause(self, service_names=None, **options):
containers = self.containers(service_names)
parallel.parallel_pause(reversed(containers), options)
return containers
def unpause(self, service_names=None, **options):
containers = self.containers(service_names)
parallel.parallel_unpause(containers, options)
return containers
def kill(self, service_names=None, **options):
parallel.parallel_kill(self.containers(service_names), options)
def remove_stopped(self, service_names=None, one_off=OneOffFilter.exclude, **options):
parallel.parallel_remove(self.containers(
service_names, stopped=True, one_off=one_off
), options)
def down(self, remove_image_type, include_volumes, remove_orphans=False):
self.stop(one_off=OneOffFilter.include)
self.find_orphan_containers(remove_orphans)
self.remove_stopped(v=include_volumes, one_off=OneOffFilter.include)
self.networks.remove()
if include_volumes:
self.volumes.remove()
self.remove_images(remove_image_type)
def remove_images(self, remove_image_type):
for service in self.get_services():
service.remove_image(remove_image_type)
def restart(self, service_names=None, **options):
containers = self.containers(service_names, stopped=True)
parallel.parallel_execute(
containers,
self.build_container_operation_with_timeout_func('restart', options),
operator.attrgetter('name'),
'Restarting')
return containers
def build(self, service_names=None, no_cache=False, pull=False, force_rm=False):
for service in self.get_services(service_names):
if service.can_be_built():
service.build(no_cache, pull, force_rm)
else:
log.info('%s uses an image, skipping' % service.name)
def create(
self,
service_names=None,
strategy=ConvergenceStrategy.changed,
do_build=BuildAction.none,
):
services = self.get_services_without_duplicate(service_names, include_deps=True)
for svc in services:
svc.ensure_image_exists(do_build=do_build)
plans = self._get_convergence_plans(services, strategy)
for service in services:
service.execute_convergence_plan(
plans[service.name],
detached=True,
start=False)
def events(self, service_names=None):
def build_container_event(event, container):
time = datetime.datetime.fromtimestamp(event['time'])
time = time.replace(
microsecond=microseconds_from_time_nano(event['timeNano']))
return {
'time': time,
'type': 'container',
'action': event['status'],
'id': container.id,
'service': container.service,
'attributes': {
'name': container.name,
'image': event['from'],
},
'container': container,
}
service_names = set(service_names or self.service_names)
for event in self.client.events(
filters={'label': self.labels()},
decode=True
):
# The first part of this condition is a guard against some events
# broadcasted by swarm that don't have a status field.
# See https://github.com/docker/compose/issues/3316
if 'status' not in event or event['status'] in IMAGE_EVENTS:
# We don't receive any image events because labels aren't applied
# to images
continue
# TODO: get labels from the API v1.22 , see github issue 2618
try:
# this can fail if the container has been removed
container = Container.from_id(self.client, event['id'])
except APIError:
continue
if container.service not in service_names:
continue
yield build_container_event(event, container)
def up(self,
service_names=None,
start_deps=True,
strategy=ConvergenceStrategy.changed,
do_build=BuildAction.none,
timeout=None,
detached=False,
remove_orphans=False):
warn_for_swarm_mode(self.client)
self.initialize()
self.find_orphan_containers(remove_orphans)
services = self.get_services_without_duplicate(
service_names,
include_deps=start_deps)
for svc in services:
svc.ensure_image_exists(do_build=do_build)
plans = self._get_convergence_plans(services, strategy)
def do(service):
return service.execute_convergence_plan(
plans[service.name],
timeout=timeout,
detached=detached
)
def get_deps(service):
return {
(self.get_service(dep), config)
for dep, config in service.get_dependency_configs().items()
}
results, errors = parallel.parallel_execute(
services,
do,
operator.attrgetter('name'),
None,
get_deps
)
if errors:
raise ProjectError(
'Encountered errors while bringing up the project.'
)
return [
container
for svc_containers in results
if svc_containers is not None
for container in svc_containers
]
def initialize(self):
self.networks.initialize()
self.volumes.initialize()
def _get_convergence_plans(self, services, strategy):
plans = {}
for service in services:
updated_dependencies = [
name
for name in service.get_dependency_names()
if name in plans and
plans[name].action in ('recreate', 'create')
]
if updated_dependencies and strategy.allows_recreate:
log.debug('%s has upstream changes (%s)',
service.name,
", ".join(updated_dependencies))
plan = service.convergence_plan(ConvergenceStrategy.always)
else:
plan = service.convergence_plan(strategy)
plans[service.name] = plan
return plans
def pull(self, service_names=None, ignore_pull_failures=False):
for service in self.get_services(service_names, include_deps=False):
service.pull(ignore_pull_failures)
def push(self, service_names=None, ignore_push_failures=False):
for service in self.get_services(service_names, include_deps=False):
service.push(ignore_push_failures)
def _labeled_containers(self, stopped=False, one_off=OneOffFilter.exclude):
return list(filter(None, [
Container.from_ps(self.client, container)
for container in self.client.containers(
all=stopped,
filters={'label': self.labels(one_off=one_off)})])
)
def containers(self, service_names=None, stopped=False, one_off=OneOffFilter.exclude):
if service_names:
self.validate_service_names(service_names)
else:
service_names = self.service_names
containers = self._labeled_containers(stopped, one_off)
def matches_service_names(container):
return container.labels.get(LABEL_SERVICE) in service_names
return [c for c in containers if matches_service_names(c)]
def find_orphan_containers(self, remove_orphans):
def _find():
containers = self._labeled_containers()
for ctnr in containers:
service_name = ctnr.labels.get(LABEL_SERVICE)
if service_name not in self.service_names:
yield ctnr
orphans = list(_find())
if not orphans:
return
if remove_orphans:
for ctnr in orphans:
log.info('Removing orphan container "{0}"'.format(ctnr.name))
ctnr.kill()
ctnr.remove(force=True)
else:
log.warning(
'Found orphan containers ({0}) for this project. If '
'you removed or renamed this service in your compose '
'file, you can run this command with the '
'--remove-orphans flag to clean it up.'.format(
', '.join(["{}".format(ctnr.name) for ctnr in orphans])
)
)
def _inject_deps(self, acc, service):
dep_names = service.get_dependency_names()
if len(dep_names) > 0:
dep_services = self.get_services(
service_names=list(set(dep_names)),
include_deps=True
)
else:
dep_services = []
dep_services.append(service)
return acc + dep_services
def build_container_operation_with_timeout_func(self, operation, options):
def container_operation_with_timeout(container):
if options.get('timeout') is None:
service = self.get_service(container.service)
options['timeout'] = service.stop_timeout(None)
return getattr(container, operation)(**options)
return container_operation_with_timeout
def get_volumes_from(project, service_dict):
volumes_from = service_dict.pop('volumes_from', None)
if not volumes_from:
return []
def build_volume_from(spec):
if spec.type == 'service':
try:
return spec._replace(source=project.get_service(spec.source))
except NoSuchService:
pass
if spec.type == 'container':
try:
container = Container.from_id(project.client, spec.source)
return spec._replace(source=container)
except APIError:
pass
raise ConfigurationError(
"Service \"{}\" mounts volumes from \"{}\", which is not the name "
"of a service or container.".format(
service_dict['name'],
spec.source))
return [build_volume_from(vf) for vf in volumes_from]
def get_secrets(service, service_secrets, secret_defs):
secrets = []
for secret in service_secrets:
secret_def = secret_defs.get(secret.source)
if not secret_def:
raise ConfigurationError(
"Service \"{service}\" uses an undefined secret \"{secret}\" "
.format(service=service, secret=secret.source))
if secret_def.get('external_name'):
log.warn("Service \"{service}\" uses secret \"{secret}\" which is external. "
"External secrets are not available to containers created by "
"docker-compose.".format(service=service, secret=secret.source))
continue
if secret.uid or secret.gid or secret.mode:
log.warn("Service \"{service}\" uses secret \"{secret}\" with uid, "
"gid, or mode. These fields are not supported by this "
"implementation of the Compose file".format(
service=service, secret=secret.source))
secrets.append({'secret': secret, 'file': secret_def.get('file')})
return secrets
def warn_for_swarm_mode(client):
info = client.info()
if info.get('Swarm', {}).get('LocalNodeState') == 'active':
if info.get('ServerVersion', '').startswith('ucp'):
# UCP does multi-node scheduling with traditional Compose files.
return
log.warn(
"The Docker Engine you're using is running in swarm mode.\n\n"
"Compose does not use swarm mode to deploy services to multiple nodes in a swarm. "
"All containers will be scheduled on the current node.\n\n"
"To deploy your application across the swarm, "
"use `docker stack deploy`.\n"
)
class NoSuchService(Exception):
def __init__(self, name):
self.name = name
self.msg = "No such service: %s" % self.name
def __str__(self):
return self.msg
class ProjectError(Exception):
def __init__(self, msg):
self.msg = msg

File diff suppressed because it is too large Load diff

View file

View file

@ -1,96 +0,0 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
'''
timeparse.py
(c) Will Roberts <wildwilhelm@gmail.com> 1 February, 2014
This is a vendored and modified copy of:
github.com/wroberts/pytimeparse @ cc0550d
It has been modified to mimic the behaviour of
https://golang.org/pkg/time/#ParseDuration
'''
# MIT LICENSE
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation files
# (the "Software"), to deal in the Software without restriction,
# including without limitation the rights to use, copy, modify, merge,
# publish, distribute, sublicense, and/or sell copies of the Software,
# and to permit persons to whom the Software is furnished to do so,
# subject to the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from __future__ import absolute_import
from __future__ import unicode_literals
import re
HOURS = r'(?P<hours>[\d.]+)h'
MINS = r'(?P<mins>[\d.]+)m'
SECS = r'(?P<secs>[\d.]+)s'
MILLI = r'(?P<milli>[\d.]+)ms'
MICRO = r'(?P<micro>[\d.]+)(?:us|µs)'
NANO = r'(?P<nano>[\d.]+)ns'
def opt(x):
return r'(?:{x})?'.format(x=x)
TIMEFORMAT = r'{HOURS}{MINS}{SECS}{MILLI}{MICRO}{NANO}'.format(
HOURS=opt(HOURS),
MINS=opt(MINS),
SECS=opt(SECS),
MILLI=opt(MILLI),
MICRO=opt(MICRO),
NANO=opt(NANO),
)
MULTIPLIERS = dict([
('hours', 60 * 60),
('mins', 60),
('secs', 1),
('milli', 1.0 / 1000),
('micro', 1.0 / 1000.0 / 1000),
('nano', 1.0 / 1000.0 / 1000.0 / 1000.0),
])
def timeparse(sval):
"""Parse a time expression, returning it as a number of seconds. If
possible, the return value will be an `int`; if this is not
possible, the return will be a `float`. Returns `None` if a time
expression cannot be parsed from the given string.
Arguments:
- `sval`: the string value to parse
>>> timeparse('1m24s')
84
>>> timeparse('1.2 minutes')
72
>>> timeparse('1.2 seconds')
1.2
"""
match = re.match(r'\s*' + TIMEFORMAT + r'\s*$', sval, re.I)
if not match or not match.group(0).strip():
return
mdict = match.groupdict()
return sum(
MULTIPLIERS[k] * cast(v) for (k, v) in mdict.items() if v is not None)
def cast(value):
return int(value, 10) if value.isdigit() else float(value)

View file

@ -1,135 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import codecs
import hashlib
import json
import json.decoder
import logging
import ntpath
import six
from .errors import StreamParseError
from .timeparse import timeparse
json_decoder = json.JSONDecoder()
log = logging.getLogger(__name__)
def get_output_stream(stream):
if six.PY3:
return stream
return codecs.getwriter('utf-8')(stream)
def stream_as_text(stream):
"""Given a stream of bytes or text, if any of the items in the stream
are bytes convert them to text.
This function can be removed once docker-py returns text streams instead
of byte streams.
"""
for data in stream:
if not isinstance(data, six.text_type):
data = data.decode('utf-8', 'replace')
yield data
def line_splitter(buffer, separator=u'\n'):
index = buffer.find(six.text_type(separator))
if index == -1:
return None
return buffer[:index + 1], buffer[index + 1:]
def split_buffer(stream, splitter=None, decoder=lambda a: a):
"""Given a generator which yields strings and a splitter function,
joins all input, splits on the separator and yields each chunk.
Unlike string.split(), each chunk includes the trailing
separator, except for the last one if none was found on the end
of the input.
"""
splitter = splitter or line_splitter
buffered = six.text_type('')
for data in stream_as_text(stream):
buffered += data
while True:
buffer_split = splitter(buffered)
if buffer_split is None:
break
item, buffered = buffer_split
yield item
if buffered:
try:
yield decoder(buffered)
except Exception as e:
log.error(
'Compose tried decoding the following data chunk, but failed:'
'\n%s' % repr(buffered)
)
raise StreamParseError(e)
def json_splitter(buffer):
"""Attempt to parse a json object from a buffer. If there is at least one
object, return it and the rest of the buffer, otherwise return None.
"""
buffer = buffer.strip()
try:
obj, index = json_decoder.raw_decode(buffer)
rest = buffer[json.decoder.WHITESPACE.match(buffer, index).end():]
return obj, rest
except ValueError:
return None
def json_stream(stream):
"""Given a stream of text, return a stream of json objects.
This handles streams which are inconsistently buffered (some entries may
be newline delimited, and others are not).
"""
return split_buffer(stream, json_splitter, json_decoder.decode)
def json_hash(obj):
dump = json.dumps(obj, sort_keys=True, separators=(',', ':'))
h = hashlib.sha256()
h.update(dump.encode('utf8'))
return h.hexdigest()
def microseconds_from_time_nano(time_nano):
return int(time_nano % 1000000000 / 1000)
def nanoseconds_from_time_seconds(time_seconds):
return time_seconds * 1000000000
def parse_seconds_float(value):
return timeparse(value or '')
def parse_nanoseconds_int(value):
parsed = timeparse(value or '')
if parsed is None:
return None
return int(parsed * 1000000000)
def build_string_dict(source_dict):
return dict((k, str(v if v is not None else '')) for k, v in source_dict.items())
def splitdrive(path):
if len(path) == 0:
return ('', '')
if path[0] in ['.', '\\', '/', '~']:
return ('', path)
return ntpath.splitdrive(path)

View file

@ -1,151 +0,0 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import logging
from docker.errors import NotFound
from docker.utils import version_lt
from .config import ConfigurationError
from .const import LABEL_PROJECT
from .const import LABEL_VOLUME
log = logging.getLogger(__name__)
class Volume(object):
def __init__(self, client, project, name, driver=None, driver_opts=None,
external_name=None, labels=None):
self.client = client
self.project = project
self.name = name
self.driver = driver
self.driver_opts = driver_opts
self.external_name = external_name
self.labels = labels
def create(self):
return self.client.create_volume(
self.full_name, self.driver, self.driver_opts, labels=self._labels
)
def remove(self):
if self.external:
log.info("Volume %s is external, skipping", self.full_name)
return
log.info("Removing volume %s", self.full_name)
return self.client.remove_volume(self.full_name)
def inspect(self):
return self.client.inspect_volume(self.full_name)
def exists(self):
try:
self.inspect()
except NotFound:
return False
return True
@property
def external(self):
return bool(self.external_name)
@property
def full_name(self):
if self.external_name:
return self.external_name
return '{0}_{1}'.format(self.project, self.name)
@property
def _labels(self):
if version_lt(self.client._version, '1.23'):
return None
labels = self.labels.copy() if self.labels else {}
labels.update({
LABEL_PROJECT: self.project,
LABEL_VOLUME: self.name,
})
return labels
class ProjectVolumes(object):
def __init__(self, volumes):
self.volumes = volumes
@classmethod
def from_config(cls, name, config_data, client):
config_volumes = config_data.volumes or {}
volumes = {
vol_name: Volume(
client=client,
project=name,
name=vol_name,
driver=data.get('driver'),
driver_opts=data.get('driver_opts'),
external_name=data.get('external_name'),
labels=data.get('labels')
)
for vol_name, data in config_volumes.items()
}
return cls(volumes)
def remove(self):
for volume in self.volumes.values():
try:
volume.remove()
except NotFound:
log.warn("Volume %s not found.", volume.full_name)
def initialize(self):
try:
for volume in self.volumes.values():
volume_exists = volume.exists()
if volume.external:
log.debug(
'Volume {0} declared as external. No new '
'volume will be created.'.format(volume.name)
)
if not volume_exists:
raise ConfigurationError(
'Volume {name} declared as external, but could'
' not be found. Please create the volume manually'
' using `{command}{name}` and try again.'.format(
name=volume.full_name,
command='docker volume create --name='
)
)
continue
if not volume_exists:
log.info(
'Creating volume "{0}" with {1} driver'.format(
volume.full_name, volume.driver or 'default'
)
)
volume.create()
else:
driver = volume.inspect()['Driver']
if volume.driver is not None and driver != volume.driver:
raise ConfigurationError(
'Configuration for volume {0} specifies driver '
'{1}, but a volume with the same name uses a '
'different driver ({3}). If you wish to use the '
'new configuration, please remove the existing '
'volume "{2}" first:\n'
'$ docker volume rm {2}'.format(
volume.name, volume.driver, volume.full_name,
volume.inspect()['Driver']
)
)
except NotFound:
raise ConfigurationError(
'Volume %s specifies nonexistent driver %s' % (volume.name, volume.driver)
)
def namespace_spec(self, volume_spec):
if not volume_spec.is_named_volume:
return volume_spec
volume = self.volumes[volume_spec.external]
return volume_spec._replace(external=volume.full_name)

View file

@ -1,574 +0,0 @@
#!bash
#
# bash completion for docker-compose
#
# This work is based on the completion for the docker command.
#
# This script provides completion of:
# - commands and their options
# - service names
# - filepaths
#
# To enable the completions either:
# - place this file in /etc/bash_completion.d
# or
# - copy this file to e.g. ~/.docker-compose-completion.sh and add the line
# below to your .bashrc after bash completion features are loaded
# . ~/.docker-compose-completion.sh
__docker_compose_q() {
docker-compose 2>/dev/null $daemon_options "$@"
}
# Transforms a multiline list of strings into a single line string
# with the words separated by "|".
__docker_compose_to_alternatives() {
local parts=( $1 )
local IFS='|'
echo "${parts[*]}"
}
# Transforms a multiline list of options into an extglob pattern
# suitable for use in case statements.
__docker_compose_to_extglob() {
local extglob=$( __docker_compose_to_alternatives "$1" )
echo "@($extglob)"
}
# suppress trailing whitespace
__docker_compose_nospace() {
# compopt is not available in ancient bash versions
type compopt &>/dev/null && compopt -o nospace
}
# Extracts all service names from the compose file.
___docker_compose_all_services_in_compose_file() {
__docker_compose_q config --services
}
# All services, even those without an existing container
__docker_compose_services_all() {
COMPREPLY=( $(compgen -W "$(___docker_compose_all_services_in_compose_file)" -- "$cur") )
}
# All services that have an entry with the given key in their compose_file section
___docker_compose_services_with_key() {
# flatten sections under "services" to one line, then filter lines containing the key and return section name
__docker_compose_q config \
| sed -n -e '/^services:/,/^[^ ]/p' \
| sed -n 's/^ //p' \
| awk '/^[a-zA-Z0-9]/{printf "\n"};{printf $0;next;}' \
| awk -F: -v key=": +$1:" '$0 ~ key {print $1}'
}
# All services that are defined by a Dockerfile reference
__docker_compose_services_from_build() {
COMPREPLY=( $(compgen -W "$(___docker_compose_services_with_key build)" -- "$cur") )
}
# All services that are defined by an image
__docker_compose_services_from_image() {
COMPREPLY=( $(compgen -W "$(___docker_compose_services_with_key image)" -- "$cur") )
}
# The services for which containers have been created, optionally filtered
# by a boolean expression passed in as argument.
__docker_compose_services_with() {
local containers names
containers="$(__docker_compose_q ps -q)"
names=$(docker 2>/dev/null inspect -f "{{if ${1:-true}}}{{range \$k, \$v := .Config.Labels}}{{if eq \$k \"com.docker.compose.service\"}}{{\$v}}{{end}}{{end}}{{end}}" $containers)
COMPREPLY=( $(compgen -W "$names" -- "$cur") )
}
# The services for which at least one paused container exists
__docker_compose_services_paused() {
__docker_compose_services_with '.State.Paused'
}
# The services for which at least one running container exists
__docker_compose_services_running() {
__docker_compose_services_with '.State.Running'
}
# The services for which at least one stopped container exists
__docker_compose_services_stopped() {
__docker_compose_services_with 'not .State.Running'
}
_docker_compose_build() {
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--force-rm --help --no-cache --pull" -- "$cur" ) )
;;
*)
__docker_compose_services_from_build
;;
esac
}
_docker_compose_bundle() {
case "$prev" in
--output|-o)
_filedir
return
;;
esac
COMPREPLY=( $( compgen -W "--push-images --help --output -o" -- "$cur" ) )
}
_docker_compose_config() {
COMPREPLY=( $( compgen -W "--help --quiet -q --services" -- "$cur" ) )
}
_docker_compose_create() {
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--force-recreate --help --no-build --no-recreate" -- "$cur" ) )
;;
*)
__docker_compose_services_all
;;
esac
}
_docker_compose_docker_compose() {
case "$prev" in
--tlscacert|--tlscert|--tlskey)
_filedir
return
;;
--file|-f)
_filedir "y?(a)ml"
return
;;
$(__docker_compose_to_extglob "$daemon_options_with_args") )
return
;;
esac
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "$daemon_boolean_options $daemon_options_with_args --help -h --verbose --version -v" -- "$cur" ) )
;;
*)
COMPREPLY=( $( compgen -W "${commands[*]}" -- "$cur" ) )
;;
esac
}
_docker_compose_down() {
case "$prev" in
--rmi)
COMPREPLY=( $( compgen -W "all local" -- "$cur" ) )
return
;;
esac
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--help --rmi --volumes -v --remove-orphans" -- "$cur" ) )
;;
esac
}
_docker_compose_events() {
case "$prev" in
--json)
return
;;
esac
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--help --json" -- "$cur" ) )
;;
*)
__docker_compose_services_all
;;
esac
}
_docker_compose_exec() {
case "$prev" in
--index|--user)
return
;;
esac
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "-d --help --index --privileged -T --user" -- "$cur" ) )
;;
*)
__docker_compose_services_running
;;
esac
}
_docker_compose_help() {
COMPREPLY=( $( compgen -W "${commands[*]}" -- "$cur" ) )
}
_docker_compose_kill() {
case "$prev" in
-s)
COMPREPLY=( $( compgen -W "SIGHUP SIGINT SIGKILL SIGUSR1 SIGUSR2" -- "$(echo $cur | tr '[:lower:]' '[:upper:]')" ) )
return
;;
esac
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--help -s" -- "$cur" ) )
;;
*)
__docker_compose_services_running
;;
esac
}
_docker_compose_logs() {
case "$prev" in
--tail)
return
;;
esac
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--follow -f --help --no-color --tail --timestamps -t" -- "$cur" ) )
;;
*)
__docker_compose_services_all
;;
esac
}
_docker_compose_pause() {
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--help" -- "$cur" ) )
;;
*)
__docker_compose_services_running
;;
esac
}
_docker_compose_port() {
case "$prev" in
--protocol)
COMPREPLY=( $( compgen -W "tcp udp" -- "$cur" ) )
return;
;;
--index)
return;
;;
esac
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--help --index --protocol" -- "$cur" ) )
;;
*)
__docker_compose_services_all
;;
esac
}
_docker_compose_ps() {
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--help -q" -- "$cur" ) )
;;
*)
__docker_compose_services_all
;;
esac
}
_docker_compose_pull() {
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--help --ignore-pull-failures" -- "$cur" ) )
;;
*)
__docker_compose_services_from_image
;;
esac
}
_docker_compose_push() {
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--help --ignore-push-failures" -- "$cur" ) )
;;
*)
__docker_compose_services_all
;;
esac
}
_docker_compose_restart() {
case "$prev" in
--timeout|-t)
return
;;
esac
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--help --timeout -t" -- "$cur" ) )
;;
*)
__docker_compose_services_running
;;
esac
}
_docker_compose_rm() {
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--force -f --help -v" -- "$cur" ) )
;;
*)
__docker_compose_services_stopped
;;
esac
}
_docker_compose_run() {
case "$prev" in
-e)
COMPREPLY=( $( compgen -e -- "$cur" ) )
__docker_compose_nospace
return
;;
--entrypoint|--name|--user|-u|--workdir|-w)
return
;;
esac
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "-d --entrypoint -e --help --name --no-deps --publish -p --rm --service-ports -T --user -u --workdir -w" -- "$cur" ) )
;;
*)
__docker_compose_services_all
;;
esac
}
_docker_compose_scale() {
case "$prev" in
=)
COMPREPLY=("$cur")
return
;;
--timeout|-t)
return
;;
esac
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--help --timeout -t" -- "$cur" ) )
;;
*)
COMPREPLY=( $(compgen -S "=" -W "$(___docker_compose_all_services_in_compose_file)" -- "$cur") )
__docker_compose_nospace
;;
esac
}
_docker_compose_start() {
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--help" -- "$cur" ) )
;;
*)
__docker_compose_services_stopped
;;
esac
}
_docker_compose_stop() {
case "$prev" in
--timeout|-t)
return
;;
esac
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--help --timeout -t" -- "$cur" ) )
;;
*)
__docker_compose_services_running
;;
esac
}
_docker_compose_top() {
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--help" -- "$cur" ) )
;;
*)
__docker_compose_services_running
;;
esac
}
_docker_compose_unpause() {
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--help" -- "$cur" ) )
;;
*)
__docker_compose_services_paused
;;
esac
}
_docker_compose_up() {
case "$prev" in
--timeout|-t)
return
;;
esac
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--abort-on-container-exit --build -d --force-recreate --help --no-build --no-color --no-deps --no-recreate --timeout -t --remove-orphans" -- "$cur" ) )
;;
*)
__docker_compose_services_all
;;
esac
}
_docker_compose_version() {
case "$cur" in
-*)
COMPREPLY=( $( compgen -W "--short" -- "$cur" ) )
;;
esac
}
_docker_compose() {
local previous_extglob_setting=$(shopt -p extglob)
shopt -s extglob
local commands=(
build
bundle
config
create
down
events
exec
help
kill
logs
pause
port
ps
pull
push
restart
rm
run
scale
start
stop
top
unpause
up
version
)
# options for the docker daemon that have to be passed to secondary calls to
# docker-compose executed by this script
local daemon_boolean_options="
--skip-hostname-check
--tls
--tlsverify
"
local daemon_options_with_args="
--file -f
--host -H
--project-name -p
--tlscacert
--tlscert
--tlskey
"
COMPREPLY=()
local cur prev words cword
_get_comp_words_by_ref -n : cur prev words cword
# search subcommand and invoke its handler.
# special treatment of some top-level options
local command='docker_compose'
local daemon_options=()
local counter=1
while [ $counter -lt $cword ]; do
case "${words[$counter]}" in
$(__docker_compose_to_extglob "$daemon_boolean_options") )
local opt=${words[counter]}
daemon_options+=($opt)
;;
$(__docker_compose_to_extglob "$daemon_options_with_args") )
local opt=${words[counter]}
local arg=${words[++counter]}
daemon_options+=($opt $arg)
;;
-*)
;;
*)
command="${words[$counter]}"
break
;;
esac
(( counter++ ))
done
local completions_func=_docker_compose_${command//-/_}
declare -F $completions_func >/dev/null && $completions_func
eval "$previous_extglob_setting"
return 0
}
complete -F _docker_compose docker-compose

View file

@ -1,469 +0,0 @@
#compdef docker-compose
# Description
# -----------
# zsh completion for docker-compose
# https://github.com/sdurrheimer/docker-compose-zsh-completion
# -------------------------------------------------------------------------
# Version
# -------
# 1.5.0
# -------------------------------------------------------------------------
# Authors
# -------
# * Steve Durrheimer <s.durrheimer@gmail.com>
# -------------------------------------------------------------------------
# Inspiration
# -----------
# * @albers docker-compose bash completion script
# * @felixr docker zsh completion script : https://github.com/felixr/docker-zsh-completion
# -------------------------------------------------------------------------
__docker-compose_q() {
docker-compose 2>/dev/null $compose_options "$@"
}
# All services defined in docker-compose.yml
__docker-compose_all_services_in_compose_file() {
local already_selected
local -a services
already_selected=$(echo $words | tr " " "|")
__docker-compose_q config --services \
| grep -Ev "^(${already_selected})$"
}
# All services, even those without an existing container
__docker-compose_services_all() {
[[ $PREFIX = -* ]] && return 1
integer ret=1
services=$(__docker-compose_all_services_in_compose_file)
_alternative "args:services:($services)" && ret=0
return ret
}
# All services that have an entry with the given key in their docker-compose.yml section
__docker-compose_services_with_key() {
local already_selected
local -a buildable
already_selected=$(echo $words | tr " " "|")
# flatten sections to one line, then filter lines containing the key and return section name.
__docker-compose_q config \
| sed -n -e '/^services:/,/^[^ ]/p' \
| sed -n 's/^ //p' \
| awk '/^[a-zA-Z0-9]/{printf "\n"};{printf $0;next;}' \
| grep " \+$1:" \
| cut -d: -f1 \
| grep -Ev "^(${already_selected})$"
}
# All services that are defined by a Dockerfile reference
__docker-compose_services_from_build() {
[[ $PREFIX = -* ]] && return 1
integer ret=1
buildable=$(__docker-compose_services_with_key build)
_alternative "args:buildable services:($buildable)" && ret=0
return ret
}
# All services that are defined by an image
__docker-compose_services_from_image() {
[[ $PREFIX = -* ]] && return 1
integer ret=1
pullable=$(__docker-compose_services_with_key image)
_alternative "args:pullable services:($pullable)" && ret=0
return ret
}
__docker-compose_get_services() {
[[ $PREFIX = -* ]] && return 1
integer ret=1
local kind
declare -a running paused stopped lines args services
docker_status=$(docker ps > /dev/null 2>&1)
if [ $? -ne 0 ]; then
_message "Error! Docker is not running."
return 1
fi
kind=$1
shift
[[ $kind =~ (stopped|all) ]] && args=($args -a)
lines=(${(f)"$(_call_program commands docker $docker_options ps $args)"})
services=(${(f)"$(_call_program commands docker-compose 2>/dev/null $compose_options ps -q)"})
# Parse header line to find columns
local i=1 j=1 k header=${lines[1]}
declare -A begin end
while (( j < ${#header} - 1 )); do
i=$(( j + ${${header[$j,-1]}[(i)[^ ]]} - 1 ))
j=$(( i + ${${header[$i,-1]}[(i) ]} - 1 ))
k=$(( j + ${${header[$j,-1]}[(i)[^ ]]} - 2 ))
begin[${header[$i,$((j-1))]}]=$i
end[${header[$i,$((j-1))]}]=$k
done
lines=(${lines[2,-1]})
# Container ID
local line s name
local -a names
for line in $lines; do
if [[ ${services[@]} == *"${line[${begin[CONTAINER ID]},${end[CONTAINER ID]}]%% ##}"* ]]; then
names=(${(ps:,:)${${line[${begin[NAMES]},-1]}%% *}})
for name in $names; do
s="${${name%_*}#*_}:${(l:15:: :::)${${line[${begin[CREATED]},${end[CREATED]}]/ ago/}%% ##}}"
s="$s, ${line[${begin[CONTAINER ID]},${end[CONTAINER ID]}]%% ##}"
s="$s, ${${${line[${begin[IMAGE]},${end[IMAGE]}]}/:/\\:}%% ##}"
if [[ ${line[${begin[STATUS]},${end[STATUS]}]} = Exit* ]]; then
stopped=($stopped $s)
else
if [[ ${line[${begin[STATUS]},${end[STATUS]}]} = *\(Paused\)* ]]; then
paused=($paused $s)
fi
running=($running $s)
fi
done
fi
done
[[ $kind =~ (running|all) ]] && _describe -t services-running "running services" running "$@" && ret=0
[[ $kind =~ (paused|all) ]] && _describe -t services-paused "paused services" paused "$@" && ret=0
[[ $kind =~ (stopped|all) ]] && _describe -t services-stopped "stopped services" stopped "$@" && ret=0
return ret
}
__docker-compose_pausedservices() {
[[ $PREFIX = -* ]] && return 1
__docker-compose_get_services paused "$@"
}
__docker-compose_stoppedservices() {
[[ $PREFIX = -* ]] && return 1
__docker-compose_get_services stopped "$@"
}
__docker-compose_runningservices() {
[[ $PREFIX = -* ]] && return 1
__docker-compose_get_services running "$@"
}
__docker-compose_services() {
[[ $PREFIX = -* ]] && return 1
__docker-compose_get_services all "$@"
}
__docker-compose_caching_policy() {
oldp=( "$1"(Nmh+1) ) # 1 hour
(( $#oldp ))
}
__docker-compose_commands() {
local cache_policy
zstyle -s ":completion:${curcontext}:" cache-policy cache_policy
if [[ -z "$cache_policy" ]]; then
zstyle ":completion:${curcontext}:" cache-policy __docker-compose_caching_policy
fi
if ( [[ ${+_docker_compose_subcommands} -eq 0 ]] || _cache_invalid docker_compose_subcommands) \
&& ! _retrieve_cache docker_compose_subcommands;
then
local -a lines
lines=(${(f)"$(_call_program commands docker-compose 2>&1)"})
_docker_compose_subcommands=(${${${lines[$((${lines[(i)Commands:]} + 1)),${lines[(I) *]}]}## #}/ ##/:})
(( $#_docker_compose_subcommands > 0 )) && _store_cache docker_compose_subcommands _docker_compose_subcommands
fi
_describe -t docker-compose-commands "docker-compose command" _docker_compose_subcommands
}
__docker-compose_subcommand() {
local opts_help opts_force_recreate opts_no_recreate opts_no_build opts_remove_orphans opts_timeout opts_no_color opts_no_deps
opts_help='(: -)--help[Print usage]'
opts_force_recreate="(--no-recreate)--force-recreate[Recreate containers even if their configuration and image haven't changed. Incompatible with --no-recreate.]"
opts_no_recreate="(--force-recreate)--no-recreate[If containers already exist, don't recreate them. Incompatible with --force-recreate.]"
opts_no_build="(--build)--no-build[Don't build an image, even if it's missing.]"
opts_remove_orphans="--remove-orphans[Remove containers for services not defined in the Compose file]"
opts_timeout=('(-t --timeout)'{-t,--timeout}"[Specify a shutdown timeout in seconds. (default: 10)]:seconds: ")
opts_no_color='--no-color[Produce monochrome output.]'
opts_no_deps="--no-deps[Don't start linked services.]"
integer ret=1
case "$words[1]" in
(build)
_arguments \
$opts_help \
'--force-rm[Always remove intermediate containers.]' \
'--no-cache[Do not use cache when building the image.]' \
'--pull[Always attempt to pull a newer version of the image.]' \
'*:services:__docker-compose_services_from_build' && ret=0
;;
(bundle)
_arguments \
$opts_help \
'--push-images[Automatically push images for any services which have a `build` option specified.]' \
'(--output -o)'{--output,-o}'[Path to write the bundle file to. Defaults to "<project name>.dab".]:file:_files' && ret=0
;;
(config)
_arguments \
$opts_help \
'(--quiet -q)'{--quiet,-q}"[Only validate the configuration, don't print anything.]" \
'--services[Print the service names, one per line.]' && ret=0
;;
(create)
_arguments \
$opts_help \
$opts_force_recreate \
$opts_no_recreate \
$opts_no_build \
"(--no-build)--build[Build images before creating containers.]" \
'*:services:__docker-compose_services_all' && ret=0
;;
(down)
_arguments \
$opts_help \
"--rmi[Remove images. Type must be one of: 'all': Remove all images used by any service. 'local': Remove only images that don't have a custom tag set by the \`image\` field.]:type:(all local)" \
'(-v --volumes)'{-v,--volumes}"[Remove named volumes declared in the \`volumes\` section of the Compose file and anonymous volumes attached to containers.]" \
$opts_remove_orphans && ret=0
;;
(events)
_arguments \
$opts_help \
'--json[Output events as a stream of json objects]' \
'*:services:__docker-compose_services_all' && ret=0
;;
(exec)
_arguments \
$opts_help \
'-d[Detached mode: Run command in the background.]' \
'--privileged[Give extended privileges to the process.]' \
'--user=[Run the command as this user.]:username:_users' \
'-T[Disable pseudo-tty allocation. By default `docker-compose exec` allocates a TTY.]' \
'--index=[Index of the container if there are multiple instances of a service \[default: 1\]]:index: ' \
'(-):running services:__docker-compose_runningservices' \
'(-):command: _command_names -e' \
'*::arguments: _normal' && ret=0
;;
(help)
_arguments ':subcommand:__docker-compose_commands' && ret=0
;;
(kill)
_arguments \
$opts_help \
'-s[SIGNAL to send to the container. Default signal is SIGKILL.]:signal:_signals' \
'*:running services:__docker-compose_runningservices' && ret=0
;;
(logs)
_arguments \
$opts_help \
'(-f --follow)'{-f,--follow}'[Follow log output]' \
$opts_no_color \
'--tail=[Number of lines to show from the end of the logs for each container.]:number of lines: ' \
'(-t --timestamps)'{-t,--timestamps}'[Show timestamps]' \
'*:services:__docker-compose_services_all' && ret=0
;;
(pause)
_arguments \
$opts_help \
'*:running services:__docker-compose_runningservices' && ret=0
;;
(port)
_arguments \
$opts_help \
'--protocol=[tcp or udp \[default: tcp\]]:protocol:(tcp udp)' \
'--index=[index of the container if there are multiple instances of a service \[default: 1\]]:index: ' \
'1:running services:__docker-compose_runningservices' \
'2:port:_ports' && ret=0
;;
(ps)
_arguments \
$opts_help \
'-q[Only display IDs]' \
'*:services:__docker-compose_services_all' && ret=0
;;
(pull)
_arguments \
$opts_help \
'--ignore-pull-failures[Pull what it can and ignores images with pull failures.]' \
'*:services:__docker-compose_services_from_image' && ret=0
;;
(push)
_arguments \
$opts_help \
'--ignore-push-failures[Push what it can and ignores images with push failures.]' \
'*:services:__docker-compose_services' && ret=0
;;
(rm)
_arguments \
$opts_help \
'(-f --force)'{-f,--force}"[Don't ask to confirm removal]" \
'-v[Remove any anonymous volumes attached to containers]' \
'*:stopped services:__docker-compose_stoppedservices' && ret=0
;;
(run)
_arguments \
$opts_help \
'-d[Detached mode: Run container in the background, print new container name.]' \
'*-e[KEY=VAL Set an environment variable (can be used multiple times)]:environment variable KEY=VAL: ' \
'--entrypoint[Overwrite the entrypoint of the image.]:entry point: ' \
'--name=[Assign a name to the container]:name: ' \
$opts_no_deps \
'(-p --publish)'{-p,--publish=}"[Publish a container's port(s) to the host]" \
'--rm[Remove container after run. Ignored in detached mode.]' \
"--service-ports[Run command with the service's ports enabled and mapped to the host.]" \
'-T[Disable pseudo-tty allocation. By default `docker-compose run` allocates a TTY.]' \
'(-u --user)'{-u,--user=}'[Run as specified username or uid]:username or uid:_users' \
'(-w --workdir)'{-w,--workdir=}'[Working directory inside the container]:workdir: ' \
'(-):services:__docker-compose_services' \
'(-):command: _command_names -e' \
'*::arguments: _normal' && ret=0
;;
(scale)
_arguments \
$opts_help \
$opts_timeout \
'*:running services:__docker-compose_runningservices' && ret=0
;;
(start)
_arguments \
$opts_help \
'*:stopped services:__docker-compose_stoppedservices' && ret=0
;;
(stop|restart)
_arguments \
$opts_help \
$opts_timeout \
'*:running services:__docker-compose_runningservices' && ret=0
;;
(top)
_arguments \
$opts_help \
'*:running services:__docker-compose_runningservices' && ret=0
;;
(unpause)
_arguments \
$opts_help \
'*:paused services:__docker-compose_pausedservices' && ret=0
;;
(up)
_arguments \
$opts_help \
'(--abort-on-container-exit)-d[Detached mode: Run containers in the background, print new container names. Incompatible with --abort-on-container-exit.]' \
$opts_no_color \
$opts_no_deps \
$opts_force_recreate \
$opts_no_recreate \
$opts_no_build \
"(--no-build)--build[Build images before starting containers.]" \
"(-d)--abort-on-container-exit[Stops all containers if any container was stopped. Incompatible with -d.]" \
'(-t --timeout)'{-t,--timeout}"[Use this timeout in seconds for container shutdown when attached or when containers are already running. (default: 10)]:seconds: " \
$opts_remove_orphans \
'*:services:__docker-compose_services_all' && ret=0
;;
(version)
_arguments \
$opts_help \
"--short[Shows only Compose's version number.]" && ret=0
;;
(*)
_message 'Unknown sub command' && ret=1
;;
esac
return ret
}
_docker-compose() {
# Support for subservices, which allows for `compdef _docker docker-shell=_docker_containers`.
# Based on /usr/share/zsh/functions/Completion/Unix/_git without support for `ret`.
if [[ $service != docker-compose ]]; then
_call_function - _$service
return
fi
local curcontext="$curcontext" state line
integer ret=1
typeset -A opt_args
local file_description
if [[ -n ${words[(r)-f]} || -n ${words[(r)--file]} ]] ; then
file_description="Specify an override docker-compose file (default: docker-compose.override.yml)"
else
file_description="Specify an alternate docker-compose file (default: docker-compose.yml)"
fi
_arguments -C \
'(- :)'{-h,--help}'[Get help]' \
'*'{-f,--file}"[${file_description}]:file:_files -g '*.yml'" \
'(-p --project-name)'{-p,--project-name}'[Specify an alternate project name (default: directory name)]:project name:' \
'--verbose[Show more output]' \
'(- :)'{-v,--version}'[Print version and exit]' \
'(-H --host)'{-H,--host}'[Daemon socket to connect to]:host:' \
'--tls[Use TLS; implied by --tlsverify]' \
'--tlscacert=[Trust certs signed only by this CA]:ca path:' \
'--tlscert=[Path to TLS certificate file]:client cert path:' \
'--tlskey=[Path to TLS key file]:tls key path:' \
'--tlsverify[Use TLS and verify the remote]' \
"--skip-hostname-check[Don't check the daemon's hostname against the name specified in the client certificate (for example if your docker host is an IP address)]" \
'(-): :->command' \
'(-)*:: :->option-or-argument' && ret=0
local -a relevant_compose_flags relevant_docker_flags compose_options docker_options
relevant_compose_flags=(
"--file" "-f"
"--host" "-H"
"--project-name" "-p"
"--tls"
"--tlscacert"
"--tlscert"
"--tlskey"
"--tlsverify"
"--skip-hostname-check"
)
relevant_docker_flags=(
"--host" "-H"
"--tls"
"--tlscacert"
"--tlscert"
"--tlskey"
"--tlsverify"
)
for k in "${(@k)opt_args}"; do
if [[ -n "${relevant_docker_flags[(r)$k]}" ]]; then
docker_options+=$k
if [[ -n "$opt_args[$k]" ]]; then
docker_options+=$opt_args[$k]
fi
fi
if [[ -n "${relevant_compose_flags[(r)$k]}" ]]; then
compose_options+=$k
if [[ -n "$opt_args[$k]" ]]; then
compose_options+=$opt_args[$k]
fi
fi
done
case $state in
(command)
__docker-compose_commands && ret=0
;;
(option-or-argument)
curcontext=${curcontext%:*:*}:docker-compose-$words[1]:
__docker-compose_subcommand && ret=0
;;
esac
return ret
}
_docker-compose "$@"

View file

@ -1,173 +0,0 @@
#!/usr/bin/env python
"""
Migrate a Compose file from the V1 format in Compose 1.5 to the V2 format
supported by Compose 1.6+
"""
from __future__ import absolute_import
from __future__ import unicode_literals
import argparse
import logging
import sys
import ruamel.yaml
from compose.config.types import VolumeSpec
log = logging.getLogger('migrate')
def migrate(content):
data = ruamel.yaml.load(content, ruamel.yaml.RoundTripLoader)
service_names = data.keys()
for name, service in data.items():
warn_for_links(name, service)
warn_for_external_links(name, service)
rewrite_net(service, service_names)
rewrite_build(service)
rewrite_logging(service)
rewrite_volumes_from(service, service_names)
services = {name: data.pop(name) for name in data.keys()}
data['version'] = "2"
data['services'] = services
create_volumes_section(data)
return data
def warn_for_links(name, service):
links = service.get('links')
if links:
example_service = links[0].partition(':')[0]
log.warn(
"Service {name} has links, which no longer create environment "
"variables such as {example_service_upper}_PORT. "
"If you are using those in your application code, you should "
"instead connect directly to the hostname, e.g. "
"'{example_service}'."
.format(name=name, example_service=example_service,
example_service_upper=example_service.upper()))
def warn_for_external_links(name, service):
external_links = service.get('external_links')
if external_links:
log.warn(
"Service {name} has external_links: {ext}, which now work "
"slightly differently. In particular, two containers must be "
"connected to at least one network in common in order to "
"communicate, even if explicitly linked together.\n\n"
"Either connect the external container to your app's default "
"network, or connect both the external container and your "
"service's containers to a pre-existing network. See "
"https://docs.docker.com/compose/networking/ "
"for more on how to do this."
.format(name=name, ext=external_links))
def rewrite_net(service, service_names):
if 'net' in service:
network_mode = service.pop('net')
# "container:<service name>" is now "service:<service name>"
if network_mode.startswith('container:'):
name = network_mode.partition(':')[2]
if name in service_names:
network_mode = 'service:{}'.format(name)
service['network_mode'] = network_mode
def rewrite_build(service):
if 'dockerfile' in service:
service['build'] = {
'context': service.pop('build'),
'dockerfile': service.pop('dockerfile'),
}
def rewrite_logging(service):
if 'log_driver' in service:
service['logging'] = {'driver': service.pop('log_driver')}
if 'log_opt' in service:
service['logging']['options'] = service.pop('log_opt')
def rewrite_volumes_from(service, service_names):
for idx, volume_from in enumerate(service.get('volumes_from', [])):
if volume_from.split(':', 1)[0] not in service_names:
service['volumes_from'][idx] = 'container:%s' % volume_from
def create_volumes_section(data):
named_volumes = get_named_volumes(data['services'])
if named_volumes:
log.warn(
"Named volumes ({names}) must be explicitly declared. Creating a "
"'volumes' section with declarations.\n\n"
"For backwards-compatibility, they've been declared as external. "
"If you don't mind the volume names being prefixed with the "
"project name, you can remove the 'external' option from each one."
.format(names=', '.join(list(named_volumes))))
data['volumes'] = named_volumes
def get_named_volumes(services):
volume_specs = [
VolumeSpec.parse(volume)
for service in services.values()
for volume in service.get('volumes', [])
]
names = {
spec.external
for spec in volume_specs
if spec.is_named_volume
}
return {name: {'external': True} for name in names}
def write(stream, new_format, indent, width):
ruamel.yaml.dump(
new_format,
stream,
Dumper=ruamel.yaml.RoundTripDumper,
indent=indent,
width=width)
def parse_opts(args):
parser = argparse.ArgumentParser()
parser.add_argument("filename", help="Compose file filename.")
parser.add_argument("-i", "--in-place", action='store_true')
parser.add_argument(
"--indent", type=int, default=2,
help="Number of spaces used to indent the output yaml.")
parser.add_argument(
"--width", type=int, default=80,
help="Number of spaces used as the output width.")
return parser.parse_args()
def main(args):
logging.basicConfig(format='\033[33m%(levelname)s:\033[37m %(message)s\033[0m\n')
opts = parse_opts(args)
with open(opts.filename, 'r') as fh:
new_format = migrate(fh.read())
if opts.in_place:
output = open(opts.filename, 'w')
else:
output = sys.stdout
write(output, new_format, opts.indent, opts.width)
if __name__ == "__main__":
main(sys.argv)

View file

@ -1,56 +0,0 @@
# -*- mode: python -*-
block_cipher = None
a = Analysis(['bin/docker-compose'],
pathex=['.'],
hiddenimports=[],
hookspath=None,
runtime_hooks=None,
cipher=block_cipher)
pyz = PYZ(a.pure, cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
[
(
'compose/config/config_schema_v1.json',
'compose/config/config_schema_v1.json',
'DATA'
),
(
'compose/config/config_schema_v2.0.json',
'compose/config/config_schema_v2.0.json',
'DATA'
),
(
'compose/config/config_schema_v2.1.json',
'compose/config/config_schema_v2.1.json',
'DATA'
),
(
'compose/config/config_schema_v3.0.json',
'compose/config/config_schema_v3.0.json',
'DATA'
),
(
'compose/config/config_schema_v3.1.json',
'compose/config/config_schema_v3.1.json',
'DATA'
),
(
'compose/GITSHA',
'compose/GITSHA',
'DATA'
)
],
name='docker-compose',
debug=False,
strip=None,
upx=True,
console=True)

1
docs/.gitignore-gh-pages Normal file
View file

@ -0,0 +1 @@
/_site

1
docs/CNAME Normal file
View file

@ -0,0 +1 @@
www.fig.sh

10
docs/Dockerfile Normal file
View file

@ -0,0 +1,10 @@
FROM ubuntu:13.10
RUN apt-get -qq update && apt-get install -y ruby1.8 bundler python
RUN locale-gen en_US.UTF-8
ADD Gemfile /code/
ADD Gemfile.lock /code/
WORKDIR /code
RUN bundle install
ADD . /code
EXPOSE 4000
CMD bundle exec jekyll build

3
docs/Gemfile Normal file
View file

@ -0,0 +1,3 @@
source 'https://rubygems.org'
gem 'github-pages'

62
docs/Gemfile.lock Normal file
View file

@ -0,0 +1,62 @@
GEM
remote: https://rubygems.org/
specs:
RedCloth (4.2.9)
blankslate (2.1.2.4)
classifier (1.3.3)
fast-stemmer (>= 1.0.0)
colorator (0.1)
commander (4.1.5)
highline (~> 1.6.11)
fast-stemmer (1.0.2)
ffi (1.9.3)
github-pages (12)
RedCloth (= 4.2.9)
jekyll (= 1.4.2)
kramdown (= 1.2.0)
liquid (= 2.5.4)
maruku (= 0.7.0)
rdiscount (= 2.1.7)
redcarpet (= 2.3.0)
highline (1.6.20)
jekyll (1.4.2)
classifier (~> 1.3)
colorator (~> 0.1)
commander (~> 4.1.3)
liquid (~> 2.5.2)
listen (~> 1.3)
maruku (~> 0.7.0)
pygments.rb (~> 0.5.0)
redcarpet (~> 2.3.0)
safe_yaml (~> 0.9.7)
toml (~> 0.1.0)
kramdown (1.2.0)
liquid (2.5.4)
listen (1.3.1)
rb-fsevent (>= 0.9.3)
rb-inotify (>= 0.9)
rb-kqueue (>= 0.2)
maruku (0.7.0)
parslet (1.5.0)
blankslate (~> 2.0)
posix-spawn (0.3.8)
pygments.rb (0.5.4)
posix-spawn (~> 0.3.6)
yajl-ruby (~> 1.1.0)
rb-fsevent (0.9.4)
rb-inotify (0.9.3)
ffi (>= 0.5.0)
rb-kqueue (0.2.0)
ffi (>= 0.5.0)
rdiscount (2.1.7)
redcarpet (2.3.0)
safe_yaml (0.9.7)
toml (0.1.0)
parslet (~> 1.5.0)
yajl-ruby (1.1.0)
PLATFORMS
ruby
DEPENDENCIES
github-pages

View file

@ -1,16 +0,0 @@
# The docs have been moved!
The documentation for Compose has been merged into
[the general documentation repo](https://github.com/docker/docker.github.io).
The docs for Compose are now here:
https://github.com/docker/docker.github.io/tree/master/compose
Please submit pull requests for unpublished features on the `vnext-compose` branch (https://github.com/docker/docker.github.io/tree/vnext-compose).
If you submit a PR to this codebase that has a docs impact, create a second docs PR on `docker.github.io`. Use the docs PR template provided (coming soon - watch this space).
PRs for typos, additional information, etc. for already-published features should be labeled as `okay-to-publish` (we are still settling on a naming convention, will provide a label soon). You can submit these PRs either to `vnext-compose` or directly to `master` on `docker.github.io`
As always, the docs remain open-source and we appreciate your feedback and
pull requests!

3
docs/_config.yml Normal file
View file

@ -0,0 +1,3 @@
markdown: redcarpet
encoding: utf-8

View file

@ -0,0 +1,75 @@
<!DOCTYPE html>
<html lang="en-gb">
<head>
<meta charset="utf-8">
<title>{{ page.title }}</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<link href='http://fonts.googleapis.com/css?family=Lilita+One|Lato:300,400,700' rel='stylesheet' type='text/css'>
<link rel="stylesheet" type="text/css" href="css/bootstrap.min.css">
<link rel="stylesheet" type="text/css" href="css/fig.css?{{ site.time | date:'%Y%m%d%U%H%N%S' }}">
<link rel="canonical" href="http://www.fig.sh{% if page.url =="/index.html" %}/{% else %}{{ page.url }}{% endif %}">
</head>
<body>
<div class="container">
<nav class="deprecation">
<p>Fig has been replaced by Docker Compose, and is now deprecated. The new documentation is on the <a href="http://docs.docker.com/compose{% if page.url =="/index.html" %}/{% else %}{{ page.url | remove:'.html' }}{% endif %}">Docker website</a>.</p>
</nav>
<div class="logo mobile-logo">
<a href="index.html">
<img src="img/logo.png">
Fig
</a>
</div>
<div class="content">{{ content }}</div>
<div class="sidebar">
<h1 class="logo">
<a href="index.html">
<img src="img/logo.png">
Fig
</a>
</h1>
<ul class="nav">
<li><a href="index.html">Home</a></li>
<li><a href="install.html">Install</a></li>
<li><a href="rails.html">Get started with Rails</a></li>
<li><a href="django.html">Get started with Django</a></li>
<li><a href="wordpress.html">Get started with Wordpress</a></li>
</ul>
<ul class="nav">
<li>Reference:</li>
<ul>
<li><a href="yml.html">fig.yml</a></li>
<li><a href="cli.html">Commands</a></li>
<li><a href="env.html">Environment variables</a></li>
</ul>
</ul>
<ul class="nav">
<li><a href="https://github.com/docker/fig">Fig on GitHub</a></li>
<li><a href="http://webchat.freenode.net/?channels=%23docker-fig&uio=d4">#docker-fig on Freenode</a></li>
</ul>
<p>Fig is a project from <a href="https://www.docker.com">Docker</a>.</p>
<div class="badges">
<iframe src="http://ghbtns.com/github-btn.html?user=docker&amp;repo=fig&amp;type=watch&amp;count=true" allowtransparency="true" frameborder="0" scrolling="0" width="100" height="20"></iframe>
<a href="https://twitter.com/share" class="twitter-share-button" data-url="http://www.fig.sh/">Tweet</a>
</div>
</div>
</div>
<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-43996733-3', 'auto');
ga('send', 'pageview');
</script>
</body>
</html>

142
docs/cli.md Normal file
View file

@ -0,0 +1,142 @@
---
layout: default
title: Fig CLI reference
---
CLI reference
=============
Most commands are run against one or more services. If the service is omitted, it will apply to all services.
Run `fig [COMMAND] --help` for full usage.
## Options
### --verbose
Show more output
### --version
Print version and exit
### -f, --file FILE
Specify an alternate fig file (default: fig.yml)
### -p, --project-name NAME
Specify an alternate project name (default: directory name)
## Commands
### build
Build or rebuild services.
Services are built once and then tagged as `project_service`, e.g. `figtest_db`. If you change a service's `Dockerfile` or the contents of its build directory, you can run `fig build` to rebuild it.
### help
Get help on a command.
### kill
Force stop service containers.
### logs
View output from services.
### port
Print the public port for a port binding
### ps
List containers.
### pull
Pulls service images.
### rm
Remove stopped service containers.
### run
Run a one-off command on a service.
For example:
$ fig run web python manage.py shell
By default, linked services will be started, unless they are already running.
One-off commands are started in new containers with the same config as a normal container for that service, so volumes, links, etc will all be created as expected. The only thing different to a normal container is the command will be overridden with the one specified and no ports will be created in case they collide.
Links are also created between one-off commands and the other containers for that service so you can do stuff like this:
$ fig run db psql -h db -U docker
If you do not want linked containers to be started when running the one-off command, specify the `--no-deps` flag:
$ fig run --no-deps web python manage.py shell
### scale
Set number of containers to run for a service.
Numbers are specified in the form `service=num` as arguments.
For example:
$ fig scale web=2 worker=3
### start
Start existing containers for a service.
### stop
Stop running containers without removing them. They can be started again with `fig start`.
### up
Build, (re)create, start and attach to containers for a service.
Linked services will be started, unless they are already running.
By default, `fig up` will aggregate the output of each container, and when it exits, all containers will be stopped. If you run `fig up -d`, it'll start the containers in the background and leave them running.
By default if there are existing containers for a service, `fig up` will stop and recreate them (preserving mounted volumes with [volumes-from]), so that changes in `fig.yml` are picked up. If you do no want containers to be stopped and recreated, use `fig up --no-recreate`. This will still start any stopped containers, if needed.
[volumes-from]: http://docs.docker.io/en/latest/use/working_with_volumes/
## Environment Variables
Several environment variables can be used to configure Fig's behaviour.
Variables starting with `DOCKER_` are the same as those used to configure the Docker command-line client. If you're using boot2docker, `$(boot2docker shellinit)` will set them to their correct values.
### FIG\_PROJECT\_NAME
Set the project name, which is prepended to the name of every container started by Fig. Defaults to the `basename` of the current working directory.
### FIG\_FILE
Set the path to the `fig.yml` to use. Defaults to `fig.yml` in the current working directory.
### DOCKER\_HOST
Set the URL to the docker daemon. Defaults to `unix:///var/run/docker.sock`, as with the docker client.
### DOCKER\_TLS\_VERIFY
When set to anything other than an empty string, enables TLS communication with the daemon.
### DOCKER\_CERT\_PATH
Configure the path to the `ca.pem`, `cert.pem` and `key.pem` files used for TLS verification. Defaults to `~/.docker`.

7
docs/css/bootstrap.min.css vendored Normal file

File diff suppressed because one or more lines are too long

196
docs/css/fig.css Normal file
View file

@ -0,0 +1,196 @@
body {
padding-top: 20px;
padding-bottom: 60px;
font-family: 'Lato', sans-serif;
font-weight: 300;
font-size: 18px;
color: #362;
}
h1, h2, h3, h4, h5, h6 {
font-family: 'Lato', sans-serif;
font-weight: 400;
color: #25594D;
}
h2, h3, h4, h5, h6 {
margin-top: 1.5em;
}
p {
margin: 20px 0;
}
a, a:hover, a:visited {
color: #4D9900;
text-decoration: underline;
}
pre, code {
border: none;
background: #D5E1B4;
}
code, pre code {
color: #484F40;
}
pre {
border-bottom: 2px solid #bec9a1;
font-size: 14px;
}
code {
font-size: 0.84em;
}
pre code {
background: none;
}
img {
max-width: 100%;
}
.container {
margin-left: 0;
}
.deprecation {
background: #EDCAAC;
padding: 1px 30px;
}
.deprecation, .deprecation a {
color: #a41211;
}
.logo {
font-family: 'Lilita One', sans-serif;
font-size: 64px;
margin: 20px 0 40px 0;
}
.logo a {
color: #a41211;
text-decoration: none;
}
.logo img {
width: 60px;
vertical-align: -8px;
}
.mobile-logo {
text-align: center;
}
.sidebar {
font-size: 15px;
color: #777;
}
.sidebar a {
color: #a41211;
}
.sidebar p {
margin: 10px 0;
}
@media (max-width: 767px) {
.sidebar {
text-align: center;
margin-top: 40px;
}
.sidebar .logo {
display: none;
}
}
@media (min-width: 768px) {
.mobile-logo {
display: none;
}
.logo {
margin-top: 30px;
margin-bottom: 30px;
}
.content h1 {
margin: 60px 0 55px 0;
}
.sidebar {
position: fixed;
top: 0;
left: 0;
bottom: 0;
width: 280px;
overflow-y: auto;
padding-left: 40px;
padding-right: 10px;
border-right: 1px solid #ccc;
}
.deprecation, .content {
margin-left: 320px;
max-width: 650px;
}
}
.nav {
margin: 15px 0;
}
.nav li a {
display: block;
padding: 5px 0;
line-height: 1.2;
text-decoration: none;
}
.nav li a:hover, .nav li a:focus {
text-decoration: underline;
background: none;
}
.nav ul {
padding-left: 20px;
list-style: none;
}
.badges {
margin: 40px 0;
}
a.btn {
background: #25594D;
color: white;
text-transform: uppercase;
text-decoration: none;
}
a.btn:hover {
color: white;
}
.strapline {
font-size: 30px;
}
@media (min-width: 768px) {
.strapline {
font-size: 40px;
display: block;
line-height: 1.2;
margin-top: 25px;
margin-bottom: 35px;
}
}
strong {
font-weight: 700;
}

91
docs/django.md Normal file
View file

@ -0,0 +1,91 @@
---
layout: default
title: Getting started with Fig and Django
---
Getting started with Fig and Django
===================================
Let's use Fig to set up and run a Django/PostgreSQL app. Before starting, you'll need to have [Fig installed](install.html).
Let's set up the three files that'll get us started. First, our app is going to be running inside a Docker container which contains all of its dependencies. We can define what goes inside that Docker container using a file called `Dockerfile`. It'll contain this to start with:
FROM python:2.7
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN pip install -r requirements.txt
ADD . /code/
That'll install our application inside an image with Python installed alongside all of our Python dependencies. For more information on how to write Dockerfiles, see the [Docker user guide](https://docs.docker.com/userguide/dockerimages/#building-an-image-from-a-dockerfile) and the [Dockerfile reference](http://docs.docker.com/reference/builder/).
Second, we define our Python dependencies in a file called `requirements.txt`:
Django
psycopg2
Simple enough. Finally, this is all tied together with a file called `fig.yml`. It describes the services that our app comprises of (a web server and database), what Docker images they use, how they link together, what volumes will be mounted inside the containers and what ports they expose.
db:
image: postgres
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
links:
- db
See the [`fig.yml` reference](yml.html) for more information on how it works.
We can now start a Django project using `fig run`:
$ fig run web django-admin.py startproject figexample .
First, Fig will build an image for the `web` service using the `Dockerfile`. It will then run `django-admin.py startproject figexample .` inside a container using that image.
This will generate a Django app inside the current directory:
$ ls
Dockerfile fig.yml figexample manage.py requirements.txt
First thing we need to do is set up the database connection. Replace the `DATABASES = ...` definition in `figexample/settings.py` to read:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'postgres',
'USER': 'postgres',
'HOST': 'db',
'PORT': 5432,
}
}
These settings are determined by the [postgres](https://registry.hub.docker.com/_/postgres/) Docker image we are using.
Then, run `fig up`:
Recreating myapp_db_1...
Recreating myapp_web_1...
Attaching to myapp_db_1, myapp_web_1
myapp_db_1 |
myapp_db_1 | PostgreSQL stand-alone backend 9.1.11
myapp_db_1 | 2014-01-27 12:17:03 UTC LOG: database system is ready to accept connections
myapp_db_1 | 2014-01-27 12:17:03 UTC LOG: autovacuum launcher started
myapp_web_1 | Validating models...
myapp_web_1 |
myapp_web_1 | 0 errors found
myapp_web_1 | January 27, 2014 - 12:12:40
myapp_web_1 | Django version 1.6.1, using settings 'figexample.settings'
myapp_web_1 | Starting development server at http://0.0.0.0:8000/
myapp_web_1 | Quit the server with CONTROL-C.
And your Django app should be running at port 8000 on your docker daemon (if you're using boot2docker, `boot2docker ip` will tell you its address).
You can also run management commands with Docker. To set up your database, for example, run `fig up` and in another terminal run:
$ fig run web python manage.py syncdb

33
docs/env.md Normal file
View file

@ -0,0 +1,33 @@
---
layout: default
title: Fig environment variables reference
---
Environment variables reference
===============================
**Note:** Environment variables are no longer the recommended method for connecting to linked services. Instead, you should use the link name (by default, the name of the linked service) as the hostname to connect to. See the [fig.yml documentation](yml.html#links) for details.
Fig uses [Docker links] to expose services' containers to one another. Each linked container injects a set of environment variables, each of which begins with the uppercase name of the container.
To see what environment variables are available to a service, run `fig run SERVICE env`.
<b><i>name</i>\_PORT</b><br>
Full URL, e.g. `DB_PORT=tcp://172.17.0.5:5432`
<b><i>name</i>\_PORT\_<i>num</i>\_<i>protocol</i></b><br>
Full URL, e.g. `DB_PORT_5432_TCP=tcp://172.17.0.5:5432`
<b><i>name</i>\_PORT\_<i>num</i>\_<i>protocol</i>\_ADDR</b><br>
Container's IP address, e.g. `DB_PORT_5432_TCP_ADDR=172.17.0.5`
<b><i>name</i>\_PORT\_<i>num</i>\_<i>protocol</i>\_PORT</b><br>
Exposed port number, e.g. `DB_PORT_5432_TCP_PORT=5432`
<b><i>name</i>\_PORT\_<i>num</i>\_<i>protocol</i>\_PROTO</b><br>
Protocol (tcp or udp), e.g. `DB_PORT_5432_TCP_PROTO=tcp`
<b><i>name</i>\_NAME</b><br>
Fully qualified container name, e.g. `DB_1_NAME=/myapp_web_1/myapp_db_1`
[Docker links]: http://docs.docker.com/userguide/dockerlinks/

8
docs/fig.yml Normal file
View file

@ -0,0 +1,8 @@
jekyll:
build: .
ports:
- "4000:4000"
volumes:
- .:/code
environment:
- LANG=en_US.UTF-8

BIN
docs/img/logo.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 130 KiB

140
docs/index.md Normal file
View file

@ -0,0 +1,140 @@
---
layout: default
title: Fig | Fast, isolated development environments using Docker
---
<strong class="strapline">Fast, isolated development environments using Docker.</strong>
Define your app's environment with a `Dockerfile` so it can be reproduced anywhere:
FROM python:2.7
ADD . /code
WORKDIR /code
RUN pip install -r requirements.txt
Define the services that make up your app in `fig.yml` so they can be run together in an isolated environment:
```yaml
web:
build: .
command: python app.py
links:
- db
ports:
- "8000:8000"
db:
image: postgres
```
(No more installing Postgres on your laptop!)
Then type `fig up`, and Fig will start and run your entire app:
![example fig run](https://orchardup.com/static/images/fig-example-large.gif)
There are commands to:
- start, stop and rebuild services
- view the status of running services
- tail running services' log output
- run a one-off command on a service
Quick start
-----------
Let's get a basic Python web app running on Fig. It assumes a little knowledge of Python, but the concepts should be clear if you're not familiar with it.
First, [install Docker and Fig](install.html).
You'll want to make a directory for the project:
$ mkdir figtest
$ cd figtest
Inside this directory, create `app.py`, a simple web app that uses the Flask framework and increments a value in Redis:
```python
from flask import Flask
from redis import Redis
import os
app = Flask(__name__)
redis = Redis(host='redis', port=6379)
@app.route('/')
def hello():
redis.incr('hits')
return 'Hello World! I have been seen %s times.' % redis.get('hits')
if __name__ == "__main__":
app.run(host="0.0.0.0", debug=True)
```
We define our Python dependencies in a file called `requirements.txt`:
flask
redis
Next, we want to create a Docker image containing all of our app's dependencies. We specify how to build one using a file called `Dockerfile`:
FROM python:2.7
ADD . /code
WORKDIR /code
RUN pip install -r requirements.txt
This tells Docker to install Python, our code and our Python dependencies inside a Docker image. For more information on how to write Dockerfiles, see the [Docker user guide](https://docs.docker.com/userguide/dockerimages/#building-an-image-from-a-dockerfile) and the [Dockerfile reference](http://docs.docker.com/reference/builder/).
We then define a set of services using `fig.yml`:
web:
build: .
command: python app.py
ports:
- "5000:5000"
volumes:
- .:/code
links:
- redis
redis:
image: redis
This defines two services:
- `web`, which is built from `Dockerfile` in the current directory. It also says to run the command `python app.py` inside the image, forward the exposed port 5000 on the container to port 5000 on the host machine, connect up the Redis service, and mount the current directory inside the container so we can work on code without having to rebuild the image.
- `redis`, which uses the public image [redis](https://registry.hub.docker.com/_/redis/).
Now if we run `fig up`, it'll pull a Redis image, build an image for our own code, and start everything up:
$ fig up
Pulling image redis...
Building web...
Starting figtest_redis_1...
Starting figtest_web_1...
redis_1 | [8] 02 Jan 18:43:35.576 # Server started, Redis version 2.8.3
web_1 | * Running on http://0.0.0.0:5000/
The web app should now be listening on port 5000 on your docker daemon (if you're using boot2docker, `boot2docker ip` will tell you its address).
If you want to run your services in the background, you can pass the `-d` flag to `fig up` and use `fig ps` to see what is currently running:
$ fig up -d
Starting figtest_redis_1...
Starting figtest_web_1...
$ fig ps
Name Command State Ports
-------------------------------------------------------------------
figtest_redis_1 /usr/local/bin/run Up
figtest_web_1 /bin/sh -c python app.py Up 5000->5000/tcp
`fig run` allows you to run one-off commands for your services. For example, to see what environment variables are available to the `web` service:
$ fig run web env
See `fig --help` other commands that are available.
If you started Fig with `fig up -d`, you'll probably want to stop your services once you've finished with them:
$ fig stop
That's more-or-less how Fig works. See the reference section below for full details on the commands, configuration file and environment variables. If you have any thoughts or suggestions, [open an issue on GitHub](https://github.com/docker/fig).

27
docs/install.md Normal file
View file

@ -0,0 +1,27 @@
---
layout: default
title: Installing Fig
---
Installing Fig
==============
First, install Docker version 1.3 or greater.
If you're on OS X, you can use the [OS X installer](https://docs.docker.com/installation/mac/) to install both Docker and boot2docker. Once boot2docker is running, set the environment variables that'll configure Docker and Fig to talk to it:
$(boot2docker shellinit)
To persist the environment variables across shell sessions, you can add that line to your `~/.bashrc` file.
There are also guides for [Ubuntu](https://docs.docker.com/installation/ubuntulinux/) and [other platforms](https://docs.docker.com/installation/) in Dockers documentation.
Next, install Fig:
curl -L https://github.com/docker/fig/releases/download/1.0.1/fig-`uname -s`-`uname -m` > /usr/local/bin/fig; chmod +x /usr/local/bin/fig
Releases are available for OS X and 64-bit Linux. Fig is also available as a Python package if you're on another platform (or if you prefer that sort of thing):
$ sudo pip install -U fig
That should be all you need! Run `fig --version` to see if it worked.

98
docs/rails.md Normal file
View file

@ -0,0 +1,98 @@
---
layout: default
title: Getting started with Fig and Rails
---
Getting started with Fig and Rails
==================================
We're going to use Fig to set up and run a Rails/PostgreSQL app. Before starting, you'll need to have [Fig installed](install.html).
Let's set up the three files that'll get us started. First, our app is going to be running inside a Docker container which contains all of its dependencies. We can define what goes inside that Docker container using a file called `Dockerfile`. It'll contain this to start with:
FROM ruby
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev
RUN mkdir /myapp
WORKDIR /myapp
ADD Gemfile /myapp/Gemfile
RUN bundle install
ADD . /myapp
That'll put our application code inside an image with Ruby, Bundler and all our dependencies. For more information on how to write Dockerfiles, see the [Docker user guide](https://docs.docker.com/userguide/dockerimages/#building-an-image-from-a-dockerfile) and the [Dockerfile reference](http://docs.docker.com/reference/builder/).
Next, we have a bootstrap `Gemfile` which just loads Rails. It'll be overwritten in a moment by `rails new`.
source 'https://rubygems.org'
gem 'rails', '4.0.2'
Finally, `fig.yml` is where the magic happens. It describes what services our app comprises (a database and a web app), how to get each one's Docker image (the database just runs on a pre-made PostgreSQL image, and the web app is built from the current directory), and the configuration we need to link them together and expose the web app's port.
db:
image: postgres
ports:
- "5432"
web:
build: .
command: bundle exec rackup -p 3000
volumes:
- .:/myapp
ports:
- "3000:3000"
links:
- db
With those files in place, we can now generate the Rails skeleton app using `fig run`:
$ fig run web rails new . --force --database=postgresql --skip-bundle
First, Fig will build the image for the `web` service using the `Dockerfile`. Then it'll run `rails new` inside a new container, using that image. Once it's done, you should have a fresh app generated:
$ ls
Dockerfile app fig.yml tmp
Gemfile bin lib vendor
Gemfile.lock config log
README.rdoc config.ru public
Rakefile db test
Uncomment the line in your new `Gemfile` which loads `therubyracer`, so we've got a Javascript runtime:
gem 'therubyracer', platforms: :ruby
Now that we've got a new `Gemfile`, we need to build the image again. (This, and changes to the Dockerfile itself, should be the only times you'll need to rebuild).
$ fig build
The app is now bootable, but we're not quite there yet. By default, Rails expects a database to be running on `localhost` - we need to point it at the `db` container instead. We also need to change the database and username to align with the defaults set by the `postgres` image.
Open up your newly-generated `database.yml`. Replace its contents with the following:
development: &default
adapter: postgresql
encoding: unicode
database: postgres
pool: 5
username: postgres
password:
host: db
test:
<<: *default
database: myapp_test
We can now boot the app.
$ fig up
If all's well, you should see some PostgreSQL output, and then—after a few seconds—the familiar refrain:
myapp_web_1 | [2014-01-17 17:16:29] INFO WEBrick 1.3.1
myapp_web_1 | [2014-01-17 17:16:29] INFO ruby 2.0.0 (2013-11-22) [x86_64-linux-gnu]
myapp_web_1 | [2014-01-17 17:16:29] INFO WEBrick::HTTPServer#start: pid=1 port=3000
Finally, we just need to create the database. In another terminal, run:
$ fig run web rake db:create
And we're rolling—your app should now be running on port 3000 on your docker daemon (if you're using boot2docker, `boot2docker ip` will tell you its address).
![Screenshot of Rails' stock index.html](https://orchardup.com/static/images/fig-rails-screenshot.png)

91
docs/wordpress.md Normal file
View file

@ -0,0 +1,91 @@
---
layout: default
title: Getting started with Fig and Wordpress
---
Getting started with Fig and Wordpress
======================================
Fig makes it nice and easy to run Wordpress in an isolated environment. [Install Fig](install.html), then download Wordpress into the current directory:
$ curl https://wordpress.org/latest.tar.gz | tar -xvzf -
This will create a directory called `wordpress`, which you can rename to the name of your project if you wish. Inside that directory, we create `Dockerfile`, a file that defines what environment your app is going to run in:
```
FROM orchardup/php5
ADD . /code
```
This instructs Docker on how to build an image that contains PHP and Wordpress. For more information on how to write Dockerfiles, see the [Docker user guide](https://docs.docker.com/userguide/dockerimages/#building-an-image-from-a-dockerfile) and the [Dockerfile reference](http://docs.docker.com/reference/builder/).
Next up, `fig.yml` starts our web service and a separate MySQL instance:
```
web:
build: .
command: php -S 0.0.0.0:8000 -t /code
ports:
- "8000:8000"
links:
- db
volumes:
- .:/code
db:
image: orchardup/mysql
environment:
MYSQL_DATABASE: wordpress
```
Two supporting files are needed to get this working - first up, `wp-config.php` is the standard Wordpress config file with a single change to point the database configuration at the `db` container:
```
<?php
define('DB_NAME', 'wordpress');
define('DB_USER', 'root');
define('DB_PASSWORD', '');
define('DB_HOST', "db:3306");
define('DB_CHARSET', 'utf8');
define('DB_COLLATE', '');
define('AUTH_KEY', 'put your unique phrase here');
define('SECURE_AUTH_KEY', 'put your unique phrase here');
define('LOGGED_IN_KEY', 'put your unique phrase here');
define('NONCE_KEY', 'put your unique phrase here');
define('AUTH_SALT', 'put your unique phrase here');
define('SECURE_AUTH_SALT', 'put your unique phrase here');
define('LOGGED_IN_SALT', 'put your unique phrase here');
define('NONCE_SALT', 'put your unique phrase here');
$table_prefix = 'wp_';
define('WPLANG', '');
define('WP_DEBUG', false);
if ( !defined('ABSPATH') )
define('ABSPATH', dirname(__FILE__) . '/');
require_once(ABSPATH . 'wp-settings.php');
```
Finally, `router.php` tells PHP's built-in web server how to run Wordpress:
```
<?php
$root = $_SERVER['DOCUMENT_ROOT'];
chdir($root);
$path = '/'.ltrim(parse_url($_SERVER['REQUEST_URI'])['path'],'/');
set_include_path(get_include_path().':'.__DIR__);
if(file_exists($root.$path))
{
if(is_dir($root.$path) && substr($path,strlen($path) - 1, 1) !== '/')
$path = rtrim($path,'/').'/index.php';
if(strpos($path,'.php') === false) return false;
else {
chdir(dirname($root.$path));
require_once $root.$path;
}
}else include_once 'index.php';
```
With those four files in place, run `fig up` inside your Wordpress directory and it'll pull and build the images we need, and then start the web and database containers. You'll then be able to visit Wordpress at port 8000 on your docker daemon (if you're using boot2docker, `boot2docker ip` will tell you its address).

159
docs/yml.md Normal file
View file

@ -0,0 +1,159 @@
---
layout: default
title: fig.yml reference
---
fig.yml reference
=================
Each service defined in `fig.yml` must specify exactly one of `image` or `build`. Other keys are optional, and are analogous to their `docker run` command-line counterparts.
As with `docker run`, options specified in the Dockerfile (e.g. `CMD`, `EXPOSE`, `VOLUME`, `ENV`) are respected by default - you don't need to specify them again in `fig.yml`.
###image
Tag or partial image ID. Can be local or remote - Fig will attempt to pull if it doesn't exist locally.
```
image: ubuntu
image: orchardup/postgresql
image: a4bc65fd
```
### build
Path to a directory containing a Dockerfile. Fig will build and tag it with a generated name, and use that image thereafter.
```
build: /path/to/build/dir
```
### command
Override the default command.
```
command: bundle exec thin -p 3000
```
<a name="links"></a>
### links
Link to containers in another service. Either specify both the service name and the link alias (`SERVICE:ALIAS`), or just the service name (which will also be used for the alias).
```
links:
- db
- db:database
- redis
```
An entry with the alias' name will be created in `/etc/hosts` inside containers for this service, e.g:
```
172.17.2.186 db
172.17.2.186 database
172.17.2.187 redis
```
Environment variables will also be created - see the [environment variable reference](env.html) for details.
### ports
Expose ports. Either specify both ports (`HOST:CONTAINER`), or just the container port (a random host port will be chosen).
**Note:** When mapping ports in the `HOST:CONTAINER` format, you may experience erroneous results when using a container port lower than 60, because YAML will parse numbers in the format `xx:yy` as sexagesimal (base 60). For this reason, we recommend always explicitly specifying your port mappings as strings.
```
ports:
- "3000"
- "8000:8000"
- "49100:22"
- "127.0.0.1:8001:8001"
```
### expose
Expose ports without publishing them to the host machine - they'll only be accessible to linked services. Only the internal port can be specified.
```
expose:
- "3000"
- "8000"
```
### volumes
Mount paths as volumes, optionally specifying a path on the host machine
(`HOST:CONTAINER`), or an access mode (`HOST:CONTAINER:ro`).
```
volumes:
- /var/lib/mysql
- cache/:/tmp/cache
- ~/configs:/etc/configs/:ro
```
### volumes_from
Mount all of the volumes from another service or container.
```
volumes_from:
- service_name
- container_name
```
### environment
Add environment variables. You can use either an array or a dictionary.
Environment variables with only a key are resolved to their values on the machine Fig is running on, which can be helpful for secret or host-specific values.
```
environment:
RACK_ENV: development
SESSION_SECRET:
environment:
- RACK_ENV=development
- SESSION_SECRET
```
### net
Networking mode. Use the same values as the docker client `--net` parameter.
```
net: "bridge"
net: "none"
net: "container:[name or id]"
net: "host"
```
### dns
Custom DNS servers. Can be a single value or a list.
```
dns: 8.8.8.8
dns:
- 8.8.8.8
- 9.9.9.9
```
### working\_dir, entrypoint, user, hostname, domainname, mem\_limit, privileged
Each of these is a single value, analogous to its [docker run](https://docs.docker.com/reference/run/) counterpart.
```
working_dir: /code
entrypoint: /code/entrypoint.sh
user: postgresql
hostname: foo
domainname: foo.com
mem_limit: 1000000000
privileged: true
```

View file

@ -1,5 +0,0 @@
# Experimental: Compose, Swarm and Multi-Host Networking
Compose now supports multi-host networking as standard. Read more here:
https://docs.docker.com/compose/networking

4
fig/__init__.py Normal file
View file

@ -0,0 +1,4 @@
from __future__ import unicode_literals
from .service import Service # noqa:flake8
__version__ = '1.0.1'

View file

@ -1,8 +1,4 @@
from __future__ import absolute_import
from __future__ import unicode_literals from __future__ import unicode_literals
import colorama
NAMES = [ NAMES = [
'grey', 'grey',
'red', 'red',
@ -33,7 +29,6 @@ def make_color_fn(code):
return lambda s: ansi_color(code, s) return lambda s: ansi_color(code, s)
colorama.init(strip=False)
for (name, code) in get_pairs(): for (name, code) in get_pairs():
globals()[name] = make_color_fn(code) globals()[name] = make_color_fn(code)

114
fig/cli/command.py Normal file
View file

@ -0,0 +1,114 @@
from __future__ import unicode_literals
from __future__ import absolute_import
from requests.exceptions import ConnectionError, SSLError
import errno
import logging
import os
import re
import yaml
import six
from ..project import Project
from ..service import ConfigError
from .docopt_command import DocoptCommand
from .utils import call_silently, is_mac, is_ubuntu
from .docker_client import docker_client
from . import verbose_proxy
from . import errors
from .. import __version__
log = logging.getLogger(__name__)
class Command(DocoptCommand):
base_dir = '.'
def dispatch(self, *args, **kwargs):
try:
super(Command, self).dispatch(*args, **kwargs)
except SSLError, e:
raise errors.UserError('SSL error: %s' % e)
except ConnectionError:
if call_silently(['which', 'docker']) != 0:
if is_mac():
raise errors.DockerNotFoundMac()
elif is_ubuntu():
raise errors.DockerNotFoundUbuntu()
else:
raise errors.DockerNotFoundGeneric()
elif call_silently(['which', 'boot2docker']) == 0:
raise errors.ConnectionErrorBoot2Docker()
else:
raise errors.ConnectionErrorGeneric(self.get_client().base_url)
def perform_command(self, options, handler, command_options):
if options['COMMAND'] == 'help':
# Skip looking up the figfile.
handler(None, command_options)
return
explicit_config_path = options.get('--file') or os.environ.get('FIG_FILE')
project = self.get_project(
self.get_config_path(explicit_config_path),
project_name=options.get('--project-name'),
verbose=options.get('--verbose'))
handler(project, command_options)
def get_client(self, verbose=False):
client = docker_client()
if verbose:
version_info = six.iteritems(client.version())
log.info("Fig version %s", __version__)
log.info("Docker base_url: %s", client.base_url)
log.info("Docker version: %s",
", ".join("%s=%s" % item for item in version_info))
return verbose_proxy.VerboseProxy('docker', client)
return client
def get_config(self, config_path):
try:
with open(config_path, 'r') as fh:
return yaml.safe_load(fh)
except IOError as e:
if e.errno == errno.ENOENT:
raise errors.FigFileNotFound(os.path.basename(e.filename))
raise errors.UserError(six.text_type(e))
def get_project(self, config_path, project_name=None, verbose=False):
try:
return Project.from_config(
self.get_project_name(config_path, project_name),
self.get_config(config_path),
self.get_client(verbose=verbose))
except ConfigError as e:
raise errors.UserError(six.text_type(e))
def get_project_name(self, config_path, project_name=None):
def normalize_name(name):
return re.sub(r'[^a-zA-Z0-9]', '', name)
project_name = project_name or os.environ.get('FIG_PROJECT_NAME')
if project_name is not None:
return normalize_name(project_name)
project = os.path.basename(os.path.dirname(os.path.abspath(config_path)))
if project:
return normalize_name(project)
return 'default'
def get_config_path(self, file_path=None):
if file_path:
return os.path.join(self.base_dir, file_path)
if os.path.exists(os.path.join(self.base_dir, 'fig.yaml')):
log.warning("Fig just read the file 'fig.yaml' on startup, rather "
"than 'fig.yml'")
log.warning("Please be aware that fig.yml the expected extension "
"in most cases, and using .yaml can cause compatibility "
"issues in future")
return os.path.join(self.base_dir, 'fig.yaml')
return os.path.join(self.base_dir, 'fig.yml')

34
fig/cli/docker_client.py Normal file
View file

@ -0,0 +1,34 @@
from docker import Client
from docker import tls
import ssl
import os
def docker_client():
"""
Returns a docker-py client configured using environment variables
according to the same logic as the official Docker client.
"""
cert_path = os.environ.get('DOCKER_CERT_PATH', '')
if cert_path == '':
cert_path = os.path.join(os.environ.get('HOME', ''), '.docker')
base_url = os.environ.get('DOCKER_HOST')
tls_config = None
if os.environ.get('DOCKER_TLS_VERIFY', '') != '':
parts = base_url.split('://', 1)
base_url = '%s://%s' % ('https', parts[1])
client_cert = (os.path.join(cert_path, 'cert.pem'), os.path.join(cert_path, 'key.pem'))
ca_cert = os.path.join(cert_path, 'ca.pem')
tls_config = tls.TLSConfig(
ssl_version=ssl.PROTOCOL_TLSv1,
verify=True,
assert_hostname=False,
client_cert=client_cert,
ca_cert=ca_cert,
)
return Client(base_url=base_url, tls=tls_config)

54
fig/cli/docopt_command.py Normal file
View file

@ -0,0 +1,54 @@
from __future__ import unicode_literals
from __future__ import absolute_import
import sys
from inspect import getdoc
from docopt import docopt, DocoptExit
def docopt_full_help(docstring, *args, **kwargs):
try:
return docopt(docstring, *args, **kwargs)
except DocoptExit:
raise SystemExit(docstring)
class DocoptCommand(object):
def docopt_options(self):
return {'options_first': True}
def sys_dispatch(self):
self.dispatch(sys.argv[1:], None)
def dispatch(self, argv, global_options):
self.perform_command(*self.parse(argv, global_options))
def perform_command(self, options, handler, command_options):
handler(command_options)
def parse(self, argv, global_options):
options = docopt_full_help(getdoc(self), argv, **self.docopt_options())
command = options['COMMAND']
if command is None:
raise SystemExit(getdoc(self))
if not hasattr(self, command):
raise NoSuchCommand(command, self)
handler = getattr(self, command)
docstring = getdoc(handler)
if docstring is None:
raise NoSuchCommand(command, self)
command_options = docopt_full_help(docstring, options['ARGS'], options_first=True)
return options, handler, command_options
class NoSuchCommand(Exception):
def __init__(self, command, supercommand):
super(NoSuchCommand, self).__init__("No such command: %s" % command)
self.command = command
self.supercommand = supercommand

62
fig/cli/errors.py Normal file
View file

@ -0,0 +1,62 @@
from __future__ import absolute_import
from textwrap import dedent
class UserError(Exception):
def __init__(self, msg):
self.msg = dedent(msg).strip()
def __unicode__(self):
return self.msg
__str__ = __unicode__
class DockerNotFoundMac(UserError):
def __init__(self):
super(DockerNotFoundMac, self).__init__("""
Couldn't connect to Docker daemon. You might need to install docker-osx:
https://github.com/noplay/docker-osx
""")
class DockerNotFoundUbuntu(UserError):
def __init__(self):
super(DockerNotFoundUbuntu, self).__init__("""
Couldn't connect to Docker daemon. You might need to install Docker:
http://docs.docker.io/en/latest/installation/ubuntulinux/
""")
class DockerNotFoundGeneric(UserError):
def __init__(self):
super(DockerNotFoundGeneric, self).__init__("""
Couldn't connect to Docker daemon. You might need to install Docker:
http://docs.docker.io/en/latest/installation/
""")
class ConnectionErrorBoot2Docker(UserError):
def __init__(self):
super(ConnectionErrorBoot2Docker, self).__init__("""
Couldn't connect to Docker daemon - you might need to run `boot2docker up`.
""")
class ConnectionErrorGeneric(UserError):
def __init__(self, url):
super(ConnectionErrorGeneric, self).__init__("""
Couldn't connect to Docker daemon at %s - is it running?
If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
""" % url)
class FigFileNotFound(UserError):
def __init__(self, filename):
super(FigFileNotFound, self).__init__("""
Can't find %s. Are you in the right directory?
""" % filename)

23
fig/cli/formatter.py Normal file
View file

@ -0,0 +1,23 @@
from __future__ import unicode_literals
from __future__ import absolute_import
import os
import texttable
def get_tty_width():
tty_size = os.popen('stty size', 'r').read().split()
if len(tty_size) != 2:
return 80
_, width = tty_size
return int(width)
class Formatter(object):
def table(self, headers, rows):
table = texttable.Texttable(max_width=get_tty_width())
table.set_cols_dtype(['t' for h in headers])
table.add_rows([headers] + rows)
table.set_deco(table.HEADER)
table.set_chars(['-', '|', '+', '-'])
return table.draw()

79
fig/cli/log_printer.py Normal file
View file

@ -0,0 +1,79 @@
from __future__ import unicode_literals
from __future__ import absolute_import
import sys
from itertools import cycle
from .multiplexer import Multiplexer, STOP
from . import colors
from .utils import split_buffer
class LogPrinter(object):
def __init__(self, containers, attach_params=None, output=sys.stdout, monochrome=False):
self.containers = containers
self.attach_params = attach_params or {}
self.prefix_width = self._calculate_prefix_width(containers)
self.generators = self._make_log_generators(monochrome)
self.output = output
def run(self):
mux = Multiplexer(self.generators)
for line in mux.loop():
self.output.write(line)
def _calculate_prefix_width(self, containers):
"""
Calculate the maximum width of container names so we can make the log
prefixes line up like so:
db_1 | Listening
web_1 | Listening
"""
prefix_width = 0
for container in containers:
prefix_width = max(prefix_width, len(container.name_without_project))
return prefix_width
def _make_log_generators(self, monochrome):
color_fns = cycle(colors.rainbow())
generators = []
for container in self.containers:
if monochrome:
color_fn = lambda s: s
else:
color_fn = color_fns.next()
generators.append(self._make_log_generator(container, color_fn))
return generators
def _make_log_generator(self, container, color_fn):
prefix = color_fn(self._generate_prefix(container)).encode('utf-8')
# Attach to container before log printer starts running
line_generator = split_buffer(self._attach(container), '\n')
for line in line_generator:
yield prefix + line
exit_code = container.wait()
yield color_fn("%s exited with code %s\n" % (container.name, exit_code))
yield STOP
def _generate_prefix(self, container):
"""
Generate the prefix for a log line without colour
"""
name = container.name_without_project
padding = ' ' * (self.prefix_width - len(name))
return ''.join([name, padding, ' | '])
def _attach(self, container):
params = {
'stdout': True,
'stderr': True,
'stream': True,
}
params.update(self.attach_params)
params = dict((name, 1 if value else 0) for (name, value) in list(params.items()))
return container.attach(**params)

449
fig/cli/main.py Normal file
View file

@ -0,0 +1,449 @@
from __future__ import print_function
from __future__ import unicode_literals
import logging
import sys
import re
import signal
from operator import attrgetter
from inspect import getdoc
import dockerpty
from .. import __version__
from ..project import NoSuchService, ConfigurationError
from ..service import BuildError, CannotBeScaledError
from .command import Command
from .formatter import Formatter
from .log_printer import LogPrinter
from .utils import yesno
from docker.errors import APIError
from .errors import UserError
from .docopt_command import NoSuchCommand
log = logging.getLogger(__name__)
def main():
setup_logging()
try:
command = TopLevelCommand()
command.sys_dispatch()
except KeyboardInterrupt:
log.error("\nAborting.")
sys.exit(1)
except (UserError, NoSuchService, ConfigurationError) as e:
log.error(e.msg)
sys.exit(1)
except NoSuchCommand as e:
log.error("No such command: %s", e.command)
log.error("")
log.error("\n".join(parse_doc_section("commands:", getdoc(e.supercommand))))
sys.exit(1)
except APIError as e:
log.error(e.explanation)
sys.exit(1)
except BuildError as e:
log.error("Service '%s' failed to build: %s" % (e.service.name, e.reason))
sys.exit(1)
def setup_logging():
console_handler = logging.StreamHandler(sys.stderr)
console_handler.setFormatter(logging.Formatter())
console_handler.setLevel(logging.INFO)
root_logger = logging.getLogger()
root_logger.addHandler(console_handler)
root_logger.setLevel(logging.DEBUG)
# Disable requests logging
logging.getLogger("requests").propagate = False
# stolen from docopt master
def parse_doc_section(name, source):
pattern = re.compile('^([^\n]*' + name + '[^\n]*\n?(?:[ \t].*?(?:\n|$))*)',
re.IGNORECASE | re.MULTILINE)
return [s.strip() for s in pattern.findall(source)]
class TopLevelCommand(Command):
"""Fast, isolated development environments using Docker.
Usage:
fig [options] [COMMAND] [ARGS...]
fig -h|--help
Options:
--verbose Show more output
--version Print version and exit
-f, --file FILE Specify an alternate fig file (default: fig.yml)
-p, --project-name NAME Specify an alternate project name (default: directory name)
Commands:
build Build or rebuild services
help Get help on a command
kill Kill containers
logs View output from containers
port Print the public port for a port binding
ps List containers
pull Pulls service images
rm Remove stopped containers
run Run a one-off command
scale Set number of containers for a service
start Start services
stop Stop services
restart Restart services
up Create and start containers
"""
def docopt_options(self):
options = super(TopLevelCommand, self).docopt_options()
options['version'] = "fig %s" % __version__
return options
def build(self, project, options):
"""
Build or rebuild services.
Services are built once and then tagged as `project_service`,
e.g. `figtest_db`. If you change a service's `Dockerfile` or the
contents of its build directory, you can run `fig build` to rebuild it.
Usage: build [options] [SERVICE...]
Options:
--no-cache Do not use cache when building the image.
"""
no_cache = bool(options.get('--no-cache', False))
project.build(service_names=options['SERVICE'], no_cache=no_cache)
def help(self, project, options):
"""
Get help on a command.
Usage: help COMMAND
"""
command = options['COMMAND']
if not hasattr(self, command):
raise NoSuchCommand(command, self)
raise SystemExit(getdoc(getattr(self, command)))
def kill(self, project, options):
"""
Force stop service containers.
Usage: kill [SERVICE...]
"""
project.kill(service_names=options['SERVICE'])
def logs(self, project, options):
"""
View output from containers.
Usage: logs [options] [SERVICE...]
Options:
--no-color Produce monochrome output.
"""
containers = project.containers(service_names=options['SERVICE'], stopped=True)
monochrome = options['--no-color']
print("Attaching to", list_containers(containers))
LogPrinter(containers, attach_params={'logs': True}, monochrome=monochrome).run()
def port(self, project, options):
"""
Print the public port for a port binding.
Usage: port [options] SERVICE PRIVATE_PORT
Options:
--protocol=proto tcp or udp (defaults to tcp)
--index=index index of the container if there are multiple
instances of a service (defaults to 1)
"""
service = project.get_service(options['SERVICE'])
try:
container = service.get_container(number=options.get('--index') or 1)
except ValueError as e:
raise UserError(str(e))
print(container.get_local_port(
options['PRIVATE_PORT'],
protocol=options.get('--protocol') or 'tcp') or '')
def ps(self, project, options):
"""
List containers.
Usage: ps [options] [SERVICE...]
Options:
-q Only display IDs
"""
containers = sorted(
project.containers(service_names=options['SERVICE'], stopped=True) +
project.containers(service_names=options['SERVICE'], one_off=True),
key=attrgetter('name'))
if options['-q']:
for container in containers:
print(container.id)
else:
headers = [
'Name',
'Command',
'State',
'Ports',
]
rows = []
for container in containers:
command = container.human_readable_command
if len(command) > 30:
command = '%s ...' % command[:26]
rows.append([
container.name,
command,
container.human_readable_state,
container.human_readable_ports,
])
print(Formatter().table(headers, rows))
def pull(self, project, options):
"""
Pulls images for services.
Usage: pull [options] [SERVICE...]
Options:
--allow-insecure-ssl Allow insecure connections to the docker
registry
"""
insecure_registry = options['--allow-insecure-ssl']
project.pull(
service_names=options['SERVICE'],
insecure_registry=insecure_registry
)
def rm(self, project, options):
"""
Remove stopped service containers.
Usage: rm [options] [SERVICE...]
Options:
--force Don't ask to confirm removal
-v Remove volumes associated with containers
"""
all_containers = project.containers(service_names=options['SERVICE'], stopped=True)
stopped_containers = [c for c in all_containers if not c.is_running]
if len(stopped_containers) > 0:
print("Going to remove", list_containers(stopped_containers))
if options.get('--force') \
or yesno("Are you sure? [yN] ", default=False):
project.remove_stopped(
service_names=options['SERVICE'],
v=options.get('-v', False)
)
else:
print("No stopped containers")
def run(self, project, options):
"""
Run a one-off command on a service.
For example:
$ fig run web python manage.py shell
By default, linked services will be started, unless they are already
running. If you do not want to start linked services, use
`fig run --no-deps SERVICE COMMAND [ARGS...]`.
Usage: run [options] [-e KEY=VAL...] SERVICE [COMMAND] [ARGS...]
Options:
--allow-insecure-ssl Allow insecure connections to the docker
registry
-d Detached mode: Run container in the background, print
new container name.
--entrypoint CMD Override the entrypoint of the image.
-e KEY=VAL Set an environment variable (can be used multiple times)
--no-deps Don't start linked services.
--rm Remove container after run. Ignored in detached mode.
-T Disable pseudo-tty allocation. By default `fig run`
allocates a TTY.
"""
service = project.get_service(options['SERVICE'])
insecure_registry = options['--allow-insecure-ssl']
if not options['--no-deps']:
deps = service.get_linked_names()
if len(deps) > 0:
project.up(
service_names=deps,
start_links=True,
recreate=False,
)
tty = True
if options['-d'] or options['-T'] or not sys.stdin.isatty():
tty = False
if options['COMMAND']:
command = [options['COMMAND']] + options['ARGS']
else:
command = service.options.get('command')
container_options = {
'command': command,
'tty': tty,
'stdin_open': not options['-d'],
}
if options['-e']:
for option in options['-e']:
if 'environment' not in service.options:
service.options['environment'] = {}
k, v = option.split('=', 1)
service.options['environment'][k] = v
if options['--entrypoint']:
container_options['entrypoint'] = options.get('--entrypoint')
container = service.create_container(
one_off=True,
insecure_registry=insecure_registry,
**container_options
)
if options['-d']:
service.start_container(container, ports=None, one_off=True)
print(container.name)
else:
service.start_container(container, ports=None, one_off=True)
dockerpty.start(project.client, container.id, interactive=not options['-T'])
exit_code = container.wait()
if options['--rm']:
log.info("Removing %s..." % container.name)
project.client.remove_container(container.id)
sys.exit(exit_code)
def scale(self, project, options):
"""
Set number of containers to run for a service.
Numbers are specified in the form `service=num` as arguments.
For example:
$ fig scale web=2 worker=3
Usage: scale [SERVICE=NUM...]
"""
for s in options['SERVICE=NUM']:
if '=' not in s:
raise UserError('Arguments to scale should be in the form service=num')
service_name, num = s.split('=', 1)
try:
num = int(num)
except ValueError:
raise UserError('Number of containers for service "%s" is not a '
'number' % service_name)
try:
project.get_service(service_name).scale(num)
except CannotBeScaledError:
raise UserError(
'Service "%s" cannot be scaled because it specifies a port '
'on the host. If multiple containers for this service were '
'created, the port would clash.\n\nRemove the ":" from the '
'port definition in fig.yml so Docker can choose a random '
'port for each container.' % service_name)
def start(self, project, options):
"""
Start existing containers.
Usage: start [SERVICE...]
"""
project.start(service_names=options['SERVICE'])
def stop(self, project, options):
"""
Stop running containers without removing them.
They can be started again with `fig start`.
Usage: stop [SERVICE...]
"""
project.stop(service_names=options['SERVICE'])
def restart(self, project, options):
"""
Restart running containers.
Usage: restart [SERVICE...]
"""
project.restart(service_names=options['SERVICE'])
def up(self, project, options):
"""
Build, (re)create, start and attach to containers for a service.
By default, `fig up` will aggregate the output of each container, and
when it exits, all containers will be stopped. If you run `fig up -d`,
it'll start the containers in the background and leave them running.
If there are existing containers for a service, `fig up` will stop
and recreate them (preserving mounted volumes with volumes-from),
so that changes in `fig.yml` are picked up. If you do not want existing
containers to be recreated, `fig up --no-recreate` will re-use existing
containers.
Usage: up [options] [SERVICE...]
Options:
--allow-insecure-ssl Allow insecure connections to the docker
registry
-d Detached mode: Run containers in the background,
print new container names.
--no-color Produce monochrome output.
--no-deps Don't start linked services.
--no-recreate If containers already exist, don't recreate them.
"""
insecure_registry = options['--allow-insecure-ssl']
detached = options['-d']
monochrome = options['--no-color']
start_links = not options['--no-deps']
recreate = not options['--no-recreate']
service_names = options['SERVICE']
project.up(
service_names=service_names,
start_links=start_links,
recreate=recreate,
insecure_registry=insecure_registry,
)
to_attach = [c for s in project.get_services(service_names) for c in s.containers()]
if not detached:
print("Attaching to", list_containers(to_attach))
log_printer = LogPrinter(to_attach, attach_params={"logs": True}, monochrome=monochrome)
try:
log_printer.run()
finally:
def handler(signal, frame):
project.kill(service_names=service_names)
sys.exit(0)
signal.signal(signal.SIGINT, handler)
print("Gracefully stopping... (press Ctrl+C again to force)")
project.stop(service_names=service_names)
def list_containers(containers):
return ", ".join(c.name for c in containers)

42
fig/cli/multiplexer.py Normal file
View file

@ -0,0 +1,42 @@
from __future__ import absolute_import
from threading import Thread
try:
from Queue import Queue, Empty
except ImportError:
from queue import Queue, Empty # Python 3.x
# Yield STOP from an input generator to stop the
# top-level loop without processing any more input.
STOP = object()
class Multiplexer(object):
def __init__(self, generators):
self.generators = generators
self.queue = Queue()
def loop(self):
self._init_readers()
while True:
try:
item = self.queue.get(timeout=0.1)
if item is STOP:
break
else:
yield item
except Empty:
pass
def _init_readers(self):
for generator in self.generators:
t = Thread(target=_enqueue_output, args=(generator, self.queue))
t.daemon = True
t.start()
def _enqueue_output(generator, queue):
for item in generator:
queue.put(item)

103
fig/cli/utils.py Normal file
View file

@ -0,0 +1,103 @@
from __future__ import unicode_literals
from __future__ import absolute_import
from __future__ import division
import datetime
import os
import subprocess
import platform
def yesno(prompt, default=None):
"""
Prompt the user for a yes or no.
Can optionally specify a default value, which will only be
used if they enter a blank line.
Unrecognised input (anything other than "y", "n", "yes",
"no" or "") will return None.
"""
answer = raw_input(prompt).strip().lower()
if answer == "y" or answer == "yes":
return True
elif answer == "n" or answer == "no":
return False
elif answer == "":
return default
else:
return None
# http://stackoverflow.com/a/5164027
def prettydate(d):
diff = datetime.datetime.utcnow() - d
s = diff.seconds
if diff.days > 7 or diff.days < 0:
return d.strftime('%d %b %y')
elif diff.days == 1:
return '1 day ago'
elif diff.days > 1:
return '{0} days ago'.format(diff.days)
elif s <= 1:
return 'just now'
elif s < 60:
return '{0} seconds ago'.format(s)
elif s < 120:
return '1 minute ago'
elif s < 3600:
return '{0} minutes ago'.format(s / 60)
elif s < 7200:
return '1 hour ago'
else:
return '{0} hours ago'.format(s / 3600)
def mkdir(path, permissions=0o700):
if not os.path.exists(path):
os.mkdir(path)
os.chmod(path, permissions)
return path
def split_buffer(reader, separator):
"""
Given a generator which yields strings and a separator string,
joins all input, splits on the separator and yields each chunk.
Unlike string.split(), each chunk includes the trailing
separator, except for the last one if none was found on the end
of the input.
"""
buffered = str('')
separator = str(separator)
for data in reader:
buffered += data
while True:
index = buffered.find(separator)
if index == -1:
break
yield buffered[:index + 1]
buffered = buffered[index + 1:]
if len(buffered) > 0:
yield buffered
def call_silently(*args, **kwargs):
"""
Like subprocess.call(), but redirects stdout and stderr to /dev/null.
"""
with open(os.devnull, 'w') as shutup:
return subprocess.call(*args, stdout=shutup, stderr=shutup, **kwargs)
def is_mac():
return platform.system() == 'Darwin'
def is_ubuntu():
return platform.system() == 'Linux' and platform.linux_distribution()[0] == 'Ubuntu'

View file

@ -1,10 +1,8 @@
from __future__ import absolute_import
from __future__ import unicode_literals
import functools import functools
from itertools import chain
import logging import logging
import pprint import pprint
from itertools import chain
import six import six

View file

@ -1,14 +1,8 @@
from __future__ import absolute_import
from __future__ import unicode_literals from __future__ import unicode_literals
from __future__ import absolute_import
from functools import reduce
import six import six
from .const import LABEL_CONTAINER_NUMBER
from .const import LABEL_PROJECT
from .const import LABEL_SERVICE
class Container(object): class Container(object):
""" """
@ -19,27 +13,24 @@ class Container(object):
self.client = client self.client = client
self.dictionary = dictionary self.dictionary = dictionary
self.has_been_inspected = has_been_inspected self.has_been_inspected = has_been_inspected
self.log_stream = None
@classmethod @classmethod
def from_ps(cls, client, dictionary, **kwargs): def from_ps(cls, client, dictionary, **kwargs):
""" """
Construct a container object from the output of GET /containers/json. Construct a container object from the output of GET /containers/json.
""" """
name = get_container_name(dictionary)
if name is None:
return None
new_dictionary = { new_dictionary = {
'Id': dictionary['Id'], 'Id': dictionary['Id'],
'Image': dictionary['Image'], 'Image': dictionary['Image'],
'Name': '/' + name,
} }
for name in dictionary.get('Names', []):
if len(name.split('/')) == 2:
new_dictionary['Name'] = name
return cls(client, new_dictionary, **kwargs) return cls(client, new_dictionary, **kwargs)
@classmethod @classmethod
def from_id(cls, client, id): def from_id(cls, client, id):
return cls(client, client.inspect_container(id), has_been_inspected=True) return cls(client, client.inspect_container(id))
@classmethod @classmethod
def create(cls, client, **options): def create(cls, client, **options):
@ -54,38 +45,24 @@ class Container(object):
def image(self): def image(self):
return self.dictionary['Image'] return self.dictionary['Image']
@property
def image_config(self):
return self.client.inspect_image(self.image)
@property @property
def short_id(self): def short_id(self):
return self.id[:12] return self.id[:10]
@property @property
def name(self): def name(self):
return self.dictionary['Name'][1:] return self.dictionary['Name'][1:]
@property
def service(self):
return self.labels.get(LABEL_SERVICE)
@property @property
def name_without_project(self): def name_without_project(self):
project = self.labels.get(LABEL_PROJECT) return '_'.join(self.dictionary['Name'].split('_')[1:])
if self.name.startswith('{0}_{1}'.format(project, self.service)):
return '{0}_{1}'.format(self.service, self.number)
else:
return self.name
@property @property
def number(self): def number(self):
number = self.labels.get(LABEL_CONTAINER_NUMBER) try:
if not number: return int(self.name.split('_')[-1])
raise ValueError("Container {0} does not have a {1} label".format( except ValueError:
self.short_id, LABEL_CONTAINER_NUMBER)) return None
return int(number)
@property @property
def ports(self): def ports(self):
@ -103,24 +80,8 @@ class Container(object):
return ', '.join(format_port(*item) return ', '.join(format_port(*item)
for item in sorted(six.iteritems(self.ports))) for item in sorted(six.iteritems(self.ports)))
@property
def labels(self):
return self.get('Config.Labels') or {}
@property
def stop_signal(self):
return self.get('Config.StopSignal')
@property
def log_config(self):
return self.get('HostConfig.LogConfig') or None
@property @property
def human_readable_state(self): def human_readable_state(self):
if self.is_paused:
return 'Paused'
if self.is_restarting:
return 'Restarting'
if self.is_running: if self.is_running:
return 'Ghost' if self.get('State.Ghost') else 'Up' return 'Ghost' if self.get('State.Ghost') else 'Up'
else: else:
@ -134,44 +95,12 @@ class Container(object):
@property @property
def environment(self): def environment(self):
def parse_env(var): return dict(var.split("=", 1) for var in self.get('Config.Env') or [])
if '=' in var:
return var.split("=", 1)
return var, None
return dict(parse_env(var) for var in self.get('Config.Env') or [])
@property
def exit_code(self):
return self.get('State.ExitCode')
@property @property
def is_running(self): def is_running(self):
return self.get('State.Running') return self.get('State.Running')
@property
def is_restarting(self):
return self.get('State.Restarting')
@property
def is_paused(self):
return self.get('State.Paused')
@property
def log_driver(self):
return self.get('HostConfig.LogConfig.Type')
@property
def has_api_logs(self):
log_type = self.log_driver
return not log_type or log_type in ('json-file', 'journald')
def attach_log_stream(self):
"""A log stream can only be attached if the container uses a json-file
log driver.
"""
if self.has_api_logs:
self.log_stream = self.attach(stdout=True, stderr=True, stream=True)
def get(self, key): def get(self, key):
"""Return a value from the container or None if the value is not set. """Return a value from the container or None if the value is not set.
@ -189,48 +118,21 @@ class Container(object):
port = self.ports.get("%s/%s" % (port, protocol)) port = self.ports.get("%s/%s" % (port, protocol))
return "{HostIp}:{HostPort}".format(**port[0]) if port else None return "{HostIp}:{HostPort}".format(**port[0]) if port else None
def get_mount(self, mount_dest):
for mount in self.get('Mounts'):
if mount['Destination'] == mount_dest:
return mount
return None
def start(self, **options): def start(self, **options):
return self.client.start(self.id, **options) return self.client.start(self.id, **options)
def stop(self, **options): def stop(self, **options):
return self.client.stop(self.id, **options) return self.client.stop(self.id, **options)
def pause(self, **options): def kill(self):
return self.client.pause(self.id, **options) return self.client.kill(self.id)
def unpause(self, **options): def restart(self):
return self.client.unpause(self.id, **options) return self.client.restart(self.id)
def kill(self, **options):
return self.client.kill(self.id, **options)
def restart(self, **options):
return self.client.restart(self.id, **options)
def remove(self, **options): def remove(self, **options):
return self.client.remove_container(self.id, **options) return self.client.remove_container(self.id, **options)
def create_exec(self, command, **options):
return self.client.exec_create(self.id, command, **options)
def start_exec(self, exec_id, **options):
return self.client.exec_start(exec_id, **options)
def rename_to_tmp_name(self):
"""Rename the container to a hopefully unique temporary container name
by prepending the short id.
"""
self.client.rename(
self.id,
'%s_%s' % (self.short_id, self.name)
)
def inspect_if_not_inspected(self): def inspect_if_not_inspected(self):
if not self.has_been_inspected: if not self.has_been_inspected:
self.inspect() self.inspect()
@ -246,27 +148,25 @@ class Container(object):
self.has_been_inspected = True self.has_been_inspected = True
return self.dictionary return self.dictionary
def links(self):
links = []
for container in self.client.containers():
for name in container['Names']:
bits = name.split('/')
if len(bits) > 2 and bits[1] == self.name:
links.append(bits[2])
return links
def attach(self, *args, **kwargs): def attach(self, *args, **kwargs):
return self.client.attach(self.id, *args, **kwargs) return self.client.attach(self.id, *args, **kwargs)
def attach_socket(self, **kwargs):
return self.client.attach_socket(self.id, **kwargs)
def __repr__(self): def __repr__(self):
return '<Container: %s (%s)>' % (self.name, self.id[:6]) return '<Container: %s>' % self.name
def __eq__(self, other): def __eq__(self, other):
if type(self) != type(other): if type(self) != type(other):
return False return False
return self.id == other.id return self.id == other.id
def __hash__(self):
return self.id.__hash__()
def get_container_name(container):
if not container.get('Name') and not container.get('Names'):
return None
# inspect
if 'Name' in container:
return container['Name']
# ps
shortest_name = min(container['Names'], key=lambda n: len(n.split('/')))
return shortest_name.split('/')[-1]

85
fig/progress_stream.py Normal file
View file

@ -0,0 +1,85 @@
import json
import os
import codecs
class StreamOutputError(Exception):
pass
def stream_output(output, stream):
is_terminal = hasattr(stream, 'fileno') and os.isatty(stream.fileno())
stream = codecs.getwriter('utf-8')(stream)
all_events = []
lines = {}
diff = 0
for chunk in output:
event = json.loads(chunk)
all_events.append(event)
if 'progress' in event or 'progressDetail' in event:
image_id = event.get('id')
if not image_id:
continue
if image_id in lines:
diff = len(lines) - lines[image_id]
else:
lines[image_id] = len(lines)
stream.write("\n")
diff = 0
if is_terminal:
# move cursor up `diff` rows
stream.write("%c[%dA" % (27, diff))
print_output_event(event, stream, is_terminal)
if 'id' in event and is_terminal:
# move cursor back down
stream.write("%c[%dB" % (27, diff))
stream.flush()
return all_events
def print_output_event(event, stream, is_terminal):
if 'errorDetail' in event:
raise StreamOutputError(event['errorDetail']['message'])
terminator = ''
if is_terminal and 'stream' not in event:
# erase current line
stream.write("%c[2K\r" % 27)
terminator = "\r"
pass
elif 'progressDetail' in event:
return
if 'time' in event:
stream.write("[%s] " % event['time'])
if 'id' in event:
stream.write("%s: " % event['id'])
if 'from' in event:
stream.write("(from %s) " % event['from'])
status = event.get('status', '')
if 'progress' in event:
stream.write("%s %s%s" % (status, event['progress'], terminator))
elif 'progressDetail' in event:
detail = event['progressDetail']
if 'current' in detail:
percentage = float(detail['current']) / float(detail['total']) * 100
stream.write('%s (%.1f%%)%s' % (status, percentage, terminator))
else:
stream.write('%s%s' % (status, terminator))
elif 'stream' in event:
stream.write("%s%s" % (event['stream'], terminator))
else:
stream.write("%s%s\n" % (status, terminator))

229
fig/project.py Normal file
View file

@ -0,0 +1,229 @@
from __future__ import unicode_literals
from __future__ import absolute_import
import logging
from .service import Service
from .container import Container
from docker.errors import APIError
log = logging.getLogger(__name__)
def sort_service_dicts(services):
# Topological sort (Cormen/Tarjan algorithm).
unmarked = services[:]
temporary_marked = set()
sorted_services = []
get_service_names = lambda links: [link.split(':')[0] for link in links]
def visit(n):
if n['name'] in temporary_marked:
if n['name'] in get_service_names(n.get('links', [])):
raise DependencyError('A service can not link to itself: %s' % n['name'])
if n['name'] in n.get('volumes_from', []):
raise DependencyError('A service can not mount itself as volume: %s' % n['name'])
else:
raise DependencyError('Circular import between %s' % ' and '.join(temporary_marked))
if n in unmarked:
temporary_marked.add(n['name'])
dependents = [m for m in services if (n['name'] in get_service_names(m.get('links', []))) or (n['name'] in m.get('volumes_from', []))]
for m in dependents:
visit(m)
temporary_marked.remove(n['name'])
unmarked.remove(n)
sorted_services.insert(0, n)
while unmarked:
visit(unmarked[-1])
return sorted_services
class Project(object):
"""
A collection of services.
"""
def __init__(self, name, services, client):
self.name = name
self.services = services
self.client = client
@classmethod
def from_dicts(cls, name, service_dicts, client):
"""
Construct a ServiceCollection from a list of dicts representing services.
"""
project = cls(name, [], client)
for service_dict in sort_service_dicts(service_dicts):
links = project.get_links(service_dict)
volumes_from = project.get_volumes_from(service_dict)
project.services.append(Service(client=client, project=name, links=links, volumes_from=volumes_from, **service_dict))
return project
@classmethod
def from_config(cls, name, config, client):
dicts = []
for service_name, service in list(config.items()):
if not isinstance(service, dict):
raise ConfigurationError('Service "%s" doesn\'t have any configuration options. All top level keys in your fig.yml must map to a dictionary of configuration options.')
service['name'] = service_name
dicts.append(service)
return cls.from_dicts(name, dicts, client)
def get_service(self, name):
"""
Retrieve a service by name. Raises NoSuchService
if the named service does not exist.
"""
for service in self.services:
if service.name == name:
return service
raise NoSuchService(name)
def get_services(self, service_names=None, include_links=False):
"""
Returns a list of this project's services filtered
by the provided list of names, or all services if service_names is None
or [].
If include_links is specified, returns a list including the links for
service_names, in order of dependency.
Preserves the original order of self.services where possible,
reordering as needed to resolve links.
Raises NoSuchService if any of the named services do not exist.
"""
if service_names is None or len(service_names) == 0:
return self.get_services(
service_names=[s.name for s in self.services],
include_links=include_links
)
else:
unsorted = [self.get_service(name) for name in service_names]
services = [s for s in self.services if s in unsorted]
if include_links:
services = reduce(self._inject_links, services, [])
uniques = []
[uniques.append(s) for s in services if s not in uniques]
return uniques
def get_links(self, service_dict):
links = []
if 'links' in service_dict:
for link in service_dict.get('links', []):
if ':' in link:
service_name, link_name = link.split(':', 1)
else:
service_name, link_name = link, None
try:
links.append((self.get_service(service_name), link_name))
except NoSuchService:
raise ConfigurationError('Service "%s" has a link to service "%s" which does not exist.' % (service_dict['name'], service_name))
del service_dict['links']
return links
def get_volumes_from(self, service_dict):
volumes_from = []
if 'volumes_from' in service_dict:
for volume_name in service_dict.get('volumes_from', []):
try:
service = self.get_service(volume_name)
volumes_from.append(service)
except NoSuchService:
try:
container = Container.from_id(self.client, volume_name)
volumes_from.append(container)
except APIError:
raise ConfigurationError('Service "%s" mounts volumes from "%s", which is not the name of a service or container.' % (service_dict['name'], volume_name))
del service_dict['volumes_from']
return volumes_from
def start(self, service_names=None, **options):
for service in self.get_services(service_names):
service.start(**options)
def stop(self, service_names=None, **options):
for service in reversed(self.get_services(service_names)):
service.stop(**options)
def kill(self, service_names=None, **options):
for service in reversed(self.get_services(service_names)):
service.kill(**options)
def restart(self, service_names=None, **options):
for service in self.get_services(service_names):
service.restart(**options)
def build(self, service_names=None, no_cache=False):
for service in self.get_services(service_names):
if service.can_be_built():
service.build(no_cache)
else:
log.info('%s uses an image, skipping' % service.name)
def up(self, service_names=None, start_links=True, recreate=True, insecure_registry=False):
running_containers = []
for service in self.get_services(service_names, include_links=start_links):
if recreate:
for (_, container) in service.recreate_containers(insecure_registry=insecure_registry):
running_containers.append(container)
else:
for container in service.start_or_create_containers(insecure_registry=insecure_registry):
running_containers.append(container)
return running_containers
def pull(self, service_names=None, insecure_registry=False):
for service in self.get_services(service_names, include_links=True):
service.pull(insecure_registry=insecure_registry)
def remove_stopped(self, service_names=None, **options):
for service in self.get_services(service_names):
service.remove_stopped(**options)
def containers(self, service_names=None, stopped=False, one_off=False):
return [Container.from_ps(self.client, container)
for container in self.client.containers(all=stopped)
for service in self.get_services(service_names)
if service.has_container(container, one_off=one_off)]
def _inject_links(self, acc, service):
linked_names = service.get_linked_names()
if len(linked_names) > 0:
linked_services = self.get_services(
service_names=linked_names,
include_links=True
)
else:
linked_services = []
linked_services.append(service)
return acc + linked_services
class NoSuchService(Exception):
def __init__(self, name):
self.name = name
self.msg = "No such service: %s" % self.name
def __str__(self):
return self.msg
class ConfigurationError(Exception):
def __init__(self, msg):
self.msg = msg
def __str__(self):
return self.msg
class DependencyError(ConfigurationError):
pass

525
fig/service.py Normal file
View file

@ -0,0 +1,525 @@
from __future__ import unicode_literals
from __future__ import absolute_import
from collections import namedtuple
import logging
import re
import os
from operator import attrgetter
import sys
from docker.errors import APIError
from .container import Container
from .progress_stream import stream_output, StreamOutputError
log = logging.getLogger(__name__)
DOCKER_CONFIG_KEYS = ['image', 'command', 'hostname', 'domainname', 'user', 'detach', 'stdin_open', 'tty', 'mem_limit', 'ports', 'environment', 'dns', 'volumes', 'entrypoint', 'privileged', 'volumes_from', 'net', 'working_dir']
DOCKER_CONFIG_HINTS = {
'link' : 'links',
'port' : 'ports',
'privilege' : 'privileged',
'priviliged': 'privileged',
'privilige' : 'privileged',
'volume' : 'volumes',
'workdir' : 'working_dir',
}
VALID_NAME_CHARS = '[a-zA-Z0-9]'
class BuildError(Exception):
def __init__(self, service, reason):
self.service = service
self.reason = reason
class CannotBeScaledError(Exception):
pass
class ConfigError(ValueError):
pass
VolumeSpec = namedtuple('VolumeSpec', 'external internal mode')
ServiceName = namedtuple('ServiceName', 'project service number')
class Service(object):
def __init__(self, name, client=None, project='default', links=None, volumes_from=None, **options):
if not re.match('^%s+$' % VALID_NAME_CHARS, name):
raise ConfigError('Invalid service name "%s" - only %s are allowed' % (name, VALID_NAME_CHARS))
if not re.match('^%s+$' % VALID_NAME_CHARS, project):
raise ConfigError('Invalid project name "%s" - only %s are allowed' % (project, VALID_NAME_CHARS))
if 'image' in options and 'build' in options:
raise ConfigError('Service %s has both an image and build path specified. A service can either be built to image or use an existing image, not both.' % name)
supported_options = DOCKER_CONFIG_KEYS + ['build', 'expose']
for k in options:
if k not in supported_options:
msg = "Unsupported config option for %s service: '%s'" % (name, k)
if k in DOCKER_CONFIG_HINTS:
msg += " (did you mean '%s'?)" % DOCKER_CONFIG_HINTS[k]
raise ConfigError(msg)
self.name = name
self.client = client
self.project = project
self.links = links or []
self.volumes_from = volumes_from or []
self.options = options
def containers(self, stopped=False, one_off=False):
return [Container.from_ps(self.client, container)
for container in self.client.containers(all=stopped)
if self.has_container(container, one_off=one_off)]
def has_container(self, container, one_off=False):
"""Return True if `container` was created to fulfill this service."""
name = get_container_name(container)
if not name or not is_valid_name(name, one_off):
return False
project, name, _number = parse_name(name)
return project == self.project and name == self.name
def get_container(self, number=1):
"""Return a :class:`fig.container.Container` for this service. The
container must be active, and match `number`.
"""
for container in self.client.containers():
if not self.has_container(container):
continue
_, _, container_number = parse_name(get_container_name(container))
if container_number == number:
return Container.from_ps(self.client, container)
raise ValueError("No container found for %s_%s" % (self.name, number))
def start(self, **options):
for c in self.containers(stopped=True):
self.start_container_if_stopped(c, **options)
def stop(self, **options):
for c in self.containers():
log.info("Stopping %s..." % c.name)
c.stop(**options)
def kill(self, **options):
for c in self.containers():
log.info("Killing %s..." % c.name)
c.kill(**options)
def restart(self, **options):
for c in self.containers():
log.info("Restarting %s..." % c.name)
c.restart(**options)
def scale(self, desired_num):
"""
Adjusts the number of containers to the specified number and ensures they are running.
- creates containers until there are at least `desired_num`
- stops containers until there are at most `desired_num` running
- starts containers until there are at least `desired_num` running
- removes all stopped containers
"""
if not self.can_be_scaled():
raise CannotBeScaledError()
# Create enough containers
containers = self.containers(stopped=True)
while len(containers) < desired_num:
containers.append(self.create_container())
running_containers = []
stopped_containers = []
for c in containers:
if c.is_running:
running_containers.append(c)
else:
stopped_containers.append(c)
running_containers.sort(key=lambda c: c.number)
stopped_containers.sort(key=lambda c: c.number)
# Stop containers
while len(running_containers) > desired_num:
c = running_containers.pop()
log.info("Stopping %s..." % c.name)
c.stop(timeout=1)
stopped_containers.append(c)
# Start containers
while len(running_containers) < desired_num:
c = stopped_containers.pop(0)
log.info("Starting %s..." % c.name)
self.start_container(c)
running_containers.append(c)
self.remove_stopped()
def remove_stopped(self, **options):
for c in self.containers(stopped=True):
if not c.is_running:
log.info("Removing %s..." % c.name)
c.remove(**options)
def create_container(self, one_off=False, insecure_registry=False, **override_options):
"""
Create a container for this service. If the image doesn't exist, attempt to pull
it.
"""
container_options = self._get_container_create_options(override_options, one_off=one_off)
try:
return Container.create(self.client, **container_options)
except APIError as e:
if e.response.status_code == 404 and e.explanation and 'No such image' in str(e.explanation):
log.info('Pulling image %s...' % container_options['image'])
output = self.client.pull(
container_options['image'],
stream=True,
insecure_registry=insecure_registry
)
stream_output(output, sys.stdout)
return Container.create(self.client, **container_options)
raise
def recreate_containers(self, insecure_registry=False, **override_options):
"""
If a container for this service doesn't exist, create and start one. If there are
any, stop them, create+start new ones, and remove the old containers.
"""
containers = self.containers(stopped=True)
if not containers:
log.info("Creating %s..." % self._next_container_name(containers))
container = self.create_container(insecure_registry=insecure_registry, **override_options)
self.start_container(container)
return [(None, container)]
else:
tuples = []
for c in containers:
log.info("Recreating %s..." % c.name)
tuples.append(self.recreate_container(c, insecure_registry=insecure_registry, **override_options))
return tuples
def recreate_container(self, container, **override_options):
"""Recreate a container. An intermediate container is created so that
the new container has the same name, while still supporting
`volumes-from` the original container.
"""
try:
container.stop()
except APIError as e:
if (e.response.status_code == 500
and e.explanation
and 'no such process' in str(e.explanation)):
pass
else:
raise
intermediate_container = Container.create(
self.client,
image=container.image,
entrypoint=['/bin/echo'],
command=[],
)
intermediate_container.start(volumes_from=container.id)
intermediate_container.wait()
container.remove()
options = dict(override_options)
new_container = self.create_container(**options)
self.start_container(new_container, intermediate_container=intermediate_container)
intermediate_container.remove()
return (intermediate_container, new_container)
def start_container_if_stopped(self, container, **options):
if container.is_running:
return container
else:
log.info("Starting %s..." % container.name)
return self.start_container(container, **options)
def start_container(self, container=None, intermediate_container=None, **override_options):
container = container or self.create_container(**override_options)
options = dict(self.options, **override_options)
ports = dict(split_port(port) for port in options.get('ports') or [])
volume_bindings = dict(
build_volume_binding(parse_volume_spec(volume))
for volume in options.get('volumes') or []
if ':' in volume)
privileged = options.get('privileged', False)
net = options.get('net', 'bridge')
dns = options.get('dns', None)
container.start(
links=self._get_links(link_to_self=options.get('one_off', False)),
port_bindings=ports,
binds=volume_bindings,
volumes_from=self._get_volumes_from(intermediate_container),
privileged=privileged,
network_mode=net,
dns=dns,
)
return container
def start_or_create_containers(self, insecure_registry=False):
containers = self.containers(stopped=True)
if not containers:
log.info("Creating %s..." % self._next_container_name(containers))
new_container = self.create_container(insecure_registry=insecure_registry)
return [self.start_container(new_container)]
else:
return [self.start_container_if_stopped(c) for c in containers]
def get_linked_names(self):
return [s.name for (s, _) in self.links]
def _next_container_name(self, all_containers, one_off=False):
bits = [self.project, self.name]
if one_off:
bits.append('run')
return '_'.join(bits + [str(self._next_container_number(all_containers))])
def _next_container_number(self, all_containers):
numbers = [parse_name(c.name).number for c in all_containers]
return 1 if not numbers else max(numbers) + 1
def _get_links(self, link_to_self):
links = []
for service, link_name in self.links:
for container in service.containers():
links.append((container.name, link_name or service.name))
links.append((container.name, container.name))
links.append((container.name, container.name_without_project))
if link_to_self:
for container in self.containers():
links.append((container.name, self.name))
links.append((container.name, container.name))
links.append((container.name, container.name_without_project))
return links
def _get_volumes_from(self, intermediate_container=None):
volumes_from = []
for volume_source in self.volumes_from:
if isinstance(volume_source, Service):
containers = volume_source.containers(stopped=True)
if not containers:
volumes_from.append(volume_source.create_container().id)
else:
volumes_from.extend(map(attrgetter('id'), containers))
elif isinstance(volume_source, Container):
volumes_from.append(volume_source.id)
if intermediate_container:
volumes_from.append(intermediate_container.id)
return volumes_from
def _get_container_create_options(self, override_options, one_off=False):
container_options = dict((k, self.options[k]) for k in DOCKER_CONFIG_KEYS if k in self.options)
container_options.update(override_options)
container_options['name'] = self._next_container_name(
self.containers(stopped=True, one_off=one_off),
one_off)
# If a qualified hostname was given, split it into an
# unqualified hostname and a domainname unless domainname
# was also given explicitly. This matches the behavior of
# the official Docker CLI in that scenario.
if ('hostname' in container_options
and 'domainname' not in container_options
and '.' in container_options['hostname']):
parts = container_options['hostname'].partition('.')
container_options['hostname'] = parts[0]
container_options['domainname'] = parts[2]
if 'ports' in container_options or 'expose' in self.options:
ports = []
all_ports = container_options.get('ports', []) + self.options.get('expose', [])
for port in all_ports:
port = str(port)
if ':' in port:
port = port.split(':')[-1]
if '/' in port:
port = tuple(port.split('/'))
ports.append(port)
container_options['ports'] = ports
if 'volumes' in container_options:
container_options['volumes'] = dict(
(parse_volume_spec(v).internal, {})
for v in container_options['volumes'])
if 'environment' in container_options:
if isinstance(container_options['environment'], list):
container_options['environment'] = dict(split_env(e) for e in container_options['environment'])
container_options['environment'] = dict(resolve_env(k, v) for k, v in container_options['environment'].iteritems())
if self.can_be_built():
if len(self.client.images(name=self._build_tag_name())) == 0:
self.build()
container_options['image'] = self._build_tag_name()
# Delete options which are only used when starting
for key in ['privileged', 'net', 'dns']:
if key in container_options:
del container_options[key]
return container_options
def build(self, no_cache=False):
log.info('Building %s...' % self.name)
build_output = self.client.build(
self.options['build'],
tag=self._build_tag_name(),
stream=True,
rm=True,
nocache=no_cache,
)
try:
all_events = stream_output(build_output, sys.stdout)
except StreamOutputError, e:
raise BuildError(self, unicode(e))
image_id = None
for event in all_events:
if 'stream' in event:
match = re.search(r'Successfully built ([0-9a-f]+)', event.get('stream', ''))
if match:
image_id = match.group(1)
if image_id is None:
raise BuildError(self)
return image_id
def can_be_built(self):
return 'build' in self.options
def _build_tag_name(self):
"""
The tag to give to images built for this service.
"""
return '%s_%s' % (self.project, self.name)
def can_be_scaled(self):
for port in self.options.get('ports', []):
if ':' in str(port):
return False
return True
def pull(self, insecure_registry=False):
if 'image' in self.options:
log.info('Pulling %s (%s)...' % (self.name, self.options.get('image')))
self.client.pull(
self.options.get('image'),
insecure_registry=insecure_registry
)
NAME_RE = re.compile(r'^([^_]+)_([^_]+)_(run_)?(\d+)$')
def is_valid_name(name, one_off=False):
match = NAME_RE.match(name)
if match is None:
return False
if one_off:
return match.group(3) == 'run_'
else:
return match.group(3) is None
def parse_name(name):
match = NAME_RE.match(name)
(project, service_name, _, suffix) = match.groups()
return ServiceName(project, service_name, int(suffix))
def get_container_name(container):
if not container.get('Name') and not container.get('Names'):
return None
# inspect
if 'Name' in container:
return container['Name']
# ps
for name in container['Names']:
if len(name.split('/')) == 2:
return name[1:]
def parse_volume_spec(volume_config):
parts = volume_config.split(':')
if len(parts) > 3:
raise ConfigError("Volume %s has incorrect format, should be "
"external:internal[:mode]" % volume_config)
if len(parts) == 1:
return VolumeSpec(None, parts[0], 'rw')
if len(parts) == 2:
parts.append('rw')
external, internal, mode = parts
if mode not in ('rw', 'ro'):
raise ConfigError("Volume %s has invalid mode (%s), should be "
"one of: rw, ro." % (volume_config, mode))
return VolumeSpec(external, internal, mode)
def build_volume_binding(volume_spec):
internal = {'bind': volume_spec.internal, 'ro': volume_spec.mode == 'ro'}
external = os.path.expanduser(volume_spec.external)
return os.path.abspath(os.path.expandvars(external)), internal
def split_port(port):
parts = str(port).split(':')
if not 1 <= len(parts) <= 3:
raise ConfigError('Invalid port "%s", should be '
'[[remote_ip:]remote_port:]port[/protocol]' % port)
if len(parts) == 1:
internal_port, = parts
return internal_port, None
if len(parts) == 2:
external_port, internal_port = parts
return internal_port, external_port
external_ip, external_port, internal_port = parts
return internal_port, (external_ip, external_port or None)
def split_env(env):
if '=' in env:
return env.split('=', 1)
else:
return env, None
def resolve_env(key, val):
if val is not None:
return key, val
elif key in os.environ:
return key, os.environ[key]
else:
return key, ''

BIN
logo.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 38 KiB

Some files were not shown because too many files have changed in this diff Show more