From f039c8b43cae5fab4b8a22004ae6ab56d4d698b8 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Thu, 20 Oct 2016 17:39:55 -0700 Subject: [PATCH 01/45] Update release process document to account for recent changes. Signed-off-by: Joffrey F --- project/RELEASE-PROCESS.md | 62 ++++++++++++++++++++------------------ 1 file changed, 33 insertions(+), 29 deletions(-) diff --git a/project/RELEASE-PROCESS.md b/project/RELEASE-PROCESS.md index 930af15a..c1834f2f 100644 --- a/project/RELEASE-PROCESS.md +++ b/project/RELEASE-PROCESS.md @@ -20,18 +20,30 @@ release. As part of this script you'll be asked to: -1. Update the version in `docs/install.md` and `compose/__init__.py`. +1. Update the version in `compose/__init__.py` and `script/run/run.sh`. - If the next release will be an RC, append `rcN`, e.g. `1.4.0rc1`. + If the next release will be an RC, append `-rcN`, e.g. `1.4.0-rc1`. 2. Write release notes in `CHANGES.md`. - Almost every feature enhancement should be mentioned, with the most visible/exciting ones first. Use descriptive sentences and give context where appropriate. + Almost every feature enhancement should be mentioned, with the most + visible/exciting ones first. Use descriptive sentences and give context + where appropriate. - Bug fixes are worth mentioning if it's likely that they've affected lots of people, or if they were regressions in the previous version. + Bug fixes are worth mentioning if it's likely that they've affected lots + of people, or if they were regressions in the previous version. Improvements to the code are not worth mentioning. +3. Create a new repository on [bintray](https://bintray.com/docker-compose). + The name has to match the name of the branch (e.g. `bump-1.9.0`) and the + type should be "Generic". Other fields can be left blank. + +4. Check that the `vnext-compose` branch on + [the docs repo](https://github.com/docker/docker.github.io/) has + documentation for all the new additions in the upcoming release, and create + a PR there for what needs to be amended. + ## When a PR is merged into master that we want in the release @@ -55,8 +67,8 @@ Check out the bump branch and run the `build-binaries` script When prompted build the non-linux binaries and test them. -1. Download the osx binary from Bintray. Make sure that the latest build has - finished, otherwise you'll be downloading an old binary. +1. Download the osx binary from Bintray. Make sure that the latest Travis + build has finished, otherwise you'll be downloading an old binary. https://dl.bintray.com/docker-compose/$BRANCH_NAME/ @@ -67,22 +79,24 @@ When prompted build the non-linux binaries and test them. 3. Draft a release from the tag on GitHub (the script will open the window for you) - In the "Tag version" dropdown, select the tag you just pushed. + The tag will only be present on Github when you run the `push-release` + script in step 7, but you can pre-fill it at that point. -4. Paste in installation instructions and release notes. Here's an example - change the Compose version and Docker version as appropriate: +4. Paste in installation instructions and release notes. Here's an example - + change the Compose version and Docker version as appropriate: - Firstly, note that Compose 1.5.0 requires Docker 1.8.0 or later. + If you're a Mac or Windows user, the best way to install Compose and keep it up-to-date is **[Docker for Mac and Windows](https://www.docker.com/products/docker)**. - Secondly, if you're a Mac user, the **[Docker Toolbox](https://www.docker.com/toolbox)** will install Compose 1.5.0 for you, alongside the latest versions of the Docker Engine, Machine and Kitematic. + Note that Compose 1.9.0 requires Docker Engine 1.10.0 or later for version 2 of the Compose File format, and Docker Engine 1.9.1 or later for version 1. Docker for Mac and Windows will automatically install the latest version of Docker Engine for you. - Otherwise, you can use the usual commands to install/upgrade. Either download the binary: + Alternatively, you can use the usual commands to install or upgrade Compose: - curl -L https://github.com/docker/compose/releases/download/1.5.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose - chmod +x /usr/local/bin/docker-compose + ``` + curl -L https://github.com/docker/compose/releases/download/1.9.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose + chmod +x /usr/local/bin/docker-compose + ``` - Or install the PyPi package: - - pip install -U docker-compose==1.5.0 + See the [install docs](https://docs.docker.com/compose/install/) for more install options and instructions. Here's what's new: @@ -99,6 +113,8 @@ When prompted build the non-linux binaries and test them. ./script/release/push-release +8. Merge the bump PR. + 8. Publish the release on GitHub. 9. Check that all the binaries download (following the install instructions) and run. @@ -107,19 +123,7 @@ When prompted build the non-linux binaries and test them. ## If it’s a stable release (not an RC) -1. Merge the bump PR. - -2. Make sure `origin/release` is updated locally: - - git fetch origin - -3. Update the `docs` branch on the upstream repo: - - git push git@github.com:docker/compose.git origin/release:docs - -4. Let the docs team know that it’s been updated so they can publish it. - -5. Close the release’s milestone. +1. Close the release’s milestone. ## If it’s a minor release (1.x.0), rather than a patch release (1.x.y) From 3d8dc6f47a1484b0b6f8e3bd5cbf6115524d3892 Mon Sep 17 00:00:00 2001 From: Michal Zdrojewski Date: Mon, 24 Oct 2016 15:05:01 +0100 Subject: [PATCH 02/45] fix(docs): updated documentation links Signed-off-by: Michal Zdrojewski --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 93550f5a..5cf69b05 100644 --- a/README.md +++ b/README.md @@ -6,11 +6,11 @@ Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a Compose file to configure your application's services. Then, using a single command, you create and start all the services from your configuration. To learn more about all the features of Compose -see [the list of features](https://github.com/docker/compose/blob/release/docs/overview.md#features). +see [the list of features](https://github.com/docker/docker.github.io/blob/master/compose/overview.md#features). Compose is great for development, testing, and staging environments, as well as CI workflows. You can learn more about each case in -[Common Use Cases](https://github.com/docker/compose/blob/release/docs/overview.md#common-use-cases). +[Common Use Cases](https://github.com/docker/docker.github.io/blob/master/compose/overview.md#common-use-cases). Using Compose is basically a three-step process. @@ -35,7 +35,7 @@ A `docker-compose.yml` looks like this: image: redis For more information about the Compose file, see the -[Compose file reference](https://github.com/docker/compose/blob/release/docs/compose-file.md) +[Compose file reference](https://github.com/docker/docker.github.io/blob/master/compose/compose-file.md) Compose has commands for managing the whole lifecycle of your application: From 2c24bc3a083e138898cfe619a9cbf414f6df12d8 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 24 Oct 2016 10:55:04 -0700 Subject: [PATCH 03/45] Add missing config schema to docker-compose.spec Signed-off-by: Joffrey F --- docker-compose.spec | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docker-compose.spec b/docker-compose.spec index 3a165dd6..e57d6b7a 100644 --- a/docker-compose.spec +++ b/docker-compose.spec @@ -27,6 +27,11 @@ exe = EXE(pyz, 'compose/config/config_schema_v2.0.json', 'DATA' ), + ( + 'compose/config/config_schema_v2.1.json', + 'compose/config/config_schema_v2.1.json', + 'DATA' + ), ( 'compose/GITSHA', 'compose/GITSHA', From ea68be3441ef2c0c100b8bac9499fa4180106eb5 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 24 Oct 2016 11:36:44 -0700 Subject: [PATCH 04/45] Do not print Swarm mode warning when connecting to a UCP server Signed-off-by: Joffrey F --- compose/project.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/compose/project.py b/compose/project.py index f85e285f..60647fe9 100644 --- a/compose/project.py +++ b/compose/project.py @@ -538,6 +538,10 @@ def get_volumes_from(project, service_dict): def warn_for_swarm_mode(client): info = client.info() if info.get('Swarm', {}).get('LocalNodeState') == 'active': + if info.get('ServerVersion', '').startswith('ucp'): + # UCP does multi-node scheduling with traditional Compose files. + return + log.warn( "The Docker Engine you're using is running in swarm mode.\n\n" "Compose does not use swarm mode to deploy services to multiple nodes in a swarm. " From 43e29b41c04dfa00fee294f57ccd2e5d1a80f99a Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 24 Oct 2016 11:51:45 -0700 Subject: [PATCH 05/45] Fix schema divergence - add missing fields to compose 2.1 schema Signed-off-by: Joffrey F --- compose/config/config_schema_v2.1.json | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/compose/config/config_schema_v2.1.json b/compose/config/config_schema_v2.1.json index f30b9054..3561a8cf 100644 --- a/compose/config/config_schema_v2.1.json +++ b/compose/config/config_schema_v2.1.json @@ -140,6 +140,7 @@ "mac_address": {"type": "string"}, "mem_limit": {"type": ["number", "string"]}, "memswap_limit": {"type": ["number", "string"]}, + "mem_swappiness": {"type": "integer"}, "network_mode": {"type": "string"}, "networks": { @@ -168,6 +169,14 @@ } ] }, + "oom_score_adj": {"type": "integer", "minimum": -1000, "maximum": 1000}, + "group_add": { + "type": "array", + "items": { + "type": ["string", "number"] + }, + "uniqueItems": true + }, "pid": {"type": ["string", "null"]}, "ports": { @@ -248,6 +257,7 @@ }, "additionalProperties": false }, + "internal": {"type": "boolean"}, "enable_ipv6": {"type": "boolean"}, "labels": {"$ref": "#/definitions/list_or_dict"} }, From d2fb146913a1204b805ceb05e3d2b1b1a1006675 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 24 Oct 2016 12:12:19 -0700 Subject: [PATCH 06/45] Replace "which" calls with the portable find_executable function Signed-off-by: Joffrey F --- compose/cli/errors.py | 6 +++--- compose/cli/main.py | 6 +++--- tests/unit/cli/errors_test.py | 6 +++--- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/compose/cli/errors.py b/compose/cli/errors.py index 4fdec08a..5b977095 100644 --- a/compose/cli/errors.py +++ b/compose/cli/errors.py @@ -4,6 +4,7 @@ from __future__ import unicode_literals import contextlib import logging import socket +from distutils.spawn import find_executable from textwrap import dedent from docker.errors import APIError @@ -13,7 +14,6 @@ from requests.exceptions import SSLError from requests.packages.urllib3.exceptions import ReadTimeoutError from ..const import API_VERSION_TO_ENGINE_VERSION -from .utils import call_silently from .utils import is_docker_for_mac_installed from .utils import is_mac from .utils import is_ubuntu @@ -90,11 +90,11 @@ def exit_with_error(msg): def get_conn_error_message(url): - if call_silently(['which', 'docker']) != 0: + if find_executable('docker') is None: return docker_not_found_msg("Couldn't connect to Docker daemon.") if is_docker_for_mac_installed(): return conn_error_docker_for_mac - if call_silently(['which', 'docker-machine']) == 0: + if find_executable('docker-machine') is not None: return conn_error_docker_machine return conn_error_generic.format(url=url) diff --git a/compose/cli/main.py b/compose/cli/main.py index 58b95c2f..08e58e37 100644 --- a/compose/cli/main.py +++ b/compose/cli/main.py @@ -10,6 +10,7 @@ import pipes import re import subprocess import sys +from distutils.spawn import find_executable from inspect import getdoc from operator import attrgetter @@ -1063,9 +1064,8 @@ def exit_if(condition, message, exit_code): def call_docker(args): - try: - executable_path = subprocess.check_output(["which", "docker"]).strip() - except subprocess.CalledProcessError: + executable_path = find_executable('docker') + if not executable_path: raise UserError(errors.docker_not_found_msg("Couldn't find `docker` binary.")) args = [executable_path] + args diff --git a/tests/unit/cli/errors_test.py b/tests/unit/cli/errors_test.py index 1d454a08..a7b57562 100644 --- a/tests/unit/cli/errors_test.py +++ b/tests/unit/cli/errors_test.py @@ -16,9 +16,9 @@ def mock_logging(): yield mock_log -def patch_call_silently(side_effect): +def patch_find_executable(side_effect): return mock.patch( - 'compose.cli.errors.call_silently', + 'compose.cli.errors.find_executable', autospec=True, side_effect=side_effect) @@ -27,7 +27,7 @@ class TestHandleConnectionErrors(object): def test_generic_connection_error(self, mock_logging): with pytest.raises(errors.ConnectionError): - with patch_call_silently([0, 1]): + with patch_find_executable(['/bin/docker', None]): with handle_connection_errors(mock.Mock()): raise ConnectionError() From 60d005b055119ce976265eaf34e4daa6483ead58 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 24 Oct 2016 13:58:45 -0700 Subject: [PATCH 07/45] Improve robustness of a couple integration tests with occasional failures Signed-off-by: Joffrey F --- tests/integration/project_test.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/integration/project_test.py b/tests/integration/project_test.py index e3c34af9..3afefb5a 100644 --- a/tests/integration/project_test.py +++ b/tests/integration/project_test.py @@ -826,9 +826,9 @@ class ProjectTest(DockerClientTestCase): name='composetest', config_data=config_data ) - project.up() + project.up(detached=True) - service_container = project.get_service('web').containers()[0] + service_container = project.get_service('web').containers(stopped=True)[0] ipam_config = service_container.inspect().get( 'NetworkSettings', {} ).get( @@ -857,8 +857,8 @@ class ProjectTest(DockerClientTestCase): name='composetest', config_data=config_data ) - project.up() - service_container = project.get_service('web').containers()[0] + project.up(detached=True) + service_container = project.get_service('web').containers(stopped=True)[0] assert service_container.inspect()['HostConfig']['Isolation'] == 'default' @v2_1_only() From 99343fd76cc632d30ff2132ea715728317e10250 Mon Sep 17 00:00:00 2001 From: Albin Kerouanton Date: Tue, 25 Oct 2016 11:06:39 +0200 Subject: [PATCH 08/45] Fix path of the parent dir of COMPOSE_FILE Signed-off-by: Albin Kerouanton --- script/run/run.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/script/run/run.sh b/script/run/run.sh index 6205747a..5872b081 100755 --- a/script/run/run.sh +++ b/script/run/run.sh @@ -35,7 +35,7 @@ if [ "$(pwd)" != '/' ]; then VOLUMES="-v $(pwd):$(pwd)" fi if [ -n "$COMPOSE_FILE" ]; then - compose_dir=$(dirname $COMPOSE_FILE) + compose_dir=$(realpath $(dirname $COMPOSE_FILE)) fi # TODO: also check --file argument if [ -n "$compose_dir" ]; then From 4871523d5e113e1f16a4ce50533d9b6f0bb80237 Mon Sep 17 00:00:00 2001 From: Mike Dougherty Date: Fri, 22 Apr 2016 12:56:07 -0700 Subject: [PATCH 09/45] Update Jenkinsfile to perform existing jenkins tasks Signed-off-by: Mike Dougherty --- Jenkinsfile | 84 +++++++++++++++++++++++++++++++++++++++++++++---- script/test/all | 3 +- 2 files changed, 80 insertions(+), 7 deletions(-) diff --git a/Jenkinsfile b/Jenkinsfile index fa29520b..5de9a3fb 100644 --- a/Jenkinsfile +++ b/Jenkinsfile @@ -1,8 +1,80 @@ -// Only run on Linux atm -wrappedNode(label: 'docker') { - deleteDir() - stage "checkout" - checkout scm +#!groovy - documentationChecker("docs") +def image + +def checkDocs = { -> + wrappedNode(label: 'linux') { + deleteDir(); checkout(scm) + documentationChecker("docs") + } } + +def buildImage = { -> + wrappedNode(label: "linux && !zfs") { + stage("build image") { + deleteDir(); checkout(scm) + def imageName = "dockerbuildbot/compose:${gitCommit()}" + image = docker.image(imageName) + try { + image.pull() + } catch (Exception exc) { + image = docker.build(imageName, ".") + image.push() + } + } + } +} + +def runTests = { Map settings -> + def dockerVersions = settings.get("dockerVersions", null) + def pythonVersions = settings.get("pythonVersions", null) + + if (!pythonVersions) { + throw new Exception("Need Python versions to test. e.g.: `runTests(pythonVersions: 'py27,py34')`") + } + if (!dockerVersions) { + throw new Exception("Need Docker versions to test. e.g.: `runTests(dockerVersions: 'all')`") + } + + { -> + wrappedNode(label: "linux && !zfs") { + stage("test python=${pythonVersions} / docker=${dockerVersions}") { + deleteDir(); checkout(scm) + def storageDriver = sh(script: 'docker info | awk -F \': \' \'$1 == "Storage Driver" { print $2; exit }\'', returnStdout: true).trim() + echo "Using local system's storage driver: ${storageDriver}" + sh """docker run \\ + -t \\ + --rm \\ + --privileged \\ + --volume="\$(pwd)/.git:/code/.git" \\ + --volume="/var/run/docker.sock:/var/run/docker.sock" \\ + -e "TAG=${image.id}" \\ + -e "STORAGE_DRIVER=${storageDriver}" \\ + -e "DOCKER_VERSIONS=${dockerVersions}" \\ + -e "BUILD_NUMBER=\$BUILD_TAG" \\ + -e "PY_TEST_VERSIONS=${pythonVersions}" \\ + --entrypoint="script/ci" \\ + ${image.id} \\ + --verbose + """ + } + } + } +} + +def buildAndTest = { -> + buildImage() + // TODO: break this out into meaningful "DOCKER_VERSIONS" values instead of all + parallel( + failFast: true, + all_py27: runTests(pythonVersions: "py27", dockerVersions: "all"), + all_py34: runTests(pythonVersions: "py34", dockerVersions: "all"), + ) +} + + +parallel( + failFast: false, + docs: checkDocs, + test: buildAndTest +) diff --git a/script/test/all b/script/test/all index 08bf1618..7151a75e 100755 --- a/script/test/all +++ b/script/test/all @@ -24,6 +24,7 @@ fi BUILD_NUMBER=${BUILD_NUMBER-$USER} +PY_TEST_VERSIONS=${PY_TEST_VERSIONS:-py27,py34} for version in $DOCKER_VERSIONS; do >&2 echo "Running tests against Docker $version" @@ -58,6 +59,6 @@ for version in $DOCKER_VERSIONS; do --env="DOCKER_VERSION=$version" \ --entrypoint="tox" \ "$TAG" \ - -e py27,py34 -- "$@" + -e "$PY_TEST_VERSIONS" -- "$@" done From 046144e8f40de571b0d9c0029329e20712ca4736 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Thu, 27 Oct 2016 12:13:32 -0700 Subject: [PATCH 10/45] Bump docker-py version to include latest patch Signed-off-by: Joffrey F --- requirements.txt | 2 +- setup.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/requirements.txt b/requirements.txt index 474efbdf..e72a88e7 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,7 +1,7 @@ PyYAML==3.11 backports.ssl-match-hostname==3.5.0.1; python_version < '3' cached-property==1.2.0 -docker-py==1.10.4 +docker-py==1.10.5 dockerpty==0.4.1 docopt==0.6.1 enum34==1.0.4; python_version < '3.4' diff --git a/setup.py b/setup.py index 442bc80c..19ff5c6a 100644 --- a/setup.py +++ b/setup.py @@ -34,7 +34,7 @@ install_requires = [ 'requests >= 2.6.1, != 2.11.0, < 2.12', 'texttable >= 0.8.1, < 0.9', 'websocket-client >= 0.32.0, < 1.0', - 'docker-py >= 1.10.4, < 2.0', + 'docker-py >= 1.10.5, < 2.0', 'dockerpty >= 0.4.1, < 0.5', 'six >= 1.3.0, < 2', 'jsonschema >= 2.5.1, < 3', From ba43d08fbdc1e7ec25978cc9068c3a9a7be91aab Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Tue, 1 Nov 2016 11:09:29 -0700 Subject: [PATCH 11/45] Add whitelisted driver option added by the overlay driver to avoid breakage Signed-off-by: Joffrey F --- compose/network.py | 21 +++++++++++++++++ tests/unit/network_test.py | 47 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 68 insertions(+) create mode 100644 tests/unit/network_test.py diff --git a/compose/network.py b/compose/network.py index e581a4fe..8e38401c 100644 --- a/compose/network.py +++ b/compose/network.py @@ -12,6 +12,10 @@ from .config import ConfigurationError log = logging.getLogger(__name__) +OPTS_EXCEPTIONS = [ + 'com.docker.network.driver.overlay.vxlanid_list', +] + class Network(object): def __init__(self, client, project, name, driver=None, driver_opts=None, @@ -113,6 +117,23 @@ def create_ipam_config_from_dict(ipam_dict): ) +def check_remote_network_config(remote, local): + if local.driver and remote['Driver'] != local.driver: + raise ConfigurationError( + 'Network "{}" needs to be recreated - driver has changed' + .format(local.full_name) + ) + local_opts = local.driver_opts or {} + for k in set.union(set(remote['Options'].keys()), set(local_opts.keys())): + if k in OPTS_EXCEPTIONS: + continue + if remote['Options'].get(k) != local_opts.get(k): + raise ConfigurationError( + 'Network "{}" needs to be recreated - options have changed' + .format(local.full_name) + ) + + def build_networks(name, config_data, client): network_config = config_data.networks or {} networks = { diff --git a/tests/unit/network_test.py b/tests/unit/network_test.py new file mode 100644 index 00000000..4720b053 --- /dev/null +++ b/tests/unit/network_test.py @@ -0,0 +1,47 @@ +from __future__ import absolute_import +from __future__ import unicode_literals + +import pytest + +from .. import unittest +from compose.config import ConfigurationError +from compose.network import check_remote_network_config +from compose.network import Network + + +class NetworkTest(unittest.TestCase): + def test_check_remote_network_config_success(self): + options = {'com.docker.network.driver.foo': 'bar'} + net = Network( + None, 'compose_test', 'net1', 'bridge', + options + ) + check_remote_network_config( + {'Driver': 'bridge', 'Options': options}, net + ) + + def test_check_remote_network_config_whitelist(self): + options = {'com.docker.network.driver.foo': 'bar'} + remote_options = { + 'com.docker.network.driver.overlay.vxlanid_list': '257', + 'com.docker.network.driver.foo': 'bar' + } + net = Network( + None, 'compose_test', 'net1', 'overlay', + options + ) + check_remote_network_config( + {'Driver': 'overlay', 'Options': remote_options}, net + ) + + def test_check_remote_network_config_driver_mismatch(self): + net = Network(None, 'compose_test', 'net1', 'overlay') + with pytest.raises(ConfigurationError): + check_remote_network_config({'Driver': 'bridge', 'Options': {}}, net) + + def test_check_remote_network_config_options_mismatch(self): + net = Network(None, 'compose_test', 'net1', 'overlay') + with pytest.raises(ConfigurationError): + check_remote_network_config({'Driver': 'overlay', 'Options': { + 'com.docker.network.driver.foo': 'baz' + }}, net) From 7a430dbe96ad01831e21eab7eaebf26fdaa9249c Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Wed, 2 Nov 2016 16:43:29 -0700 Subject: [PATCH 12/45] Updated docker-py dependency to latest version Signed-off-by: Joffrey F --- requirements.txt | 2 +- setup.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/requirements.txt b/requirements.txt index e72a88e7..933146c7 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,7 +1,7 @@ PyYAML==3.11 backports.ssl-match-hostname==3.5.0.1; python_version < '3' cached-property==1.2.0 -docker-py==1.10.5 +docker-py==1.10.6 dockerpty==0.4.1 docopt==0.6.1 enum34==1.0.4; python_version < '3.4' diff --git a/setup.py b/setup.py index 19ff5c6a..672ea80e 100644 --- a/setup.py +++ b/setup.py @@ -34,7 +34,7 @@ install_requires = [ 'requests >= 2.6.1, != 2.11.0, < 2.12', 'texttable >= 0.8.1, < 0.9', 'websocket-client >= 0.32.0, < 1.0', - 'docker-py >= 1.10.5, < 2.0', + 'docker-py >= 1.10.6, < 2.0', 'dockerpty >= 0.4.1, < 0.5', 'six >= 1.3.0, < 2', 'jsonschema >= 2.5.1, < 3', From da1508051dcc7f106f891078b688bc31db7e43c5 Mon Sep 17 00:00:00 2001 From: Mike Dougherty Date: Thu, 3 Nov 2016 13:59:56 -0700 Subject: [PATCH 13/45] Remove docs checker from Jenkinsfile and use cleanWorkspace option on wrappedNode Signed-off-by: Mike Dougherty --- Jenkinsfile | 34 +++++++++------------------------- 1 file changed, 9 insertions(+), 25 deletions(-) diff --git a/Jenkinsfile b/Jenkinsfile index 5de9a3fb..e2f86daa 100644 --- a/Jenkinsfile +++ b/Jenkinsfile @@ -2,17 +2,10 @@ def image -def checkDocs = { -> - wrappedNode(label: 'linux') { - deleteDir(); checkout(scm) - documentationChecker("docs") - } -} - def buildImage = { -> - wrappedNode(label: "linux && !zfs") { + wrappedNode(label: "linux && !zfs", cleanWorkspace: true) { stage("build image") { - deleteDir(); checkout(scm) + checkout(scm) def imageName = "dockerbuildbot/compose:${gitCommit()}" image = docker.image(imageName) try { @@ -37,9 +30,9 @@ def runTests = { Map settings -> } { -> - wrappedNode(label: "linux && !zfs") { + wrappedNode(label: "linux && !zfs", cleanWorkspace: true) { stage("test python=${pythonVersions} / docker=${dockerVersions}") { - deleteDir(); checkout(scm) + checkout(scm) def storageDriver = sh(script: 'docker info | awk -F \': \' \'$1 == "Storage Driver" { print $2; exit }\'', returnStdout: true).trim() echo "Using local system's storage driver: ${storageDriver}" sh """docker run \\ @@ -62,19 +55,10 @@ def runTests = { Map settings -> } } -def buildAndTest = { -> - buildImage() - // TODO: break this out into meaningful "DOCKER_VERSIONS" values instead of all - parallel( - failFast: true, - all_py27: runTests(pythonVersions: "py27", dockerVersions: "all"), - all_py34: runTests(pythonVersions: "py34", dockerVersions: "all"), - ) -} - - +buildImage() +// TODO: break this out into meaningful "DOCKER_VERSIONS" values instead of all parallel( - failFast: false, - docs: checkDocs, - test: buildAndTest + failFast: true, + all_py27: runTests(pythonVersions: "py27", dockerVersions: "all"), + all_py34: runTests(pythonVersions: "py34", dockerVersions: "all"), ) From 10417eebd70f028f57e56ef9a04d8ed51abdad99 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Thu, 3 Nov 2016 16:28:44 -0700 Subject: [PATCH 14/45] Fix logging dict merging Signed-off-by: Joffrey F --- compose/config/config.py | 2 +- tests/unit/config/config_test.py | 45 ++++++++++++++++++++++++++++++++ 2 files changed, 46 insertions(+), 1 deletion(-) diff --git a/compose/config/config.py b/compose/config/config.py index 437ed389..9d23b34d 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -771,7 +771,7 @@ def merge_service_dicts(base, override, version): for field in ['dns', 'dns_search', 'env_file', 'tmpfs']: md.merge_field(field, merge_list_or_string) - md.merge_field('logging', merge_logging) + md.merge_field('logging', merge_logging, default={}) for field in set(ALLOWED_KEYS) - set(md): md.merge_scalar(field) diff --git a/tests/unit/config/config_test.py b/tests/unit/config/config_test.py index d205e282..66ae0147 100644 --- a/tests/unit/config/config_test.py +++ b/tests/unit/config/config_test.py @@ -1657,6 +1657,51 @@ class ConfigTest(unittest.TestCase): } } + def test_merge_logging_v2_no_base(self): + base = { + 'image': 'alpine:edge' + } + override = { + 'logging': { + 'driver': 'json-file', + 'options': { + 'frequency': '2000' + } + } + } + actual = config.merge_service_dicts(base, override, V2_0) + assert actual == { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'json-file', + 'options': { + 'frequency': '2000' + } + } + } + + def test_merge_logging_v2_no_override(self): + base = { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'syslog', + 'options': { + 'frequency': '2000' + } + } + } + override = {} + actual = config.merge_service_dicts(base, override, V2_0) + assert actual == { + 'image': 'alpine:edge', + 'logging': { + 'driver': 'syslog', + 'options': { + 'frequency': '2000' + } + } + } + def test_external_volume_config(self): config_details = build_config_details({ 'version': '2', From 4aa7d15d9771fadd22e8c1ff952526fdd6a67d77 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Fri, 4 Nov 2016 10:51:14 -0700 Subject: [PATCH 15/45] Call check_remote_network_config from Network.ensure Signed-off-by: Joffrey F --- compose/network.py | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/compose/network.py b/compose/network.py index 8e38401c..06935a46 100644 --- a/compose/network.py +++ b/compose/network.py @@ -53,14 +53,7 @@ class Network(object): try: data = self.inspect() - if self.driver and data['Driver'] != self.driver: - raise ConfigurationError( - 'Network "{}" needs to be recreated - driver has changed' - .format(self.full_name)) - if data['Options'] != (self.driver_opts or {}): - raise ConfigurationError( - 'Network "{}" needs to be recreated - options have changed' - .format(self.full_name)) + check_remote_network_config(data, self) except NotFound: driver_name = 'the default driver' if self.driver: From ba249e51796e6b35ed13e5f03716a9520a2dacfc Mon Sep 17 00:00:00 2001 From: Aanand Prasad Date: Wed, 9 Nov 2016 15:10:02 +0000 Subject: [PATCH 16/45] Test that values in 'environment' override env files Signed-off-by: Aanand Prasad --- tests/unit/config/config_test.py | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/tests/unit/config/config_test.py b/tests/unit/config/config_test.py index 66ae0147..51c5e226 100644 --- a/tests/unit/config/config_test.py +++ b/tests/unit/config/config_test.py @@ -2493,6 +2493,15 @@ class EnvTest(unittest.TestCase): {'ONE': '2', 'TWO': '1', 'THREE': '3', 'FOO': 'bar'}, ) + def test_environment_overrides_env_file(self): + self.assertEqual( + resolve_environment({ + 'environment': {'FOO': 'baz'}, + 'env_file': ['tests/fixtures/env/one.env'], + }), + {'ONE': '2', 'TWO': '1', 'THREE': '3', 'FOO': 'baz'}, + ) + def test_resolve_environment_with_multiple_env_files(self): service_dict = { 'env_file': [ From 91620ae97bbffd247e2f78132255c0fe97910c4f Mon Sep 17 00:00:00 2001 From: Jari Takkala Date: Fri, 29 Jul 2016 06:46:42 -0400 Subject: [PATCH 17/45] Add sysctl option support when creating service Closes #3765 Signed-off-by: Jari Takkala --- compose/config/config.py | 16 +++++++++++----- compose/config/config_schema_v2.1.json | 1 + compose/service.py | 2 ++ 3 files changed, 14 insertions(+), 5 deletions(-) diff --git a/compose/config/config.py b/compose/config/config.py index 9d23b34d..57039e69 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -83,6 +83,7 @@ DOCKER_CONFIG_KEYS = [ 'shm_size', 'stdin_open', 'stop_signal', + 'sysctls', 'tty', 'user', 'volume_driver', @@ -629,6 +630,9 @@ def process_service(service_config): if 'extra_hosts' in service_dict: service_dict['extra_hosts'] = parse_extra_hosts(service_dict['extra_hosts']) + if 'sysctls' in service_dict: + service_dict['sysctls'] = build_string_dict(parse_sysctls(service_dict['sysctls'])) + for field in ['dns', 'dns_search', 'tmpfs']: if field in service_dict: service_dict[field] = to_list(service_dict[field]) @@ -757,6 +761,7 @@ def merge_service_dicts(base, override, version): md.merge_mapping('labels', parse_labels) md.merge_mapping('ulimits', parse_ulimits) md.merge_mapping('networks', parse_networks) + md.merge_mapping('sysctls', parse_sysctls) md.merge_sequence('links', ServiceLink.parse) for field in ['volumes', 'devices']: @@ -831,11 +836,11 @@ def merge_environment(base, override): return env -def split_label(label): - if '=' in label: - return label.split('=', 1) +def split_kv(kvpair): + if '=' in kvpair: + return kvpair.split('=', 1) else: - return label, '' + return kvpair, '' def parse_dict_or_list(split_func, type_name, arguments): @@ -856,8 +861,9 @@ def parse_dict_or_list(split_func, type_name, arguments): parse_build_arguments = functools.partial(parse_dict_or_list, split_env, 'build arguments') parse_environment = functools.partial(parse_dict_or_list, split_env, 'environment') -parse_labels = functools.partial(parse_dict_or_list, split_label, 'labels') +parse_labels = functools.partial(parse_dict_or_list, split_kv, 'labels') parse_networks = functools.partial(parse_dict_or_list, lambda k: (k, None), 'networks') +parse_sysctls = functools.partial(parse_dict_or_list, split_kv, 'sysctls') def parse_ulimits(ulimits): diff --git a/compose/config/config_schema_v2.1.json b/compose/config/config_schema_v2.1.json index 3561a8cf..fc95f2cb 100644 --- a/compose/config/config_schema_v2.1.json +++ b/compose/config/config_schema_v2.1.json @@ -193,6 +193,7 @@ "restart": {"type": "string"}, "security_opt": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, "shm_size": {"type": ["number", "string"]}, + "sysctls": {"$ref": "#/definitions/list_or_dict"}, "stdin_open": {"type": "boolean"}, "stop_signal": {"type": "string"}, "tmpfs": {"$ref": "#/definitions/string_or_list"}, diff --git a/compose/service.py b/compose/service.py index 760d29a7..c917aac4 100644 --- a/compose/service.py +++ b/compose/service.py @@ -62,6 +62,7 @@ DOCKER_START_KEYS = [ 'restart', 'security_opt', 'shm_size', + 'sysctls', 'volumes_from', ] @@ -707,6 +708,7 @@ class Service(object): cgroup_parent=options.get('cgroup_parent'), cpu_quota=options.get('cpu_quota'), shm_size=options.get('shm_size'), + sysctls=options.get('sysctls'), tmpfs=options.get('tmpfs'), oom_score_adj=options.get('oom_score_adj'), mem_swappiness=options.get('mem_swappiness'), From 0291d9ade5574ff31aecc1a9f263f1ec21a901cf Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 14 Nov 2016 17:23:25 -0800 Subject: [PATCH 18/45] Limit testing pool to Ubuntu hosts to avoid errors with dind not starting properly. Signed-off-by: Joffrey F --- Jenkinsfile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Jenkinsfile b/Jenkinsfile index e2f86daa..51136b1f 100644 --- a/Jenkinsfile +++ b/Jenkinsfile @@ -3,7 +3,7 @@ def image def buildImage = { -> - wrappedNode(label: "linux && !zfs", cleanWorkspace: true) { + wrappedNode(label: "ubuntu && !zfs", cleanWorkspace: true) { stage("build image") { checkout(scm) def imageName = "dockerbuildbot/compose:${gitCommit()}" @@ -30,7 +30,7 @@ def runTests = { Map settings -> } { -> - wrappedNode(label: "linux && !zfs", cleanWorkspace: true) { + wrappedNode(label: "ubuntu && !zfs", cleanWorkspace: true) { stage("test python=${pythonVersions} / docker=${dockerVersions}") { checkout(scm) def storageDriver = sh(script: 'docker info | awk -F \': \' \'$1 == "Storage Driver" { print $2; exit }\'', returnStdout: true).trim() From efb4ed1b9e130ca5ac54f6e0fb23ce68c3689c1f Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 14 Nov 2016 18:03:26 -0800 Subject: [PATCH 19/45] Handle new pull failures behavior in Engine 1.13 Signed-off-by: Joffrey F --- compose/service.py | 6 +++--- tests/acceptance/cli_test.py | 7 ++++--- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/compose/service.py b/compose/service.py index 760d29a7..ad426706 100644 --- a/compose/service.py +++ b/compose/service.py @@ -10,6 +10,7 @@ from operator import attrgetter import enum import six from docker.errors import APIError +from docker.errors import NotFound from docker.utils import LogConfig from docker.utils.ports import build_port_bindings from docker.utils.ports import split_port @@ -829,12 +830,11 @@ class Service(object): repo, tag, separator = parse_repository_tag(self.options['image']) tag = tag or 'latest' log.info('Pulling %s (%s%s%s)...' % (self.name, repo, separator, tag)) - output = self.client.pull(repo, tag=tag, stream=True) - try: + output = self.client.pull(repo, tag=tag, stream=True) return progress_stream.get_digest_from_pull( stream_output(output, sys.stdout)) - except StreamOutputError as e: + except (StreamOutputError, NotFound) as e: if not ignore_pull_failures: raise else: diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index a7cd78f1..f153bd95 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -330,12 +330,13 @@ class CLITestCase(DockerClientTestCase): def test_pull_with_ignore_pull_failures(self): result = self.dispatch([ '-f', 'ignore-pull-failures.yml', - 'pull', '--ignore-pull-failures']) + 'pull', '--ignore-pull-failures'] + ) assert 'Pulling simple (busybox:latest)...' in result.stderr assert 'Pulling another (nonexisting-image:latest)...' in result.stderr - assert 'Error: image library/nonexisting-image' in result.stderr - assert 'not found' in result.stderr + assert ('repository nonexisting-image not found' in result.stderr or + 'image library/nonexisting-image:latest not found' in result.stderr) def test_build_plain(self): self.base_dir = 'tests/fixtures/simple-dockerfile' From 7f60ff5ae6b320dc0b38f8f2bcc850eca95f49ae Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Wed, 9 Nov 2016 15:38:09 -0800 Subject: [PATCH 20/45] Avoid breaking when remote driver options are null. Signed-off-by: Joffrey F --- compose/network.py | 7 ++++--- tests/unit/network_test.py | 10 +++++++++- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/compose/network.py b/compose/network.py index 06935a46..3b57cb94 100644 --- a/compose/network.py +++ b/compose/network.py @@ -111,16 +111,17 @@ def create_ipam_config_from_dict(ipam_dict): def check_remote_network_config(remote, local): - if local.driver and remote['Driver'] != local.driver: + if local.driver and remote.get('Driver') != local.driver: raise ConfigurationError( 'Network "{}" needs to be recreated - driver has changed' .format(local.full_name) ) local_opts = local.driver_opts or {} - for k in set.union(set(remote['Options'].keys()), set(local_opts.keys())): + remote_opts = remote.get('Options') or {} + for k in set.union(set(remote_opts.keys()), set(local_opts.keys())): if k in OPTS_EXCEPTIONS: continue - if remote['Options'].get(k) != local_opts.get(k): + if remote_opts.get(k) != local_opts.get(k): raise ConfigurationError( 'Network "{}" needs to be recreated - options have changed' .format(local.full_name) diff --git a/tests/unit/network_test.py b/tests/unit/network_test.py index 4720b053..12d06f41 100644 --- a/tests/unit/network_test.py +++ b/tests/unit/network_test.py @@ -37,7 +37,9 @@ class NetworkTest(unittest.TestCase): def test_check_remote_network_config_driver_mismatch(self): net = Network(None, 'compose_test', 'net1', 'overlay') with pytest.raises(ConfigurationError): - check_remote_network_config({'Driver': 'bridge', 'Options': {}}, net) + check_remote_network_config( + {'Driver': 'bridge', 'Options': {}}, net + ) def test_check_remote_network_config_options_mismatch(self): net = Network(None, 'compose_test', 'net1', 'overlay') @@ -45,3 +47,9 @@ class NetworkTest(unittest.TestCase): check_remote_network_config({'Driver': 'overlay', 'Options': { 'com.docker.network.driver.foo': 'baz' }}, net) + + def test_check_remote_network_config_null_remote(self): + net = Network(None, 'compose_test', 'net1', 'overlay') + check_remote_network_config( + {'Driver': 'overlay', 'Options': None}, net + ) From d717c88b6e81f5eb0769bc0670a6b78de842b2ce Mon Sep 17 00:00:00 2001 From: Aanand Prasad Date: Wed, 9 Nov 2016 17:16:36 +0000 Subject: [PATCH 21/45] Support version 3.0 of the Compose file format Signed-off-by: Aanand Prasad --- compose/config/config.py | 13 +- compose/config/config_schema_v3.0.json | 378 ++++++++++++++++++++++ compose/config/errors.py | 4 +- compose/config/serialize.py | 3 +- compose/const.py | 3 + tests/acceptance/cli_test.py | 55 ++++ tests/fixtures/v3-full/docker-compose.yml | 37 +++ tests/unit/config/config_test.py | 6 + 8 files changed, 491 insertions(+), 8 deletions(-) create mode 100644 compose/config/config_schema_v3.0.json create mode 100644 tests/fixtures/v3-full/docker-compose.yml diff --git a/compose/config/config.py b/compose/config/config.py index 9d23b34d..fb77436d 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -15,6 +15,7 @@ from cached_property import cached_property from ..const import COMPOSEFILE_V1 as V1 from ..const import COMPOSEFILE_V2_0 as V2_0 from ..const import COMPOSEFILE_V2_1 as V2_1 +from ..const import COMPOSEFILE_V3_0 as V3_0 from ..utils import build_string_dict from ..utils import splitdrive from .environment import env_vars_from_file @@ -175,7 +176,10 @@ class ConfigFile(namedtuple('_ConfigFile', 'filename config')): if version == '2': version = V2_0 - if version not in (V2_0, V2_1): + if version == '3': + version = V3_0 + + if version not in (V2_0, V2_1, V3_0): raise ConfigurationError( 'Version in "{}" is unsupported. {}' .format(self.filename, VERSION_EXPLANATION)) @@ -433,7 +437,7 @@ def process_config_file(config_file, environment, service_name=None): 'service', environment) - if config_file.version in (V2_0, V2_1): + if config_file.version in (V2_0, V2_1, V3_0): processed_config = dict(config_file.config) processed_config['services'] = services processed_config['volumes'] = interpolate_config_section( @@ -446,9 +450,10 @@ def process_config_file(config_file, environment, service_name=None): config_file.get_networks(), 'network', environment) - - if config_file.version == V1: + elif config_file.version == V1: processed_config = services + else: + raise Exception("Unsupported version: {}".format(repr(config_file.version))) config_file = config_file._replace(config=processed_config) validate_against_config_schema(config_file) diff --git a/compose/config/config_schema_v3.0.json b/compose/config/config_schema_v3.0.json new file mode 100644 index 00000000..9ac31b1f --- /dev/null +++ b/compose/config/config_schema_v3.0.json @@ -0,0 +1,378 @@ +{ + "$schema": "http://json-schema.org/draft-04/schema#", + "id": "config_schema_v3.0.json", + "type": "object", + "required": ["version"], + + "properties": { + "version": { + "type": "string" + }, + + "services": { + "id": "#/properties/services", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/service" + } + }, + "additionalProperties": false + }, + + "networks": { + "id": "#/properties/networks", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/network" + } + } + }, + + "volumes": { + "id": "#/properties/volumes", + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "$ref": "#/definitions/volume" + } + }, + "additionalProperties": false + } + }, + + "additionalProperties": false, + + "definitions": { + + "service": { + "id": "#/definitions/service", + "type": "object", + + "properties": { + "deploy": {"$ref": "#/definitions/deployment"}, + "build": { + "oneOf": [ + {"type": "string"}, + { + "type": "object", + "properties": { + "context": {"type": "string"}, + "dockerfile": {"type": "string"}, + "args": {"$ref": "#/definitions/list_or_dict"} + }, + "additionalProperties": false + } + ] + }, + "cap_add": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "cap_drop": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "cgroup_parent": {"type": "string"}, + "command": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "container_name": {"type": "string"}, + "depends_on": {"$ref": "#/definitions/list_of_strings"}, + "devices": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "dns": {"$ref": "#/definitions/string_or_list"}, + "dns_search": {"$ref": "#/definitions/string_or_list"}, + "domainname": {"type": "string"}, + "entrypoint": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "env_file": {"$ref": "#/definitions/string_or_list"}, + "environment": {"$ref": "#/definitions/list_or_dict"}, + + "expose": { + "type": "array", + "items": { + "type": ["string", "number"], + "format": "expose" + }, + "uniqueItems": true + }, + + "external_links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "extra_hosts": {"$ref": "#/definitions/list_or_dict"}, + "healthcheck": {"$ref": "#/definitions/healthcheck"}, + "hostname": {"type": "string"}, + "image": {"type": "string"}, + "ipc": {"type": "string"}, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + + "logging": { + "type": "object", + + "properties": { + "driver": {"type": "string"}, + "options": { + "type": "object", + "patternProperties": { + "^.+$": {"type": ["string", "number", "null"]} + } + } + }, + "additionalProperties": false + }, + + "mac_address": {"type": "string"}, + "network_mode": {"type": "string"}, + + "networks": { + "oneOf": [ + {"$ref": "#/definitions/list_of_strings"}, + { + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "oneOf": [ + { + "type": "object", + "properties": { + "aliases": {"$ref": "#/definitions/list_of_strings"}, + "ipv4_address": {"type": "string"}, + "ipv6_address": {"type": "string"} + }, + "additionalProperties": false + }, + {"type": "null"} + ] + } + }, + "additionalProperties": false + } + ] + }, + "pid": {"type": ["string", "null"]}, + + "ports": { + "type": "array", + "items": { + "type": ["string", "number"], + "format": "ports" + }, + "uniqueItems": true + }, + + "privileged": {"type": "boolean"}, + "read_only": {"type": "boolean"}, + "restart": {"type": "string"}, + "security_opt": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "shm_size": {"type": ["number", "string"]}, + "stdin_open": {"type": "boolean"}, + "stop_signal": {"type": "string"}, + "stop_grace_period": {"type": "string", "format": "duration"}, + "tmpfs": {"$ref": "#/definitions/string_or_list"}, + "tty": {"type": "boolean"}, + "ulimits": { + "type": "object", + "patternProperties": { + "^[a-z]+$": { + "oneOf": [ + {"type": "integer"}, + { + "type":"object", + "properties": { + "hard": {"type": "integer"}, + "soft": {"type": "integer"} + }, + "required": ["soft", "hard"], + "additionalProperties": false + } + ] + } + } + }, + "user": {"type": "string"}, + "volumes": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, + "working_dir": {"type": "string"} + }, + "additionalProperties": false + }, + + "healthcheck": { + "id": "#/definitions/healthcheck", + "type": ["object", "null"], + "properties": { + "interval": {"type":"string"}, + "timeout": {"type":"string"}, + "retries": {"type": "number"}, + "command": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + } + }, + "additionalProperties": false + }, + "deployment": { + "id": "#/definitions/deployment", + "type": ["object", "null"], + "properties": { + "mode": {"type": "string"}, + "replicas": {"type": "integer"}, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "update_config": { + "type": "object", + "properties": { + "parallelism": {"type": "integer"}, + "delay": {"type": "string", "format": "duration"}, + "failure_action": {"type": "string"}, + "monitor": {"type": "string", "format": "duration"}, + "max_failure_ratio": {"type": "number"} + }, + "additionalProperties": false + }, + "resources": { + "type": "object", + "properties": { + "limits": {"$ref": "#/definitions/resource"}, + "reservations": {"$ref": "#/definitions/resource"} + } + }, + "restart_policy": { + "type": "object", + "properties": { + "condition": {"type": "string"}, + "delay": {"type": "string", "format": "duration"}, + "max_attempts": {"type": "integer"}, + "window": {"type": "string", "format": "duration"} + }, + "additionalProperties": false + }, + "placement": { + "type": "object", + "properties": { + "constraints": {"type": "array", "items": {"type": "string"}} + }, + "additionalProperties": false + } + }, + "additionalProperties": false + }, + + "resource": { + "id": "#/definitions/resource", + "type": "object", + "properties": { + "cpus": {"type": "string"}, + "memory": {"type": "string"} + }, + "additionaProperties": false + }, + + "network": { + "id": "#/definitions/network", + "type": ["object", "null"], + "properties": { + "driver": {"type": "string"}, + "driver_opts": { + "type": "object", + "patternProperties": { + "^.+$": {"type": ["string", "number"]} + } + }, + "ipam": { + "type": "object", + "properties": { + "driver": {"type": "string"}, + "config": { + "type": "array", + "items": { + "type": "object", + "properties": { + "subnet": {"type": "string"} + }, + "additionalProperties": false + } + } + }, + "additionalProperties": false + }, + "external": { + "type": ["boolean", "object"], + "properties": { + "name": {"type": "string"} + }, + "additionalProperties": false + }, + "labels": {"$ref": "#/definitions/list_or_dict"} + }, + "additionalProperties": false + }, + + "volume": { + "id": "#/definitions/volume", + "type": ["object", "null"], + "properties": { + "driver": {"type": "string"}, + "driver_opts": { + "type": "object", + "patternProperties": { + "^.+$": {"type": ["string", "number"]} + } + }, + "external": { + "type": ["boolean", "object"], + "properties": { + "name": {"type": "string"} + } + } + }, + "labels": {"$ref": "#/definitions/list_or_dict"}, + "additionalProperties": false + }, + + "string_or_list": { + "oneOf": [ + {"type": "string"}, + {"$ref": "#/definitions/list_of_strings"} + ] + }, + + "list_of_strings": { + "type": "array", + "items": {"type": "string"}, + "uniqueItems": true + }, + + "list_or_dict": { + "oneOf": [ + { + "type": "object", + "patternProperties": { + ".+": { + "type": ["string", "number", "null"] + } + }, + "additionalProperties": false + }, + {"type": "array", "items": {"type": "string"}, "uniqueItems": true} + ] + }, + + "constraints": { + "service": { + "id": "#/definitions/constraints/service", + "anyOf": [ + {"required": ["build"]}, + {"required": ["image"]} + ], + "properties": { + "build": { + "required": ["context"] + } + } + } + } + } +} diff --git a/compose/config/errors.py b/compose/config/errors.py index d14cbbdd..16ed01b8 100644 --- a/compose/config/errors.py +++ b/compose/config/errors.py @@ -3,8 +3,8 @@ from __future__ import unicode_literals VERSION_EXPLANATION = ( - 'You might be seeing this error because you\'re using the wrong Compose ' - 'file version. Either specify a version of "2" (or "2.0") and place your ' + 'You might be seeing this error because you\'re using the wrong Compose file version. ' + 'Either specify a supported version ("2.0", "2.1", "3.0") and place your ' 'service definitions under the `services` key, or omit the `version` key ' 'and place your service definitions at the root of the file to use ' 'version 1.\nFor more on the Compose file format versions, see ' diff --git a/compose/config/serialize.py b/compose/config/serialize.py index 95b1387f..768f3d47 100644 --- a/compose/config/serialize.py +++ b/compose/config/serialize.py @@ -6,7 +6,6 @@ import yaml from compose.config import types from compose.config.config import V1 -from compose.config.config import V2_0 from compose.config.config import V2_1 @@ -34,7 +33,7 @@ def denormalize_config(config): del net_conf['external_name'] version = config.version - if version not in (V2_0, V2_1): + if version == V1: version = V2_1 return { diff --git a/compose/const.py b/compose/const.py index e7b1ae97..3da39855 100644 --- a/compose/const.py +++ b/compose/const.py @@ -17,15 +17,18 @@ LABEL_CONFIG_HASH = 'com.docker.compose.config-hash' COMPOSEFILE_V1 = '1' COMPOSEFILE_V2_0 = '2.0' COMPOSEFILE_V2_1 = '2.1' +COMPOSEFILE_V3_0 = '3.0' API_VERSIONS = { COMPOSEFILE_V1: '1.21', COMPOSEFILE_V2_0: '1.22', COMPOSEFILE_V2_1: '1.24', + COMPOSEFILE_V3_0: '1.24', } API_VERSION_TO_ENGINE_VERSION = { API_VERSIONS[COMPOSEFILE_V1]: '1.9.0', API_VERSIONS[COMPOSEFILE_V2_0]: '1.10.0', API_VERSIONS[COMPOSEFILE_V2_1]: '1.12.0', + API_VERSIONS[COMPOSEFILE_V3_0]: '1.12.0', } diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index f153bd95..e9a41691 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -285,6 +285,61 @@ class CLITestCase(DockerClientTestCase): 'volumes': {}, } + def test_config_v3(self): + self.base_dir = 'tests/fixtures/v3-full' + result = self.dispatch(['config']) + + assert yaml.load(result.stdout) == { + 'version': '3.0', + 'networks': {}, + 'volumes': {}, + 'services': { + 'web': { + 'image': 'busybox', + 'deploy': { + 'mode': 'replicated', + 'replicas': 6, + 'labels': ['FOO=BAR'], + 'update_config': { + 'parallelism': 3, + 'delay': '10s', + 'failure_action': 'continue', + 'monitor': '60s', + 'max_failure_ratio': 0.3, + }, + 'resources': { + 'limits': { + 'cpus': '0.001', + 'memory': '50M', + }, + 'reservations': { + 'cpus': '0.0001', + 'memory': '20M', + }, + }, + 'restart_policy': { + 'condition': 'on_failure', + 'delay': '5s', + 'max_attempts': 3, + 'window': '120s', + }, + 'placement': { + 'constraints': ['node=foo'], + }, + }, + + 'healthcheck': { + 'command': 'cat /etc/passwd', + 'interval': '10s', + 'timeout': '1s', + 'retries': 5, + }, + + 'stop_grace_period': '20s', + }, + }, + } + def test_ps(self): self.project.get_service('simple').create_container() result = self.dispatch(['ps']) diff --git a/tests/fixtures/v3-full/docker-compose.yml b/tests/fixtures/v3-full/docker-compose.yml new file mode 100644 index 00000000..1187dd7b --- /dev/null +++ b/tests/fixtures/v3-full/docker-compose.yml @@ -0,0 +1,37 @@ +version: "3" +services: + web: + image: busybox + + deploy: + mode: replicated + replicas: 6 + labels: [FOO=BAR] + update_config: + parallelism: 3 + delay: 10s + failure_action: continue + monitor: 60s + max_failure_ratio: 0.3 + resources: + limits: + cpus: '0.001' + memory: 50M + reservations: + cpus: '0.0001' + memory: 20M + restart_policy: + condition: on_failure + delay: 5s + max_attempts: 3 + window: 120s + placement: + constraints: [node=foo] + + healthcheck: + command: cat /etc/passwd + interval: 10s + timeout: 1s + retries: 5 + + stop_grace_period: 20s diff --git a/tests/unit/config/config_test.py b/tests/unit/config/config_test.py index 51c5e226..114145e1 100644 --- a/tests/unit/config/config_test.py +++ b/tests/unit/config/config_test.py @@ -18,6 +18,7 @@ from compose.config.config import resolve_environment from compose.config.config import V1 from compose.config.config import V2_0 from compose.config.config import V2_1 +from compose.config.config import V3_0 from compose.config.environment import Environment from compose.config.errors import ConfigurationError from compose.config.errors import VERSION_EXPLANATION @@ -156,9 +157,14 @@ class ConfigTest(unittest.TestCase): for version in ['2', '2.0']: cfg = config.load(build_config_details({'version': version})) assert cfg.version == V2_0 + cfg = config.load(build_config_details({'version': '2.1'})) assert cfg.version == V2_1 + for version in ['3', '3.0']: + cfg = config.load(build_config_details({'version': version})) + assert cfg.version == V3_0 + def test_v1_file_version(self): cfg = config.load(build_config_details({'web': {'image': 'busybox'}})) assert cfg.version == V1 From f75ef6862fe20f378c9702cda652deb34a49f946 Mon Sep 17 00:00:00 2001 From: Aanand Prasad Date: Thu, 10 Nov 2016 17:30:46 +0000 Subject: [PATCH 22/45] Warn if any services use 'deploy' Signed-off-by: Aanand Prasad --- compose/config/config.py | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/compose/config/config.py b/compose/config/config.py index fb77436d..940b9eb0 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -330,6 +330,14 @@ def load(config_details): for service_dict in service_dicts: match_named_volumes(service_dict, volumes) + services_using_deploy = [s for s in service_dicts if s.get('deploy')] + if services_using_deploy: + log.warn( + "Some services ({}) use the 'deploy' key, which will be ignored. " + "Compose does not support deploy configuration - use the experimental " + "`docker deploy` command to deploy to a swarm." + .format(", ".join(sorted(s['name'] for s in services_using_deploy)))) + return Config(main_file.version, service_dicts, volumes, networks) From 6cac48c0564f7f6b4e4d91055f50ec8c86505dd9 Mon Sep 17 00:00:00 2001 From: Daniel Nephin Date: Thu, 10 Nov 2016 17:38:19 -0500 Subject: [PATCH 23/45] Add a vendored and modified pytimeparse Signed-off-by: Daniel Nephin --- compose/timeparse.py | 96 ++++++++++++++++++++++++++++++++++++ tests/unit/timeparse_test.py | 52 +++++++++++++++++++ 2 files changed, 148 insertions(+) create mode 100644 compose/timeparse.py create mode 100644 tests/unit/timeparse_test.py diff --git a/compose/timeparse.py b/compose/timeparse.py new file mode 100644 index 00000000..16ef8a6d --- /dev/null +++ b/compose/timeparse.py @@ -0,0 +1,96 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- +''' +timeparse.py +(c) Will Roberts 1 February, 2014 + +This is a vendored and modified copy of: +github.com/wroberts/pytimeparse @ cc0550d + +It has been modified to mimic the behaviour of +https://golang.org/pkg/time/#ParseDuration +''' +# MIT LICENSE +# +# Permission is hereby granted, free of charge, to any person +# obtaining a copy of this software and associated documentation files +# (the "Software"), to deal in the Software without restriction, +# including without limitation the rights to use, copy, modify, merge, +# publish, distribute, sublicense, and/or sell copies of the Software, +# and to permit persons to whom the Software is furnished to do so, +# subject to the following conditions: +# +# The above copyright notice and this permission notice shall be +# included in all copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS +# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN +# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. +from __future__ import absolute_import +from __future__ import unicode_literals + +import re + +HOURS = r'(?P[\d.]+)h' +MINS = r'(?P[\d.]+)m' +SECS = r'(?P[\d.]+)s' +MILLI = r'(?P[\d.]+)ms' +MICRO = r'(?P[\d.]+)(?:us|µs)' +NANO = r'(?P[\d.]+)ns' + + +def opt(x): + return r'(?:{x})?'.format(x=x) + + +TIMEFORMAT = r'{HOURS}{MINS}{SECS}{MILLI}{MICRO}{NANO}'.format( + HOURS=opt(HOURS), + MINS=opt(MINS), + SECS=opt(SECS), + MILLI=opt(MILLI), + MICRO=opt(MICRO), + NANO=opt(NANO), +) + +MULTIPLIERS = dict([ + ('hours', 60 * 60), + ('mins', 60), + ('secs', 1), + ('milli', 1.0 / 1000), + ('micro', 1.0 / 1000.0 / 1000), + ('nano', 1.0 / 1000.0 / 1000.0 / 1000.0), +]) + + +def timeparse(sval): + """Parse a time expression, returning it as a number of seconds. If + possible, the return value will be an `int`; if this is not + possible, the return will be a `float`. Returns `None` if a time + expression cannot be parsed from the given string. + + Arguments: + - `sval`: the string value to parse + + >>> timeparse('1m24s') + 84 + >>> timeparse('1.2 minutes') + 72 + >>> timeparse('1.2 seconds') + 1.2 + """ + match = re.match(r'\s*' + TIMEFORMAT + r'\s*$', sval, re.I) + if not match or not match.group(0).strip(): + return + + mdict = match.groupdict() + return sum( + MULTIPLIERS[k] * cast(v) for (k, v) in mdict.items() if v is not None) + + +def cast(value): + return int(value, 10) if value.isdigit() else float(value) diff --git a/tests/unit/timeparse_test.py b/tests/unit/timeparse_test.py new file mode 100644 index 00000000..e9fe6c24 --- /dev/null +++ b/tests/unit/timeparse_test.py @@ -0,0 +1,52 @@ +from __future__ import absolute_import +from __future__ import unicode_literals + +from compose import timeparse + + +def test_milli(): + assert timeparse.timeparse('5ms') == 0.005 + + +def test_milli_float(): + assert timeparse.timeparse('50.5ms') == 0.0505 + + +def test_second_milli(): + assert timeparse.timeparse('200s5ms') == 200.005 + + +def test_second_milli_micro(): + assert timeparse.timeparse('200s5ms10us') == 200.00501 + + +def test_second(): + assert timeparse.timeparse('200s') == 200 + + +def test_second_as_float(): + assert timeparse.timeparse('20.5s') == 20.5 + + +def test_minute(): + assert timeparse.timeparse('32m') == 1920 + + +def test_hour_minute(): + assert timeparse.timeparse('2h32m') == 9120 + + +def test_minute_as_float(): + assert timeparse.timeparse('1.5m') == 90 + + +def test_hour_minute_second(): + assert timeparse.timeparse('5h34m56s') == 20096 + + +def test_invalid_with_space(): + assert timeparse.timeparse('5h 34m 56s') is None + + +def test_invalid_with_comma(): + assert timeparse.timeparse('5h,34m,56s') is None From 079c95c3401adb0f837e6b4e54132cdd41eada68 Mon Sep 17 00:00:00 2001 From: Daniel Nephin Date: Thu, 10 Nov 2016 17:47:05 -0500 Subject: [PATCH 24/45] Use stop grace period for container stop. Signed-off-by: Daniel Nephin --- compose/cli/main.py | 14 +++++++++----- compose/parallel.py | 4 ---- compose/project.py | 20 ++++++++++++++++---- compose/service.py | 23 ++++++++++++++++------- tests/unit/timeparse_test.py | 4 ++++ 5 files changed, 45 insertions(+), 20 deletions(-) diff --git a/compose/cli/main.py b/compose/cli/main.py index 08e58e37..cf53f6aa 100644 --- a/compose/cli/main.py +++ b/compose/cli/main.py @@ -24,7 +24,6 @@ from ..config import ConfigurationError from ..config import parse_environment from ..config.environment import Environment from ..config.serialize import serialize_config -from ..const import DEFAULT_TIMEOUT from ..const import IS_WINDOWS_PLATFORM from ..errors import StreamParseError from ..progress_stream import StreamOutputError @@ -726,7 +725,7 @@ class TopLevelCommand(object): -t, --timeout TIMEOUT Specify a shutdown timeout in seconds. (default: 10) """ - timeout = int(options.get('--timeout') or DEFAULT_TIMEOUT) + timeout = timeout_from_opts(options) for s in options['SERVICE=NUM']: if '=' not in s: @@ -760,7 +759,7 @@ class TopLevelCommand(object): -t, --timeout TIMEOUT Specify a shutdown timeout in seconds. (default: 10) """ - timeout = int(options.get('--timeout') or DEFAULT_TIMEOUT) + timeout = timeout_from_opts(options) self.project.stop(service_names=options['SERVICE'], timeout=timeout) def restart(self, options): @@ -773,7 +772,7 @@ class TopLevelCommand(object): -t, --timeout TIMEOUT Specify a shutdown timeout in seconds. (default: 10) """ - timeout = int(options.get('--timeout') or DEFAULT_TIMEOUT) + timeout = timeout_from_opts(options) containers = self.project.restart(service_names=options['SERVICE'], timeout=timeout) exit_if(not containers, 'No containers to restart', 1) @@ -831,7 +830,7 @@ class TopLevelCommand(object): start_deps = not options['--no-deps'] cascade_stop = options['--abort-on-container-exit'] service_names = options['SERVICE'] - timeout = int(options.get('--timeout') or DEFAULT_TIMEOUT) + timeout = timeout_from_opts(options) remove_orphans = options['--remove-orphans'] detached = options.get('-d') @@ -896,6 +895,11 @@ def convergence_strategy_from_opts(options): return ConvergenceStrategy.changed +def timeout_from_opts(options): + timeout = options.get('--timeout') + return None if timeout is None else int(timeout) + + def image_type_from_opt(flag, value): if not value: return ImageType.none diff --git a/compose/parallel.py b/compose/parallel.py index 7ac66b37..26718872 100644 --- a/compose/parallel.py +++ b/compose/parallel.py @@ -248,7 +248,3 @@ def parallel_unpause(containers, options): def parallel_kill(containers, options): parallel_operation(containers, 'kill', options, 'Killing') - - -def parallel_restart(containers, options): - parallel_operation(containers, 'restart', options, 'Restarting') diff --git a/compose/project.py b/compose/project.py index 60647fe9..eef2f3b8 100644 --- a/compose/project.py +++ b/compose/project.py @@ -14,7 +14,6 @@ from .config import ConfigurationError from .config.config import V1 from .config.sort_services import get_container_name_from_network_mode from .config.sort_services import get_service_name_from_network_mode -from .const import DEFAULT_TIMEOUT from .const import IMAGE_EVENTS from .const import LABEL_ONE_OFF from .const import LABEL_PROJECT @@ -250,7 +249,7 @@ class Project(object): parallel.parallel_execute( containers, - operator.methodcaller('stop', **options), + self.build_container_operation_with_timeout_func('stop', options), operator.attrgetter('name'), 'Stopping', get_deps) @@ -291,7 +290,12 @@ class Project(object): def restart(self, service_names=None, **options): containers = self.containers(service_names, stopped=True) - parallel.parallel_restart(containers, options) + + parallel.parallel_execute( + containers, + self.build_container_operation_with_timeout_func('restart', options), + operator.attrgetter('name'), + 'Restarting') return containers def build(self, service_names=None, no_cache=False, pull=False, force_rm=False): @@ -365,7 +369,7 @@ class Project(object): start_deps=True, strategy=ConvergenceStrategy.changed, do_build=BuildAction.none, - timeout=DEFAULT_TIMEOUT, + timeout=None, detached=False, remove_orphans=False): @@ -506,6 +510,14 @@ class Project(object): dep_services.append(service) return acc + dep_services + def build_container_operation_with_timeout_func(self, operation, options): + def container_operation_with_timeout(container): + if options.get('timeout') is None: + service = self.get_service(container.service) + options['timeout'] = service.stop_timeout(None) + return getattr(container, operation)(**options) + return container_operation_with_timeout + def get_volumes_from(project, service_dict): volumes_from = service_dict.pop('volumes_from', None) diff --git a/compose/service.py b/compose/service.py index ad426706..39737694 100644 --- a/compose/service.py +++ b/compose/service.py @@ -17,6 +17,7 @@ from docker.utils.ports import split_port from . import __version__ from . import progress_stream +from . import timeparse from .config import DOCKER_CONFIG_KEYS from .config import merge_environment from .config.types import VolumeSpec @@ -169,7 +170,7 @@ class Service(object): self.start_container_if_stopped(c, **options) return containers - def scale(self, desired_num, timeout=DEFAULT_TIMEOUT): + def scale(self, desired_num, timeout=None): """ Adjusts the number of containers to the specified number and ensures they are running. @@ -196,7 +197,7 @@ class Service(object): return container def stop_and_remove(container): - container.stop(timeout=timeout) + container.stop(timeout=self.stop_timeout(timeout)) container.remove() running_containers = self.containers(stopped=False) @@ -374,7 +375,7 @@ class Service(object): def execute_convergence_plan(self, plan, - timeout=DEFAULT_TIMEOUT, + timeout=None, detached=False, start=True): (action, containers) = plan @@ -421,7 +422,7 @@ class Service(object): def recreate_container( self, container, - timeout=DEFAULT_TIMEOUT, + timeout=None, attach_logs=False, start_new_container=True): """Recreate a container. @@ -432,7 +433,7 @@ class Service(object): """ log.info("Recreating %s" % container.name) - container.stop(timeout=timeout) + container.stop(timeout=self.stop_timeout(timeout)) container.rename_to_tmp_name() new_container = self.create_container( previous_container=container, @@ -446,6 +447,14 @@ class Service(object): container.remove() return new_container + def stop_timeout(self, timeout): + if timeout is not None: + return timeout + timeout = timeparse.timeparse(self.options.get('stop_grace_period') or '') + if timeout is not None: + return timeout + return DEFAULT_TIMEOUT + def start_container_if_stopped(self, container, attach_logs=False, quiet=False): if not container.is_running: if not quiet: @@ -483,10 +492,10 @@ class Service(object): link_local_ips=netdefs.get('link_local_ips', None), ) - def remove_duplicate_containers(self, timeout=DEFAULT_TIMEOUT): + def remove_duplicate_containers(self, timeout=None): for c in self.duplicate_containers(): log.info('Removing %s' % c.name) - c.stop(timeout=timeout) + c.stop(timeout=self.stop_timeout(timeout)) c.remove() def duplicate_containers(self): diff --git a/tests/unit/timeparse_test.py b/tests/unit/timeparse_test.py index e9fe6c24..9915932c 100644 --- a/tests/unit/timeparse_test.py +++ b/tests/unit/timeparse_test.py @@ -50,3 +50,7 @@ def test_invalid_with_space(): def test_invalid_with_comma(): assert timeparse.timeparse('5h,34m,56s') is None + + +def test_invalid_with_empty_string(): + assert timeparse.timeparse('') is None From b93211881b913091501a76d062f033754a653740 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Wed, 16 Nov 2016 13:35:29 -0800 Subject: [PATCH 25/45] Changelog update Signed-off-by: Joffrey F --- CHANGELOG.md | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 100 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8ec7d5b5..17487890 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,106 @@ Change log ========== +1.9.0 (2016-11-16) +----------------- + +**Breaking changes** + +- When using Compose with Docker Toolbox/Machine on Windows, volume paths are + no longer converted from `C:\Users` to `/c/Users`-style by default. To + re-enable this conversion so that your volumes keep working, set the + environment variable `COMPOSE_CONVERT_WINDOWS_PATHS=1`. Users of + Docker for Windows are not affected and do not need to set the variable. + +New Features + +- Interactive mode for `docker-compose run` and `docker-compose exec` is + now supported on Windows platforms. Please note that the `docker` binary + is required to be present on the system for this feature to work. + +- Introduced version 2.1 of the `docker-compose.yml` specification. This + version requires to be used with Docker Engine 1.12 or above. + - Added support for setting volume labels and network labels in + `docker-compose.yml`. + - Added support for the `isolation` parameter in service definitions. + - Added support for link-local IPs in the service networks definitions. + - Added support for shell-style inline defaults in variable interpolation. + The supported forms are `${FOO-default}` (fall back if FOO is unset) and + `${FOO:-default}` (fall back if FOO is unset or empty). + +- Added support for the `group_add` and `oom_score_adj` parameters in + service definitions. + +- Added support for the `internal` and `enable_ipv6` parameters in network + definitions. + +- Compose now defaults to using the `npipe` protocol on Windows. + +- Overriding a `logging` configuration will now properly merge the `options` + mappings if the `driver` values do not conflict. + +Bug Fixes + +- Fixed several bugs related to `npipe` protocol support on Windows. + +- Fixed an issue with Windows paths being incorrectly converted when + using Docker on Windows Server. + +- Fixed a bug where an empty `restart` value would sometimes result in an + exception being raised. + +- Fixed an issue where service logs containing unicode characters would + sometimes cause an error to occur. + +- Fixed a bug where unicode values in environment variables would sometimes + raise a unicode exception when retrieved. + +- Fixed an issue where Compose would incorrectly detect a configuration + mismatch for overlay networks. + + +1.8.1 (2016-09-22) +----------------- + +Bug Fixes + +- Fixed a bug where users using a credentials store were not able + to access their private images. + +- Fixed a bug where users using identity tokens to authenticate + were not able to access their private images. + +- Fixed a bug where an `HttpHeaders` entry in the docker configuration + file would cause Compose to crash when trying to build an image. + +- Fixed a few bugs related to the handling of Windows paths in volume + binding declarations. + +- Fixed a bug where Compose would sometimes crash while trying to + read a streaming response from the engine. + +- Fixed an issue where Compose would crash when encountering an API error + while streaming container logs. + +- Fixed an issue where Compose would erroneously try to output logs from + drivers not handled by the Engine's API. + +- Fixed a bug where options from the `docker-machine config` command would + not be properly interpreted by Compose. + +- Fixed a bug where the connection to the Docker Engine would + sometimes fail when running a large number of services simultaneously. + +- Fixed an issue where Compose would sometimes print a misleading + suggestion message when running the `bundle` command. + +- Fixed a bug where connection errors would not be handled properly by + Compose during the project initialization phase. + +- Fixed a bug where a misleading error would appear when encountering + a connection timeout. + + 1.8.0 (2016-06-14) ----------------- From 716a6baa59c62266af0bb7628ec88586f470140c Mon Sep 17 00:00:00 2001 From: Aanand Prasad Date: Mon, 14 Nov 2016 18:31:38 +0000 Subject: [PATCH 26/45] Implement 'healthcheck' option Signed-off-by: Aanand Prasad --- compose/config/config.py | 29 +++++++++++ compose/config/config_schema_v3.0.json | 5 +- compose/service.py | 4 +- compose/utils.py | 16 ++++++ requirements.txt | 2 +- tests/acceptance/cli_test.py | 50 +++++++++++++++++-- tests/fixtures/healthcheck/docker-compose.yml | 24 +++++++++ tests/fixtures/v3-full/docker-compose.yml | 2 +- tests/unit/config/config_test.py | 49 ++++++++++++++++++ 9 files changed, 172 insertions(+), 9 deletions(-) create mode 100644 tests/fixtures/healthcheck/docker-compose.yml diff --git a/compose/config/config.py b/compose/config/config.py index 940b9eb0..e5a37e84 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -17,6 +17,7 @@ from ..const import COMPOSEFILE_V2_0 as V2_0 from ..const import COMPOSEFILE_V2_1 as V2_1 from ..const import COMPOSEFILE_V3_0 as V3_0 from ..utils import build_string_dict +from ..utils import parse_nanoseconds_int from ..utils import splitdrive from .environment import env_vars_from_file from .environment import Environment @@ -65,6 +66,7 @@ DOCKER_CONFIG_KEYS = [ 'extra_hosts', 'group_add', 'hostname', + 'healthcheck', 'image', 'ipc', 'labels', @@ -642,6 +644,10 @@ def process_service(service_config): if 'extra_hosts' in service_dict: service_dict['extra_hosts'] = parse_extra_hosts(service_dict['extra_hosts']) + if 'healthcheck' in service_dict: + service_dict['healthcheck'] = process_healthcheck( + service_dict['healthcheck'], service_config.name) + for field in ['dns', 'dns_search', 'tmpfs']: if field in service_dict: service_dict[field] = to_list(service_dict[field]) @@ -649,6 +655,29 @@ def process_service(service_config): return service_dict +def process_healthcheck(raw, service_name): + hc = {} + + if raw.get('disable'): + if len(raw) > 1: + raise ConfigurationError( + 'Service "{}" defines an invalid healthcheck: ' + '"disable: true" cannot be combined with other options' + .format(service_name)) + hc['test'] = ['NONE'] + elif 'test' in raw: + hc['test'] = raw['test'] + + if 'interval' in raw: + hc['interval'] = parse_nanoseconds_int(raw['interval']) + if 'timeout' in raw: + hc['timeout'] = parse_nanoseconds_int(raw['timeout']) + if 'retries' in raw: + hc['retries'] = raw['retries'] + + return hc + + def finalize_service(service_config, service_names, version, environment): service_dict = dict(service_config.config) diff --git a/compose/config/config_schema_v3.0.json b/compose/config/config_schema_v3.0.json index 9ac31b1f..4edd8dd4 100644 --- a/compose/config/config_schema_v3.0.json +++ b/compose/config/config_schema_v3.0.json @@ -205,12 +205,13 @@ "interval": {"type":"string"}, "timeout": {"type":"string"}, "retries": {"type": "number"}, - "command": { + "test": { "oneOf": [ {"type": "string"}, {"type": "array", "items": {"type": "string"}} ] - } + }, + "disable": {"type": "boolean"} }, "additionalProperties": false }, diff --git a/compose/service.py b/compose/service.py index 39737694..cf52d489 100644 --- a/compose/service.py +++ b/compose/service.py @@ -17,7 +17,6 @@ from docker.utils.ports import split_port from . import __version__ from . import progress_stream -from . import timeparse from .config import DOCKER_CONFIG_KEYS from .config import merge_environment from .config.types import VolumeSpec @@ -35,6 +34,7 @@ from .parallel import parallel_start from .progress_stream import stream_output from .progress_stream import StreamOutputError from .utils import json_hash +from .utils import parse_seconds_float log = logging.getLogger(__name__) @@ -450,7 +450,7 @@ class Service(object): def stop_timeout(self, timeout): if timeout is not None: return timeout - timeout = timeparse.timeparse(self.options.get('stop_grace_period') or '') + timeout = parse_seconds_float(self.options.get('stop_grace_period')) if timeout is not None: return timeout return DEFAULT_TIMEOUT diff --git a/compose/utils.py b/compose/utils.py index 8f05e308..b8bdf732 100644 --- a/compose/utils.py +++ b/compose/utils.py @@ -11,6 +11,7 @@ import ntpath import six from .errors import StreamParseError +from .timeparse import timeparse json_decoder = json.JSONDecoder() @@ -107,6 +108,21 @@ def microseconds_from_time_nano(time_nano): return int(time_nano % 1000000000 / 1000) +def nanoseconds_from_time_seconds(time_seconds): + return time_seconds * 1000000000 + + +def parse_seconds_float(value): + return timeparse(value or '') + + +def parse_nanoseconds_int(value): + parsed = timeparse(value or '') + if parsed is None: + return None + return int(parsed * 1000000000) + + def build_string_dict(source_dict): return dict((k, str(v if v is not None else '')) for k, v in source_dict.items()) diff --git a/requirements.txt b/requirements.txt index 933146c7..63469799 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,11 +1,11 @@ PyYAML==3.11 backports.ssl-match-hostname==3.5.0.1; python_version < '3' cached-property==1.2.0 -docker-py==1.10.6 dockerpty==0.4.1 docopt==0.6.1 enum34==1.0.4; python_version < '3.4' functools32==3.2.3.post2; python_version < '3.2' +git+https://github.com/docker/docker-py.git@2ff7371ae7703033f981e1b137a3be0caf7a4f9c#egg=docker-py ipaddress==1.0.16 jsonschema==2.5.1 pypiwin32==219; sys_platform == 'win32' diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index e9a41691..97518f4f 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -21,6 +21,7 @@ from .. import mock from compose.cli.command import get_project from compose.container import Container from compose.project import OneOffFilter +from compose.utils import nanoseconds_from_time_seconds from tests.integration.testcases import DockerClientTestCase from tests.integration.testcases import get_links from tests.integration.testcases import pull_busybox @@ -329,9 +330,9 @@ class CLITestCase(DockerClientTestCase): }, 'healthcheck': { - 'command': 'cat /etc/passwd', - 'interval': '10s', - 'timeout': '1s', + 'test': 'cat /etc/passwd', + 'interval': 10000000000, + 'timeout': 1000000000, 'retries': 5, }, @@ -925,6 +926,49 @@ class CLITestCase(DockerClientTestCase): assert foo_container.get('HostConfig.NetworkMode') == \ 'container:{}'.format(bar_container.id) + def test_up_with_healthcheck(self): + def wait_on_health_status(container, status): + def condition(): + container.inspect() + return container.get('State.Health.Status') == status + + return wait_on_condition(condition, delay=0.5) + + self.base_dir = 'tests/fixtures/healthcheck' + self.dispatch(['up', '-d'], None) + + passes = self.project.get_service('passes') + passes_container = passes.containers()[0] + + assert passes_container.get('Config.Healthcheck') == { + "Test": ["CMD-SHELL", "/bin/true"], + "Interval": nanoseconds_from_time_seconds(1), + "Timeout": nanoseconds_from_time_seconds(30*60), + "Retries": 1, + } + + wait_on_health_status(passes_container, 'healthy') + + fails = self.project.get_service('fails') + fails_container = fails.containers()[0] + + assert fails_container.get('Config.Healthcheck') == { + "Test": ["CMD", "/bin/false"], + "Interval": nanoseconds_from_time_seconds(2.5), + "Retries": 2, + } + + wait_on_health_status(fails_container, 'unhealthy') + + disabled = self.project.get_service('disabled') + disabled_container = disabled.containers()[0] + + assert disabled_container.get('Config.Healthcheck') == { + "Test": ["NONE"], + } + + assert 'Health' not in disabled_container.get('State') + def test_up_with_no_deps(self): self.base_dir = 'tests/fixtures/links-composefile' self.dispatch(['up', '-d', '--no-deps', 'web'], None) diff --git a/tests/fixtures/healthcheck/docker-compose.yml b/tests/fixtures/healthcheck/docker-compose.yml new file mode 100644 index 00000000..2c45b8d8 --- /dev/null +++ b/tests/fixtures/healthcheck/docker-compose.yml @@ -0,0 +1,24 @@ +version: "3" +services: + passes: + image: busybox + command: top + healthcheck: + test: "/bin/true" + interval: 1s + timeout: 30m + retries: 1 + + fails: + image: busybox + command: top + healthcheck: + test: ["CMD", "/bin/false"] + interval: 2.5s + retries: 2 + + disabled: + image: busybox + command: top + healthcheck: + disable: true diff --git a/tests/fixtures/v3-full/docker-compose.yml b/tests/fixtures/v3-full/docker-compose.yml index 1187dd7b..b4d1b642 100644 --- a/tests/fixtures/v3-full/docker-compose.yml +++ b/tests/fixtures/v3-full/docker-compose.yml @@ -29,7 +29,7 @@ services: constraints: [node=foo] healthcheck: - command: cat /etc/passwd + test: cat /etc/passwd interval: 10s timeout: 1s retries: 5 diff --git a/tests/unit/config/config_test.py b/tests/unit/config/config_test.py index 114145e1..f7df3aee 100644 --- a/tests/unit/config/config_test.py +++ b/tests/unit/config/config_test.py @@ -24,6 +24,7 @@ from compose.config.errors import ConfigurationError from compose.config.errors import VERSION_EXPLANATION from compose.config.types import VolumeSpec from compose.const import IS_WINDOWS_PLATFORM +from compose.utils import nanoseconds_from_time_seconds from tests import mock from tests import unittest @@ -3171,6 +3172,54 @@ class BuildPathTest(unittest.TestCase): assert 'build path' in exc.exconly() +class HealthcheckTest(unittest.TestCase): + def test_healthcheck(self): + service_dict = make_service_dict( + 'test', + {'healthcheck': { + 'test': ['CMD', 'true'], + 'interval': '1s', + 'timeout': '1m', + 'retries': 3, + }}, + '.', + ) + + assert service_dict['healthcheck'] == { + 'test': ['CMD', 'true'], + 'interval': nanoseconds_from_time_seconds(1), + 'timeout': nanoseconds_from_time_seconds(60), + 'retries': 3, + } + + def test_disable(self): + service_dict = make_service_dict( + 'test', + {'healthcheck': { + 'disable': True, + }}, + '.', + ) + + assert service_dict['healthcheck'] == { + 'test': ['NONE'], + } + + def test_disable_with_other_config_is_invalid(self): + with pytest.raises(ConfigurationError) as excinfo: + make_service_dict( + 'invalid-healthcheck', + {'healthcheck': { + 'disable': True, + 'interval': '1s', + }}, + '.', + ) + + assert 'invalid-healthcheck' in excinfo.exconly() + assert 'disable' in excinfo.exconly() + + class GetDefaultConfigFilesTestCase(unittest.TestCase): files = [ From c26a2afaf36443fbc4ea813e34027bdb05ded0e1 Mon Sep 17 00:00:00 2001 From: Daniel Nephin Date: Mon, 21 Nov 2016 16:34:49 -0500 Subject: [PATCH 27/45] Update messages about docker stack deploy. Signed-off-by: Daniel Nephin --- compose/config/config.py | 4 ++-- compose/project.py | 4 +--- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/compose/config/config.py b/compose/config/config.py index 940b9eb0..5215b361 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -334,8 +334,8 @@ def load(config_details): if services_using_deploy: log.warn( "Some services ({}) use the 'deploy' key, which will be ignored. " - "Compose does not support deploy configuration - use the experimental " - "`docker deploy` command to deploy to a swarm." + "Compose does not support deploy configuration - use " + "`docker stack deploy` to deploy to a swarm." .format(", ".join(sorted(s['name'] for s in services_using_deploy)))) return Config(main_file.version, service_dicts, volumes, networks) diff --git a/compose/project.py b/compose/project.py index eef2f3b8..0178bbab 100644 --- a/compose/project.py +++ b/compose/project.py @@ -559,9 +559,7 @@ def warn_for_swarm_mode(client): "Compose does not use swarm mode to deploy services to multiple nodes in a swarm. " "All containers will be scheduled on the current node.\n\n" "To deploy your application across the swarm, " - "use the bundle feature of the Docker experimental build.\n\n" - "More info:\n" - "https://docs.docker.com/compose/bundles\n" + "use `docker stack deploy`.\n" ) From 024b5dd6da4c96822ea84e31cedcb88a8949a245 Mon Sep 17 00:00:00 2001 From: Thomas Grainger Date: Tue, 22 Nov 2016 11:14:56 +0000 Subject: [PATCH 28/45] case PyPI correctly Signed-off-by: Thomas Grainger --- CHANGELOG.md | 2 +- script/release/push-release | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 17487890..9780df98 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -814,7 +814,7 @@ Fig has been renamed to Docker Compose, or just Compose for short. This has seve - The command you type is now `docker-compose`, not `fig`. - You should rename your fig.yml to docker-compose.yml. -- If you’re installing via PyPi, the package is now `docker-compose`, so install it with `pip install docker-compose`. +- If you’re installing via PyPI, the package is now `docker-compose`, so install it with `pip install docker-compose`. Besides that, there’s a lot of new stuff in this release: diff --git a/script/release/push-release b/script/release/push-release index 33d0d777..d5ae3de9 100755 --- a/script/release/push-release +++ b/script/release/push-release @@ -54,7 +54,7 @@ git push $GITHUB_REPO $VERSION echo "Uploading the docker image" docker push docker/compose:$VERSION -echo "Uploading sdist to pypi" +echo "Uploading sdist to PyPI" pandoc -f markdown -t rst README.md -o README.rst sed -i -e 's/logo.png?raw=true/https:\/\/github.com\/docker\/compose\/raw\/master\/logo.png?raw=true/' README.rst ./script/build/write-git-sha From dc9184a90fd22b13265162c4bc8e04c9867a768d Mon Sep 17 00:00:00 2001 From: Ian Campbell Date: Fri, 25 Nov 2016 10:12:22 +0000 Subject: [PATCH 29/45] progress_stream: Avoid undefined ANSI escape codes The ANSI escape codes \e[0A (cursor up 0 lines) and \e[0B (cursor down 0 lines) are not well defined and are treated differently by different terminals. In particular xterm treats 0 as a missing parameter and therefore defaults to 1, whereas rxvt-unicode treats these escapes as a request to move 0 lines. However the use of these codes is unnecessary and were really just hiding the fact that we were not correctly computing diff when adding a new line. Having added the new line to the ids map and output the corresponding \n the correct diff would be 1 and not 0 (which xterm interprets as 1) as currently. Rather than changing the hardcoded 0 to a 1 pull the diff calculation out and always do it since it produces the correct answer in both cases. This fixes similar corruption when compose is pulling an image to that seen with `docker pull` and rxvt-unicode (and likely other terminals in that family) seen in docker/docker#28111. This is the same as the fix made to Docker's pkg/jsonmessage in https://github.com/docker/docker/pull/28238 (and I have shamelessly ripped off most of this commit message from there). Signed-off-by: Ian Campbell --- compose/progress_stream.py | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/compose/progress_stream.py b/compose/progress_stream.py index a0f5601f..5314f89f 100644 --- a/compose/progress_stream.py +++ b/compose/progress_stream.py @@ -32,12 +32,11 @@ def stream_output(output, stream): if not image_id: continue - if image_id in lines: - diff = len(lines) - lines[image_id] - else: + if image_id not in lines: lines[image_id] = len(lines) stream.write("\n") - diff = 0 + + diff = len(lines) - lines[image_id] # move cursor up `diff` rows stream.write("%c[%dA" % (27, diff)) From 6aacf5142750654a9b1f165223478fab3a424eeb Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Fri, 2 Dec 2016 16:10:15 -0800 Subject: [PATCH 30/45] Win32 interactive run - Connect container to networks before starting Signed-off-by: Joffrey F --- compose/cli/main.py | 1 + 1 file changed, 1 insertion(+) diff --git a/compose/cli/main.py b/compose/cli/main.py index cf53f6aa..c25ccbfa 100644 --- a/compose/cli/main.py +++ b/compose/cli/main.py @@ -988,6 +988,7 @@ def run_one_off_container(container_options, project, service, options): try: try: if IS_WINDOWS_PLATFORM: + service.connect_container_to_networks(container) exit_code = call_docker(["start", "--attach", "--interactive", container.id]) else: operation = RunOperation( From 62f8b1402e1bbe726cf83ea204077754385fe53a Mon Sep 17 00:00:00 2001 From: Yong Wen Chua Date: Mon, 5 Dec 2016 14:25:56 +0800 Subject: [PATCH 31/45] Implement `userns_mode` HostConfig for services Fixes #3349 This allows the key `userns_mode` to be used in service definitions. Since `userns_mode` requires API version > 1.23, this is only available in 2.1 and 3.0 versions of compose file Signed-off-by: Yong Wen Chua --- compose/config/config.py | 1 + compose/config/config_schema_v2.1.json | 1 + compose/config/config_schema_v3.0.json | 1 + compose/service.py | 4 +++- tests/integration/service_test.py | 13 +++++++++++++ 5 files changed, 19 insertions(+), 1 deletion(-) diff --git a/compose/config/config.py b/compose/config/config.py index 5215b361..b83b12bf 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -86,6 +86,7 @@ DOCKER_CONFIG_KEYS = [ 'stop_signal', 'tty', 'user', + 'userns_mode', 'volume_driver', 'volumes', 'volumes_from', diff --git a/compose/config/config_schema_v2.1.json b/compose/config/config_schema_v2.1.json index 3561a8cf..b5d614d6 100644 --- a/compose/config/config_schema_v2.1.json +++ b/compose/config/config_schema_v2.1.json @@ -217,6 +217,7 @@ } }, "user": {"type": "string"}, + "userns_mode": {"type": "string"}, "volumes": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, "volume_driver": {"type": "string"}, "volumes_from": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, diff --git a/compose/config/config_schema_v3.0.json b/compose/config/config_schema_v3.0.json index 9ac31b1f..297c60be 100644 --- a/compose/config/config_schema_v3.0.json +++ b/compose/config/config_schema_v3.0.json @@ -192,6 +192,7 @@ } }, "user": {"type": "string"}, + "userns_mode": {"type": "string"}, "volumes": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, "working_dir": {"type": "string"} }, diff --git a/compose/service.py b/compose/service.py index 39737694..6a62067b 100644 --- a/compose/service.py +++ b/compose/service.py @@ -64,6 +64,7 @@ DOCKER_START_KEYS = [ 'restart', 'security_opt', 'shm_size', + 'userns_mode', 'volumes_from', ] @@ -720,7 +721,8 @@ class Service(object): tmpfs=options.get('tmpfs'), oom_score_adj=options.get('oom_score_adj'), mem_swappiness=options.get('mem_swappiness'), - group_add=options.get('group_add') + group_add=options.get('group_add'), + userns_mode=options.get('userns_mode') ) # TODO: Add as an argument to create_host_config once it's supported diff --git a/tests/integration/service_test.py b/tests/integration/service_test.py index a5ca81ee..09758eee 100644 --- a/tests/integration/service_test.py +++ b/tests/integration/service_test.py @@ -30,6 +30,7 @@ from compose.service import ConvergencePlan from compose.service import ConvergenceStrategy from compose.service import NetworkMode from compose.service import Service +from tests.integration.testcases import v2_1_only from tests.integration.testcases import v2_only @@ -842,6 +843,18 @@ class ServiceTest(DockerClientTestCase): container = create_and_start_container(service) self.assertEqual(container.get('HostConfig.PidMode'), 'host') + @v2_1_only() + def test_userns_mode_none_defined(self): + service = self.create_service('web', userns_mode=None) + container = create_and_start_container(service) + self.assertEqual(container.get('HostConfig.UsernsMode'), '') + + @v2_1_only() + def test_userns_mode_host(self): + service = self.create_service('web', userns_mode='host') + container = create_and_start_container(service) + self.assertEqual(container.get('HostConfig.UsernsMode'), 'host') + def test_dns_no_value(self): service = self.create_service('web') container = create_and_start_container(service) From 4d0575355c1a0201ed7fc11063a4efbca6971a96 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Tue, 6 Dec 2016 17:14:05 -0800 Subject: [PATCH 32/45] Bump master version to 1.10.0dev Signed-off-by: Joffrey F --- compose/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/compose/__init__.py b/compose/__init__.py index 6e610652..6f05b282 100644 --- a/compose/__init__.py +++ b/compose/__init__.py @@ -1,4 +1,4 @@ from __future__ import absolute_import from __future__ import unicode_literals -__version__ = '1.9.0dev' +__version__ = '1.10.0dev' From e04a12b5ca9fb30d152ab6a83ae8c0ec15ac85b0 Mon Sep 17 00:00:00 2001 From: Daniel Nephin Date: Tue, 6 Dec 2016 17:01:35 -0500 Subject: [PATCH 33/45] Increase minimum version for v3. Signed-off-by: Daniel Nephin --- compose/const.py | 4 ++-- tests/acceptance/cli_test.py | 2 ++ tests/integration/testcases.py | 23 +++++++++++------------ 3 files changed, 15 insertions(+), 14 deletions(-) diff --git a/compose/const.py b/compose/const.py index 3da39855..ca8d7fe5 100644 --- a/compose/const.py +++ b/compose/const.py @@ -23,12 +23,12 @@ API_VERSIONS = { COMPOSEFILE_V1: '1.21', COMPOSEFILE_V2_0: '1.22', COMPOSEFILE_V2_1: '1.24', - COMPOSEFILE_V3_0: '1.24', + COMPOSEFILE_V3_0: '1.25', } API_VERSION_TO_ENGINE_VERSION = { API_VERSIONS[COMPOSEFILE_V1]: '1.9.0', API_VERSIONS[COMPOSEFILE_V2_0]: '1.10.0', API_VERSIONS[COMPOSEFILE_V2_1]: '1.12.0', - API_VERSIONS[COMPOSEFILE_V3_0]: '1.12.0', + API_VERSIONS[COMPOSEFILE_V3_0]: '1.13.0', } diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index e9a41691..0d5d5058 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -26,6 +26,7 @@ from tests.integration.testcases import get_links from tests.integration.testcases import pull_busybox from tests.integration.testcases import v2_1_only from tests.integration.testcases import v2_only +from tests.integration.testcases import v3_only ProcessResult = namedtuple('ProcessResult', 'stdout stderr') @@ -285,6 +286,7 @@ class CLITestCase(DockerClientTestCase): 'volumes': {}, } + @v3_only() def test_config_v3(self): self.base_dir = 'tests/fixtures/v3-full' result = self.dispatch(['config']) diff --git a/tests/integration/testcases.py b/tests/integration/testcases.py index c7743fb8..f6bc402b 100644 --- a/tests/integration/testcases.py +++ b/tests/integration/testcases.py @@ -45,11 +45,11 @@ def engine_max_version(): return V2_1 -def v2_only(): +def build_version_required_decorator(ignored_versions): def decorator(f): @functools.wraps(f) def wrapper(self, *args, **kwargs): - if engine_max_version() == V1: + if engine_max_version() in ignored_versions: skip("Engine version is too low") return return f(self, *args, **kwargs) @@ -58,17 +58,16 @@ def v2_only(): return decorator -def v2_1_only(): - def decorator(f): - @functools.wraps(f) - def wrapper(self, *args, **kwargs): - if engine_max_version() in (V1, V2_0): - skip('Engine version is too low') - return - return f(self, *args, **kwargs) - return wrapper +def v2_only(): + return build_version_required_decorator((V1,)) - return decorator + +def v2_1_only(): + return build_version_required_decorator((V1, V2_0)) + + +def v3_only(): + return build_version_required_decorator((V1, V2_0, V2_1)) class DockerClientTestCase(unittest.TestCase): From 04e5925a230d07a6ffd855b4c3812f5a6b54a523 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 5 Dec 2016 17:42:07 -0800 Subject: [PATCH 34/45] Use docker SDK 2.0 Signed-off-by: Joffrey F --- compose/cli/docker_client.py | 4 ++-- compose/network.py | 8 ++++---- compose/service.py | 2 +- requirements.txt | 2 +- setup.py | 2 +- tests/unit/bundle_test.py | 2 +- tests/unit/cli_test.py | 4 ++-- tests/unit/container_test.py | 2 +- tests/unit/project_test.py | 2 +- tests/unit/service_test.py | 10 +++++----- tests/unit/volume_test.py | 2 +- 11 files changed, 20 insertions(+), 20 deletions(-) diff --git a/compose/cli/docker_client.py b/compose/cli/docker_client.py index b196d303..018d2451 100644 --- a/compose/cli/docker_client.py +++ b/compose/cli/docker_client.py @@ -3,7 +3,7 @@ from __future__ import unicode_literals import logging -from docker import Client +from docker import APIClient from docker.errors import TLSParameterError from docker.tls import TLSConfig from docker.utils import kwargs_from_env @@ -71,4 +71,4 @@ def docker_client(environment, version=None, tls_config=None, host=None, kwargs['user_agent'] = generate_user_agent() - return Client(**kwargs) + return APIClient(**kwargs) diff --git a/compose/network.py b/compose/network.py index 3b57cb94..8a29b73e 100644 --- a/compose/network.py +++ b/compose/network.py @@ -4,8 +4,8 @@ from __future__ import unicode_literals import logging from docker.errors import NotFound -from docker.utils import create_ipam_config -from docker.utils import create_ipam_pool +from docker.types import IPAMConfig +from docker.types import IPAMPool from .config import ConfigurationError @@ -96,10 +96,10 @@ def create_ipam_config_from_dict(ipam_dict): if not ipam_dict: return None - return create_ipam_config( + return IPAMConfig( driver=ipam_dict.get('driver'), pool_configs=[ - create_ipam_pool( + IPAMPool( subnet=config.get('subnet'), iprange=config.get('ip_range'), gateway=config.get('gateway'), diff --git a/compose/service.py b/compose/service.py index cf52d489..c32788fe 100644 --- a/compose/service.py +++ b/compose/service.py @@ -11,7 +11,7 @@ import enum import six from docker.errors import APIError from docker.errors import NotFound -from docker.utils import LogConfig +from docker.types import LogConfig from docker.utils.ports import build_port_bindings from docker.utils.ports import split_port diff --git a/requirements.txt b/requirements.txt index 63469799..8f6fe169 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,11 +1,11 @@ PyYAML==3.11 backports.ssl-match-hostname==3.5.0.1; python_version < '3' cached-property==1.2.0 +docker==2.0.0 dockerpty==0.4.1 docopt==0.6.1 enum34==1.0.4; python_version < '3.4' functools32==3.2.3.post2; python_version < '3.2' -git+https://github.com/docker/docker-py.git@2ff7371ae7703033f981e1b137a3be0caf7a4f9c#egg=docker-py ipaddress==1.0.16 jsonschema==2.5.1 pypiwin32==219; sys_platform == 'win32' diff --git a/setup.py b/setup.py index 672ea80e..9dba2167 100644 --- a/setup.py +++ b/setup.py @@ -34,7 +34,7 @@ install_requires = [ 'requests >= 2.6.1, != 2.11.0, < 2.12', 'texttable >= 0.8.1, < 0.9', 'websocket-client >= 0.32.0, < 1.0', - 'docker-py >= 1.10.6, < 2.0', + 'docker >= 2.0.0, < 3.0', 'dockerpty >= 0.4.1, < 0.5', 'six >= 1.3.0, < 2', 'jsonschema >= 2.5.1, < 3', diff --git a/tests/unit/bundle_test.py b/tests/unit/bundle_test.py index 223b3b07..a279cab0 100644 --- a/tests/unit/bundle_test.py +++ b/tests/unit/bundle_test.py @@ -15,7 +15,7 @@ from compose.config.config import Config def mock_service(): return mock.create_autospec( service.Service, - client=mock.create_autospec(docker.Client), + client=mock.create_autospec(docker.APIClient), options={}) diff --git a/tests/unit/cli_test.py b/tests/unit/cli_test.py index 2c90b29b..f9b60bff 100644 --- a/tests/unit/cli_test.py +++ b/tests/unit/cli_test.py @@ -97,7 +97,7 @@ class CLITestCase(unittest.TestCase): @mock.patch('compose.cli.main.RunOperation', autospec=True) @mock.patch('compose.cli.main.PseudoTerminal', autospec=True) def test_run_interactive_passes_logs_false(self, mock_pseudo_terminal, mock_run_operation): - mock_client = mock.create_autospec(docker.Client) + mock_client = mock.create_autospec(docker.APIClient) project = Project.from_config( name='composetest', client=mock_client, @@ -128,7 +128,7 @@ class CLITestCase(unittest.TestCase): assert call_kwargs['logs'] is False def test_run_service_with_restart_always(self): - mock_client = mock.create_autospec(docker.Client) + mock_client = mock.create_autospec(docker.APIClient) project = Project.from_config( name='composetest', diff --git a/tests/unit/container_test.py b/tests/unit/container_test.py index 62e3aa2c..04f43016 100644 --- a/tests/unit/container_test.py +++ b/tests/unit/container_test.py @@ -98,7 +98,7 @@ class ContainerTest(unittest.TestCase): self.assertEqual(container.name_without_project, "custom_name_of_container") def test_inspect_if_not_inspected(self): - mock_client = mock.create_autospec(docker.Client) + mock_client = mock.create_autospec(docker.APIClient) container = Container(mock_client, dict(Id="the_id")) container.inspect_if_not_inspected() diff --git a/tests/unit/project_test.py b/tests/unit/project_test.py index 9569adc9..9a12438f 100644 --- a/tests/unit/project_test.py +++ b/tests/unit/project_test.py @@ -19,7 +19,7 @@ from compose.service import Service class ProjectTest(unittest.TestCase): def setUp(self): - self.mock_client = mock.create_autospec(docker.Client) + self.mock_client = mock.create_autospec(docker.APIClient) def test_from_config(self): config = Config( diff --git a/tests/unit/service_test.py b/tests/unit/service_test.py index 1d5aa10f..2d5b1761 100644 --- a/tests/unit/service_test.py +++ b/tests/unit/service_test.py @@ -34,7 +34,7 @@ from compose.service import warn_on_masked_volume class ServiceTest(unittest.TestCase): def setUp(self): - self.mock_client = mock.create_autospec(docker.Client) + self.mock_client = mock.create_autospec(docker.APIClient) def test_containers(self): service = Service('db', self.mock_client, 'myproject', image='foo') @@ -666,7 +666,7 @@ class ServiceTest(unittest.TestCase): class TestServiceNetwork(object): def test_connect_container_to_networks_short_aliase_exists(self): - mock_client = mock.create_autospec(docker.Client) + mock_client = mock.create_autospec(docker.APIClient) service = Service( 'db', mock_client, @@ -751,7 +751,7 @@ class NetTestCase(unittest.TestCase): def test_network_mode_service(self): container_id = 'bbbb' service_name = 'web' - mock_client = mock.create_autospec(docker.Client) + mock_client = mock.create_autospec(docker.APIClient) mock_client.containers.return_value = [ {'Id': container_id, 'Name': container_id, 'Image': 'abcd'}, ] @@ -765,7 +765,7 @@ class NetTestCase(unittest.TestCase): def test_network_mode_service_no_containers(self): service_name = 'web' - mock_client = mock.create_autospec(docker.Client) + mock_client = mock.create_autospec(docker.APIClient) mock_client.containers.return_value = [] service = Service(name=service_name, client=mock_client) @@ -783,7 +783,7 @@ def build_mount(destination, source, mode='rw'): class ServiceVolumesTest(unittest.TestCase): def setUp(self): - self.mock_client = mock.create_autospec(docker.Client) + self.mock_client = mock.create_autospec(docker.APIClient) def test_build_volume_binding(self): binding = build_volume_binding(VolumeSpec.parse('/outside:/inside', True)) diff --git a/tests/unit/volume_test.py b/tests/unit/volume_test.py index d7ad0792..24829192 100644 --- a/tests/unit/volume_test.py +++ b/tests/unit/volume_test.py @@ -10,7 +10,7 @@ from tests import mock @pytest.fixture def mock_client(): - return mock.create_autospec(docker.Client) + return mock.create_autospec(docker.APIClient) class TestVolume(object): From fb165d9c1505e2505f637cb2cede77e4d0731884 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Fri, 16 Dec 2016 12:21:59 -0800 Subject: [PATCH 35/45] Add v3_only marker to healthcheck test Signed-off-by: Joffrey F --- tests/acceptance/cli_test.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index 856b8f93..2d0ce715 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -316,8 +316,8 @@ class CLITestCase(DockerClientTestCase): 'memory': '50M', }, 'reservations': { - 'cpus': '0.0001', - 'memory': '20M', + 'cpus': '0.0001', + 'memory': '20M', }, }, 'restart_policy': { @@ -928,6 +928,7 @@ class CLITestCase(DockerClientTestCase): assert foo_container.get('HostConfig.NetworkMode') == \ 'container:{}'.format(bar_container.id) + @v3_only() def test_up_with_healthcheck(self): def wait_on_health_status(container, status): def condition(): @@ -945,7 +946,7 @@ class CLITestCase(DockerClientTestCase): assert passes_container.get('Config.Healthcheck') == { "Test": ["CMD-SHELL", "/bin/true"], "Interval": nanoseconds_from_time_seconds(1), - "Timeout": nanoseconds_from_time_seconds(30*60), + "Timeout": nanoseconds_from_time_seconds(30 * 60), "Retries": 1, } From e736151ee497a65df4e7e271ebd059731e250760 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 5 Dec 2016 16:11:21 -0800 Subject: [PATCH 36/45] Make created networks attachable for file format >=2.1 Signed-off-by: Joffrey F --- compose/network.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/compose/network.py b/compose/network.py index 8a29b73e..eb76e292 100644 --- a/compose/network.py +++ b/compose/network.py @@ -6,6 +6,7 @@ import logging from docker.errors import NotFound from docker.types import IPAMConfig from docker.types import IPAMPool +from docker.utils import version_gte from .config import ConfigurationError @@ -72,6 +73,7 @@ class Network(object): internal=self.internal, enable_ipv6=self.enable_ipv6, labels=self.labels, + attachable=version_gte(self.client._version, '1.24') or None ) def remove(self): From eb6441c8e337a1e797adee26b77c0c03465375a2 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 19 Dec 2016 20:35:09 -0800 Subject: [PATCH 37/45] Add sysctls option to 3.0 schema Signed-off-by: Joffrey F --- compose/config/config_schema_v3.0.json | 1 + 1 file changed, 1 insertion(+) diff --git a/compose/config/config_schema_v3.0.json b/compose/config/config_schema_v3.0.json index 8b8ceac0..1f93347f 100644 --- a/compose/config/config_schema_v3.0.json +++ b/compose/config/config_schema_v3.0.json @@ -167,6 +167,7 @@ "restart": {"type": "string"}, "security_opt": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, "shm_size": {"type": ["number", "string"]}, + "sysctls": {"$ref": "#/definitions/list_or_dict"}, "stdin_open": {"type": "boolean"}, "stop_signal": {"type": "string"}, "stop_grace_period": {"type": "string", "format": "duration"}, From 346802715dd3869c2a33c5d361ec8a56fa3352c5 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 24 Oct 2016 16:27:47 -0700 Subject: [PATCH 38/45] Use colorama to enable colored output on Windows Signed-off-by: Joffrey F --- compose/cli/colors.py | 4 ++++ requirements.txt | 1 + setup.py | 1 + 3 files changed, 6 insertions(+) diff --git a/compose/cli/colors.py b/compose/cli/colors.py index 3c18886f..6677a376 100644 --- a/compose/cli/colors.py +++ b/compose/cli/colors.py @@ -1,5 +1,8 @@ from __future__ import absolute_import from __future__ import unicode_literals + +import colorama + NAMES = [ 'grey', 'red', @@ -30,6 +33,7 @@ def make_color_fn(code): return lambda s: ansi_color(code, s) +colorama.init() for (name, code) in get_pairs(): globals()[name] = make_color_fn(code) diff --git a/requirements.txt b/requirements.txt index 8f6fe169..bae5d9ea 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,6 +1,7 @@ PyYAML==3.11 backports.ssl-match-hostname==3.5.0.1; python_version < '3' cached-property==1.2.0 +colorama==0.3.7 docker==2.0.0 dockerpty==0.4.1 docopt==0.6.1 diff --git a/setup.py b/setup.py index 9dba2167..8b4cf709 100644 --- a/setup.py +++ b/setup.py @@ -29,6 +29,7 @@ def find_version(*file_paths): install_requires = [ 'cached-property >= 1.2.0, < 2', + 'colorama >= 0.3.7, < 0.4', 'docopt >= 0.6.1, < 0.7', 'PyYAML >= 3.10, < 4', 'requests >= 2.6.1, != 2.11.0, < 2.12', From ba47fb99ba7670fc8435af0a958a3ed01188711a Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 5 Dec 2016 18:21:10 -0800 Subject: [PATCH 39/45] Add default labels to networks and volumes created by Compose Signed-off-by: Joffrey F --- compose/const.py | 2 ++ compose/network.py | 18 ++++++++++++++++-- compose/volume.py | 16 +++++++++++++++- tests/acceptance/cli_test.py | 8 ++++---- tests/integration/network_test.py | 17 +++++++++++++++++ tests/integration/project_test.py | 7 ++++--- tests/integration/volume_test.py | 10 ++++++++++ 7 files changed, 68 insertions(+), 10 deletions(-) create mode 100644 tests/integration/network_test.py diff --git a/compose/const.py b/compose/const.py index ca8d7fe5..1b1be5c7 100644 --- a/compose/const.py +++ b/compose/const.py @@ -11,7 +11,9 @@ LABEL_CONTAINER_NUMBER = 'com.docker.compose.container-number' LABEL_ONE_OFF = 'com.docker.compose.oneoff' LABEL_PROJECT = 'com.docker.compose.project' LABEL_SERVICE = 'com.docker.compose.service' +LABEL_NETWORK = 'com.docker.compose.network' LABEL_VERSION = 'com.docker.compose.version' +LABEL_VOLUME = 'com.docker.compose.volume' LABEL_CONFIG_HASH = 'com.docker.compose.config-hash' COMPOSEFILE_V1 = '1' diff --git a/compose/network.py b/compose/network.py index eb76e292..d98f68d2 100644 --- a/compose/network.py +++ b/compose/network.py @@ -7,8 +7,11 @@ from docker.errors import NotFound from docker.types import IPAMConfig from docker.types import IPAMPool from docker.utils import version_gte +from docker.utils import version_lt from .config import ConfigurationError +from .const import LABEL_NETWORK +from .const import LABEL_PROJECT log = logging.getLogger(__name__) @@ -72,8 +75,8 @@ class Network(object): ipam=self.ipam, internal=self.internal, enable_ipv6=self.enable_ipv6, - labels=self.labels, - attachable=version_gte(self.client._version, '1.24') or None + labels=self._labels, + attachable=version_gte(self.client._version, '1.24') or None, ) def remove(self): @@ -93,6 +96,17 @@ class Network(object): return self.external_name return '{0}_{1}'.format(self.project, self.name) + @property + def _labels(self): + if version_lt(self.client._version, '1.23'): + return None + labels = self.labels.copy() if self.labels else {} + labels.update({ + LABEL_PROJECT: self.project, + LABEL_NETWORK: self.name, + }) + return labels + def create_ipam_config_from_dict(ipam_dict): if not ipam_dict: diff --git a/compose/volume.py b/compose/volume.py index 1fd1d51c..ab6a88fa 100644 --- a/compose/volume.py +++ b/compose/volume.py @@ -4,8 +4,11 @@ from __future__ import unicode_literals import logging from docker.errors import NotFound +from docker.utils import version_lt from .config import ConfigurationError +from .const import LABEL_PROJECT +from .const import LABEL_VOLUME log = logging.getLogger(__name__) @@ -23,7 +26,7 @@ class Volume(object): def create(self): return self.client.create_volume( - self.full_name, self.driver, self.driver_opts, labels=self.labels + self.full_name, self.driver, self.driver_opts, labels=self._labels ) def remove(self): @@ -53,6 +56,17 @@ class Volume(object): return self.external_name return '{0}_{1}'.format(self.project, self.name) + @property + def _labels(self): + if version_lt(self.client._version, '1.23'): + return None + labels = self.labels.copy() if self.labels else {} + labels.update({ + LABEL_PROJECT: self.project, + LABEL_VOLUME: self.name, + }) + return labels + class ProjectVolumes(object): diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index 2d0ce715..b9766226 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -850,8 +850,8 @@ class CLITestCase(DockerClientTestCase): ] assert [n['Name'] for n in networks] == [network_with_label] - - assert networks[0]['Labels'] == {'label_key': 'label_val'} + assert 'label_key' in networks[0]['Labels'] + assert networks[0]['Labels']['label_key'] == 'label_val' @v2_1_only() def test_up_with_volume_labels(self): @@ -870,8 +870,8 @@ class CLITestCase(DockerClientTestCase): ] assert [v['Name'] for v in volumes] == [volume_with_label] - - assert volumes[0]['Labels'] == {'label_key': 'label_val'} + assert 'label_key' in volumes[0]['Labels'] + assert volumes[0]['Labels']['label_key'] == 'label_val' @v2_only() def test_up_no_services(self): diff --git a/tests/integration/network_test.py b/tests/integration/network_test.py new file mode 100644 index 00000000..2ff610fb --- /dev/null +++ b/tests/integration/network_test.py @@ -0,0 +1,17 @@ +from __future__ import absolute_import +from __future__ import unicode_literals + +from .testcases import DockerClientTestCase +from compose.const import LABEL_NETWORK +from compose.const import LABEL_PROJECT +from compose.network import Network + + +class NetworkTest(DockerClientTestCase): + def test_network_default_labels(self): + net = Network(self.client, 'composetest', 'foonet') + net.ensure() + net_data = net.inspect() + labels = net_data['Labels'] + assert labels[LABEL_NETWORK] == net.name + assert labels[LABEL_PROJECT] == net.project diff --git a/tests/integration/project_test.py b/tests/integration/project_test.py index 3afefb5a..de073239 100644 --- a/tests/integration/project_test.py +++ b/tests/integration/project_test.py @@ -942,8 +942,8 @@ class ProjectTest(DockerClientTestCase): ] assert [n['Name'] for n in networks] == ['composetest_{}'.format(network_name)] - - assert networks[0]['Labels'] == {'label_key': 'label_val'} + assert 'label_key' in networks[0]['Labels'] + assert networks[0]['Labels']['label_key'] == 'label_val' @v2_only() def test_project_up_volumes(self): @@ -1009,7 +1009,8 @@ class ProjectTest(DockerClientTestCase): assert [v['Name'] for v in volumes] == ['composetest_{}'.format(volume_name)] - assert volumes[0]['Labels'] == {'label_key': 'label_val'} + assert 'label_key' in volumes[0]['Labels'] + assert volumes[0]['Labels']['label_key'] == 'label_val' @v2_only() def test_project_up_logging_with_multiple_files(self): diff --git a/tests/integration/volume_test.py b/tests/integration/volume_test.py index a75250ac..add16962 100644 --- a/tests/integration/volume_test.py +++ b/tests/integration/volume_test.py @@ -4,6 +4,8 @@ from __future__ import unicode_literals from docker.errors import DockerException from .testcases import DockerClientTestCase +from compose.const import LABEL_PROJECT +from compose.const import LABEL_VOLUME from compose.volume import Volume @@ -94,3 +96,11 @@ class VolumeTest(DockerClientTestCase): assert vol.exists() is False vol.create() assert vol.exists() is True + + def test_volume_default_labels(self): + vol = self.create_volume('volume01') + vol.create() + vol_data = vol.inspect() + labels = vol_data['Labels'] + assert labels[LABEL_VOLUME] == vol.name + assert labels[LABEL_PROJECT] == vol.project From 0edfe08bf017841a1a2624b98e4a05bb11fc6d4f Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Tue, 13 Dec 2016 16:54:41 -0800 Subject: [PATCH 40/45] Add healthchecks to 2.1 schema. Update depends_on Signed-off-by: Joffrey F --- compose/config/config_schema_v2.1.json | 42 +++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/compose/config/config_schema_v2.1.json b/compose/config/config_schema_v2.1.json index 3057f2de..5ab9f71f 100644 --- a/compose/config/config_schema_v2.1.json +++ b/compose/config/config_schema_v2.1.json @@ -77,7 +77,28 @@ "cpu_shares": {"type": ["number", "string"]}, "cpu_quota": {"type": ["number", "string"]}, "cpuset": {"type": "string"}, - "depends_on": {"$ref": "#/definitions/list_of_strings"}, + "depends_on": { + "oneOf": [ + {"$ref": "#/definitions/list_of_strings"}, + { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-zA-Z0-9._-]+$": { + "type": "object", + "additionalProperties": false, + "properties": { + "condition": { + "type": "string", + "enum": ["service_started", "service_healthy"] + } + }, + "required": ["condition"] + } + } + } + ] + }, "devices": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, "dns": {"$ref": "#/definitions/string_or_list"}, "dns_search": {"$ref": "#/definitions/string_or_list"}, @@ -120,6 +141,7 @@ "external_links": {"type": "array", "items": {"type": "string"}, "uniqueItems": true}, "extra_hosts": {"$ref": "#/definitions/list_or_dict"}, + "healthcheck": {"$ref": "#/definitions/healthcheck"}, "hostname": {"type": "string"}, "image": {"type": "string"}, "ipc": {"type": "string"}, @@ -231,6 +253,24 @@ "additionalProperties": false }, + "healthcheck": { + "id": "#/definitions/healthcheck", + "type": "object", + "additionalProperties": false, + "properties": { + "disabled": {"type": "boolean"}, + "interval": {"type": "string"}, + "retries": {"type": "number"}, + "test": { + "oneOf": [ + {"type": "string"}, + {"type": "array", "items": {"type": "string"}} + ] + }, + "timeout": {"type": "string"} + } + }, + "network": { "id": "#/definitions/network", "type": "object", From f6edd610f36ec9f6ffdd16df970f3d6cefb1169d Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Wed, 14 Dec 2016 15:39:42 -0800 Subject: [PATCH 41/45] Add 3.0 schema to docker-compose.spec Signed-off-by: Joffrey F --- docker-compose.spec | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docker-compose.spec b/docker-compose.spec index e57d6b7a..ec5a2039 100644 --- a/docker-compose.spec +++ b/docker-compose.spec @@ -32,6 +32,11 @@ exe = EXE(pyz, 'compose/config/config_schema_v2.1.json', 'DATA' ), + ( + 'compose/config/config_schema_v3.0.json', + 'compose/config/config_schema_v3.0.json', + 'DATA' + ), ( 'compose/GITSHA', 'compose/GITSHA', From 04394b1d0a82f4507edc8863c4ab9cf64944f6d9 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 19 Dec 2016 20:20:03 -0800 Subject: [PATCH 42/45] Expand depends_on to allow different conditions (service_start, service_healthy) Rework "up" and "start" to wait on conditional state of dependent services Add integration tests Signed-off-by: Joffrey F --- compose/config/config.py | 12 +++- compose/config/validation.py | 8 ++- compose/errors.py | 21 ++++++ compose/parallel.py | 7 +- compose/project.py | 12 +++- compose/service.py | 59 ++++++++++++++-- tests/acceptance/cli_test.py | 2 +- tests/integration/project_test.py | 114 ++++++++++++++++++++++++++++++ tests/unit/config/config_test.py | 9 ++- tests/unit/parallel_test.py | 2 +- 10 files changed, 228 insertions(+), 18 deletions(-) diff --git a/compose/config/config.py b/compose/config/config.py index dccd11e0..73a34017 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -649,6 +649,8 @@ def process_service(service_config): if 'sysctls' in service_dict: service_dict['sysctls'] = build_string_dict(parse_sysctls(service_dict['sysctls'])) + service_dict = process_depends_on(service_dict) + for field in ['dns', 'dns_search', 'tmpfs']: if field in service_dict: service_dict[field] = to_list(service_dict[field]) @@ -658,6 +660,14 @@ def process_service(service_config): return service_dict +def process_depends_on(service_dict): + if 'depends_on' in service_dict and not isinstance(service_dict['depends_on'], dict): + service_dict['depends_on'] = dict([ + (svc, {'condition': 'service_started'}) for svc in service_dict['depends_on'] + ]) + return service_dict + + def process_healthcheck(service_dict, service_name): if 'healthcheck' not in service_dict: return service_dict @@ -665,7 +675,7 @@ def process_healthcheck(service_dict, service_name): hc = {} raw = service_dict['healthcheck'] - if raw.get('disable'): + if raw.get('disable') or raw.get('disabled'): if len(raw) > 1: raise ConfigurationError( 'Service "{}" defines an invalid healthcheck: ' diff --git a/compose/config/validation.py b/compose/config/validation.py index 7452e984..3f23f0a7 100644 --- a/compose/config/validation.py +++ b/compose/config/validation.py @@ -180,11 +180,13 @@ def validate_links(service_config, service_names): def validate_depends_on(service_config, service_names): - for dependency in service_config.config.get('depends_on', []): + deps = service_config.config.get('depends_on', {}) + for dependency in deps.keys(): if dependency not in service_names: raise ConfigurationError( "Service '{s.name}' depends on service '{dep}' which is " - "undefined.".format(s=service_config, dep=dependency)) + "undefined.".format(s=service_config, dep=dependency) + ) def get_unsupported_config_msg(path, error_key): @@ -201,7 +203,7 @@ def anglicize_json_type(json_type): def is_service_dict_schema(schema_id): - return schema_id in ('config_schema_v1.json', '#/properties/services') + return schema_id in ('config_schema_v1.json', '#/properties/services') def handle_error_for_schema_with_id(error, path): diff --git a/compose/errors.py b/compose/errors.py index 376cc555..415b41e7 100644 --- a/compose/errors.py +++ b/compose/errors.py @@ -10,3 +10,24 @@ class OperationFailedError(Exception): class StreamParseError(RuntimeError): def __init__(self, reason): self.msg = reason + + +class HealthCheckException(Exception): + def __init__(self, reason): + self.msg = reason + + +class HealthCheckFailed(HealthCheckException): + def __init__(self, container_id): + super(HealthCheckFailed, self).__init__( + 'Container "{}" is unhealthy.'.format(container_id) + ) + + +class NoHealthCheckConfigured(HealthCheckException): + def __init__(self, service_name): + super(NoHealthCheckConfigured, self).__init__( + 'Service "{}" is missing a healthcheck configuration'.format( + service_name + ) + ) diff --git a/compose/parallel.py b/compose/parallel.py index 26718872..b2654dcf 100644 --- a/compose/parallel.py +++ b/compose/parallel.py @@ -165,13 +165,14 @@ def feed_queue(objects, func, get_deps, results, state): for obj in pending: deps = get_deps(obj) - if any(dep in state.failed for dep in deps): + if any(dep[0] in state.failed for dep in deps): log.debug('{} has upstream errors - not processing'.format(obj)) results.put((obj, None, UpstreamError())) state.failed.add(obj) elif all( - dep not in objects or dep in state.finished - for dep in deps + dep not in objects or ( + dep in state.finished and (not ready_check or ready_check(dep)) + ) for dep, ready_check in deps ): log.debug('Starting producer thread for {}'.format(obj)) t = Thread(target=producer, args=(obj, func, results)) diff --git a/compose/project.py b/compose/project.py index 0178bbab..d99ef7c9 100644 --- a/compose/project.py +++ b/compose/project.py @@ -227,7 +227,10 @@ class Project(object): services = self.get_services(service_names) def get_deps(service): - return {self.get_service(dep) for dep in service.get_dependency_names()} + return { + (self.get_service(dep), config) + for dep, config in service.get_dependency_configs().items() + } parallel.parallel_execute( services, @@ -243,7 +246,7 @@ class Project(object): def get_deps(container): # actually returning inversed dependencies - return {other for other in containers + return {(other, None) for other in containers if container.service in self.get_service(other.service).get_dependency_names()} @@ -394,7 +397,10 @@ class Project(object): ) def get_deps(service): - return {self.get_service(dep) for dep in service.get_dependency_names()} + return { + (self.get_service(dep), config) + for dep, config in service.get_dependency_configs().items() + } results, errors = parallel.parallel_execute( services, diff --git a/compose/service.py b/compose/service.py index e3862b6e..1338f154 100644 --- a/compose/service.py +++ b/compose/service.py @@ -28,6 +28,8 @@ from .const import LABEL_PROJECT from .const import LABEL_SERVICE from .const import LABEL_VERSION from .container import Container +from .errors import HealthCheckFailed +from .errors import NoHealthCheckConfigured from .errors import OperationFailedError from .parallel import parallel_execute from .parallel import parallel_start @@ -69,6 +71,9 @@ DOCKER_START_KEYS = [ 'volumes_from', ] +CONDITION_STARTED = 'service_started' +CONDITION_HEALTHY = 'service_healthy' + class BuildError(Exception): def __init__(self, service, reason): @@ -533,10 +538,38 @@ class Service(object): def get_dependency_names(self): net_name = self.network_mode.service_name - return (self.get_linked_service_names() + - self.get_volumes_from_names() + - ([net_name] if net_name else []) + - self.options.get('depends_on', [])) + return ( + self.get_linked_service_names() + + self.get_volumes_from_names() + + ([net_name] if net_name else []) + + list(self.options.get('depends_on', {}).keys()) + ) + + def get_dependency_configs(self): + net_name = self.network_mode.service_name + configs = dict( + [(name, None) for name in self.get_linked_service_names()] + ) + configs.update(dict( + [(name, None) for name in self.get_volumes_from_names()] + )) + configs.update({net_name: None} if net_name else {}) + configs.update(self.options.get('depends_on', {})) + for svc, config in self.options.get('depends_on', {}).items(): + if config['condition'] == CONDITION_STARTED: + configs[svc] = lambda s: True + elif config['condition'] == CONDITION_HEALTHY: + configs[svc] = lambda s: s.is_healthy() + else: + # The config schema already prevents this, but it might be + # bypassed if Compose is called programmatically. + raise ValueError( + 'depends_on condition "{}" is invalid.'.format( + config['condition'] + ) + ) + + return configs def get_linked_service_names(self): return [service.name for (service, _) in self.links] @@ -871,6 +904,24 @@ class Service(object): else: log.error(six.text_type(e)) + def is_healthy(self): + """ Check that all containers for this service report healthy. + Returns false if at least one healthcheck is pending. + If an unhealthy container is detected, raise a HealthCheckFailed + exception. + """ + result = True + for ctnr in self.containers(): + ctnr.inspect() + status = ctnr.get('State.Health.Status') + if status is None: + raise NoHealthCheckConfigured(self.name) + elif status == 'starting': + result = False + elif status == 'unhealthy': + raise HealthCheckFailed(ctnr.short_id) + return result + def short_id_alias_exists(container, network): aliases = container.get( diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index b9766226..77d57840 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -928,7 +928,7 @@ class CLITestCase(DockerClientTestCase): assert foo_container.get('HostConfig.NetworkMode') == \ 'container:{}'.format(bar_container.id) - @v3_only() + @v2_1_only() def test_up_with_healthcheck(self): def wait_on_health_status(container, status): def condition(): diff --git a/tests/integration/project_test.py b/tests/integration/project_test.py index de073239..855974de 100644 --- a/tests/integration/project_test.py +++ b/tests/integration/project_test.py @@ -19,6 +19,8 @@ from compose.config.types import VolumeSpec from compose.const import LABEL_PROJECT from compose.const import LABEL_SERVICE from compose.container import Container +from compose.errors import HealthCheckFailed +from compose.errors import NoHealthCheckConfigured from compose.project import Project from compose.project import ProjectError from compose.service import ConvergenceStrategy @@ -1375,3 +1377,115 @@ class ProjectTest(DockerClientTestCase): ctnr for ctnr in project._labeled_containers() if ctnr.labels.get(LABEL_SERVICE) == 'service1' ]) == 0 + + @v2_1_only() + def test_project_up_healthy_dependency(self): + config_dict = { + 'version': '2.1', + 'services': { + 'svc1': { + 'image': 'busybox:latest', + 'command': 'top', + 'healthcheck': { + 'test': 'exit 0', + 'retries': 1, + 'timeout': '10s', + 'interval': '0.1s' + }, + }, + 'svc2': { + 'image': 'busybox:latest', + 'command': 'top', + 'depends_on': { + 'svc1': {'condition': 'service_healthy'}, + } + } + } + } + config_data = build_config(config_dict) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + project.up() + containers = project.containers() + assert len(containers) == 2 + + svc1 = project.get_service('svc1') + svc2 = project.get_service('svc2') + assert 'svc1' in svc2.get_dependency_names() + assert svc1.is_healthy() + + @v2_1_only() + def test_project_up_unhealthy_dependency(self): + config_dict = { + 'version': '2.1', + 'services': { + 'svc1': { + 'image': 'busybox:latest', + 'command': 'top', + 'healthcheck': { + 'test': 'exit 1', + 'retries': 1, + 'timeout': '10s', + 'interval': '0.1s' + }, + }, + 'svc2': { + 'image': 'busybox:latest', + 'command': 'top', + 'depends_on': { + 'svc1': {'condition': 'service_healthy'}, + } + } + } + } + config_data = build_config(config_dict) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + with pytest.raises(HealthCheckFailed): + project.up() + containers = project.containers() + assert len(containers) == 1 + + svc1 = project.get_service('svc1') + svc2 = project.get_service('svc2') + assert 'svc1' in svc2.get_dependency_names() + with pytest.raises(HealthCheckFailed): + svc1.is_healthy() + + @v2_1_only() + def test_project_up_no_healthcheck_dependency(self): + config_dict = { + 'version': '2.1', + 'services': { + 'svc1': { + 'image': 'busybox:latest', + 'command': 'top', + 'healthcheck': { + 'disabled': True + }, + }, + 'svc2': { + 'image': 'busybox:latest', + 'command': 'top', + 'depends_on': { + 'svc1': {'condition': 'service_healthy'}, + } + } + } + } + config_data = build_config(config_dict) + project = Project.from_config( + name='composetest', config_data=config_data, client=self.client + ) + with pytest.raises(NoHealthCheckConfigured): + project.up() + containers = project.containers() + assert len(containers) == 1 + + svc1 = project.get_service('svc1') + svc2 = project.get_service('svc2') + assert 'svc1' in svc2.get_dependency_names() + with pytest.raises(NoHealthCheckConfigured): + svc1.is_healthy() diff --git a/tests/unit/config/config_test.py b/tests/unit/config/config_test.py index f7df3aee..7a68333f 100644 --- a/tests/unit/config/config_test.py +++ b/tests/unit/config/config_test.py @@ -920,7 +920,10 @@ class ConfigTest(unittest.TestCase): 'build': {'context': os.path.abspath('/')}, 'image': 'example/web', 'volumes': [VolumeSpec.parse('/home/user/project:/code')], - 'depends_on': ['db', 'other'], + 'depends_on': { + 'db': {'condition': 'container_start'}, + 'other': {'condition': 'container_start'}, + }, }, { 'name': 'db', @@ -3055,7 +3058,9 @@ class ExtendsTest(unittest.TestCase): image: example """) services = load_from_filename(str(tmpdir.join('docker-compose.yml'))) - assert service_sort(services)[2]['depends_on'] == ['other'] + assert service_sort(services)[2]['depends_on'] == { + 'other': {'condition': 'container_start'} + } @pytest.mark.xfail(IS_WINDOWS_PLATFORM, reason='paths use slash') diff --git a/tests/unit/parallel_test.py b/tests/unit/parallel_test.py index 479c0f1d..2a50b718 100644 --- a/tests/unit/parallel_test.py +++ b/tests/unit/parallel_test.py @@ -25,7 +25,7 @@ deps = { def get_deps(obj): - return deps[obj] + return [(dep, None) for dep in deps[obj]] def test_parallel_execute(): From bef230853012d78e7ab58de65b653839401be974 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Mon, 19 Dec 2016 20:39:16 -0800 Subject: [PATCH 43/45] Fix condition name in config tests Signed-off-by: Joffrey F --- tests/acceptance/cli_test.py | 2 +- tests/unit/config/config_test.py | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/acceptance/cli_test.py b/tests/acceptance/cli_test.py index 77d57840..b9766226 100644 --- a/tests/acceptance/cli_test.py +++ b/tests/acceptance/cli_test.py @@ -928,7 +928,7 @@ class CLITestCase(DockerClientTestCase): assert foo_container.get('HostConfig.NetworkMode') == \ 'container:{}'.format(bar_container.id) - @v2_1_only() + @v3_only() def test_up_with_healthcheck(self): def wait_on_health_status(container, status): def condition(): diff --git a/tests/unit/config/config_test.py b/tests/unit/config/config_test.py index 7a68333f..31a888ed 100644 --- a/tests/unit/config/config_test.py +++ b/tests/unit/config/config_test.py @@ -921,8 +921,8 @@ class ConfigTest(unittest.TestCase): 'image': 'example/web', 'volumes': [VolumeSpec.parse('/home/user/project:/code')], 'depends_on': { - 'db': {'condition': 'container_start'}, - 'other': {'condition': 'container_start'}, + 'db': {'condition': 'service_started'}, + 'other': {'condition': 'service_started'}, }, }, { @@ -3059,7 +3059,7 @@ class ExtendsTest(unittest.TestCase): """) services = load_from_filename(str(tmpdir.join('docker-compose.yml'))) assert service_sort(services)[2]['depends_on'] == { - 'other': {'condition': 'container_start'} + 'other': {'condition': 'service_started'} } From f90618fc43471355702c703909cc2d92813498f2 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Wed, 4 Jan 2017 13:14:23 -0800 Subject: [PATCH 44/45] Unify healthcheck spec definition in v2 and v3 Signed-off-by: Joffrey F --- compose/config/config.py | 2 +- compose/config/config_schema_v2.1.json | 2 +- compose/config/config_schema_v3.0.json | 12 ++++++------ tests/integration/project_test.py | 2 +- 4 files changed, 9 insertions(+), 9 deletions(-) diff --git a/compose/config/config.py b/compose/config/config.py index 73a34017..fd935591 100644 --- a/compose/config/config.py +++ b/compose/config/config.py @@ -675,7 +675,7 @@ def process_healthcheck(service_dict, service_name): hc = {} raw = service_dict['healthcheck'] - if raw.get('disable') or raw.get('disabled'): + if raw.get('disable'): if len(raw) > 1: raise ConfigurationError( 'Service "{}" defines an invalid healthcheck: ' diff --git a/compose/config/config_schema_v2.1.json b/compose/config/config_schema_v2.1.json index 5ab9f71f..d0d5233a 100644 --- a/compose/config/config_schema_v2.1.json +++ b/compose/config/config_schema_v2.1.json @@ -258,7 +258,7 @@ "type": "object", "additionalProperties": false, "properties": { - "disabled": {"type": "boolean"}, + "disable": {"type": "boolean"}, "interval": {"type": "string"}, "retries": {"type": "number"}, "test": { diff --git a/compose/config/config_schema_v3.0.json b/compose/config/config_schema_v3.0.json index 1f93347f..8d075d47 100644 --- a/compose/config/config_schema_v3.0.json +++ b/compose/config/config_schema_v3.0.json @@ -202,10 +202,11 @@ "healthcheck": { "id": "#/definitions/healthcheck", - "type": ["object", "null"], + "type": "object", + "additionalProperties": false, "properties": { - "interval": {"type":"string"}, - "timeout": {"type":"string"}, + "disable": {"type": "boolean"}, + "interval": {"type": "string"}, "retries": {"type": "number"}, "test": { "oneOf": [ @@ -213,9 +214,8 @@ {"type": "array", "items": {"type": "string"}} ] }, - "disable": {"type": "boolean"} - }, - "additionalProperties": false + "timeout": {"type": "string"} + } }, "deployment": { "id": "#/definitions/deployment", diff --git a/tests/integration/project_test.py b/tests/integration/project_test.py index 855974de..c5e3cf50 100644 --- a/tests/integration/project_test.py +++ b/tests/integration/project_test.py @@ -1463,7 +1463,7 @@ class ProjectTest(DockerClientTestCase): 'image': 'busybox:latest', 'command': 'top', 'healthcheck': { - 'disabled': True + 'disable': True }, }, 'svc2': { From ecff6f1a9a6f56eeaff378da814dafde9dd53b02 Mon Sep 17 00:00:00 2001 From: Joffrey F Date: Wed, 4 Jan 2017 13:47:13 -0800 Subject: [PATCH 45/45] Bump 1.10.0-rc1 Signed-off-by: Joffrey F --- CHANGELOG.md | 40 ++++++++++++++++++++++++++++++++++++++++ compose/__init__.py | 2 +- script/run/run.sh | 2 +- 3 files changed, 42 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9780df98..3653a434 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,46 @@ Change log ========== +1.10.0 (2017-01-18) +------------------- + +### New Features + +#### Compose file version 3.0 + +- Introduced version 3.0 of the `docker-compose.yml` specification. This + version requires to be used with Docker Engine 1.13 or above and is + specifically designed to work with the `docker stack` commands. + + - Added support for the `stop_grace_period` option in service definitions. + +#### Compose file version 2.1 and up + +- Healthcheck configuration can now be done in the service definition using + the `healthcheck` parameter + +- Containers dependencies can now be set up to wait on positive healthchecks + when declared using `depends_on`. See the documentation for the updated + syntax. + **Note:** This feature will not be ported to version 3 Compose files. + +- Added support for the `sysctls` parameter in service definitions + +- Added support for the `userns_mode` parameter in service definitions + +- Compose now adds identifying labels to networks and volumes it creates + +### Bugfixes + +- Colored output now works properly on Windows. + +- Fixed a bug where docker-compose run would fail to set up link aliases + in interactive mode on Windows. + +- Networks created by Compose are now always made attachable + (Compose files v2.1 and up). + + 1.9.0 (2016-11-16) ----------------- diff --git a/compose/__init__.py b/compose/__init__.py index 6f05b282..f1bad0e1 100644 --- a/compose/__init__.py +++ b/compose/__init__.py @@ -1,4 +1,4 @@ from __future__ import absolute_import from __future__ import unicode_literals -__version__ = '1.10.0dev' +__version__ = '1.10.0-rc1' diff --git a/script/run/run.sh b/script/run/run.sh index 5872b081..5354b5f0 100755 --- a/script/run/run.sh +++ b/script/run/run.sh @@ -15,7 +15,7 @@ set -e -VERSION="1.8.0" +VERSION="1.10.0-rc1" IMAGE="docker/compose:$VERSION"