Bump 1.3.0rc3

Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>

.dockerignore

@@ -1,2 +1,4 @@
 .git
+build
+dist
 venv

CHANGES.md

@@ -1,6 +1,51 @@
 Change log
 ==========
 
+1.3.0 RC3 (2015-06-15)
+----------------------
+
+Firstly, two important notes:
+
+- **This release contains breaking changes, and you will need to either remove or migrate your existing containers before running your app** - see the [upgrading section of the install docs](https://github.com/docker/compose/blob/1.3.0rc1/docs/install.md#upgrading) for details.
+
+- Compose now requires Docker 1.6.0 or later.
+
+We've done a lot of work in this release to remove hacks and make Compose more stable:
+
+- Compose now uses Docker labels, rather than container names, to keep track of containers. This is both cleaner and more performant.
+
+- Compose no longer uses "intermediate containers" when recreating containers for a service. This makes `docker-compose up` less complex and more resilient to failure.
+
+There are some new features:
+
+- `docker-compose up` has an **experimental** new behaviour: it will only recreate containers for services whose configuration has changed in `docker-compose.yml`. This will eventually become the default, but for now you can take it for a spin:
+
+        $ docker-compose up --x-smart-recreate
+
+- When invoked in a subdirectory of a project, `docker-compose` will now climb up through parent directories until it finds a `docker-compose.yml`.
+
+Several new configuration keys have been added to `docker-compose.yml`:
+
+- `dockerfile`, like `docker build --file`, lets you specify an alternate Dockerfile to use with `build`.
+- `labels`, like `docker run --labels`, lets you add custom metadata to containers.
+- `extra_hosts`, like `docker run --add-host`, lets you add entries to a container's `/etc/hosts` file.
+- `pid: host`, like `docker run --pid=host`, lets you reuse the same PID namespace as the host machine.
+- `cpuset`, like `docker run --cpuset-cpus`, lets you specify which CPUs to allow execution in.
+- `read_only`, like `docker run --read-only`, lets you mount a container's filesystem as read-only.
+- `security_opt`, like `docker run --security-opt`, lets you specify [security options](https://docs.docker.com/reference/run/#security-configuration).
+- `log_driver`, like `docker run --log-driver`, lets you specify a [log driver](https://docs.docker.com/reference/run/#logging-drivers-log-driver).
+
+Many bugs have been fixed, including the following:
+
+- The output of `docker-compose run` was sometimes truncated, especially when running under Jenkins.
+- A service's volumes would sometimes not update after volume configuration was changed in `docker-compose.yml`.
+- Authenticating against third-party registries would sometimes fail.
+- `docker-compose run --rm` would fail to remove the container if the service had a `restart` policy in place.
+- `docker-compose scale` would refuse to scale a service beyond 1 container if it exposed a specific port number on the host.
+- Compose would refuse to create multiple volume entries with the same host path.
+
+Thanks @ahromis, @albers, @aleksandr-vin, @antoineco, @ccverak, @chernjie, @dnephin, @josephpage, @KyleJamesWalker, @lsowen, @mchasal, @sdake, @sherter, @stephenlawrence, @turtlemonvh, @vdemeester, @xuxinkun and @zwily!
+
 1.2.0 (2015-04-16)
 ------------------
 

CONTRIBUTING.md

@@ -53,23 +53,26 @@ you can specify a test directory, file, module, class or method:
 
 ## Building binaries
 
-Linux:
+`script/build-linux` will build the Linux binary inside a Docker container:
 
     $ script/build-linux
 
-OS X:
+`script/build-osx` will build the Mac OS X binary inside a virtualenv:
 
     $ script/build-osx
 
-Note that this only works on Mountain Lion, not Mavericks, due to a
-[bug in PyInstaller](http://www.pyinstaller.org/ticket/807).
+For official releases, you should build inside a Mountain Lion VM for proper
+compatibility. Run the this script first to prepare the environment before
+building - it will use Homebrew to make sure Python is installed and
+up-to-date.
+
+    $ script/prepare-osx
 
 ## Release process
 
 1. Open pull request that:
  - Updates the version in `compose/__init__.py`
  - Updates the binary URL in `docs/install.md`
- - Updates the script URL in `docs/completion.md`
  - Adds release notes to `CHANGES.md`
 2. Create unpublished GitHub release with release notes
 3. Build Linux version on any Docker host with `script/build-linux` and attach

Dockerfile

@@ -3,9 +3,11 @@ FROM debian:wheezy
 RUN set -ex; \
     apt-get update -qq; \
     apt-get install -y \
-        python \
-        python-pip \
-        python-dev \
+        gcc \
+        make \
+        zlib1g \
+        zlib1g-dev \
+        libssl-dev \
         git \
         apt-transport-https \
         ca-certificates \
@@ -15,11 +17,44 @@ RUN set -ex; \
     ; \
     rm -rf /var/lib/apt/lists/*
 
-ENV ALL_DOCKER_VERSIONS 1.6.0
+# Build Python 2.7.9 from source
+RUN set -ex; \
+    curl -LO https://www.python.org/ftp/python/2.7.9/Python-2.7.9.tgz; \
+    tar -xzf Python-2.7.9.tgz; \
+    cd Python-2.7.9; \
+    ./configure --enable-shared; \
+    make; \
+    make install; \
+    cd ..; \
+    rm -rf /Python-2.7.9; \
+    rm Python-2.7.9.tgz
+
+# Make libpython findable
+ENV LD_LIBRARY_PATH /usr/local/lib
+
+# Install setuptools
+RUN set -ex; \
+    curl -LO https://bootstrap.pypa.io/ez_setup.py; \
+    python ez_setup.py; \
+    rm ez_setup.py
+
+# Install pip
+RUN set -ex; \
+    curl -LO https://pypi.python.org/packages/source/p/pip/pip-7.0.1.tar.gz; \
+    tar -xzf pip-7.0.1.tar.gz; \
+    cd pip-7.0.1; \
+    python setup.py install; \
+    cd ..; \
+    rm -rf pip-7.0.1; \
+    rm pip-7.0.1.tar.gz
+
+ENV ALL_DOCKER_VERSIONS 1.6.0 1.7.0-rc2
 
 RUN set -ex; \
     curl https://get.docker.com/builds/Linux/x86_64/docker-1.6.0 -o /usr/local/bin/docker-1.6.0; \
-    chmod +x /usr/local/bin/docker-1.6.0
+    chmod +x /usr/local/bin/docker-1.6.0; \
+    curl https://test.docker.com/builds/Linux/x86_64/docker-1.7.0-rc2 -o /usr/local/bin/docker-1.7.0-rc2; \
+    chmod +x /usr/local/bin/docker-1.7.0-rc2
 
 # Set the default Docker to be run
 RUN ln -s /usr/local/bin/docker-1.6.0 /usr/local/bin/docker

README.md

@@ -47,6 +47,6 @@ Installation and documentation
 Contributing
 ------------
 
-[![Build Status](http://jenkins.dockerproject.com/buildStatus/icon?job=Compose Master)](http://jenkins.dockerproject.com/job/Compose%20Master/)
+[![Build Status](http://jenkins.dockerproject.org/buildStatus/icon?job=Compose%20Master)](http://jenkins.dockerproject.org/job/Compose%20Master/)
 
 Want to help build Compose? Check out our [contributing documentation](https://github.com/docker/compose/blob/master/CONTRIBUTING.md).

compose/__init__.py

@@ -1,3 +1,3 @@
 from __future__ import unicode_literals
 
-__version__ = '1.3.0dev'
+__version__ = '1.3.0rc3'

compose/cli/main.py

@@ -10,17 +10,16 @@ import sys
 from docker.errors import APIError
 import dockerpty
 
-from .. import __version__
 from .. import legacy
 from ..project import NoSuchService, ConfigurationError
-from ..service import BuildError, CannotBeScaledError, NeedsBuildError
+from ..service import BuildError, NeedsBuildError
 from ..config import parse_environment
 from .command import Command
 from .docopt_command import NoSuchCommand
 from .errors import UserError
 from .formatter import Formatter
 from .log_printer import LogPrinter
-from .utils import yesno
+from .utils import get_version_info, yesno
 
 log = logging.getLogger(__name__)
 
@@ -104,7 +103,7 @@ class TopLevelCommand(Command):
     """
     def docopt_options(self):
         options = super(TopLevelCommand, self).docopt_options()
-        options['version'] = "docker-compose %s" % __version__
+        options['version'] = get_version_info()
         return options
 
     def build(self, project, options):
@@ -170,13 +169,14 @@ class TopLevelCommand(Command):
         Usage: port [options] SERVICE PRIVATE_PORT
 
         Options:
-            --protocol=proto  tcp or udp (defaults to tcp)
+            --protocol=proto  tcp or udp [default: tcp]
             --index=index     index of the container if there are multiple
-                              instances of a service (defaults to 1)
+                              instances of a service [default: 1]
         """
+        index = int(options.get('--index'))
         service = project.get_service(options['SERVICE'])
         try:
-            container = service.get_container(number=options.get('--index') or 1)
+            container = service.get_container(number=index)
         except ValueError as e:
             raise UserError(str(e))
         print(container.get_local_port(
@@ -336,6 +336,7 @@ class TopLevelCommand(Command):
             container_options['ports'] = []
 
         container = service.create_container(
+            quiet=True,
             one_off=True,
             insecure_registry=insecure_registry,
             **container_options
@@ -348,7 +349,6 @@ class TopLevelCommand(Command):
             dockerpty.start(project.client, container.id, interactive=not options['-T'])
             exit_code = container.wait()
             if options['--rm']:
-                log.info("Removing %s..." % container.name)
                 project.client.remove_container(container.id)
             sys.exit(exit_code)
 
@@ -372,15 +372,7 @@ class TopLevelCommand(Command):
             except ValueError:
                 raise UserError('Number of containers for service "%s" is not a '
                                 'number' % service_name)
-            try:
-                project.get_service(service_name).scale(num)
-            except CannotBeScaledError:
-                raise UserError(
-                    'Service "%s" cannot be scaled because it specifies a port '
-                    'on the host. If multiple containers for this service were '
-                    'created, the port would clash.\n\nRemove the ":" from the '
-                    'port definition in docker-compose.yml so Docker can choose a random '
-                    'port for each container.' % service_name)
+            project.get_service(service_name).scale(num)
 
     def start(self, project, options):
         """

compose/cli/utils.py

@@ -5,6 +5,9 @@ import datetime
 import os
 import subprocess
 import platform
+import ssl
+
+from .. import __version__
 
 
 def yesno(prompt, default=None):
@@ -120,3 +123,11 @@ def is_mac():
 
 def is_ubuntu():
     return platform.system() == 'Linux' and platform.linux_distribution()[0] == 'Ubuntu'
+
+
+def get_version_info():
+    return '\n'.join([
+        'docker-compose version: %s' % __version__,
+        "%s version: %s" % (platform.python_implementation(), platform.python_version()),
+        "OpenSSL version: %s" % ssl.OPENSSL_VERSION,
+    ])

compose/progress_stream.py

@@ -74,8 +74,9 @@ def print_output_event(event, stream, is_terminal):
         stream.write("%s %s%s" % (status, event['progress'], terminator))
     elif 'progressDetail' in event:
         detail = event['progressDetail']
-        if 'current' in detail:
-            percentage = float(detail['current']) / float(detail['total']) * 100
+        total = detail.get('total')
+        if 'current' in detail and total:
+            percentage = float(detail['current']) / float(total) * 100
             stream.write('%s (%.1f%%)%s' % (status, percentage, terminator))
         else:
             stream.write('%s%s' % (status, terminator))

compose/project.py

@@ -171,7 +171,7 @@ class Project(object):
                     try:
                         net = Container.from_id(self.client, net_name)
                     except APIError:
-                        raise ConfigurationError('Serivce "%s" is trying to use the network of "%s", which is not the name of a service or container.' % (service_dict['name'], net_name))
+                        raise ConfigurationError('Service "%s" is trying to use the network of "%s", which is not the name of a service or container.' % (service_dict['name'], net_name))
             else:
                 net = service_dict['net']
 

compose/service.py

@@ -55,10 +55,6 @@ class BuildError(Exception):
         self.reason = reason
 
 
-class CannotBeScaledError(Exception):
-    pass
-
-
 class ConfigError(ValueError):
     pass
 
@@ -154,7 +150,9 @@ class Service(object):
         - removes all stopped containers
         """
         if not self.can_be_scaled():
-            raise CannotBeScaledError()
+            log.warn('Service %s specifies a port on the host. If multiple containers '
+                     'for this service are created on a single host, the port will clash.'
+                     % self.name)
 
         # Create enough containers
         containers = self.containers(stopped=True)
@@ -199,6 +197,7 @@ class Service(object):
                          do_build=True,
                          previous_container=None,
                          number=None,
+                         quiet=False,
                          **override_options):
         """
         Create a container for this service. If the image doesn't exist, attempt to pull
@@ -216,7 +215,7 @@ class Service(object):
             previous_container=previous_container,
         )
 
-        if 'name' in container_options:
+        if 'name' in container_options and not quiet:
             log.info("Creating %s..." % container_options['name'])
 
         return Container.create(self.client, **container_options)
@@ -378,6 +377,7 @@ class Service(object):
             do_build=False,
             previous_container=container,
             number=container.labels.get(LABEL_CONTAINER_NUMBER),
+            quiet=True,
         )
         self.start_container(new_container)
         container.remove()
@@ -394,21 +394,6 @@ class Service(object):
         container.start()
         return container
 
-    def start_or_create_containers(
-            self,
-            insecure_registry=False,
-            do_build=True):
-        containers = self.containers(stopped=True)
-
-        if not containers:
-            new_container = self.create_container(
-                insecure_registry=insecure_registry,
-                do_build=do_build,
-            )
-            return [self.start_container(new_container)]
-        else:
-            return [self.start_container_if_stopped(c) for c in containers]
-
     def config_hash(self):
         return json_hash(self.config_dict())
 
@@ -708,6 +693,47 @@ class Service(object):
         stream_output(output, sys.stdout)
 
 
+# Names
+
+
+def build_container_name(project, service, number, one_off=False):
+    bits = [project, service]
+    if one_off:
+        bits.append('run')
+    return '_'.join(bits + [str(number)])
+
+
+# Images
+
+
+def parse_repository_tag(s):
+    if ":" not in s:
+        return s, ""
+    repo, tag = s.rsplit(":", 1)
+    if "/" in tag:
+        return s, ""
+    return repo, tag
+
+
+# Volumes
+
+
+def merge_volume_bindings(volumes_option, previous_container):
+    """Return a list of volume bindings for a container. Container data volumes
+    are replaced by those from the previous container.
+    """
+    volume_bindings = dict(
+        build_volume_binding(parse_volume_spec(volume))
+        for volume in volumes_option or []
+        if ':' in volume)
+
+    if previous_container:
+        volume_bindings.update(
+            get_container_data_volumes(previous_container, volumes_option))
+
+    return volume_bindings.values()
+
+
 def get_container_data_volumes(container, volumes_option):
     """Find the container data volumes that are in `volumes_option`, and return
     a mapping of volume bindings for those volumes.
@@ -736,51 +762,8 @@ def get_container_data_volumes(container, volumes_option):
     return dict(volumes)
 
 
-def merge_volume_bindings(volumes_option, previous_container):
-    """Return a list of volume bindings for a container. Container data volumes
-    are replaced by those from the previous container.
-    """
-    volume_bindings = dict(
-        build_volume_binding(parse_volume_spec(volume))
-        for volume in volumes_option or []
-        if ':' in volume)
-
-    if previous_container:
-        volume_bindings.update(
-            get_container_data_volumes(previous_container, volumes_option))
-
-    return volume_bindings
-
-
-def build_container_name(project, service, number, one_off=False):
-    bits = [project, service]
-    if one_off:
-        bits.append('run')
-    return '_'.join(bits + [str(number)])
-
-
-def build_container_labels(label_options, service_labels, number, one_off=False):
-    labels = label_options or {}
-    labels.update(label.split('=', 1) for label in service_labels)
-    labels[LABEL_CONTAINER_NUMBER] = str(number)
-    labels[LABEL_VERSION] = __version__
-    return labels
-
-
-def parse_restart_spec(restart_config):
-    if not restart_config:
-        return None
-    parts = restart_config.split(':')
-    if len(parts) > 2:
-        raise ConfigError("Restart %s has incorrect format, should be "
-                          "mode[:max_retry]" % restart_config)
-    if len(parts) == 2:
-        name, max_retry_count = parts
-    else:
-        name, = parts
-        max_retry_count = 0
-
-    return {'Name': name, 'MaximumRetryCount': int(max_retry_count)}
+def build_volume_binding(volume_spec):
+    return volume_spec.internal, "{}:{}:{}".format(*volume_spec)
 
 
 def parse_volume_spec(volume_config):
@@ -803,18 +786,7 @@ def parse_volume_spec(volume_config):
     return VolumeSpec(external, internal, mode)
 
 
-def parse_repository_tag(s):
-    if ":" not in s:
-        return s, ""
-    repo, tag = s.rsplit(":", 1)
-    if "/" in tag:
-        return s, ""
-    return repo, tag
-
-
-def build_volume_binding(volume_spec):
-    internal = {'bind': volume_spec.internal, 'ro': volume_spec.mode == 'ro'}
-    return volume_spec.external, internal
+# Ports
 
 
 def build_port_bindings(ports):
@@ -845,6 +817,39 @@ def split_port(port):
     return internal_port, (external_ip, external_port or None)
 
 
+# Labels
+
+
+def build_container_labels(label_options, service_labels, number, one_off=False):
+    labels = label_options or {}
+    labels.update(label.split('=', 1) for label in service_labels)
+    labels[LABEL_CONTAINER_NUMBER] = str(number)
+    labels[LABEL_VERSION] = __version__
+    return labels
+
+
+# Restart policy
+
+
+def parse_restart_spec(restart_config):
+    if not restart_config:
+        return None
+    parts = restart_config.split(':')
+    if len(parts) > 2:
+        raise ConfigError("Restart %s has incorrect format, should be "
+                          "mode[:max_retry]" % restart_config)
+    if len(parts) == 2:
+        name, max_retry_count = parts
+    else:
+        name, = parts
+        max_retry_count = 0
+
+    return {'Name': name, 'MaximumRetryCount': int(max_retry_count)}
+
+
+# Extra hosts
+
+
 def build_extra_hosts(extra_hosts_config):
     if not extra_hosts_config:
         return {}

contrib/completion/bash/docker-compose

@@ -104,7 +104,7 @@ _docker-compose_docker-compose() {
 
 	case "$cur" in
 		-*)
-			COMPREPLY=( $( compgen -W "--help -h --verbose --version --file -f --project-name -p" -- "$cur" ) )
+			COMPREPLY=( $( compgen -W "--help -h --verbose --version -v --file -f --project-name -p" -- "$cur" ) )
 			;;
 		*)
 			COMPREPLY=( $( compgen -W "${commands[*]}" -- "$cur" ) )
@@ -293,7 +293,7 @@ _docker-compose_up() {
 
 	case "$cur" in
 		-*)
-			COMPREPLY=( $( compgen -W "--allow-insecure-ssl -d --no-build --no-color --no-deps --no-recreate -t --timeout" -- "$cur" ) )
+			COMPREPLY=( $( compgen -W "--allow-insecure-ssl -d --no-build --no-color --no-deps --no-recreate -t --timeout --x-smart-recreate" -- "$cur" ) )
 			;;
 		*)
 			__docker-compose_services_all

contrib/completion/zsh/_docker-compose

@@ -0,0 +1,304 @@
+#compdef docker-compose
+
+# Description
+# -----------
+#  zsh completion for docker-compose
+#  https://github.com/sdurrheimer/docker-compose-zsh-completion
+# -------------------------------------------------------------------------
+# Version
+# -------
+#  0.1.0
+# -------------------------------------------------------------------------
+# Authors
+# -------
+#  * Steve Durrheimer <s.durrheimer@gmail.com>
+# -------------------------------------------------------------------------
+# Inspiration
+# -----------
+#  * @albers docker-compose bash completion script
+#  * @felixr docker zsh completion script : https://github.com/felixr/docker-zsh-completion
+# -------------------------------------------------------------------------
+
+# For compatibility reasons, Compose and therefore its completion supports several
+# stack compositon files as listed here, in descending priority.
+# Support for these filenames might be dropped in some future version.
+__docker-compose_compose_file() {
+    local file
+    for file in docker-compose.y{,a}ml fig.y{,a}ml ; do
+        [ -e $file ] && {
+            echo $file
+            return
+        }
+    done
+    echo docker-compose.yml
+}
+
+# Extracts all service names from docker-compose.yml.
+___docker-compose_all_services_in_compose_file() {
+    local already_selected
+    local -a services
+    already_selected=$(echo ${words[@]} | tr " " "|")
+    awk -F: '/^[a-zA-Z0-9]/{print $1}' "${compose_file:-$(__docker-compose_compose_file)}" 2>/dev/null | grep -Ev "$already_selected"
+}
+
+# All services, even those without an existing container
+__docker-compose_services_all() {
+    services=$(___docker-compose_all_services_in_compose_file)
+    _alternative "args:services:($services)"
+}
+
+# All services that have an entry with the given key in their docker-compose.yml section
+___docker-compose_services_with_key() {
+    local already_selected
+    local -a buildable
+    already_selected=$(echo ${words[@]} | tr " " "|")
+    # flatten sections to one line, then filter lines containing the key and return section name.
+    awk '/^[a-zA-Z0-9]/{printf "\n"};{printf $0;next;}' "${compose_file:-$(__docker-compose_compose_file)}" 2>/dev/null | awk -F: -v key=": +$1:" '$0 ~ key {print $1}' 2>/dev/null | grep -Ev "$already_selected"
+}
+
+# All services that are defined by a Dockerfile reference
+__docker-compose_services_from_build() {
+    buildable=$(___docker-compose_services_with_key build)
+    _alternative "args:buildable services:($buildable)"
+}
+
+# All services that are defined by an image
+__docker-compose_services_from_image() {
+    pullable=$(___docker-compose_services_with_key image)
+    _alternative "args:pullable services:($pullable)"
+}
+
+__docker-compose_get_services() {
+    local kind expl
+    declare -a running stopped lines args services
+
+    docker_status=$(docker ps > /dev/null 2>&1)
+    if [ $? -ne 0 ]; then
+        _message "Error! Docker is not running."
+        return 1
+    fi
+
+    kind=$1
+    shift
+    [[ $kind = (stopped|all) ]] && args=($args -a)
+
+    lines=(${(f)"$(_call_program commands docker ps ${args})"})
+    services=(${(f)"$(_call_program commands docker-compose 2>/dev/null ${compose_file:+-f $compose_file} ${compose_project:+-p $compose_project} ps -q)"})
+
+    # Parse header line to find columns
+    local i=1 j=1 k header=${lines[1]}
+    declare -A begin end
+    while (( $j < ${#header} - 1 )) {
+        i=$(( $j + ${${header[$j,-1]}[(i)[^ ]]} - 1))
+        j=$(( $i + ${${header[$i,-1]}[(i)  ]} - 1))
+        k=$(( $j + ${${header[$j,-1]}[(i)[^ ]]} - 2))
+        begin[${header[$i,$(($j-1))]}]=$i
+        end[${header[$i,$(($j-1))]}]=$k
+    }
+    lines=(${lines[2,-1]})
+
+    # Container ID
+    local line s name
+    local -a names
+    for line in $lines; do
+        if [[ $services == *"${line[${begin[CONTAINER ID]},${end[CONTAINER ID]}]%% ##}"* ]]; then
+            names=(${(ps:,:)${${line[${begin[NAMES]},-1]}%% *}})
+            for name in $names; do
+                s="${${name%_*}#*_}:${(l:15:: :::)${${line[${begin[CREATED]},${end[CREATED]}]/ ago/}%% ##}}"
+                s="$s, ${line[${begin[CONTAINER ID]},${end[CONTAINER ID]}]%% ##}"
+                s="$s, ${${${line[$begin[IMAGE],$end[IMAGE]]}/:/\\:}%% ##}"
+                if [[ ${line[${begin[STATUS]},${end[STATUS]}]} = Exit* ]]; then
+                    stopped=($stopped $s)
+                else
+                    running=($running $s)
+                fi
+            done
+        fi
+    done
+
+    [[ $kind = (running|all) ]] && _describe -t services-running "running services" running
+    [[ $kind = (stopped|all) ]] && _describe -t services-stopped "stopped services" stopped
+}
+
+__docker-compose_stoppedservices() {
+    __docker-compose_get_services stopped "$@"
+}
+
+__docker-compose_runningservices() {
+    __docker-compose_get_services running "$@"
+}
+
+__docker-compose_services () {
+    __docker-compose_get_services all "$@"
+}
+
+__docker-compose_caching_policy() {
+    oldp=( "$1"(Nmh+1) )     # 1 hour
+    (( $#oldp ))
+}
+
+__docker-compose_commands () {
+    local cache_policy
+
+    zstyle -s ":completion:${curcontext}:" cache-policy cache_policy
+    if [[ -z "$cache_policy" ]]; then
+        zstyle ":completion:${curcontext}:" cache-policy __docker-compose_caching_policy
+    fi
+
+    if ( [[ ${+_docker_compose_subcommands} -eq 0 ]] || _cache_invalid docker_compose_subcommands) \
+        && ! _retrieve_cache docker_compose_subcommands;
+    then
+        local -a lines
+        lines=(${(f)"$(_call_program commands docker-compose 2>&1)"})
+        _docker_compose_subcommands=(${${${lines[$((${lines[(i)Commands:]} + 1)),${lines[(I)  *]}]}## #}/ ##/:})
+        _store_cache docker_compose_subcommands _docker_compose_subcommands
+    fi
+    _describe -t docker-compose-commands "docker-compose command" _docker_compose_subcommands
+}
+
+__docker-compose_subcommand () {
+    local -a _command_args
+    integer ret=1
+    case "$words[1]" in
+        (build)
+            _arguments \
+                '--no-cache[Do not use cache when building the image]' \
+                '*:services:__docker-compose_services_from_build' && ret=0
+            ;;
+        (help)
+            _arguments ':subcommand:__docker-compose_commands' && ret=0
+            ;;
+        (kill)
+            _arguments \
+                '-s[SIGNAL to send to the container. Default signal is SIGKILL.]:signal:_signals' \
+                '*:running services:__docker-compose_runningservices' && ret=0
+            ;;
+        (logs)
+            _arguments \
+                '--no-color[Produce monochrome output.]' \
+                '*:services:__docker-compose_services_all' && ret=0
+            ;;
+        (migrate-to-labels)
+            _arguments \
+                '(-):Recreate containers to add labels' && ret=0
+            ;;
+        (port)
+            _arguments \
+                '--protocol=-[tcp or udap (defaults to tcp)]:protocol:(tcp udp)' \
+                '--index=-[index of the container if there are mutiple instances of a service (defaults to 1)]:index: ' \
+                '1:running services:__docker-compose_runningservices' \
+                '2:port:_ports' && ret=0
+            ;;
+        (ps)
+            _arguments \
+                '-q[Only display IDs]' \
+                '*:services:__docker-compose_services_all' && ret=0
+            ;;
+        (pull)
+            _arguments \
+                '--allow-insecure-ssl[Allow insecure connections to the docker registry]' \
+                '*:services:__docker-compose_services_from_image' && ret=0
+            ;;
+        (rm)
+            _arguments \
+                '(-f --force)'{-f,--force}"[Don't ask to confirm removal]" \
+                '-v[Remove volumes associated with containers]' \
+                '*:stopped services:__docker-compose_stoppedservices' && ret=0
+            ;;
+        (run)
+            _arguments \
+                '--allow-insecure-ssl[Allow insecure connections to the docker registry]' \
+                '-d[Detached mode: Run container in the background, print new container name.]' \
+                '--entrypoint[Overwrite the entrypoint of the image.]:entry point: ' \
+                '*-e[KEY=VAL Set an environment variable (can be used multiple times)]:environment variable KEY=VAL: ' \
+                '(-u --user)'{-u,--user=-}'[Run as specified username or uid]:username or uid:_users' \
+                "--no-deps[Don't start linked services.]" \
+                '--rm[Remove container after run. Ignored in detached mode.]' \
+                "--service-ports[Run command with the service's ports enabled and mapped to the host.]" \
+                '-T[Disable pseudo-tty allocation. By default `docker-compose run` allocates a TTY.]' \
+                '(-):services:__docker-compose_services' \
+                '(-):command: _command_names -e' \
+                '*::arguments: _normal' && ret=0
+            ;;
+        (scale)
+            _arguments '*:running services:__docker-compose_runningservices' && ret=0
+            ;;
+        (start)
+            _arguments '*:stopped services:__docker-compose_stoppedservices' && ret=0
+            ;;
+        (stop|restart)
+            _arguments \
+                '(-t --timeout)'{-t,--timeout}"[Specify a shutdown timeout in seconds. (default: 10)]:seconds: " \
+                '*:running services:__docker-compose_runningservices' && ret=0
+            ;;
+        (up)
+            _arguments \
+                '--allow-insecure-ssl[Allow insecure connections to the docker registry]' \
+                '-d[Detached mode: Run containers in the background, print new container names.]' \
+                '--no-color[Produce monochrome output.]' \
+                "--no-deps[Don't start linked services.]" \
+                "--no-recreate[If containers already exist, don't recreate them.]" \
+                "--no-build[Don't build an image, even if it's missing]" \
+                '(-t --timeout)'{-t,--timeout}"[Specify a shutdown timeout in seconds. (default: 10)]:seconds: " \
+                "--x-smart-recreate[Only recreate containers whose configuration or image needs to be updated. (EXPERIMENTAL)]" \
+                '*:services:__docker-compose_services_all' && ret=0
+            ;;
+        (*)
+            _message 'Unknown sub command'
+    esac
+
+    return ret
+}
+
+_docker-compose () {
+    # Support for subservices, which allows for `compdef _docker docker-shell=_docker_containers`.
+    # Based on /usr/share/zsh/functions/Completion/Unix/_git without support for `ret`.
+    if [[ $service != docker-compose ]]; then
+        _call_function - _$service
+        return
+    fi
+
+    local curcontext="$curcontext" state line ret=1
+    typeset -A opt_args
+
+    _arguments -C \
+        '(- :)'{-h,--help}'[Get help]' \
+        '--verbose[Show more output]' \
+        '(- :)'{-v,--version}'[Print version and exit]' \
+        '(-f --file)'{-f,--file}'[Specify an alternate docker-compose file (default: docker-compose.yml)]:file:_files -g "*.yml"' \
+        '(-p --project-name)'{-p,--project-name}'[Specify an alternate project name (default: directory name)]:project name:' \
+        '(-): :->command' \
+        '(-)*:: :->option-or-argument' && ret=0
+
+    local counter=1
+    #local compose_file compose_project
+    while [ $counter -lt ${#words[@]} ]; do
+        case "${words[$counter]}" in
+            -f|--file)
+                (( counter++ ))
+                compose_file="${words[$counter]}"
+                ;;
+            -p|--project-name)
+                (( counter++ ))
+                compose_project="${words[$counter]}"
+                ;;
+            *)
+                ;;
+        esac
+        (( counter++ ))
+    done
+
+    case $state in
+        (command)
+            __docker-compose_commands && ret=0
+            ;;
+        (option-or-argument)
+            curcontext=${curcontext%:*:*}:docker-compose-$words[1]:
+            __docker-compose_subcommand && ret=0
+            ;;
+    esac
+
+    return ret
+}
+
+_docker-compose "$@"

docs/Dockerfile

@@ -1,15 +1,24 @@
-FROM docs/base:latest
-MAINTAINER Sven Dowideit <SvenDowideit@docker.com> (@SvenDowideit)
+FROM docs/base:hugo
+MAINTAINER Mary Anthony <mary@docker.com> (@moxiegirl)
 
-# to get the git info for this repo
+# To get the git info for this repo
 COPY . /src
 
-# Reset the /docs dir so we can replace the theme meta with the new repo's git info
-RUN git reset --hard
+COPY . /docs/content/compose/
 
-RUN grep "__version" /src/compose/__init__.py | sed "s/.*'\(.*\)'/\1/" > /docs/VERSION
-COPY docs/* /docs/sources/compose/
-COPY docs/mkdocs.yml /docs/mkdocs-compose.yml
-
-# Then build everything together, ready for mkdocs
-RUN /docs/build.sh
+# Sed to process GitHub Markdown
+# 1-2 Remove comment code from metadata block
+# 3 Remove .md extension from link text
+# 4 Change ](/ to ](/project/ in links
+# 5 Change ](word) to ](/project/word)
+# 6 Change ](../../ to ](/project/
+# 7 Change ](../ to ](/project/word)
+# 
+# 
+RUN find /docs/content/compose -type f -name "*.md" -exec sed -i.old \
+    -e '/^<!.*metadata]>/g' \
+    -e '/^<!.*end-metadata.*>/g' \
+    -e 's/\([(]\)\(.*\)\(\.md\)/\1\2/g' \
+    -e 's/\(\]\)\([(]\)\(\/\)/\1\2\/compose\//g' \
+    -e 's/\(\][(]\)\([A-z]*[)]\)/\]\(\/compose\/\2/g' \
+    -e 's/\(\][(]\)\(\.\.\/\)/\1\/compose\//g' {} \;

docs/Makefile

@@ -0,0 +1,55 @@
+.PHONY: all binary build cross default docs docs-build docs-shell shell test test-unit test-integration test-integration-cli test-docker-py validate
+
+# env vars passed through directly to Docker's build scripts
+# to allow things like `make DOCKER_CLIENTONLY=1 binary` easily
+# `docs/sources/contributing/devenvironment.md ` and `project/PACKAGERS.md` have some limited documentation of some of these
+DOCKER_ENVS := \
+	-e BUILDFLAGS \
+	-e DOCKER_CLIENTONLY \
+	-e DOCKER_EXECDRIVER \
+	-e DOCKER_GRAPHDRIVER \
+	-e TESTDIRS \
+	-e TESTFLAGS \
+	-e TIMEOUT
+# note: we _cannot_ add "-e DOCKER_BUILDTAGS" here because even if it's unset in the shell, that would shadow the "ENV DOCKER_BUILDTAGS" set in our Dockerfile, which is very important for our official builds
+
+# to allow `make DOCSDIR=docs docs-shell` (to create a bind mount in docs)
+DOCS_MOUNT := $(if $(DOCSDIR),-v $(CURDIR)/$(DOCSDIR):/$(DOCSDIR))
+
+# to allow `make DOCSPORT=9000 docs`
+DOCSPORT := 8000
+
+# Get the IP ADDRESS
+DOCKER_IP=$(shell python -c "import urlparse ; print urlparse.urlparse('$(DOCKER_HOST)').hostname or ''")
+HUGO_BASE_URL=$(shell test -z "$(DOCKER_IP)" && echo localhost || echo "$(DOCKER_IP)")
+HUGO_BIND_IP=0.0.0.0
+
+GIT_BRANCH := $(shell git rev-parse --abbrev-ref HEAD 2>/dev/null)
+DOCKER_IMAGE := docker$(if $(GIT_BRANCH),:$(GIT_BRANCH))
+DOCKER_DOCS_IMAGE := docs-base$(if $(GIT_BRANCH),:$(GIT_BRANCH))
+
+
+DOCKER_RUN_DOCS := docker run --rm -it $(DOCS_MOUNT) -e AWS_S3_BUCKET -e NOCACHE
+
+# for some docs workarounds (see below in "docs-build" target)
+GITCOMMIT := $(shell git rev-parse --short HEAD 2>/dev/null)
+
+default: docs
+
+docs: docs-build
+	$(DOCKER_RUN_DOCS) -p $(if $(DOCSPORT),$(DOCSPORT):)8000 -e DOCKERHOST "$(DOCKER_DOCS_IMAGE)" hugo server --port=$(DOCSPORT) --baseUrl=$(HUGO_BASE_URL) --bind=$(HUGO_BIND_IP)
+
+docs-draft: docs-build
+	$(DOCKER_RUN_DOCS) -p $(if $(DOCSPORT),$(DOCSPORT):)8000 -e DOCKERHOST "$(DOCKER_DOCS_IMAGE)" hugo server --buildDrafts="true" --port=$(DOCSPORT) --baseUrl=$(HUGO_BASE_URL) --bind=$(HUGO_BIND_IP)
+
+
+docs-shell: docs-build
+	$(DOCKER_RUN_DOCS) -p $(if $(DOCSPORT),$(DOCSPORT):)8000 "$(DOCKER_DOCS_IMAGE)" bash
+
+
+docs-build:
+#	( git remote | grep -v upstream ) || git diff --name-status upstream/release..upstream/docs ./ > ./changed-files
+#	echo "$(GIT_BRANCH)" > GIT_BRANCH
+#	echo "$(AWS_S3_BUCKET)" > AWS_S3_BUCKET
+#	echo "$(GITCOMMIT)" > GITCOMMIT
+	docker build -t "$(DOCKER_DOCS_IMAGE)" .

docs/README.md

@@ -0,0 +1,77 @@
+# Contributing to the Docker Compose documentation
+
+The documentation in this directory is part of the [https://docs.docker.com](https://docs.docker.com) website.  Docker uses [the Hugo static generator](http://gohugo.io/overview/introduction/) to convert project Markdown files to a static HTML site. 
+
+You don't need to be a Hugo expert to contribute to the compose documentation. If you are familiar with Markdown, you can modify the content in the `docs` files.  
+
+If you want to add a new file or change the location of the document in the menu, you do need to know a little more.
+
+## Documentation contributing workflow
+
+1. Edit a Markdown file in the tree.
+
+2. Save your changes.
+
+3. Make sure you in your `docs` subdirectory.
+
+4. Build the documentation.
+
+        $ make docs
+         ---> ffcf3f6c4e97
+        Removing intermediate container a676414185e8
+        Successfully built ffcf3f6c4e97
+        docker run --rm -it  -e AWS_S3_BUCKET -e NOCACHE -p 8000:8000 -e DOCKERHOST "docs-base:test-tooling" hugo server --port=8000 --baseUrl=192.168.59.103 --bind=0.0.0.0
+        ERROR: 2015/06/13 MenuEntry's .Url is deprecated and will be removed in Hugo 0.15. Use .URL instead.
+        0 of 4 drafts rendered
+        0 future content 
+        12 pages created
+        0 paginator pages created
+        0 tags created
+        0 categories created
+        in 55 ms
+        Serving pages from /docs/public
+        Web Server is available at http://0.0.0.0:8000/
+        Press Ctrl+C to stop
+
+5. Open the available server in your browser.
+
+    The documentation server has the complete menu but only the Docker Compose
+    documentation resolves.  You can't access the other project docs from this
+    localized build.
+
+## Tips on Hugo metadata and menu positioning
+
+The top of each Docker Compose documentation file contains TOML metadata. The metadata is commented out to prevent it from appears in GitHub.
+
+    <!--[metadata]>
+    +++
+    title = "Extending services in Compose"
+    description = "How to use Docker Compose's extends keyword to share configuration between files and projects"
+    keywords = ["fig, composition, compose, docker, orchestration, documentation, docs"]
+    [menu.main]
+    parent="smn_workw_compose"
+    weight=2
+    +++
+    <![end-metadata]-->  
+
+The metadata alone has this structure:
+
+    +++
+    title = "Extending services in Compose"
+    description = "How to use Docker Compose's extends keyword to share configuration between files and projects"
+    keywords = ["fig, composition, compose, docker, orchestration, documentation, docs"]
+    [menu.main]
+    parent="smn_workw_compose"
+    weight=2
+    +++
+    
+The `[menu.main]` section refers to navigation defined [in the main Docker menu](https://github.com/docker/docs-base/blob/hugo/config.toml). This metadata says *add a menu item called* Extending services in Compose *to the menu with the* `smn_workdw_compose` *identifier*.  If you locate the menu in the configuration, you'll find *Create multi-container applications* is the menu title.
+
+You can move an article in the tree by specifying a new parent. You can shift the location of the item by changing its weight.  Higher numbers are heavier and shift the item to the bottom of menu. Low or no numbers shift it up.
+
+
+## Other key documentation repositories
+
+The `docker/docs-base` repository contains [the Hugo theme and menu configuration](https://github.com/docker/docs-base). If you open the `Dockerfile` you'll see the `make docs` relies on this as a base image for building the Compose documentation.
+    
+The `docker/docs.docker.com` repository contains [build system for building the Docker documentation site](https://github.com/docker/docs.docker.com). Fork this repository to build the entire documentation site.

docs/cli.md

@@ -1,9 +1,16 @@
-page_title: Compose CLI reference
-page_description: Compose CLI reference
-page_keywords: fig, composition, compose, docker, orchestration, cli, reference
+<!--[metadata]>
++++
+title = "Compose CLI reference"
+description = "Compose CLI reference"
+keywords = ["fig, composition, compose, docker, orchestration, cli,  reference"]
+[menu.main]
+identifier = "smn_install_compose"
+parent = "smn_compose_ref"	
++++
+<![end-metadata]-->
 
 
-# CLI reference
+# Compose CLI reference
 
 Most Docker Compose commands are run against one or more services. If
 the service is not specified, the command will apply to all services.
@@ -95,7 +102,9 @@ specify the `--no-deps` flag:
 
 Similarly, if you do want the service's ports to be created and mapped to the
 host, specify the `--service-ports` flag:
-	$ docker-compose run --service-ports web python manage.py shell
+
+    $ docker-compose run --service-ports web python manage.py shell
+
 
 ### scale
 
@@ -155,7 +164,7 @@ By default, if there are existing containers for a service, `docker-compose up`
 Several environment variables are available for you to configure Compose's behaviour.
 
 Variables starting with `DOCKER_` are the same as those used to configure the
-Docker command-line client. If you're using boot2docker, `$(boot2docker shellinit)`
+Docker command-line client. If you're using boot2docker, `eval "$(boot2docker shellinit)"`
 will set them to their correct values.
 
 ### COMPOSE\_PROJECT\_NAME
@@ -183,7 +192,7 @@ Configures the path to the `ca.pem`, `cert.pem`, and `key.pem` files used for TL
 
 ## Compose documentation
 
-- [User guide](index.md)
+- [User guide](compose-overview.md)
 - [Installing Compose](install.md)
 - [Get started with Django](django.md)
 - [Get started with Rails](rails.md)

docs/completion.md

@@ -1,25 +1,52 @@
----
-layout: default
-title: Command Completion
----
+<!--[metadata]>
++++
+title = "Command Completion"
+description = "Compose CLI reference"
+keywords = ["fig, composition, compose, docker, orchestration, cli,  reference"]
+[menu.main]
+parent="smn_workw_compose"
+weight=3
++++
+<![end-metadata]-->
 
-#Command Completion
+# Command Completion
 
 Compose comes with [command completion](http://en.wikipedia.org/wiki/Command-line_completion)
-for the bash shell.
+for the bash and zsh shell.
 
-##Installing Command Completion
+## Installing Command Completion
+
+### Bash
 
 Make sure bash completion is installed. If you use a current Linux in a non-minimal installation, bash completion should be available.
 On a Mac, install with `brew install bash-completion`
- 
-Place the completion script in `/etc/bash_completion.d/` (`/usr/local/etc/bash_completion.d/` on a Mac), using e.g. 
 
-     curl -L https://raw.githubusercontent.com/docker/compose/1.2.0/contrib/completion/bash/docker-compose > /etc/bash_completion.d/docker-compose
- 
+Place the completion script in `/etc/bash_completion.d/` (`/usr/local/etc/bash_completion.d/` on a Mac), using e.g.
+
+     curl -L https://raw.githubusercontent.com/docker/compose/$(docker-compose --version | awk '{print $2}')/contrib/completion/bash/docker-compose > /etc/bash_completion.d/docker-compose
+
 Completion will be available upon next login.
 
-##Available completions
+### Zsh
+
+Place the completion script in your `/path/to/zsh/completion`, using e.g. `~/.zsh/completion/`
+
+    mkdir -p ~/.zsh/completion
+    curl -L https://raw.githubusercontent.com/docker/compose/$(docker-compose --version | awk '{print $2}')/contrib/completion/zsh/_docker-compose > ~/.zsh/completion/_docker-compose
+
+Include the directory in your `$fpath`, e.g. by adding in `~/.zshrc`
+
+    fpath=(~/.zsh/completion $fpath)
+
+Make sure `compinit` is loaded or do it by adding in `~/.zshrc`
+
+    autoload -Uz compinit && compinit -i
+
+Then reload your shell
+
+    exec $SHELL -l
+
+## Available completions
 
 Depending on what you typed on the command line so far, it will complete
 
@@ -32,11 +59,11 @@ Enjoy working with Compose faster and with less typos!
 
 ## Compose documentation
 
-- [User guide](index.md)
+- [User guide](compose-overview.md)
 - [Installing Compose](install.md)
 - [Get started with Django](django.md)
 - [Get started with Rails](rails.md)
 - [Get started with Wordpress](wordpress.md)
 - [Command line reference](cli.md)
 - [Yaml file reference](yml.md)
-- [Compose environment variables](env.md)
+- [Compose environment variables](env.md)

docs/compose-overview.md

@@ -1,11 +1,15 @@
-page_title: Compose: Multi-container orchestration for Docker
-page_description: Introduction and Overview of Compose
-page_keywords: documentation, docs,  docker, compose, orchestration, containers
+<!--[metadata]>
++++
+title = "Overview of Docker Compose"
+description = "Introduction and Overview of Compose"
+keywords = ["documentation, docs,  docker, compose, orchestration,  containers"]
+[menu.main]
+parent="smn_workw_compose"
++++
+<![end-metadata]-->
 
 
-# Docker Compose
-
-## Overview
+# Overview of Docker Compose
 
 Compose is a tool for defining and running multi-container applications with
 Docker. With Compose, you define a multi-container application in a single

docs/django.md

@@ -1,10 +1,16 @@
-page_title: Quickstart Guide: Compose and Django
-page_description: Getting started with Docker Compose and Django
-page_keywords: documentation, docs,  docker, compose, orchestration, containers,
-django
+<!--[metadata]>
++++
+title = "Quickstart Guide: Compose and Django"
+description = "Getting started with Docker Compose and Django"
+keywords = ["documentation, docs,  docker, compose, orchestration, containers"]
+[menu.main]
+parent="smn_workw_compose"
+weight=4
++++
+<![end-metadata]-->
 
 
-## Getting started with Compose and Django
+## Quickstart Guide: Compose and Django
 
 
 This Quick-start Guide will demonstrate how to use Compose to set up and run a
@@ -119,7 +125,7 @@ example, run `docker-compose up` and in another terminal run:
 
 ## More Compose documentation
 
-- [User guide](index.md)
+- [User guide](compose-overview.md)
 - [Installing Compose](install.md)
 - [Get started with Django](django.md)
 - [Get started with Rails](rails.md)

docs/env.md

@@ -1,9 +1,15 @@
----
-layout: default
-title: Compose environment variables reference
----
+<!--[metadata]>
++++
+title = "Compose environment variables reference"
+description = "Compose CLI reference"
+keywords = ["fig, composition, compose, docker, orchestration, cli,  reference"]
+[menu.main]
+parent="smn_compose_ref"
+weight=3
++++
+<![end-metadata]-->
 
-Environment variables reference
+# Compose environment variables reference
 ===============================
 
 **Note:** Environment variables are no longer the recommended method for connecting to linked services. Instead, you should use the link name (by default, the name of the linked service) as the hostname to connect to. See the [docker-compose.yml documentation](yml.md#links) for details.
@@ -34,7 +40,7 @@ Fully qualified container name, e.g. `DB_1_NAME=/myapp_web_1/myapp_db_1`
 
 ## Compose documentation
 
-- [User guide](index.md)
+- [User guide](compose-overview.md)
 - [Installing Compose](install.md)
 - [Get started with Django](django.md)
 - [Get started with Rails](rails.md)

docs/extends.md

@@ -1,6 +1,13 @@
-page_title: Extending services in Compose
-page_description: How to use Docker Compose's "extends" keyword to share configuration between files and projects
-page_keywords: fig, composition, compose, docker, orchestration, documentation, docs
+<!--[metadata]>
++++
+title = "Extending services in Compose"
+description = "How to use Docker Compose's extends keyword to share configuration between files and projects"
+keywords = ["fig, composition, compose, docker, orchestration, documentation, docs"]
+[menu.main]
+parent="smn_workw_compose"
+weight=2
++++
+<![end-metadata]-->
 
 
 ## Extending services in Compose
@@ -79,7 +86,7 @@ For full details on how to use `extends`, refer to the [reference](#reference).
 ### Example use case
 
 In this example, you’ll repurpose the example app from the [quick start
-guide](index.md). (If you're not familiar with Compose, it's recommended that
+guide](compose-overview.md). (If you're not familiar with Compose, it's recommended that
 you go through the quick start first.) This example assumes you want to use
 Compose both to develop an application locally and then deploy it to a
 production environment.
@@ -364,7 +371,7 @@ volumes:
 
 ## Compose documentation
 
-- [User guide](index.md)
+- [User guide](compose-overview.md)
 - [Installing Compose](install.md)
 - [Get started with Django](django.md)
 - [Get started with Rails](rails.md)

docs/install.md

@@ -1,14 +1,21 @@
-page_title: Installing Compose
-page_description: How to install Docker Compose
-page_keywords: compose, orchestration, install, installation, docker, documentation
+<!--[metadata]>
++++
+title = "Docker Compose"
+description = "How to install Docker Compose"
+keywords = ["compose, orchestration, install, installation, docker, documentation"]
+[menu.main]
+parent="mn_install"
+weight=4
++++
+<![end-metadata]-->
 
 
-## Installing Compose
+# Install Docker Compose
 
 To install Compose, you'll need to install Docker first. You'll then install
 Compose with a `curl` command. 
 
-### Install Docker
+## Install Docker
 
 First, install Docker version 1.6 or greater:
 
@@ -16,11 +23,11 @@ First, install Docker version 1.6 or greater:
 - [Instructions for Ubuntu](http://docs.docker.com/installation/ubuntulinux/)
 - [Instructions for other systems](http://docs.docker.com/installation/)
 
-### Install Compose
+## Install Compose
 
 To install Compose, run the following commands:
 
-    curl -L https://github.com/docker/compose/releases/download/1.2.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
+    curl -L https://github.com/docker/compose/releases/download/1.3.0rc3/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
     chmod +x /usr/local/bin/docker-compose
 
 > Note: If you get a "Permission denied" error, your `/usr/local/bin` directory probably isn't writable and you'll need to install Compose as the superuser. Run `sudo -i`, then the two commands above, then `exit`.
@@ -36,9 +43,21 @@ Compose can also be installed as a Python package:
 No further steps are required; Compose should now be successfully installed.
 You can test the installation by running `docker-compose --version`.
 
+### Upgrading
+
+If you're coming from Compose 1.2 or earlier, you'll need to remove or migrate your existing containers after upgrading Compose. This is because, as of version 1.3, Compose uses Docker labels to keep track of containers, and so they need to be recreated with labels added.
+
+If Compose detects containers that were created without labels, it will refuse to run so that you don't end up with two sets of them. If you want to keep using your existing containers (for example, because they have data volumes you want to preserve) you can migrate them with the following command:
+
+    docker-compose migrate-to-labels
+
+Alternatively, if you're not worried about keeping them, you can remove them - Compose will just create new ones.
+
+    docker rm -f myapp_web_1 myapp_db_1 ...
+
 ## Compose documentation
 
-- [User guide](index.md)
+- [User guide](compose-overview.md)
 - [Get started with Django](django.md)
 - [Get started with Rails](rails.md)
 - [Get started with Wordpress](wordpress.md)

docs/mkdocs.yml

@@ -1,12 +0,0 @@
-
-- ['compose/index.md', 'User Guide', 'Docker Compose' ]
-- ['compose/production.md', 'User Guide', 'Using Compose in production' ]
-- ['compose/extends.md', 'User Guide', 'Extending services in Compose']
-- ['compose/install.md', 'Installation', 'Docker Compose']
-- ['compose/cli.md', 'Reference', 'Compose command line']
-- ['compose/yml.md', 'Reference', 'Compose yml']
-- ['compose/env.md', 'Reference', 'Compose ENV variables']
-- ['compose/completion.md', 'Reference', 'Compose commandline completion']
-- ['compose/django.md', 'Examples', 'Getting started with Compose and Django']
-- ['compose/rails.md', 'Examples', 'Getting started with Compose and Rails']
-- ['compose/wordpress.md', 'Examples', 'Getting started with Compose and Wordpress']

docs/production.md

@@ -1,6 +1,13 @@
-page_title: Using Compose in production
-page_description: Guide to using Docker Compose in production
-page_keywords: documentation, docs,  docker, compose, orchestration, containers, production
+<!--[metadata]>
++++
+title = "Using Compose in production"
+description = "Guide to using Docker Compose in production"
+keywords = ["documentation, docs,  docker, compose, orchestration, containers,  production"]
+[menu.main]
+parent="smn_workw_compose"
+weight=1
++++
+<![end-metadata]-->
 
 
 ## Using Compose in production

docs/rails.md

@@ -1,10 +1,15 @@
-page_title: Quickstart Guide: Compose and Rails
-page_description: Getting started with Docker Compose and Rails
-page_keywords: documentation, docs,  docker, compose, orchestration, containers,
-rails
+<!--[metadata]>
++++
+title = "Quickstart Guide: Compose and Rails"
+description = "Getting started with Docker Compose and Rails"
+keywords = ["documentation, docs,  docker, compose, orchestration, containers"]
+[menu.main]
+parent="smn_workw_compose"
+weight=5
++++
+<![end-metadata]-->
 
-
-## Getting started with Compose and Rails
+## Quickstart Guide: Compose and Rails
 
 This Quickstart guide will show you how to use Compose to set up and run a Rails/PostgreSQL app. Before starting, you'll need to have [Compose installed](install.md).
 
@@ -119,7 +124,7 @@ you're using Boot2docker, `boot2docker ip` will tell you its address).
 
 ## More Compose documentation
 
-- [User guide](index.md)
+- [User guide](compose-overview.md)
 - [Installing Compose](install.md)
 - [Get started with Django](django.md)
 - [Get started with Rails](rails.md)

docs/wordpress.md

@@ -1,14 +1,21 @@
-page_title: Quickstart Guide: Compose and Wordpress
-page_description: Getting started with Docker Compose and Rails
-page_keywords: documentation, docs,  docker, compose, orchestration, containers,
-wordpress
+<!--[metadata]>
++++
+title = "Quickstart Guide: Compose and Wordpress"
+description = "Getting started with Compose and Wordpress"
+keywords = ["documentation, docs,  docker, compose, orchestration, containers"]
+[menu.main]
+parent="smn_workw_compose"
+weight=6
++++
+<![end-metadata]-->
 
-## Getting started with Compose and Wordpress
+
+# Quickstart Guide: Compose and Wordpress
 
 You can use Compose to easily run Wordpress in an isolated environment built
 with Docker containers. 
 
-### Define the project
+## Define the project
 
 First, [Install Compose](install.md) and then download Wordpress into the
 current directory:
@@ -114,7 +121,7 @@ address).
 
 ## More Compose documentation
 
-- [User guide](index.md)
+- [User guide](compose-overview.md)
 - [Installing Compose](install.md)
 - [Get started with Django](django.md)
 - [Get started with Rails](rails.md)

docs/yml.md

@@ -1,10 +1,13 @@
----
-layout: default
-title: docker-compose.yml reference
-page_title: docker-compose.yml reference
-page_description: docker-compose.yml reference
-page_keywords: fig, composition, compose, docker
----
+<!--[metadata]>
++++
+title = "docker-compose.yml reference"
+description = "docker-compose.yml reference"
+keywords = ["fig, composition, compose,  docker"]
+[menu.main]
+parent="smn_compose_ref"
++++
+<![end-metadata]-->
+
 
 # docker-compose.yml reference
 
@@ -390,7 +393,7 @@ read_only: true
 
 ## Compose documentation
 
-- [User guide](index.md)
+- [User guide](compose-overview.md)
 - [Installing Compose](install.md)
 - [Get started with Django](django.md)
 - [Get started with Rails](rails.md)

requirements.txt

@@ -1,6 +1,6 @@
 PyYAML==3.10
-docker-py==1.2.2
-dockerpty==0.3.3
+docker-py==1.2.3-rc1
+dockerpty==0.3.4
 docopt==0.6.1
 requests==2.6.1
 six==1.7.3

script/build-osx

@@ -1,7 +1,10 @@
 #!/bin/bash
 set -ex
+
+PATH="/usr/local/bin:$PATH"
+
 rm -rf venv
-virtualenv venv
+virtualenv -p /usr/local/bin/python venv
 venv/bin/pip install -r requirements.txt
 venv/bin/pip install -r requirements-dev.txt
 venv/bin/pip install .

script/prepare-osx

@@ -0,0 +1,53 @@
+#!/bin/bash
+
+set -ex
+
+python_version() {
+  python -V 2>&1
+}
+
+openssl_version() {
+  python -c "import ssl; print ssl.OPENSSL_VERSION"
+}
+
+desired_python_version="2.7.9"
+desired_python_brew_version="2.7.9"
+python_formula="https://raw.githubusercontent.com/Homebrew/homebrew/1681e193e4d91c9620c4901efd4458d9b6fcda8e/Library/Formula/python.rb"
+
+desired_openssl_version="1.0.1j"
+desired_openssl_brew_version="1.0.1j_1"
+openssl_formula="https://raw.githubusercontent.com/Homebrew/homebrew/62fc2a1a65e83ba9dbb30b2e0a2b7355831c714b/Library/Formula/openssl.rb"
+
+PATH="/usr/local/bin:$PATH"
+
+if !(which brew); then
+  ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
+fi
+
+brew update
+
+if !(python_version | grep "$desired_python_version"); then
+  if brew list | grep python; then
+    brew unlink python
+  fi
+
+  brew install "$python_formula"
+  brew switch python "$desired_python_brew_version"
+fi
+
+if !(openssl_version | grep "$desired_openssl_version"); then
+  if brew list | grep openssl; then
+    brew unlink openssl
+  fi
+
+  brew install "$openssl_formula"
+  brew switch openssl "$desired_openssl_brew_version"
+fi
+
+echo "*** Using $(python_version)"
+echo "*** Using $(openssl_version)"
+
+if !(which virtualenv); then
+  pip install virtualenv
+fi
+

script/test

@@ -9,9 +9,9 @@ docker build -t "$TAG" .
 docker run \
   --rm \
   --volume="/var/run/docker.sock:/var/run/docker.sock" \
-  --volume="$(pwd):/code" \
   -e DOCKER_VERSIONS \
   -e "TAG=$TAG" \
+  -e "affinity:image==$TAG" \
   --entrypoint="script/test-versions" \
   "$TAG" \
   "$@"

setup.py

@@ -30,8 +30,8 @@ install_requires = [
     'requests >= 2.6.1, < 2.7',
     'texttable >= 0.8.1, < 0.9',
     'websocket-client >= 0.11.0, < 1.0',
-    'docker-py >= 1.2.2, < 1.3',
-    'dockerpty >= 0.3.3, < 0.4',
+    'docker-py >= 1.2.3-rc1, < 1.3',
+    'dockerpty >= 0.3.4, < 0.4',
     'six >= 1.3.0, < 2',
 ]
 

tests/fixtures/ports-composefile-scale/docker-compose.yml

@@ -0,0 +1,6 @@
+
+simple:
+  image: busybox:latest
+  command: /bin/sleep 300
+  ports:
+    - '3000'

tests/integration/cli_test.py

@@ -1,6 +1,8 @@
 from __future__ import absolute_import
+from operator import attrgetter
 import sys
 import os
+import shlex
 
 from six import StringIO
 from mock import patch
@@ -240,8 +242,8 @@ class CLITestCase(DockerClientTestCase):
         service = self.project.get_service(name)
         container = service.containers(stopped=True, one_off=True)[0]
         self.assertEqual(
-            container.human_readable_command,
-            u'/bin/echo helloworld'
+            shlex.split(container.human_readable_command),
+            [u'/bin/echo', u'helloworld'],
         )
 
     @patch('dockerpty.start')
@@ -360,22 +362,22 @@ class CLITestCase(DockerClientTestCase):
         self.assertEqual(len(service.containers(stopped=True)), 1)
         self.assertFalse(service.containers(stopped=True)[0].is_running)
 
-    def test_kill_signal_sigint(self):
+    def test_kill_signal_sigstop(self):
         self.command.dispatch(['up', '-d'], None)
         service = self.project.get_service('simple')
         self.assertEqual(len(service.containers()), 1)
         self.assertTrue(service.containers()[0].is_running)
 
-        self.command.dispatch(['kill', '-s', 'SIGINT'], None)
+        self.command.dispatch(['kill', '-s', 'SIGSTOP'], None)
 
         self.assertEqual(len(service.containers()), 1)
-        # The container is still running. It has been only interrupted
+        # The container is still running. It has only been paused
         self.assertTrue(service.containers()[0].is_running)
 
-    def test_kill_interrupted_service(self):
+    def test_kill_stopped_service(self):
         self.command.dispatch(['up', '-d'], None)
         service = self.project.get_service('simple')
-        self.command.dispatch(['kill', '-s', 'SIGINT'], None)
+        self.command.dispatch(['kill', '-s', 'SIGSTOP'], None)
         self.assertTrue(service.containers()[0].is_running)
 
         self.command.dispatch(['kill', '-s', 'SIGKILL'], None)
@@ -435,6 +437,27 @@ class CLITestCase(DockerClientTestCase):
         self.assertEqual(get_port(3001), "0.0.0.0:49152")
         self.assertEqual(get_port(3002), "")
 
+    def test_port_with_scale(self):
+
+        self.command.base_dir = 'tests/fixtures/ports-composefile-scale'
+        self.command.dispatch(['scale', 'simple=2'], None)
+        containers = sorted(
+            self.project.containers(service_names=['simple']),
+            key=attrgetter('name'))
+
+        @patch('sys.stdout', new_callable=StringIO)
+        def get_port(number, mock_stdout, index=None):
+            if index is None:
+                self.command.dispatch(['port', 'simple', str(number)], None)
+            else:
+                self.command.dispatch(['port', '--index=' + str(index), 'simple', str(number)], None)
+            return mock_stdout.getvalue().rstrip()
+
+        self.assertEqual(get_port(3000), containers[0].get_local_port(3000))
+        self.assertEqual(get_port(3000, index=1), containers[0].get_local_port(3000))
+        self.assertEqual(get_port(3000, index=2), containers[1].get_local_port(3000))
+        self.assertEqual(get_port(3002), "")
+
     def test_env_file_relative_to_compose_file(self):
         config_path = os.path.abspath('tests/fixtures/env-file/docker-compose.yml')
         self.command.dispatch(['-f', config_path, 'up', '-d'], None)

tests/integration/resilience_test.py

@@ -0,0 +1,37 @@
+from __future__ import unicode_literals
+from __future__ import absolute_import
+
+import mock
+
+from compose.project import Project
+from .testcases import DockerClientTestCase
+
+
+class ResilienceTest(DockerClientTestCase):
+    def test_recreate_fails(self):
+        db = self.create_service('db', volumes=['/var/db'], command='top')
+        project = Project('composetest', [db], self.client)
+
+        container = db.create_container()
+        db.start_container(container)
+        host_path = container.get('Volumes')['/var/db']
+
+        project.up()
+        container = db.containers()[0]
+        self.assertEqual(container.get('Volumes')['/var/db'], host_path)
+
+        with mock.patch('compose.service.Service.create_container', crash):
+            with self.assertRaises(Crash):
+                project.up()
+
+        project.up()
+        container = db.containers()[0]
+        self.assertEqual(container.get('Volumes')['/var/db'], host_path)
+
+
+class Crash(Exception):
+    pass
+
+
+def crash(*args, **kwargs):
+    raise Crash()

tests/integration/service_test.py

@@ -17,7 +17,6 @@ from compose.const import (
     LABEL_VERSION,
 )
 from compose.service import (
-    CannotBeScaledError,
     ConfigError,
     Service,
     build_extra_hosts,
@@ -502,10 +501,10 @@ class ServiceTest(DockerClientTestCase):
             ],
         })
 
-    def test_start_with_image_id(self):
+    def test_create_with_image_id(self):
         # Image id for the current busybox:latest
         service = self.create_service('foo', image='8c2e06607696')
-        self.assertTrue(service.start_or_create_containers())
+        service.create_container()
 
     def test_scale(self):
         service = self.create_service('web')
@@ -526,10 +525,6 @@ class ServiceTest(DockerClientTestCase):
         service.scale(0)
         self.assertEqual(len(service.containers()), 0)
 
-    def test_scale_on_service_that_cannot_be_scaled(self):
-        service = self.create_service('web', ports=['8000:8000'])
-        self.assertRaises(CannotBeScaledError, lambda: service.scale(1))
-
     def test_scale_sets_ports(self):
         service = self.create_service('web', ports=['8000'])
         service.scale(2)

tests/unit/progress_stream_test.py

@@ -17,3 +17,21 @@ class ProgressStreamTestCase(unittest.TestCase):
         ]
         events = progress_stream.stream_output(output, StringIO())
         self.assertEqual(len(events), 1)
+
+    def test_stream_output_div_zero(self):
+        output = [
+            '{"status": "Downloading", "progressDetail": {"current": '
+            '0, "start": 1413653874, "total": 0}, '
+            '"progress": "..."}',
+        ]
+        events = progress_stream.stream_output(output, StringIO())
+        self.assertEqual(len(events), 1)
+
+    def test_stream_output_null_total(self):
+        output = [
+            '{"status": "Downloading", "progressDetail": {"current": '
+            '0, "start": 1413653874, "total": null}, '
+            '"progress": "..."}',
+        ]
+        events = progress_stream.stream_output(output, StringIO())
+        self.assertEqual(len(events), 1)

tests/unit/service_test.py

@@ -331,9 +331,7 @@ class ServiceVolumesTest(unittest.TestCase):
 
     def test_build_volume_binding(self):
         binding = build_volume_binding(parse_volume_spec('/outside:/inside'))
-        self.assertEqual(
-            binding,
-            ('/outside', dict(bind='/inside', ro=False)))
+        self.assertEqual(binding, ('/inside', '/outside:/inside:rw'))
 
     def test_get_container_data_volumes(self):
         options = [
@@ -360,8 +358,8 @@ class ServiceVolumesTest(unittest.TestCase):
         }, has_been_inspected=True)
 
         expected = {
-            '/var/lib/docker/aaaaaaaa': {'bind': '/existing/volume', 'ro': False},
-            '/var/lib/docker/cccccccc': {'bind': '/mnt/image/data', 'ro': False},
+            '/existing/volume': '/var/lib/docker/aaaaaaaa:/existing/volume:rw',
+            '/mnt/image/data': '/var/lib/docker/cccccccc:/mnt/image/data:rw',
         }
 
         binds = get_container_data_volumes(container, options)
@@ -384,11 +382,78 @@ class ServiceVolumesTest(unittest.TestCase):
             'Volumes': {'/existing/volume': '/var/lib/docker/aaaaaaaa'},
         }, has_been_inspected=True)
 
-        expected = {
-            '/host/volume': {'bind': '/host/volume', 'ro': True},
-            '/host/rw/volume': {'bind': '/host/rw/volume', 'ro': False},
-            '/var/lib/docker/aaaaaaaa': {'bind': '/existing/volume', 'ro': False},
-        }
+        expected = [
+            '/host/volume:/host/volume:ro',
+            '/host/rw/volume:/host/rw/volume:rw',
+            '/var/lib/docker/aaaaaaaa:/existing/volume:rw',
+        ]
 
         binds = merge_volume_bindings(options, intermediate_container)
-        self.assertEqual(binds, expected)
+        self.assertEqual(set(binds), set(expected))
+
+    def test_mount_same_host_path_to_two_volumes(self):
+        service = Service(
+            'web',
+            image='busybox',
+            volumes=[
+                '/host/path:/data1',
+                '/host/path:/data2',
+            ],
+            client=self.mock_client,
+        )
+
+        self.mock_client.inspect_image.return_value = {
+            'Id': 'ababab',
+            'ContainerConfig': {
+                'Volumes': {}
+            }
+        }
+
+        create_options = service._get_container_create_options(
+            override_options={},
+            number=1,
+        )
+
+        self.assertEqual(
+            set(create_options['host_config']['Binds']),
+            set([
+                '/host/path:/data1:rw',
+                '/host/path:/data2:rw',
+            ]),
+        )
+
+    def test_different_host_path_in_container_json(self):
+        service = Service(
+            'web',
+            image='busybox',
+            volumes=['/host/path:/data'],
+            client=self.mock_client,
+        )
+
+        self.mock_client.inspect_image.return_value = {
+            'Id': 'ababab',
+            'ContainerConfig': {
+                'Volumes': {
+                    '/data': {},
+                }
+            }
+        }
+
+        self.mock_client.inspect_container.return_value = {
+            'Id': '123123123',
+            'Image': 'ababab',
+            'Volumes': {
+                '/data': '/mnt/sda1/host/path',
+            },
+        }
+
+        create_options = service._get_container_create_options(
+            override_options={},
+            number=1,
+            previous_container=Container(self.mock_client, {'Id': '123123123'}),
+        )
+
+        self.assertEqual(
+            create_options['host_config']['Binds'],
+            ['/mnt/sda1/host/path:/data:rw'],
+        )