- The `raises` kwarg to the `cls_check` decorator was being used
incorrectly (it should be an exception class, not an object).
- We need to check for `error.cause` and get the message out of the
exception object.
NB: The particular case where validation fails in the case of `ports` is
only when ranges don't match in length - no further validation is
currently performed client-side.
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
We were essentially only testing that *at least one* of the invalid
values fails the validation check, rather than that *all* of them fail.
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
Split env resolving into two phases. The first phase is to expand the paths
of env_files, which is done before merging extends. Once all files are merged
together, the final phase is to read the env_files and use them as the base
for environment variables.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Previously these empty objects would hit a bug in splitting objects causing it crash.
With this fix the empty objects are returned properly.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
See http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html
Environment variable names used by the utilities in the Shell and
Utilities volume of POSIX.1-2008 consist solely of uppercase letters,
digits, and the <underscore> ( '_' ) from the characters defined in
Portable Character Set and do not begin with a digit. Other characters may
be permitted by an implementation; applications shall tolerate the
presence of such names.
Signed-off-by: Joffrey F <joffrey@docker.com>
Refactors config validation of a service to use a ServiceConfig data object.
Instead of passing around a bunch of related scalars, we can use the
ServiceConfig object as a parameter to most of the service validation functions.
This allows for a fix to the config schema, where the name is a field in the
schema, but not actually in the configuration. My passing the name around as
part of the ServiceConfig object, we don't need to add it to the config options.
Fixes#2299
validate_against_service_schema() is moved from a conditional branch in
ServiceExtendsResolver() to happen as one of the last steps after all
configuration is merged. This schema only contains constraints which only need
to be true at the very end of merging.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
This bug can be seen by the change to the test case. When the extended service
uses a different name, the error was reported incorrectly.
By fixing this bug we can simplify self.signature and self.detect_cycles to
always use self.service_name.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
ServiceLoader has evolved to be not really all that related to "loading" a
service. It's responsibility is more to do with handling the `extends`
field, which is only part of loading. The class and its primary method
(make_service_dict()) were renamed to better reflect their responsibility.
As part of that change process_container_options() was removed from
make_service_dict() and renamed to process_service(). It contains logic for
handling the non-extends options.
This change allows us to remove the hacks from testcase.py and only call
the functions we need to format a service dict correctly for integration tests.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Consolidates all the top level config handling into `process_config_file` which
is now used for both files and merge sources.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Includes some refactoring of log_printer_test to support checking for flush(), and so that each test calls the unit-under-test directly, instead of through a helper function.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
It's a flag passed to docker build that removes the intermediate
containers left behind on fail builds.
Signed-off-by: Adrian Budau <budau.adi@gmail.com>
Remove some unnecessary newlines.
Remove a unittest that was attempting to test behaviour that was removed a while ago, so isn't testing anything.
Updated some unit tests to use mocks instead of a custom fake.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
These cli tests are now a different kind of that that run the compose binary. They are not the same as integration tests that test some internal interface.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
- make it a positional arg, since it's required
- make it the first argument for all functions that require it
- remove an unnecessary one-line function that was only called in one place
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Mutable objects are harder to debug and harder to reason about. ServiceLoader was almost immutable. There was just a single function which set fields for a second function. Instead of mutating the object, we can pass those values as parameters to the next function.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
- fixed a docstring to make it PEP257 compliant
- wrapped some long lines
- used a more specific error
Signed-off-by: Daniel Nephin <dnephin@docker.com>
It has been an unnecessary wrapper around container.start() for a little while now, so we can call it directly.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Ensure config_hash is updated when volumes_from mode is changed, and
service is recreated on next up as a result.
Signed-off-by: Joffrey F <joffrey@docker.com>
We were outputting an extra line, which in *some* cases, on *some*
terminals, was causing the output of parallel actions to get messed up.
In particular, it would happen when the terminal had just been cleared
or hadn't yet filled up with a screen's worth of text.
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
Start restructuring extends docs in preparation for adding documentation about using multiple compose files.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Code cleanup. We no longer need this as the api returns a 304 for any
stopped containers, which doesn't raise an error.
Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
So we're not displaying output of all previous logs for a container, we attach,
if possible, to a container before the container is started.
LogPrinter checks if a container has a log_stream already attached and
print from that rather than always attempting to attach one itself.
Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
Environment keys that contain no value, get populated with values taken
from the environment not from the build phase but from running the command `up`.
Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
People are likely to run into their env vars being set to empty strings,
if they're not aware that they need to escape them for Compose to not
interpolate them.
Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
I'm not sure if it should be there at all, but at least it should hardly be where it currently is located.
Signed-off-by: Per Persson <per.persson@exertisztorm.com>
If we get back an error that wasn't an APIError, it was causing the
thread to hang. This catch all, while I appreciate feels risky to
have a catch all, is better than not catching and silently failing,
with a never ending thread.
If something worse than an APIError has gone wrong, we want to stop
the incredible journey of what we're doing.
Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
Show a warning when a relative path is specified without "./"
(cherry picked from commit 52733f6996)
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
Conflicts:
compose/config.py
tests/unit/config_test.py
Contributors can build public docs with compose docs in context
(cherry picked from commit 487eae3b7b)
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
Show an error on 'run' when there are legacy one-off containers
(cherry picked from commit 81707ef1ad)
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
Fix smart recreate when 'image' is changed to something nonexistent
(cherry picked from commit 2bc10db545)
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
Fix bug where duplicate container is leftover after 'up' fails
(cherry picked from commit f42fd6a3ad)
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
Stop 'rm' and 'ps' listing services not defined in the current file
(cherry picked from commit d85688892c)
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
Validate that service names passed to Project.containers aren't bogus.
(cherry picked from commit bc14c473c9)
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
Test that data volumes now survive a crash when recreating
(cherry picked from commit 87c30ae6e4)
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
Possible division by zero error when pulling an image - fixes#1463
(cherry picked from commit d0e87929a1)
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
> **Note:**Compose's networking support is experimental, and must be explicitly enabled with the `docker-compose --x-networking` flag.
> **Note:**This document only applies if you're using v2 of the [Compose file format](compose-file.md). Networking features are not supported for legacy Compose files.
Compose sets up a single default
By default Compose sets up a single
[network](/engine/reference/commandline/network_create.md) for your app. Each
[network](/engine/reference/commandline/network_create.md) for your app. Each
container for a service joins the default network and is both *reachable* by
container for a service joins the default network and is both *reachable* by
other containers on that network, and *discoverable* by them at a hostname
other containers on that network, and *discoverable* by them at a hostname
identical to the container name.
identical to the container name.
> **Note:** Your app's network is given the same name as the "project name", which is based on the name of the directory it lives in. See the [Command line overview](reference/docker-compose.md) for how to override it.
> **Note:** Your app's network is given a name based on the "project name", which is based on the name of the directory it lives in. You can override the project name with either the [`--project-name` flag](reference/docker-compose.md) or the [`COMPOSE_PROJECT_NAME` environment variable](reference/overview.md#compose-project-name).
For example, suppose your app is in a directory called `myapp`, and your `docker-compose.yml` looks like this:
For example, suppose your app is in a directory called `myapp`, and your `docker-compose.yml` looks like this:
version: 2
services:
web:
web:
build: .
build: .
ports:
ports:
@ -31,64 +34,103 @@ For example, suppose your app is in a directory called `myapp`, and your `docker
db:
db:
image: postgres
image: postgres
When you run `docker-compose --x-networking up`, the following happens:
When you run `docker-compose up`, the following happens:
1. A network called `myapp` is created.
1. A network called `myapp_default` is created.
2. A container is created using `web`'s configuration. It joins the network
2. A container is created using `web`'s configuration. It joins the network
`myapp` under the name `myapp_web_1`.
`myapp_default` under the name `web`.
3. A container is created using `db`'s configuration. It joins the network
3. A container is created using `db`'s configuration. It joins the network
`myapp` under the name `myapp_db_1`.
`myapp_default` under the name `db`.
Each container can now look up the hostname `myapp_web_1` or `myapp_db_1` and
Each container can now look up the hostname `web` or `db` and
get back the appropriate container's IP address. For example, `web`'s
get back the appropriate container's IP address. For example, `web`'s
application code could connect to the URL `postgres://myapp_db_1:5432` and start
application code could connect to the URL `postgres://db:5432` and start
using the Postgres database.
using the Postgres database.
Because `web` explicitly maps a port, it's also accessible from the outside world via port 8000 on your Docker host's network interface.
Because `web` explicitly maps a port, it's also accessible from the outside world via port 8000 on your Docker host's network interface.
> **Note:** in the next release there will be additional aliases for the
> container, including a short name without the project name and container
> index. The full container name will remain as one of the alias for backwards
> compatibility.
## Updating containers
## Updating containers
If you make a configuration change to a service and run `docker-compose up` to update it, the old container will be removed and the new one will join the network under a different IP address but the same name. Running containers will be able to look up that name and connect to the new address, but the old address will stop working.
If you make a configuration change to a service and run `docker-compose up` to update it, the old container will be removed and the new one will join the network under a different IP address but the same name. Running containers will be able to look up that name and connect to the new address, but the old address will stop working.
If any containers have connections open to the old container, they will be closed. It is a container's responsibility to detect this condition, look up the name again and reconnect.
If any containers have connections open to the old container, they will be closed. It is a container's responsibility to detect this condition, look up the name again and reconnect.
## Configure how services are published
By default, containers for each service are published on the network with the
container name. If you want to change the name, or stop containers from being
discoverable at all, you can use the `container_name` option:
web:
build: .
container_name: "my-web-application"
## Links
## Links
Docker links are a one-way, single-host communication system. They should now be considered deprecated, and you should update your app to use networking instead. In the majority of cases, this will simply involve removing the `links` sections from your `docker-compose.yml`.
Docker links are a one-way, single-host communication system. They should now be considered deprecated, and as part of upgrading your app to the v2 format, you must remove any `links` sections from your `docker-compose.yml` and use service names (e.g. `web`, `db`) as the hostnames to connect to.
## Specifying the network driver
By default, Compose uses the `bridge` driver when creating the app’s network. The Docker Engine provides one other driver out-of-the-box: `overlay`, which implements secure communication between containers on different hosts (see the next section for how to set up and use the `overlay` driver). Docker also allows you to install [custom network drivers](/engine/extend/plugins_network.md).
You can specify which one to use with the `--x-network-driver` flag:
$ docker-compose --x-networking --x-network-driver=overlay up
<!--[metadata]>
## Multi-host networking
## Multi-host networking
(TODO: talk about Swarm and the overlay driver)
When deploying a Compose application to a Swarm cluster, you can make use of the built-in `overlay` driver to enable multi-host communication between containers with no changes to application code. Consult the [Getting started with multi-host networking](/engine/userguide/networking/get-started-overlay.md) to see how to set up the overlay driver, and then specify `driver: overlay` in your networking config (see the sections below for how to do this).
<![end-metadata]-->
## Specifying custom networks
Instead of just using the default app network, you can specify your own networks with the top-level `networks` key. This lets you create more complex topologies and specify [custom network drivers](/engine/extend/plugins_network.md) and options. You can also use it to connect services to externally-created networks which aren't managed by Compose.
Each service can specify what networks to connect to with the *service-level*`networks` key, which is a list of names referencing entries under the *top-level*`networks` key.
Here's an example Compose file defining several networks. The `proxy` service is the gateway to the outside world, via a network called `outside` which is expected to already exist. `proxy` is isolated from the `db` service, because they do not share a network in common - only `app` can talk to both.
version: 2
services:
proxy:
build: ./proxy
networks:
- outside
- front
app:
build: ./app
networks:
- front
- back
db:
image: postgres
networks:
- back
networks:
front:
# Use the overlay driver for multi-host communication
driver: overlay
back:
# Use a custom driver which takes special options
driver: my-custom-driver
options:
foo: "1"
bar: "2"
outside:
# The 'outside' network is expected to already exist - Compose will not
# attempt to create it
external: true
## Configuring the default network
Instead of (or as well as) specifying your own networks, you can also change the settings of the app-wide default network by defining an entry under `networks` named `default`:
version: 2
services:
web:
build: .
ports:
- "8000:8000"
db:
image: postgres
networks:
default:
# Use the overlay driver for multi-host communication
driver: overlay
## Custom container network modes
## Custom container network modes
Compose allows you to specify a custom network mode for a service with the `net` option - for example, `net: "host"` specifies that its containers should use the same network namespace as the Docker host, and `net: "none"` specifies that they should have no networking capabilities.
The `docker` CLI command allows you to specify a custom network mode for a container with the `--net` option - for example, `--net=host` specifies that the container should use the same network namespace as the Docker host, and `--net=none` specifies that it should have no networking capabilities.
If a service specifies the `net` option, its containers will *not* join the app’s network and will not be able to communicate with other services in the app.
To make use of this in Compose, specify a `networks` list with a single item `host`, `bridge` or `none`:
If *all* services in an app specify the `net` option, a network will not be created at all.
app:
build: ./app
networks: ["host"]
There is no equivalent to `--net=container:CONTAINER_NAME` in the v2 Compose file format. You should instead use networks to enable communication.
