April 7, 2016

Docker Volumes and Networks with Compose

In my previous article on Docker Compose, I showed how to build a two-container application using a MySQL container and a Ghost container using official Docker hub images. In part two of this Docker Compose series, I will look at a few Docker Compose commands to manage the application, and I will introduce Docker Volumes and Docker Networks, which can be specified in the YAML file describing our Compose application.

Docker Compose Commands

Last time I showed how to bring up the application with the single `docker-compose up -d` command. Let’s first look at what this does and how you could modify it.

The `-d` option made the `docker-compose` command return. If you omit it, then `docker-compose` will not return, and you will see the logs of the two containers in stdout. Try it.

This is helpful when you start writing your Compose file and you need to see why some containers may not be starting. The second particularity is that Compose automatically looked for your application in the file `docker-compose.yml` or with the `.yaml` extension. If you had an application in a file with a different name, you could specify it with the `-f` option like `docker-compose -f otherfile.yaml up`.

Now, enter `docker-compose -h` at the command line. The complete usage of the command will return, and you will see many more commands than just `up`. We saw `ps` already, which is an equivalent to `docker ps`. The most interesting point about these Compose commands is that you can manage individual services. For example, if you wanted to stop one of the services in your application, you would use `stop`:

$ docker-compose stop ghost

The container would stop. You would bring it back up with `start` or `restart`:

$ docker-compose start ghost

The name of the service is the name you gave it in your Compose file. Restarting services might be needed if there is a dependency between them and one stopped because another one was not ready yet.

The Docker `exec` command can be used via the Compose `run` command; for instance, to run `/bin/date` in your ghost service, do the following:

$ docker-compose run ghost /bin/date

Mon Mar 21 10:16:24 UTC 2016

There are lots of commands; make sure you check the usage and experiment. Finally, to bring down the entire application and remove the containers, images, volumes, and networks entirely, use the `down` command.

$ docker-compose down

During testing, bringing down the entire application is very useful and avoids confusion. If you only stop containers, they will be restarted without changes. If you make changes to an image or want to rebuild an image, make sure that your containers are indeed taking into account your changes. In a future post, we will look at the `scale` command, which starts multiple containers of a specific service. This is particularly useful if you use Compose in conjunction with Swarm and run your application in a cluster of Docker hosts.

Docker Volumes

In the previous article, we built a new image for Ghost. We did this to write the proper configuration file and add it to the Docker image started by Compose. This illustrated the use of the `build` directive in a Compose file. However, we can directly mount a configuration file in a running container using volumes, by replacing a `build` directive with `image` and `volumes` like so:

[yaml]


ghost:  

  image: ghost

  volumes:

    - ./ghost/config.js:/var/lib/ghost/config.js

...


In this YAML snippet, we defined a volume for the ghost container, which mounted the local `config.js` file into the `/var/lib/ghost/config.js` file in the container. Docker created a volume for the `/var/lib/ghost` directory and pointed the container `config.js` file to the one we have in our project directory. If you were to edit the file on the host and restart the container, the changes would take effect immediately.

The other use of volumes in Docker is for persistent data. So far, our database is not persistent. If we remove the MySQL container, we will lose all the data we put into our blog. Less than ideal! To avoid this situation, we can create a Docker volume and mount it in `/var/lib/mysql` of the database container. The life of this volume would be totally separate from the container lifecycle.

Thankfully, Compose can help us with managing these so-called named volumes. They need to be defined under the `volumes` key in a Compose file and can be used in a service definition.

Below is a snippet of our modified Compose file, which creates a `mysql` named volume and uses it in the `mysql` service.

version: '2'

services:

 mysql:  

  image: mysql

  container_name: mysql

  volumes:

    - mysql:/var/lib/mysql

...

volumes:

 mysql:


Compose will automatically create this named volume, and you will be able to see it with the `docker volume ls` command as well as find its path with `docker volume inspect <volume_name>`. Here is an example:

$ docker volume ls | grep mysql

local               vagrant_mysql

$ docker volume inspect vagrant_mysql

[

   {

       "Name": "vagrant_mysql",

       "Driver": "local",

       "Mountpoint": "/var/lib/docker/volumes/vagrant_mysql/_data"

   }

]

Be careful, however. If you bring down the application with `docker-compose down`, the persistent volume will be deleted and you will lose your data.

Docker Networks

To finish up this article in our Docker Compose series, I’ll illustrate the use of Docker networks. Similarly to volumes, Docker can manage networks. When defining services in our Compose file, we can specify a network for each one. This allows us to build tiered applications, where each container can live in its own network providing added isolation between each service.

To showcase this functionality, we are going to add an Nginx container to our application. Nginx will proxy the requests to the Ghost service, which will access the database container. Nginx will be in its own `proxy` network. The database will be in its own `db` network. And, the ghost service will have access to both `proxy` and `db` network.

We can do this by defining both networks under a `networks` directive similar to `services` and `volumes` and adding a `networks` map in each service.

For the Nginx service, we will write a local configuration file `default.conf` which will be mounted inside the Nginx container at `/etc/nginx/conf.d/default.conf`. The file will be minimal, containing only:


server {

   listen       80;

   location / {

     proxy_pass http://ghost:2368;

   }

}

It instructs Nginx to listen on port 80 and proxy all requests to the hostname `ghost` on port 2368. Our final Compose file, containing volumes and networks definitions is below:

[yaml]


version: '2'


services:

 nginx:

   image: nginx

   depends_on:

     - ghost

   volumes:

     - ./default.conf:/etc/nginx/conf.d/default.conf

   ports:

     - "80:80"    

   networks:

     - proxy

 mysql:  

   image: mysql

   container_name: mysql

   volumes:

     - mysql:/var/lib/mysql

   environment:

     - MYSQL_ROOT_PASSWORD=root

     - MYSQL_DATABASE=ghost

     - MYSQL_USER=ghost

     - MYSQL_PASSWORD=password

   networks:

     - db

 ghost:  

   image: ghost

   volumes:

     - ./ghost/config.js:/var/lib/ghost/config.js

   depends_on:

     - mysql

   networks:

     - db

     - proxy


volumes:

 mysql:


networks:

 proxy:

 db:


Note that, in this Compose file, we removed the port definitions for MySQL and Ghost because each service will try to reach the other on the default port, which will be reachable on each isolated network.

And, we’re done! We have moved from a simple Compose file that created a blog using Ghost to a three-container application, containing volumes for configurations and persistent data as well as isolated networks for a tiered application.

Read my next article, How to Use Docker Machine to Create a Swarm Cluster, for how to start this application on a Docker Swarm cluster and scale the Ghost application.

 

 

Click Here!