This article will cover some really cool techniques and useful tricks which I learned during developing Docker containers.

Make Use of Docker’s Cache

Docker saves each commit (each RUN) in a separate layer. Make use of this caching mechanism while planning your application containers.

Let’s consider an PHP application example. The libraries for this application are installed through the Composer package manager. To make use of the cache we will add the following to our Dockerfile:

1 # Install libraries through composer
2 # Until the composer.json
3 ADD ./app/composer.json /usr/lib/composer/composer.json
4 
5 # Install all Wordpress dependencies
6 ENV COMPOSER_HOME '/usr/lib/composer'
7 RUN cd /usr/lib/composer && composer install

Later in our Docker start/init script (which is used while running the container) we symlink those files/libraries where they are supposed to be:

# Create symlinks to libraries downloaded through composer
ln -s /usr/lib/composer/vendor /var/www/vendor

This allows us to spin up containers really fast!

If we would run composer install for the first time in our start/init script the run process would take at least as long as the composer install, which usually takes a lot of time.

Note
This scenario implies that a container is build for a specific application.

Read more about this technique here.

Minimize Commit/Layer Number… Wisely

Remember that each commit represents a separate file system layer and that the number of commits is limited.

You can check the maximum limit of commits with the following script:

#!/bin/bash

# Remove the commit if it exists
docker rmi czerasz/aufs-layers-test

# Create a base container
docker run --name="aufs-layers-test-container" \
           ubuntu /bin/mkdir /data

# Commit the changes to a new image
docker commit --message='First commit' 'aufs-layers-test-container' czerasz/aufs-layers-test

# Remove the container
docker rm 'aufs-layers-test-container'

for i in {1..1000}
do

echo -e "\n$i build\n"

# Add another commit
cat <<-EOF |
FROM czerasz/aufs-layers-test

RUN /bin/echo -e "test\n" >> /data/test
EOF
docker build -t czerasz/aufs-layers-test -

done

On my machine it returned:

Cannot create container with more than 127 parents

Which means that there are only 127 commits possible.

This is the reason why one should combine the RUN commands in the Dockerfile.

A real word example (which I use in my base image) is presented below:

RUN apt-get install -y build-essential && \
    apt-get install -y software-properties-common && \
    apt-get install -y pwgen \
                       python-software-properties \
                       vim \
                       curl \
                       wget \
                       git \
                       unzip \
                       tree

The downside of this construction is that whenever I need to remove/add a package, Docker is not able to use it’s cache feature and all packages need to be installed from the scratch. And this takes time.

Separate Installation and Configuration

Add configuration files in the Dockerfile as late as possible. The configuration is something which changes quite often and it would revalidate the cache with each change.

Analyze this Wordpress Dockerfile as an example:

# --- PHP ---

# Install PHP
RUN apt-get install -y php5-fpm \
                       php5-mysql \
                       php-apc

# Wordpress Requirements
RUN apt-get install -y php5-curl \
                       php5-gd \
                       php5-intl \
                       php-pear \
                       php5-imagick \
                       php5-imap \
                       php5-mcrypt \
                       php5-memcache \
                       php5-ming \
                       php5-ps \
                       php5-pspell \
                       php5-recode \
                       php5-sqlite \
                       php5-tidy \
                       php5-xmlrpc \
                       php5-xsl

# --- NGINX ---

....

# ------ PHP-FPM CONFIGURATION ------
RUN mkdir -p /var/log/php5-fpm/
# Configure the php5-fpm  process and the default pool
ADD ./config/php-fpm/php-fpm.conf /etc/php5/fpm/php-fpm.conf
ADD ./config/php-fpm/pool.d/www.conf /etc/php5/fpm/pool.d/www.conf
ADD ./config/php-fpm/php.ini /etc/php5/fpm/php.ini

When we change the PHP configuration the build process will still work really fast, because the most time consuming part, the package installation is cached.

I even tend to split the installation and configuration between separate Dockerfiles.

Always Name Containers

While running a new containers rigorously use the --name= option. This way you will always know what this container was ment for.

Some real world examples are presented below: - czerasz-web - czerasz-database - czerasz-data-container

Note
A human readable name is very useful while linking containers and while working with data containers.

Use REFRESHED_AT Variable for Better Cache Control

Consider the following Dockerfile:

FROM ubuntu

# Set the reset cache variable
ENV REFRESHED_AT 2014-11-01

...

By changing the value of the REFRESHED_AT variable you are able to flush the whole cache.

Use a Supervisor for Multiple Proccesses

If Your container wraps multiple applications/processes use a process control system tool like supervisord.

A sample /etc/supervisor/conf.d/supervisord.conf which manages Serf, PHP-FPM, Nginx is presented below:

[supervisord]
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=true                ; (start in foreground if true;default false)


[program:php5-fpm]
command=/usr/sbin/php5-fpm -c /etc/php5/fpm
autorestart=true

[program:nginx]
command=nginx
autostart=true
autorestart=true

[program:serf]
command=/etc/serf/serf-start.sh
numprocs=1
autostart=true
autorestart=true

[program:serf-join]
command=/etc/serf/serf-join.sh
autorestart=true
exitcodes=0

Use Image Inheritance

Don’t be afraid of creating Docker images. Separate your logic into different images which inherit from each other.

An example is presented below:

ubuntu:latest → czerasz/base → czerasz/wordpress-base → czerasz/czerasz-wordpress-blog

Use Data Containers

A data container is a concept which is based on a simple idea to keep the data in a special container.

Then by using the option --volumes-from we attach the files from the data container to our application/database container.

The data container’s status will be Exited but don’t fear that someone will delete it by accident. Until it’s used by another container the command docker rm data-container will not work.

A real word example is a Wordpress Docker setup in which the uploads are kept in a separate data container. Let’s see how it works!

Create data container:

docker run -v /data/uploads \
           --name czerasz-data-container \
           -d debian chown -R www-data:www-data /data/uploads

Create a Wordpress container and attach to it the /data/uploads directory from the data container:

docker run \
       -v `pwd`/app:/var/www \
       --volumes-from czerasz-data-container \
       --link czerasz-database:db \
       -p 80:80 \
       --name czerasz-site \
       -d czerasz/wordpress-base \
       /usr/local/bin/init.sh production

Tip
Access data container files like this:

$ ls -al `docker inspect -f '' czerasz-data-container`
drwx------ 4 www-data www-data 4096 Nov  1 22:04 .
drwx------ 3 root     root     4096 Nov  1 20:14 ..
drwx------ 8 www-data www-data 4096 Nov  2 01:25 2014

Read more about data containers here.

Use Files for Environment Variables

If you have a lot of environment varaiables which need to be passed to the container use the --env-file option. It allows you to import variables from a file formated like this:

primary=mongo_1532
number_of_replicas=3
...

Always Remove Temporary Containers

There is always a situation when you need a container just for a small tasks. It could be checking a specific command, exporting files from a data container or creating a database dump.

Remember to use the --rm option. It will remove the container after it’s job is done and you will never have to deal with a container necropolis after calling docker ps -a.

A real world example (which was borrowed from here) is presented below:

docker run -it \
            --link some-mysql:mysql \
            --rm mysql sh -c 'exec mysql -h"$MYSQL_PORT_3306_TCP_ADDR" -P"$MYSQL_PORT_3306_TCP_PORT" -uroot -p"$MYSQL_ENV_MYSQL_ROOT_PASSWORD"'

Simplify Communication by Using the Hostname

Whenever you can, use the -h, --hostname="" option to give your container a logical hostname. This will make networking tasks much easier.

The example below shows a simple usecase. First we spin up a server container:

docker run --hostname="web-server" \
           --rm \
           --name host-communication-test-web-server \
           -it ubuntu bash

root@web-server:/# echo "Response from: "`hostname` > index.html && python3 -m http.server 8000

Then we spin up a client server which requests the web server:

docker run --hostname="web-client" \
           --rm \
           --link host-communication-test-web-server:web-server \
           -it ubuntu bash

root@web-client:/# apt-get update && apt-get install -y curl
root@web-client:/# curl -v web-server:8000/index.html
* Hostname was NOT found in DNS cache
*   Trying 172.17.1.57...
* Connected to web-server (172.17.1.57) port 8000 (#0)
> GET /index.html HTTP/1.1
> User-Agent: curl/7.35.0
> Host: web-server:8000
> Accept: */*
>
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Server: SimpleHTTP/0.6 Python/3.4.0
< Date: Tue, 25 Nov 2014 11:41:52 GMT
< Content-type: text/html
< Content-Length: 26
< Last-Modified: Tue, 25 Nov 2014 11:40:14 GMT
<
Response from: web-server
* Closing connection 0

As you can see we don’t have to use any environment variables (which we by the way don’t have if no port is exposed). We can just rely on the hostname.

What is a Docker Start/Init Script?

A Docker start/init script is simply a script which is executed when the container starts.

The script can be called manually while one runs a container:

docker run -it user_name/image_name /usr/lib/init-script-name.sh

Or through one of the Dockerfile directives: - ENTRYPOINT - CMD

Resources: