docker

Install Docker on Google Cloud Platform with Ubuntu Vivid 15.04

Ubuntu 15.04 (Vivid Vervet) has switched the init manager from Upstart to systemd. Will the Docker installation change? Let’s give a try and install it on a Google Compute Engine instance.

Install Docker:

1
$ curl -sSL https://get.docker.com/ | sh

Add user to Docker group for running commands without sudo:

1
$ sudo usermod -aG docker ${USER}

Log out, then log back in.

Verify the installation:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
$ docker run hello-world
Hello from Docker.
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker Hub account:
https://hub.docker.com
For more examples and ideas, visit:
https://docs.docker.com/userguide/

For upgrading, the process is the same as installing:

1
$ curl -sSL https://get.docker.com/ | sh

Fail to Upload Large File to Google App Engine Managed VMs

After deploying a Node.js app with custom runtime to Google App Engine, when uploading file larger than 1MB, the server returns HTTP 413 status code, meaning the server is refusing to process the request because the uploading entity size is too large.

I had the following setup:

  • The App Engine application is Google managed
  • The application is Node.js app with custom Dockerfile
  • The application Docker container runs node command directly on port 8080

After accessing the server (which changed from Google managed to user managed), and checked the Docker containers:

1
2
$ sudo docker ps -a
gcr.io/google_appengine/nginx-proxy:latest      "/usr/sbin/nginx"

It looks like Nginx is running as the proxy to Node.js application. When uploading file that is larger than 1MB, the response is HTTP 413 error:

1
2
3
4
5
6
7
8
9
10
11
12
<html>
    <head>
        <title>413 Request Entity Too Large</title>
    </head>
    <body bgcolor="white">
        <center>
            <h1>413 Request Entity Too Large</h1>
        </center>
        <hr>
            <center>nginx</center>
        </body>
    </html>

And the Nginx access log from the server:

1
2
$ tail /var/log/nginx/error.log 
2015/06/25 12:00:00 [error] 7#0: *101623 client intended to send too large body: 1696805 bytes

By default, Nginx limits uploading size or client_max_body_size to be 1MB. The default size is just too small. A short video upload could easily exceed the limit. So, how to accept file upload larger than 1MB via Google Managed VMs in App Engine? Is there a way to change the value through app.yamlcustom runtime configuration file? Or a custom Nginx Docker container can be deployed along with the application Docker container?

Install Docker and then Run Docker Commands without Sudo

Commands:

1
2
3
4
$ curl -sSL https://get.docker.io/ubuntu/ | sudo sh
$ sudo usermod -a -G docker $USER
$ sudo service docker restart
$ logout

Explanation:

  1. One line command installation
  2. Add the user to the group docker
  3. Restart the daemon
  4. Make sure to logout and log back in, in order to sudoless Docker taking effect.

Docker Save, Load and Deploy

Need to deploy private Docker containers without a private registry? Try docker save and docker load.

Working set:

1
2
3
4
5
$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
realguess/jq        latest              1f3f837970bf        3 months ago        6.107 MB
realguess/jq        1.4                 6071e18eae76        3 months ago        6.107 MB
busybox             ubuntu-14.04        f6169d24347d        6 weeks ago         5.609 MB

Actions to perform:

  1. Save an image with a single tag
  2. Save an image with multiple tags
  3. Save multiple images

Save a single tagged image:

1
$ docker save realguess/jq:latest > realguess-jq-latest.tar

Save a single image with all tags:

1
$ docker save realguess/jq > realguess-jq.tar

The tagged one is slightly less in size:

1
2
3
$ ls -lh realguess*.tar
-rw-rw-r-- 1 chao chao 6.0M Feb 1 12:00 realguess-jq-latest.tar
-rw-rw-r-- 1 chao chao 6.5M Feb 1 12:00 realguess-jq.tar

Save multiple images:

1
$ docker save busybox realguess/jq > busy-realguess-jq-box.tar

The size is almost twice as much comparing to a single image tar:

1
2
$ ls -lh busy*
-rw-rw-r--  1 chao chao  12M Feb 1 12:00 busy-realguess-jq-box.tar

Compress it:

1
$ out=busy-realguess-jq-box.tar && docker save busybox realguess/jq > $out && gzip $out

Much better in size:

1
2
$ ls -lh busy*
-rw-rw-r-- 1 chao chao 5.7M Feb 1 12:00 busy-realguess-jq-box.tar.gz

Do the reverse, load the tarred Docker images with docker load.

Mount Multiple Data Volumes and Multiple Data Volume Containers in Docker

Multiple Data Volumes from a Single Container

Create and run multiple data volumes in a container:

1
$ docker run -d --name vol -v /vol1 -v /vol2 ubuntu

Mount the data volumes in a new container:

1
$ docker run -it --rm --name foo --volumes-from=vol ubuntu

Both volumes will be mounted under the root directory as /vol1 and /vol2.

Multiple Data Volume Containers

Create and run multiple data volume containers:

1
$ for i in {1..2}; do docker run -d --name vol${i} -v /vol${i} ubuntu; done

Mount multiple data volume containers:

1
$ docker run -it --rm --name foo --volumes-from=vol1 --volumes-from=vol2 ubuntu

Now there are also two volumes mounted under the root directory as /vol1 and /vol2, one from each container.

Multiple Data Volume Containers But Sharing the Same Data Volume Name

Create multiple data volume containers with the same data volume name /vol:

1
$ for i in {1..2}; do docker run -d --name vol${i} -v /vol ubuntu; done

Mount multiple data volume containers:

1
$ docker run -it --rm --name foo --volumes-from=vol1 --volumes-from=vol2 ubuntu

Create a jq Docker Image with Automated Build

I have created a jq Docker image based on BusyBox with automated builds. BusyBox is really really small in size, so the jq image I have created is also very small, just a little over 6 MB.

Here is the source code and image repositories:

Building Docker image is pretty straight-forward, just follow the instruction:

https://docs.docker.com/userguide/dockerrepos/#automated-builds

I have also created a tag v1.4 in my GitHub repository to match the release of jq binary. This should also be reflected in Docker registry. After couple tries, here is the build details for adding both latest and 1.4 tags:

1
2
3
4
Type     Name     Dockerfile Location   Tag Name
------------------------------------------------
Tag      v1.4     /                     1.4
Branch   master   /                     latest

The first two columns match git branch and tag, and the last column reflects Docker tags. See https://github.com/realguess/docker-jq/tags and https://registry.hub.docker.com/u/realguess/jq/tags/manage/.

This is done by starting an automated build with type of tag instead of branch. Everything is done via the Docker Hub website.

If I pull down this repository:

1
$ docker pull realguess/jq

It should give me two image layers with the two different image IDs, which makes sense, as latest commit usually is not the same as the tagged one.

After a while, the index should be built, and I can search it via:

1
2
3
$ docker search jq
NAME             DESCRIPTION              STARS    OFFICIAL   AUTOMATED
realguess/jq                              1                   [OK]

The image is listed as AUTOMATED, but the description is missing here in the search command.

There are two types of descriptions:

  • Short description
  • Full description

Full description will be generated automatically via README.md file. I thought the short description can also be generated via README-short.txt, however, this is not the case. You can add it in the settings page for example:

https://registry.hub.docker.com/u/realguess/jq/settings/

Automated builds are triggered automatically with GitHub and BitBucket repository. Once any commit is pushed to either repository, a trigger will be sent to Docker Hub, and automated build will start.