How to handle security updates within Docker containers?

When deploying applications onto servers, there is typically a separation between what the application bundles with itself and what it expects from the platform (operating system and installed packages) to provide. One point of this is that the platform can be updated independently of the application. This is useful for example when security updates need to be applied urgently to packages provided by the platform without rebuilding the entire application.

Traditionally security updates have been applied simply by executing a package manager command to install updated versions of packages on the operating system (for example "yum update" on RHEL). But with the advent of container technology such as Docker where container images essentially bundle both the application and the platform, what is the canonical way of keeping a system with containers up to date? Both the host and containers have their own, independent, sets of packages that need updating and updating on the host will not update any packages inside the containers. With the release of RHEL 7 where Docker containers are especially featured, it would be interesting to hear what Redhat's recommended way to handle security updates of containers is.

Thoughts on a few of the options:

  • Letting the package manager update packages on the host will not update packages inside the containers.
  • Having to regenerate all container images to apply updates seems to break the separation between the application and the platform (updating the platform requires access to the application build process which generates the Docker images).
  • Running manual commands inside each of the running containers seems cumbersome and changes are at risk of being overwritten the next time containers are updated from the application release artifacts.

So none of these approaches seems satisfactory.

A Docker image bundles application and "platform", that's correct. But usually the image is composed of a base image and the actual application.

So the canonical way to handle security updates is to update the base image, then rebuild your application image.

The containers are supposed to be lightweight and interchangeable. If your container has a security problem, you rebuild a version of the container that's patched and deploy the new container. (many containers use a standard base image that uses standard package management tools like apt-get to install their dependencies, rebuilding will pull the updates from the repositories)

While you could patch inside containers, that's not going to scale well.

This is handled automatically in SUSE Enterprise Linux using zypper-docker(1)

SUSE/zypper-docker

Docker Quick Start

First of all, many of your updates that you traditionally ran in the past will simply not be inside the container itself. The container should be a fairly lightweight and small subset of the full file system your accustomed to seeing in the past. The packages you should have to update would be those that are a part of your DockerFile, and since you have the DockerFile, you should be able to to keep track of those packages and container IDs that need updates. Cloudstein's UI that will be released soon keeps track of these DockerFile ingredients for you so that one can build the update scheme that fits best for their containers. Hope this helps

The best answer here helps a lot because there is a script which contains main commandlines to do exactly what Johannes Ziemke said:

The best idea for this I’ve seen so far is Project Atomic. I don’t think it’s quite ready for prime time though.

Valko, what workflow did you end up with? I’m running long-term containers (hosting php-cgi, for instance) and what I’ve found so far is: docker pull debian/jessie to update the image, then rebuild my existing image(s), then stop the containers and run them again (with the new image). The images I build have the same name as previous ones, so the starting is done via the script. I then remove “unnamed” images. I would surely appreciate a better workflow.

Interesting question. I wonder about it myself. If you have 20 applications running on one docker host, you have to upgrade base images, rebuild and restart! 20 applications and you don’t even know if the security update affected them all, or just one of them. You have to rebuild image for say Apache when the security update affected only libpng for example. So you end up with unnecessary rebuilds and restarts…

miha: That sounds similar to what I have ended up doing. Basically continuously updating and rebuilding all images as part of making new releases. And restarting the containers using the new images.

I don’t have the answer, but in case anyone wants a simple script that can help automate checking for base image updates: dockcheck