How to set Linux capabilities on docker swarm mode service invocations

english
#1

I'm looking into the notion of vault running under swarm (1.12.x).

A single container would be started with: docker run -d --cap-add IPC_LOCK -p 8200:8200 -p 8215:8125 --name vault --volume /vagrant/vault:/vagrant/vault vault server -config=/path/to/vault.hcl

but when I want to run this in swarm as a service, there appears to be no way to specify the IPC_LOCK capability, in order to lock down encrypted swapping for the vault service in this case.

How can I set --cap-add flags when starting a swarm mode service with the docker service create command?

#2

As of 20.10, this is available from docker service create with --cap-add:

$ docker service create --help
...
      --cap-add list                       Add Linux capabilities
      --cap-drop list                      Drop Linux capabilities

Or in a compose file used with docker stack deploy with the same syntax from the version 2 file:

version: "3.9"
services:
  app:
    image: your-image:tag
    cap_add:
    - CAP_NAME1
    - CAP_NAME2
    cap_drop:
    - CAP_NAME3
    - CAP_NAME4

[ Original answer from before 20.10 ]

It's currently not supported, but Docker is working on a solution. The logic behind not including the --cap-add option blindly is in a large cluster, there could be security concerns of a manager submitting containers with added privileges to a worker. The worker may trust running secure containers that can't access the host, but not want to allow remote root access to the host via a privileged container.

Discussion on this is over on github at:

https://github.com/docker/docker/pull/26849#issuecomment-252704844

https://github.com/docker/swarmkit/issues/1030

https://github.com/docker/swarmkit/pull/1722

https://github.com/moby/moby/issues/25885#issuecomment-557790402 and https://github.com/docker/cli/pull/2199

#3

All of the other answers here are old. Docker 20.10.0 and newer now supports specifying capabilities for Swarm services via the docker service command line and the Docker Stack YAML file format.

On the command line, you just specify --cap-add [capability] or --cap-drop [capability].

And here is an example for adding a capability in a Docker Stack YAML file:

version: "3.9"
services:
  your-service:
    cap_add:
      - CAP_SYS_ADMIN
#4

I found a solution to solve the problem and I can use cap_net_admin in swarm mode.

You can modify the runtime source code to add the capabilities that you need (it will be a local default setting).

For example I added the CAP_NET_ADMIN to my runtime (used nvidia-container-runtime) wanyvic/nvidia-container-runtime.

After that I rebuilt it, started a container (use swarm mode), input: capsh --print and CAP_NET_ADMIN can be found:

root@25303a54ebb3:/# capsh --print
Current:=cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_admin,cap_net_raw,cap_sys_chroot,cap_mknod,cap_audit_write,cap_setfcap+eip
Bounding set =cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_admin,cap_net_raw,cap_sys_chroot,cap_mknod,cap_audit_write,cap_setfcap
Securebits: 00/0x0/1'b0
 secure-noroot: no (unlocked)
 secure-no-suid-fixup: no (unlocked)
 secure-keep-caps: no (unlocked)
uid=0(root)
gid=0(root)
groups=

But this method is not good.

I also can't set cap_add or cap_drop in docker-compose.yml, but I can't find a way to solve it.

#5

See https://hub.docker.com/r/ixdotai/swarm-launcher

That repo is based on this comment/idea: https://github.com/moby/moby/issues/25885#issuecomment-573355530

Depending on the use case, a workaround is to bind-mount /var/run/docker.sock from the swarm host(s) to the service, then run docker run --privileged ... or docker run --cap-add ... from the service for executing your actual privileged commands. (You’ll have to install docker cli in the image for the service.) The innermost container that you docker run in this way will have the privileges/capabilities of the swarm host rather than of the service, and the service just becomes a thin container layer.