Send mail from Docker container with host's Postfix

english
#1

I am running an Ubuntu 14.04 (Linux) server. I have installed and configured Postfix and OpenDKIM very nicely on the server; I can send emails to myself with commands such as echo hi | sendmail root, and postfix/opendkim will add headers such as Message-Id, Date, and DKIM-Signature, forward the email to my personal email address, and everything works great.

Now I would like to create an application that runs in a Docker container and can send emails with the same ease. In particular, I don't want to worry about adding headers like Message-Id, and I don't want to do very much configuration or software installation inside the container itself.

What is the best way to do this?

Is there any way to let the container run the sendmail exectuable on the host?

I tried making a connection to Postfix from a container using the SMTP protocol on port 25, but Postfix seems to treat messages received in that way differently; I think it didn't add any headers so the message got outright rejected as spam by gmail (it wasn't even good enough to be placed into my Spam folder).

Here the maillog content

Sep 28 23:35:52 dantooine postfix/smtpd[4306]: connect from unknown[172.17.0.95]
Sep 28 23:35:52 dantooine postfix/smtpd[4306]: DD457889B: client=unknown[172.17.0.95]
Sep 28 23:35:52 dantooine postfix/cleanup[4309]: DD457889B: message-id=<>
Sep 28 23:35:52 dantooine spamd[3175]: spamd: connection from localhost [::1]:59471 to port 783, fd 6
Sep 28 23:35:52 dantooine spamd[3175]: spamd: handle_user (getpwnam) unable to find user: 'someone'
Sep 28 23:35:52 dantooine spamd[3175]: spamd: still running as root: user not specified with -u, not found, or set to root, falling back to nobody
Sep 28 23:35:52 dantooine spamd[3175]: spamd: processing message (unknown) for someone:65534
Sep 28 23:35:52 dantooine spamd[3175]: spamd: clean message (2.5/5.0) for someone:65534 in 0.0 seconds, 331 bytes.
Sep 28 23:35:52 dantooine spamd[3175]: spamd: result: . 2 - MISSING_DATE,MISSING_FROM,MISSING_MID,UNPARSEABLE_RELAY scantime=0.0,size=331,user=someone,uid=65534,required_score=5.0,rhost=localhost,raddr=::1,rport=59471,mid=(unknown),autolearn=no autolearn_force=no
Sep 28 23:35:52 dantooine opendkim[3179]: DD457889B: can't determine message sender; accepting
Sep 28 23:35:53 dantooine postfix/qmgr[3664]: DD457889B: from=<whoever@example.com>, size=275, nrcpt=1 (queue active)
Sep 28 23:35:53 dantooine postfix/smtpd[4306]: disconnect from unknown[172.17.0.95]
Sep 28 23:35:53 dantooine postfix/smtp[4311]: DD457889B: to=<someone@gmail.com>, relay=gmail-smtp-in.l.google.com[2607:f8b0:4003:c05::1b]:25, delay=0.25, delays=0.12/0.01/0.03/0.09, dsn=5.7.1, status=bounced (host gmail-smtp-in.l.google.com[2607:f8b0:4003:c05::1b] said: 550-5.7.1 [fd17:8b70:893a:44bf:fe73:6c21] Our system has detected that 550-5.7.1 this message is likely unsolicited mail. To reduce the amount of spam 550-5.7.1 sent to Gmail, this message has been blocked. Please visit 550-5.7.1 http://support.google.com/mail/bin/answer.py?hl=en&answer=188131 for 550 5.7.1 more information. su20si7357528oeb.94 - gsmtp (in reply to end of DATA command))
Sep 28 23:35:53 dantooine postfix/cleanup[4309]: 254E688A0: message-id=<20140928233553.254E688A0@myserver.example.com>
Sep 28 23:35:53 dantooine postfix/bounce[4330]: DD457889B: sender non-delivery notification: 254E688A0
Sep 28 23:35:53 dantooine postfix/qmgr[3664]: 254E688A0: from=<>, size=3374, nrcpt=1 (queue active)
Sep 28 23:35:53 dantooine postfix/qmgr[3664]: DD457889B: removed
Sep 28 23:35:53 dantooine postfix/virtual[4331]: 254E688A0: to=<whoever@example.com>, relay=virtual, delay=0.01, delays=0/0/0/0, dsn=2.0.0, status=sent (delivered to maildir)
Sep 28 23:35:53 dantooine postfix/qmgr[3664]: 254E688A0: removed
#2

You have to point inet_interfaces to docker bridge (docker0) in postfix config located at set /etc/postfix/main.cf

inet_interfaces = <docker0_ip>

More internal working detail at sending-email-from-docker-through-postfix-installed-on-the-host

#3

Because you have a working solution, here I will try to explain different behavior when you telnet to postfix (SMTP) and when you use sendmail (non-SMTP).

FYI, OpenDKIM will invoked by postfix with Milter mechanism. You can get some info how milter implementation in postfix via this official documentation. Here the diagram of milter hook in postfix.

             SMTP-only       non-SMTP
             filters         filters
                ^ |            ^ |
                | v            | |
Network ->  smtpd(8)           | |
                       \       | V
Network ->  qmqpd(8)    ->  cleanup(8)  ->  incoming
                       /
            pickup(8)
               :
Local   ->  sendmail(1)

You can see that sendmail-way (non-SMTP) and telnet-way (SMTP) has different processing order.

  • The non-SMTP email will processed by cleanup before injected to milter. Cleanup daemon was responsible for adding missing headers: (Resent-) From:, To:, Message-Id:, and Date:. Therefore your email will have complete header when injected to OpenDKIM milter even original email had incomplete header.

  • The SMTP email will injected to OpenDKIM milter before any cleanup processing take place. Therefore, if your original email had incomplete header then opendkim may refuse to sign the email. The From: header was mandatory (see RFC 6376) and if an email doesn't have it, OpenDKIM will refuse to sign the email and give you a warning

      can't determine message sender; accepting

As I never use docker, than I don't know what limitation on sendmail/pickup inside the a container. I think David Grayson's workaround was safe enough to ensure that OpenDKIM signing the message.

#4

I decided that the way the container will send mail is to write it to a file in a particular directory, which will be accessible from both the container and the host as a Docker "volume".

I made a shell script called mailsender.sh that reads mails from a specified directory, sends them to sendmail, and then deletes them:

#!/bin/bash
# Runs on the host system, reading mails files from a directory
# and piping them to sendmail -t and then deleting them.

DIR=$1

if [ \! \( -d "$DIR" -a -w "$DIR" \) ]
then
  echo "Invalid directory given: $DIR"
  exit 1
fi

echo "`date`: Starting mailsender on directory $DIR"

cd $DIR

while :
do
  for file in `find . -maxdepth 1 -type f`
  do
    echo "`date`: Sending $file"
    sendmail -t < $file
    rm $file
  done
  sleep 1
done

Ubuntu uses upstart so I created a file named /etc/init/mailsender.conf to turn this script into a daemon:

description "sends mails from directory"
start on stopped rc RUNLEVEL=[2345]
stop on runlevel[!2345]
respawn
exec start-stop-daemon --start --make-pidfile --pidfile /var/run/mailsender.pid --exec
/path/to/mailsender.sh /var/mailsend

I can start the service with start mailsender and stop it with stop mailsender. I can look at its logs in /var/log/upstart/mailsender.log, and of course I can monitor it using the PID file.

You need to create the /var/mailsend directory and then make it accessible from the Docker container by adding the argument -v /var/mailsend:/var/mailsend to your docker run command.

#5

This is a half-answer, or at least a half-tested one, since I'm currently working through the same problem. I'm hoping someone can help flesh out what I've missed.

The answer from the OP (David Grayson) sounds to me like a re-invention of the postdrop mail-spool, but using that mail spool sounds like a promising approach, so here's where I've gotten to.

The /usr/bin/sendmail compatibility interface provided by postfix passes mail to postdrop, which is sgid postdrop, allowing it to store mail into the maildrop queue at /var/spool/postfix/maildrop. This should occur in the docker container. The rest of postfix should hopefully not have to run in the container.

So, I'm host mounting /var/spool/postfix/maildrop and /var/spool/postfix/public. I can get mail delivered to /var/spool/postfix/maildrop in the host environment, since I have mounted the maildrop queue directory. Because I have mounted /var/spool/postfix/public, maildrop can signal pickup to collect the mail from the queue. Unfortunately, the uids and gids involved unless I take care of that, meaning that pickup in the host directory can't read the spool files, and worse the postfix installation messes up the permissions on the maildrop directory in the host environment.

Still, this seems to work:

$ cat Dockerfile 
FROM debian:jessie
# Ids from parent environment

    RUN groupadd -g 124 postfix && \
        groupadd -g 125 postdrop && \
    useradd -u 116 -g 124 postfix

    RUN apt-get update && \
      DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y \
        postfix \
        bsd-mailx

    CMD echo test mail | mail myemail@example.com

$ sudo docker build   .
...
Successfully built 16316fcd44b6

$ sudo docker run   -v /var/spool/postfix/maildrop:/var/spool/postfix/maildrop \
  -v /var/spool/postfix/public:/var/spool/postfix/public 16316fcd44b6

While it works, I'm not terribly happy with hard coding the uids and gids. This means that the same container can't be counted to run the same everywhere. I figure though that if instead of mounting the volume from the host I mount it from a container which runs postfix, then it won't ever conflict, and I only need one postfix installation to get mail out from many containers. I'd set those uids and gids in a base image which all my containers inherit from.

I do wonder though if this is really a good approach. With such a simple mail configuration, and no daemon in use on the container for re-trying delivery, a simpler local MTA like msmtp might be more appropriate. It would deliver via TCP to a relay on the same host, where spooling would occur.

Concerns with the msmtp approach include:

  • more possibility of losing mail if the smtp relay it sends to is not available. If that's a relay on the same host, then the chance of network problems is low, but I'd have to be careful about how I restarted the relay container.
  • performance?
  • If a large burst of mail goes through, does mail start to get dropped?

In general, the shared postfix spool approach seems more likely to be a fragile configuration to set up, but less likely to fail at run time (relay unavailable, so mail dropped).

#6

Please post the header of your email (the one which misidentified as spam by GMAIL)

#7

The email I was trying to send just had a To header, Subject header, and a one-line body. I’m not sure how to tell what headers it had after Postfix ran it through the milters, do you know how? Here is the output in /var/log/syslog showing how it was processed by Postfix and refused by Gmail: log.txt · GitHub