So I am pleasantly surprised in some respects to see the Jenkins Docker plugin "pushing" Docker images to my Docker host metal but it is confusing too because my builds are taking place in Docker Slave Containers that are running on the Docker host metal. Even my Jenkins master runs in a Docker container, not directly on metal...
Following this popular Jenkins master/slave guide I arrived to the point where I had Jenkins builds working in ephemeral Docker containers.
This means that when I do Jenkins a build of some source code software component/service of mine, the build is kicked off in a Jenkins slave which happens to be a Docker Container spun up by the Jenkins Docker Plugin.
The Jenkins work space is in this slave container, the Jenkins master with Docker Plugin installed, will dispose of this slave container once the build is complete. See a diagram I made to help explain:
Some important follow up points after you have digested this diagram:
- The Jenkins Master and Jenkins Slave are running on the same Docker Host Metal at this point, as I am just at the beginning stages of getting this system running
- I am using the Docker Plugin and SSH Slaves plugin to accomplish this setup
So within this Docker Slave my software component/service build artifact is created, it could be, for example, a .dll or a .war. It happens to be the case though that my build artifact will be a Docker image. To be clear, I am building a Docker image inside a running Docker Container (the Jenkins Slave).
My confusion starts with my expectation that I should have to explicitly run a cmd to push my software component Docker image build artifact to a Docker registry. Otherwise, when the Jenkins build job is complete, the Docker plugin will shutdown the Docker container slave, dispose of (rm) the the slave container and than I will lose the build artifact inside that slave container.
What actually happens, and why I am pleasantly surprised, at least in the short term while I am getting devops up and running, is that the build artifact Docker image shows up on the Docker host metal, docker image ls.
I am surprised that the Docker plugin would go to this level of assumption/help... I know the Docker plugin allows you to configure a Docker registry and that you can add a build step to Build/Publish to a Docker Cloud which I assume that cloud is treated as a registry for images and perhaps a place to also run those images as well:
What is particularly interesting is that I am not using the Docker Plugin for any build steps, I just use the Docker Plugin to configure a Slave Container for build Jenkins Item:
The only build step I have is that I Execute a Shell Script, yes this script happens to ultimately build a Docker image but the Docker Plugin would not know this:
The Docker Plugin spins up the Docker Slave Containers, I configure the Docker Plugin and tell it a Docker Host (my metal in my situation) a Cloud is what the Docker Plugin calls a Docker Host and also Docker slave images to use on that Docker Host/Cloud:
Do I just have misunderstandings about how isolated a Jenkins build work space happens to be when inside a Docker slave container?
Is the Docker Plugin just defaulting to using the one and only Docker Cloud (my Docker host metal) I have setup for all any docker commands I happen to run inside a Jenkins Docker slave container? (a slave container by the way that does have Docker-CE installed on it)
My Jenkins Master Dockerfile:
#reference
#https://engineering.riotgames.com/news/putting-jenkins-docker-container
FROM jenkins:2.60.1
MAINTAINER Brian Ogden
USER root
#Timezone
ENV TZ=America/Los_Angeles
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
# Prep Jenkins Directories
RUN mkdir /var/log/jenkins
RUN mkdir /var/cache/jenkins
RUN chown -R jenkins:jenkins /var/log/jenkins
RUN chown -R jenkins:jenkins /var/cache/jenkins
# Copy in local config filesfiles
COPY plugins.sh /usr/local/bin/plugins.sh
RUN chmod +x /usr/local/bin/plugins.sh
# Install default plugins
# Set list of plugins to download / update in plugins.txt like this
# pluginID:version
# credentials:1.18
# maven-plugin:2.7.1
# ...
# NOTE : Just set pluginID to download latest version of plugin.
# NOTE : All plugins need to be listed as there is no transitive dependency resolution.
COPY plugins.txt /tmp/plugins.txt
RUN /usr/local/bin/plugins.sh /tmp/plugins.txt
USER jenkins
#give Jenkins a nice 8 GB memory pool and room to handle garbage collection
#ENV JAVA_OPTS="-Xmx8192m"
#give Jenkins a nice base pool of handlers and a cap
#ENV JENKINS_OPTS="--handlerCountStartup=100 --handlerCountMax=300"
ENV JENKINS_OPTS="--logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war"
I use docker-compose and Docker volume with my Jenkins Master, my docker-compose.yml:
version: '2'
services:
data:
build: data
image: tsl.devops.jenkins.data.image
container_name: tsl.devops.jenkins.data.container
master:
build: master
image: tsl.devops.jenkins.master.image
container_name: tsl.devops.jenkins.master.container
volumes_from:
- data
ports:
- "50000:50000"
#network_mode: "host"
nginx:
build: nginx
image: tsl.devops.jenkins.nginx.image
container_name: tsl.devops.jenkins.nginx.container
ports:
- "80:80"
links:
- master:jenkins-master
slavebasic:
build:
context: ./slaves
dockerfile: basic/Dockerfile
image: tsl.devops.jenkins.slave.basic.image
container_name: tsl.devops.jenkins.slave.basic.container
slavedotnetcore:
build:
context: ./slaves
dockerfile: dotnetcore/Dockerfile
image: tsl.devops.jenkins.slave.dotnetcore.image
container_name: tsl.devops.jenkins.slave.dotnetcore.container
My Jenkins Master volume/drive Dockerfile:
#reference
#https://engineering.riotgames.com/news/docker-jenkins-data-persists
FROM centos:7
MAINTAINER Brian Ogden
#create the Jenkins user in this container
RUN useradd -d "/var/jenkins_home" -u 1000 -m -s /bin/bash jenkins
#NOTE: we set the UID here to the same one the Cloudbees Jenkins image uses
#so we can match UIDs across containers, which is essential if you want
#to preserve file permissions between the containers. We also use the same home directory and bash settings.
#Jenkins log directory
RUN mkdir -p /var/log/jenkins
RUN chown -R jenkins:jenkins /var/log/jenkins
#Docker volume magic
VOLUME ["/var/log/jenkins", "/var/jenkins_home"]
USER jenkins
#just a little output reminder of the container's purpose
CMD ["echo", "Data container for Jenkins"]
My Slave Dockerfile:
FROM centos:7
MAINTAINER Brian Ogden
#the USER will be root by default just explicitly
#expressing it for better documentation
USER root
# Install Essentials
RUN yum update -y && \
yum clean all
#############################################
# Jenkins Slave setup
#############################################
RUN yum install -y \
git \
wget \
openssh-server \
java-1.8.0-openjdk \
sudo \
make && \
yum clean all
# gen dummy keys, centos doesn't autogen them like ubuntu does
RUN /usr/bin/ssh-keygen -A
# Set SSH Configuration to allow remote logins without /proc write access
RUN sed -ri 's/^session\s+required\s+pam_loginuid.so$/session optional pam_loginuid.so/' /etc/pam.d/sshd
# Create Jenkins User
RUN useradd jenkins -m -s /bin/bash
# Add public key for Jenkins login
RUN mkdir /home/jenkins/.ssh
COPY /files/id_rsa.pub /home/jenkins/.ssh/authorized_keys
#setup permissions for the new folders and files
RUN chown -R jenkins /home/jenkins
RUN chgrp -R jenkins /home/jenkins
RUN chmod 600 /home/jenkins/.ssh/authorized_keys
RUN chmod 700 /home/jenkins/.ssh
# Add the jenkins user to sudoers
RUN echo "jenkins ALL=(ALL) ALL" >> etc/sudoers
#############################################
#############################################
# Docker and Docker Compose Install
#############################################
#install required packages
RUN yum install -y \
yum-utils \
device-mapper-persistent-data \
lvm2 \
curl && \
yum clean all
#add Docker CE stable repository
RUN yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo
#Update the yum package index.
RUN yum makecache fast
#install Docker CE
RUN yum install -y docker-ce-17.06.0.ce-1.el7.centos
#install Docker Compose 1.14.0
#download Docker Compose binary from github repo
RUN curl -L https://github.com/docker/compose/releases/download/1.14.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
#Apply executable permissions to the binary
RUN chmod +x /usr/local/bin/docker-compose
#############################################
#############################################
# .NET Core SDK
#############################################
RUN yum install -y \
libunwind \
libicu
RUN curl -sSL -o dotnet.tar.gz https://go.microsoft.com/fwlink/?linkid=848821
RUN mkdir -p /opt/dotnet && tar zxf dotnet.tar.gz -C /opt/dotnet
RUN ln -s /opt/dotnet/dotnet /usr/local/bin
#add Trade Service Nuget Server
RUN mkdir -p /home/jenkins/.nuget/NuGet
COPY /files/NuGet.Config /home/jenkins/.nuget/NuGet/NuGet.Config
RUN chown -R jenkins /home/jenkins/.nuget
RUN chgrp -R jenkins /home/jenkins/.nuget
RUN chmod 600 /home/jenkins/.nuget/NuGet/NuGet.Config
RUN chmod 700 /home/jenkins/.nuget/NuGet
#speed up dotnet core builds
ENV NUGET_XMLDOC_MODE skip
ENV DOTNET_SKIP_FIRST_TIME_EXPERIENCE true
#############################################
# Expose SSH port and run SSHD
EXPOSE 22
#Technically, the Docker Plugin enforces this call when it starts containers by overriding the entry command.
#I place this here because I want this build slave to run locally as it would if it was started in the build farm.
CMD ["/usr/sbin/sshd","-D"]
An example software/component Dockerfile that will create a Docker image build artifact inside a Jenkins Slave Docker container:
FROM centos:7
MAINTAINER Brian Ogden
#Timezone
ENV TZ=America/Los_Angeles
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN yum update -y && \
yum clean all
#############################################
# .NET Core SDK
#############################################
RUN yum install -y \
libunwind \
libicu
RUN curl -sSL -o dotnet.tar.gz https://go.microsoft.com/fwlink/?linkid=848821
RUN mkdir -p /opt/dotnet && tar zxf dotnet.tar.gz -C /opt/dotnet
RUN ln -s /opt/dotnet/dotnet /usr/local/bin
#speed up dotnet core builds
ENV NUGET_XMLDOC_MODE skip
ENV DOTNET_SKIP_FIRST_TIME_EXPERIENCE true
#############################################
#############################################
# .NET Sevrice setup
#############################################
ARG ASPNETCORE_ENVIRONMENT
# Copy our code from the "/src/MyWebApi/bin/Debug/netcoreapp1.1/publish" folder to the "/app" folder in our container
WORKDIR /app
COPY ./src/TSL.Security.Service/bin/Debug/netcoreapp1.1/publish .
# Expose port 5000 for the Web API traffic
ENV ASPNETCORE_URLS http://+:5000
ENV ASPNETCORE_ENVIRONMENT $ASPNETCORE_ENVIRONMENT
EXPOSE 5000
# Run the dotnet application against a DLL from within the container
# Don't forget to publish your application or this won't work
ENTRYPOINT ["dotnet", "TSL.Security.Service.dll"]
#############################################