I wrote a number of IRC bots a number of years ago, hosting them on my infrastructure. Since building a 3 node docker swarm, I decided that these would be good candidates to use in learning Jenkins for both auto building the software, and building containers. I hadn’t made my own dockerfiles before, nor had I setup proper builds outside of my IDE for these bots before.
Dockerfile for Jenkins
In order for Jenkins to build a docker container, it needs access to docker itself. This can be done by building a custom docker container for Jenkins that has the required dependencies. Since the plan was to use Ant to build the bots as well, this too was included in the Jenkins container. Below is the resulting Dockerfile.
FROM jenkins/jenkins:lts
USER root
RUN apt-get update && \
apt-get -y install apt-transport-https \
ca-certificates \
curl \
ant \
gnupg2 \
software-properties-common && \
curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg > /tmp/dkey; apt-key add /tmp/dkey && \
add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \
$(lsb_release -cs) \
stable" && \
apt-get update && \
apt-get -y install docker-ce
RUN apt-get install -y docker-ce
RUN usermod -a -G docker jenkins
USER jenkins
Jenkins Plugins for Java/Docker Pipeline
On first start of Jenkins, the default plugins were installed for use and the following plugins were added:
- Docker Pipeline
- Auth Tokens API
- Docker Commons
I didn’t use any other plugins for my jenkins/docker pipeline to get everything going.
Docker-Compose for Jenkins
We needed to deploy Jenkins to the docker swarm cluster. This is a very simple compose file just pointing the cluster at the local build of Jenkins, providing a port to access the web GUI and a location for its data to exist within the gluster filesystem.
---
version: "3.2"
services:
jenkins:
image: 127.0.0.1:5000/jenkins
ports:
- "8070:8080"
- "50000:50000"
volumes:
- /mnt/data/jenkins:/var/jenkins_home
- /var/run/docker.sock:/var/run/docker.sock
One aspect in this compose file is that docker.sock probably not needed since jenkins isn’t being used to update the docker services itself, though might be worth seeing if it can in a pipeline. This could be additional capability added in the future.
Note: All of the docker-compose files included in this are deployed using docker stack deploy.
Jenkins Rebuilding Jenkins and Reloading
I created a pipeline to build the Jenkins container from above and push it to the docker registry. Since this dockerfile is a part of a general GIT repo I have, I’ll need to get to the correct folder in that GIT repo before building. This is about the only interesting part of this Jenkins pipeline.
pipeline {
environment {
registry = "127.0.0.1:5000/jenkins"
}
agent any
stages {
stage('Git Pull') {
steps{
git 'https://github.com/AeroSteveO/Bash-Scripts.git'
}
}
stage('Building image') {
steps{
dir("${env.WORKSPACE}/Docker-Compose/jenkins"){
script {
dockerImage = docker.build registry + ":latest"
}
}
}
}
stage('Deploy Image') {
steps{
script {
docker.withRegistry( '' ) {
dockerImage.push()
}
}
}
}
}
}
Jenkins Autobuild for Wheatley
The IRC bot projects were originally managed manually with NetBeans doing the builds, and manually pushing the new jar files to the server. I wanted a generic way to build it without using NetBeans, but still somewhat compatible, so I setup some ant scripts for building it that could be used with Jenkins.
Since Ant is also capable of doing more than just build scripts, we’ll be using it to run the java applications as well. This at least gives us a single build and run script using Ant.
<project name="Wheatley" basedir="." default="main">
<property name="src.dir" value="src"/>
<property name="build.dir" value="build"/>
<property name="classes.dir" value="${build.dir}/classes"/>
<property name="jar.dir" value="${build.dir}/jar"/>
<property name="main-class" value="rapternet.irc.bots.wheatley.listeners.WheatleyMain"/>
<property name="lib.dir" value="lib"/>
<path id="classpath">
<fileset dir="${lib.dir}" includes="**/*.jar"/>
</path>
<target name="clean">
<delete dir="${build.dir}"/>
</target>
<target name="compile">
<mkdir dir="${classes.dir}"/>
<javac srcdir="${src.dir}" destdir="${classes.dir}" classpathref="classpath"/>
</target>
<target name="jar" depends="compile">
<mkdir dir="${jar.dir}"/>
<jar destfile="${jar.dir}/${ant.project.name}.jar" basedir="${classes.dir}">
<manifest>
<attribute name="Main-Class" value="${main-class}"/>
</manifest>
</jar>
</target>
<target name="run" depends="jar">
<java fork="true" classname="${main-class}">
<classpath>
<path refid="classpath"/>
<path location="${jar.dir}/${ant.project.name}.jar"/>
</classpath>
</java>
</target>
<target name="clean-build" depends="clean,jar"/>
<target name="main" depends="clean,run"/>
</project>
Jenkins Docker Container Build for Wheatley
After getting my projects to run with a general build script, I needed to work on a way to build a docker container for them. This was done with a simple base of using OpenJDK 8, copying in my libraries and source, and building everything.
FROM openjdk:8-alpine RUN apk add --no-cache apache-ant COPY . /usr/src/wheatley WORKDIR /usr/src/wheatley RUN ant clean-build CMD ["ant", "run"]
The pipeline to do the Jenkins build is simple enough, it performs a git pull, runs docker build, and then pushes that container into the registry.
pipeline {
environment {
registry = "127.0.0.1:5000/wheatley"
}
agent any
stages {
stage('Git Pull') {
steps{
git 'https://github.com/AeroSteveO/Wheatley.git'
}
}
stage('Building image') {
steps{
script {
dockerImage = docker.build registry + ":latest"
}
}
}
stage('Deploy Image') { steps{ script { docker.withRegistry( '' ) { dockerImage.push() } } } }
}
}
Updating Code for use with Docker
The IRC bots were simple enough to update for running inside a docker container. When they were originally written, all of the input files were referenced from the directory the bots were running from. So step 1 of updating the code for docker was to provide a default path for the configuration files to be found at, and a way to tell them what that path was. This was done by using an environment variable to provide the path for the configurations to be stored at. Additional environment variables were added to provide a way to get defaults into the bot on first run before the configuration files are all populated.
Docker Compose for the BotNet
Once the containers are being auto-built and sent to the registry, we need to start them up and provide for auto-updating. For docker-swarm, we can use shepherd as the auto update service. Depending on the setup, we could use other services too
- Shepherd: Docker Swarm Update Service
- WatchTower: Single node docker update service
---
version: "3.2"
services:
wheatley:
image: 127.0.0.1:5000/wheatley
environment:
BOT_CONFIG_FOLDER: /config/
volumes:
- /mnt/data/wheatley:/config
triviabot:
image: 127.0.0.1:5000/triviabot
environment:
BOT_CONFIG_FOLDER: /config/
volumes:
- /mnt/data/wheatley:/config
shepherd:
build: .
image: mazzolino/shepherd
environment:
TZ: 'US/Eastern'
SLEEP_TIME: '5m'
FILTER_SERVICES: ''
VERBOSE: 'true'
BLACKLIST_SERVICES: 'znc znc_znc'
volumes:
- /var/run/docker.sock:/var/run/docker.sock
deploy:
placement:
constraints:
- node.role == manager
Conclusion
The project was mostly a success. I have the two IRC bots I wrote auto building with Jenkins, deploying to my docker swarm cluster and starting up. This makes it much easier to deploy new versions rather than SCPing a new jar file to the server, killing the process and restarting it. The one aspect that isn’t working quite as expected is shepherd, it took a bit of configuration to keep it from trying to upgrade all containers on the swarm (focusing only on the locally created containers first). The log file shows that shepherd is running into issues when dealing with the local docker registry. This can be taken care of later, as we have the build working.