(Local) Jenkins Instance with Multiple Slaves via Docker

I have several jobs that I currently work on for a living (I've got serious bills to pay...), but my day-to-day job involves creating and maintaining test automation for some enterprise software (basically it's a bunch of microservices and a proprietary database). Currently one of my tasks is migrating our older Jenkins Freestyle jobs to Jenkins Pipeline jobs (the benefit is questionable, but it pays the bills).

As part of a migration test for some parallel work, I wanted to run tests locally instead of on the work Jenkins instance.

There are a few reasons for this:

• I don't have permissions to install any plugins
• I don't want to break anything I can't fix on a Friday

As far as tools to do this, I figured Docker was a good enough choice since I have it on my laptop... but also I was far too lazy to start a new VM.

Setting up (Local) Jenkins with Docker

I don't have any special instructions for setting up Jenkins master on a Docker host. However I did follow a tutorial for Jenkins BlueOcean which installs a local Jenkins instance via the docker run command and saves your Jenkins information via a mount.

The command to do this is below:

docker run \
    --rm \
    -u root \
    -p 8080:8080 \
    -v jenkins-data:/var/jenkins_home \ 
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v "$HOME":/home \ 
    jenkinsci/blueocean

Simple enough (right?). I should probably mention that I'm on Windows 10, so I substituted $USERPROFILE for $HOME. It seemed to work all the same though.

After clicking my way through the setup, I took that command and created a docker-compose.yml file in the same directory as my (newly created) jenkins-data folder:

version: '3.3'

services:
  jenkins-blueocean:
    image: jenkinsci/blueocean
    networks:
      jenkins:
        aliases:
          - jenkins
    ports:
      - 8080:8080
    user: root
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "./jenkins-data:/var/jenkins_home"
      - "$USERPROFILE:/home" # Windows 10 Again!

networks:
  jenkins:

Now it's possible to start up the Jenkins instance with docker-compose up -d.

This is great and all, however what I really want to do is test different parallel setups/plugins across a set of Jenkins slave nodes. How can we do that locally?

Creating a Docker image to use with (Local) Jenkins

The normal process people use to start up Jenkins slave nodes is to create a new machine then set up the machine to be connected to via SSH. If I'm far too lazy for a VM (via Vagrant), I'm far too lazy to click through menus to add a new node. Enter the Self-Organizing Swarm Modules Plugin.

The plugin works by starting a Java CLI client that discovers your Jenkins master and joins it. You can install via the Plugin Manager (currently it's on version 3.16).

Now to create the nodes. I needed a way to create new nodes easily, but scale them for my purposes. So I created a new Dockerfile and entered the following:

FROM ubuntu:latest

ENV SWARM_PLUGIN_VERSION=3.16

WORKDIR /var/run/jenkins
COPY . .
RUN apt-get update \
    && apt-get install -y default-jre git wget \
    && rm -rf /var/lib/apt/lists* \
    && wget "https://repo.jenkins-ci.org/releases/org/jenkins-ci/plugins/swarm-client/${SWARM_PLUGIN_VERSION}/swarm-client-${SWARM_PLUGIN_VERSION}.jar" \
    && chmod 755 entrypoint.sh

ENTRYPOINT [ "./entrypoint.sh" ]

This will perform the following:

• Use latest Ubuntu server (I'm more familiar with it)
• Update/Install Java/Git/wget
• Download the swarm-client for version 3.16
• Run the entrypoint.sh script.

Well, now I need an entrypoint.sh script...

#!/bin/bash

java -jar "swarm-client-$SWARM_PLUGIN_VERSION.jar" \
	-master "http://jenkins:8080/" \
    -name "$HOSTNAME" -labels "$HOSTNAME" \
    -username "my-jenkins-user" -password "my-jenkins-password"

This will start the plugin via Java, set the name/label to the container $HOSTNAME and try to connect the Jenkins master.

Bringing it all together

Docker is all about containerization of apps and just starting this randomly isn't going to have everything magically work. To do this, let's modify our docker-compose.yml file one last time

version: '3.3'

services:
  jenkins-blueocean:
    image: jenkinsci/blueocean
    networks:
      jenkins:
        aliases:
          - jenkins
    ports:
      - 8080:8080
    user: root
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "./jenkins-data:/var/jenkins_home"
      - "$USERPROFILE:/home"

  jenkins-node:
    build: ./jenkins-node
    image: jenkins-node
    networks:
      jenkins: {}

networks:
  jenkins:

Now if everything somehow worked smoothly, you should be able to run the following commands:

docker-compose up -d jenkins-blueocean
docker-compose up --build -d --scale jenkins-node=6

And your Jenkins Executor Status should look like this (6 nodes!):

I would never run a production environment like this (hence why I put "Local" before Jenkins), however to test things like Lockable Resources and 100 parallel jobs it's kind of nice to have. At least I won't break our critical jobs (again).