Skip to content

Latest commit

 

History

History
193 lines (149 loc) · 8.86 KB

ch09-cicd.adoc

File metadata and controls

193 lines (149 loc) · 8.86 KB
Raw

CI/CD using Docker

Table of Contents

PURPOSE: This chapter explains how to use Jenkins and Docker to run continuous integration and continuous delivery.

There are several possible approaches to run Docker builds with Jenkins:

  1. Install Jenkins on your Docker host machine. Run Docker commands from your build, either using one of the several Jenkins Docker plugins, or by running Docker commands from a build step.

  2. Install Jenkins on your host machine and have a Jenkins slave machine with Docker installed to run your Docker builds

  3. Run Jenkins on Docker and use the underlying Docker installed on the host to run Docker commands.

Note
 Another option is running Jenkins on Docker and do a complete Docker installation inside the Jenkins Docker container. This technique is called Docker in Docker and it is usually a bad idea. There are several discussions about the problems with this approach, like this one: http://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/ . A better approach is using Docker outside of Docker, as explained here: http://container-solutions.com/running-docker-in-jenkins-in-docker/

Run Jenkins on Docker

In this example, we will run Jenkins on Docker and use the underlying Docker installed on the host to run Docker commands. This technique is known as Docker outside of Docker.

First, clone the project:

git clone https://github.com/fabianenardon/jenkins-docker-demo

Then, in the project folder, run:

docker-compose up
Note
 if running on Windows, run dos2unix plugins.txt before running docker-compose up, to avoid possible errors if your file is in the Windows format.

Wait for jenkins to start and then go to the browser and open http://localhost:8081. Jenkins should be running.

The Jenkins installation on this lab comes pre-configured. To login use the username jenkins and the password jenkins.

Running integration tests Jenkins

For this Continuous Integration demo, we will run a simple application that saves data on MongoDB. We will then run integration tests to check if the data was correctly saved on the database.

When running integration tests, you want to test your application in an environment as close to production as possible, so you can test interactions between the several components, services, databases, network communication, etc. Fortunately, docker can help you a lot with integration tests. There are several strategies to run integration tests, but in this application we are going to use the following:

  1. Start the services with a docker-compose.yml file created for testing purposes. This file won’t have any volumes mapped, so when the test is over, no state will be saved. The test docker-compose.yml file won’t publish any port on the host machine, so we can run simultaneous tests.

  2. Run the application, using the services started with the docker-compose.yml test file.

  3. Run Maven integration tests to check if the application execution produced the expected results. This will be done by checking what was saved on the MongoDB database.

  4. Stop the services. No state will be stored, so next time you run the integration tests, you will have a clean environment.

Create a new job on jenkins:

  1. Select Freestyle project

    docker ci cd 01

  2. In Source Code Management, select Git and add the repository URL: https://github.com/fabianenardon/mongo-docker-demo.git

    docker ci cd 02

  3. In Build, select Add build step and select Execute shell

    docker ci cd 03

  4. In the shell Command, add these instructions:

    cd sample

# Generates the images
sudo /var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/maven/bin/mvn clean install -Papp-docker-image

# Starts the mongo service. The -p option allows multiple builds to run at the same time,
# since we can start multiple instances of the containers
sudo docker-compose -p app-$BUILD_NUMBER --file src/test/resources/docker-compose.yml up -d mongo

# Waits for containers to start
sleep 30

# Run the application
sudo docker-compose -p app-$BUILD_NUMBER --file src/test/resources/docker-compose.yml \
     run mongo-docker-demo \
     java -jar /maven/jar/mongo-docker-demo-1.0-SNAPSHOT-jar-with-dependencies.jar mongo

# Run the integration tests
sudo docker-compose -p app-$BUILD_NUMBER --file src/test/resources/docker-compose.yml \
     run mongo-docker-demo-tests \
     mvn -f /maven/code/pom.xml -Dmaven.repo.local=/m2/repository \
     -Pintegration-test verify

  5. Click on Add post-build action and select Execute a set of scripts

    docker ci cd 04

  6. In Post-build Actions, select Execute shell

    docker ci cd 05

  7. In the Command box, add:

    cd sample
sudo docker-compose -p app-$BUILD_NUMBER --file src/test/resources/docker-compose.yml down

  8. Uncheck the Execute script only if build succeeds and Execute script only if build fails options, so this script will run always when the build ends. This way, we make sure to always stop the services.

    Note

    1. The -p app-$BUILD_NUMBER option allows multiple builds to run at the same time, since we can start multiple instances of the containers. We are using Jenkins $BUILD_NUMBER variable to isolate the containers. This way, each set of services will run on its own network.

    2. We are running the commands with sudo because we are actually running the Docker socket on the host. Jenkins runs with the jenkins user and we added the jenkins user to the sudoers list in our image. Obviously, this can have security consequences in a production environment, since one could create a build that would access root level services on the host. You can avoid this by configuring the jenkins user on the host, so it will have access to the Docker socket and then run the commands without sudo.

  9. Save the build and then click on Build now to run it. You should see a progress bar when the build is running. You can click on the progress bar to see the build console output.

    docker ci cd 06

  10. If the build is successful, you should see this on the build console output:

    docker ci cd 07

Run integration tests outside Jenkins

When creating integration tests, it is useful to be able to run and debug them outside Jenkins. In order to do that, you can simply run the same commands you ran in the Jenkins build:

cd jenkins_home/workspace/ci-test/sample

# Generates the images
mvn -f pom.xml clean install -Papp-docker-image

# Starts mongo service
docker-compose --file src/test/resources/docker-compose.yml up -d mongo

# Waits for services do start
sleep 30

# Run our application
docker-compose --file src/test/resources/docker-compose.yml \
               run mongo-docker-demo \
               java -jar /maven/jar/mongo-docker-demo-1.0-SNAPSHOT-jar-with-dependencies.jar mongo

# Run our integration tests
docker-compose --file src/test/resources/docker-compose.yml \
               run mongo-docker-demo-tests mvn -f /maven/code/pom.xml \
               -Dmaven.repo.local=/m2/repository -Pintegration-test verify

# Stop all the services
docker-compose --file src/test/resources/docker-compose.yml down

Debug integration tests outside Jenkins

You can debug integration tests. This can be achieved by making your test wait for a connection at a port for debugging. You can then attach your IDE to this port and debug. The source code for this project is available at https://github.com/fabianenardon/mongo-docker-demo.git.

You need to start Mongo service and run the application again as shown in previous section.

Here is how you can debug tests in NetBeans:

  1. Run the integration tests in debug mode with the command shown below:

    # Run integration tests in debug mode
docker run -v ~/.m2/repository:/m2/repository \
       -p 5005:5005 --link mongo:mongo \
       --net resources_default mongo-docker-demo-tests \
       mvn -f /maven/code/pom.xml \
       -Dmaven.repo.local=/m2/repository \
       -Pintegration-test verify -Dmaven.failsafe.debug

    This command will wait for a connection at port 5005 for debugger.

  2. Open the project in NetBeans and setup breakpoint in your test code:

    docker ci cd 08

  3. Use Debug menu item to attach the debugger:

    docker ci cd 09

  4. Attach the debugger by specifying the correct value of host and port:

    docker ci cd 10

  5. Hit a breakpoint in tests:

    docker ci cd 11

Continuous delivery with Jenkins

Continuous Delivery strategies depend greatly on the application architecture. With a dockerized application like the one in our demo, the continuous delivery strategy could be to publish a new version of the application image if the tests passed. This way, next time the application runs on production, the new image will be downloaded and automatically deployed. You can publish images with Jenkins just like you invoked all the other docker commands in the build.