Setup Jenkins to build python projects with tests

22 Jul

Update January 2018:
As this post gets a few hits from Google I thought I could do an update and finally finish it.

I’ve recently moved a couple of administrative tasks in my home network to a raspberry pi 3. As my private projects aren’t crucial I’d also like to run Jenkins on the Pi so that it can run a daily cronjob to checkout my projects and run the tests. I’m using Debian 9 on the Pi.

Thanks to steves tutorial for the setup.


Get Jenkins

The first step is to install Jenkins. Simply doing an “<package manager> install jenkins” works on most unix derivates but often leaves you with an outdated version of Jenkins (e.g. 1.4.something). To use the current stable version you can rely to the official Jenkins repository. You can either use it in e.g. a Docker container or run it directly. There are also weekly builds available if you prefer to be “a bit more bleeding edge”.

I don’t but I still like to have some of the new features therefore I’m using the stable debian branch:

wget -q -O - | sudo apt-key add -

This adds the key of the Jenkins repository to apt. The second step is then to add the repository to your /etc/apt/sources.list:

deb binary/

Now you can get the available packages and install the newest available stable Jenkins package with:

sudo apt-get update
sudo apt-get install jenkins

Depending on your system and internet connection (~80MB) this might take a while.


Get assistant tools

To get the source code everytime something changes we will us the state of the art: git. I think this is self explaining here.

For other parts of the jobs I want to get done by Jenkins I’m using the most common tools and libraries – to get a quick and stable setup hopefully without any major problems or disruptions. Therefore I am using:

  1. code management: “git”
  2. code coverage: “coverage”
  3. code quality: “pylint”
  4. unit tests: “unittest”
  5. testing: “nose”
  6. mocking: “mock”

All of these tools can easily be pulled with your package manager:

sudo apt-get install python-mock python-nose python-coverage pylint git
All of these are also available for the ARM architecture.


Configuration of Jenkins

Open a browser and use the ip address of your server system. Add the port 8080 at the end to select Jenkins. E.g.

You are then forced to enter the installation password that was generated during the installation process of the package. You can find it either in the installation log or the file located at:

cat /var/lib/jenkins/secrets/initialAdminPassword

Afterwards you are asked if you’d like to continue with a list of suggested plugins to install or if you’d like to select the single plugins on your own. I’m sticking here to the common plugins as I’m also planning to use Jenkins for different other programming languages and they might come in handy.

jenkins plugin choiceAfter the plugins are installed you are forced to create an admin user and you are done and can start using Jenkins.

Install new plugins

 To get new plugins now, select “Available” at the top and choose the following:
  • Git Plugin
  • Cobertura Plugin
  • Violations
 and install them.
Afterwards go to the general “Configure System” tab and scroll down to “Git plugin” where you can enter the two following or other values:
  1. Global Config Value = Jenkins
  2. Global Config Value = Jenkins@localhost


Create a new job

To create a job click on “New Item” and choose “Freestyle project” here.

jenkins_job_createCheck “Git” under “Source Code Management” and enter the repository URL and the necessary credentials.

The second part is to check the “Build Triggers” where we check “Poll SCM” and enter 5x* (* * * * *). This lets jenkins check the repository every minute for a new commit.

The third part is to configure the build itself. Therefore click on the dropdown “Add build step” under the “Build” section and select “Execute Shell”. Now paste the following into it:

nosetests --with-xunit --all-modules --traverse-namespace --with-coverage --cover-package=project1 --cover-inclusive
python -m coverage xml --include=project1*
pylint -f parseable -d I0011,R0801 project1 | tee pylint.out

To be continued…