Set Jenkins Build Description With a System Groovy Script

System Groovy scripts as a pre SCM build step, pre build step or post build step are an easy way extend the functionality of Jenkins without writing a complete Jenkins plugin.

In this blog post we will set the build description using two different techniques using the Groovy plugin.

Option 1: Using Build Parameters

In option one we use build parameters in the build description to set.

def currentBuild = Thread.currentThread().executable
def variable1 = build.buildVariableResolver.resolve('variable1')
def description = "$variable1"
currentBuild.setDescription(description)

Option 2: Using Environment Variables

In option two we use environment variables in the build description to set.

def currentBuild = Thread.currentThread().executable
def variable1 = build.getEnvironment(listener).get('variable1')
def description = "$variable1"
currentBuild.setDescription(description)

Example: Set the build description with a link back to an Atlassian Stash Pull Request build by a Jenkins job.

Here the URL to the Stash pull request (PULL_REQUEST_URL) and the id of the pull request (PULL_REQUEST_ID) are passed as parameters from Stash to Jenkins using the Pull Request Notifier for Stash add-on. A step-by-step how to integrate Stash and Jenkins for building pull requests can be found here.

def currentBuild = Thread.currentThread().executable
def PULL_REQUEST_URL = build.buildVariableResolver.resolve('PULL_REQUEST_URL')
def PULL_REQUEST_ID = build.buildVariableResolver.resolve('PULL_REQUEST_ID')
def description = "<a href='$PULL_REQUEST_URL'>PR #$PULL_REQUEST_ID</a>"
currentBuild.setDescription(description)
Advertisements

Continuous Integration for Pull Requests with Jenkins and Bitbucket Server (Stash)

Continuous integration and pull requests are two important concepts for almost any development team. Besides many other benefits, ensuring code stability and quality, ease of collaboration with other developers and fast release cycles are some of the key aspects. This post describes a solution to combine and Jenkins for automated builds of pull requests and reporting back the status to Stash. Thereby we build not the last commit of the pull request but merge the pull request automatically into the target branch. This gives a much better view if the result of merging the pull request will compile and if other coding guidelines are followed, e.g. static code analysis checks.

[Update 1: 2015-08-02]

[Update 2: 2015-08-13]

  • Starting with version 1.18 of Pull Request Notifier for Stash the url for the pull request is now provided as a parameter (PULL_REQUEST_URL). The parameter can now be past directly and we don’t need to build the parameter manually.
  • In order to set the build description also when there is a problem with merging the source and target branch we now set the build parameter already before we clone the repository. For this the Pre SCM Buildstep Plugin is used.

Workflow

Here the workflow we try to achieve:

  1. Users creates a topic branch (e.g. feature, bugfix).
  2. After completing his development work and pushing his changes to Stash the user creates a pull request.
  3. In order to approve a pull request we require at least one successful Jenkins build. Thereby we would like to get not only the build result of the code checked in for the pull request but get the build status after the code has been merged with the target branch.
  4. When a pull request is created/updated Jenkins shall be triggered automatically for real continuous integration.
  5. The source of the pull request shall be automatically merged with the target branch.
  6. Set the build description with the pull request ID and a link back to the Stash pull request.
  7. The build result shall be reported back to Stash.
  8. Only if the build was successful and the number of successful builds configured in Stash is reached the pull request shall be approved and merged.

Design Criteria

The following design criteria and features are addressed by this solution:

  • Re-use existing plugins as much as possible
  • Keep configuration to a minimum
  • Clear responsibility split between Stash (PR management and notification) and Jenkins (Job management and configuration)
  • Automatic merge into the correct target branch
  • Support for different Jenkins job configurations (target branches) in parallel
  • Different jobs for regular continuous integration and pull request pre-merge builds.

Ingredients

The following ingredients are necessary for above mentioned workflow (versions available at the time or writing):

Step-by-Step Configuration

Here are the steps necessary to achieve the above mentioned workflow.
  1. Install and configure Pull Request Notifier for Stash
  2. Configure Pull Request Configuration in Stash
  3. Install and configure Jenkins Git Plugin
  4. Install and configure Jenkins Stash Notifier Plugin
  5. Create/Update Jenkins job

Install and configure Pull Request Notifier for Stash

  1. Install the Pull Request Notifier for Stash add-on via the Universal Plugin Manager or manually by downloading from the Atlassian Marketplace.
  2. Choose Administration > Manage Add-ons > Pull Request Notifier > Configure to configure the plugin.
  3. Select the following triggers. They will trigger a new build when a pull request is opened, reopened, or either the target or source branch content has changed.
    • OPENED
    • REOPENED
    • RESCOPED
    • RESCOPED_FROM
    • RESCOPED_TO
  4. Enter Jenkins and proxy credentials as required
  5. Choose GET as HTTP method
  6. Enter http://localhost:8080/jenkins/git/notifyCommit?url=${PULL_REQUEST_TO_SSH_CLONE_URL}&branches=pr/${PULL_REQUEST_TO_BRANCH}&sha1=${PULL_REQUEST_FROM_HASH}&PULL_REQUEST_URL=${PULL_REQUEST_URL}&PULL_REQUEST_ID=${PULL_REQUEST_ID} as the URL to invoke. The trick here is the pr prefix in the branch name which is then used also in the branch name in the Jenkins job configuration. By this we can differentiate between regular commits to the target branch and the pull requests and can have two jobs in parallel. This notification will trigger automatically the correct builds using the Push Notification Support of the Jenkins Git Plugin based on the target branch and automatically merges the pull request with the target branch. Please adjust the Jenkins URL (http://localhost:8080/jenkins) to match with your environment.  jenkins_stash_pr_stash_trigger_configuration
  7. Save the trigger configuration with Save.

Configure Pull Request Configuration in Stash

  1. Choose in your Stash repository Settings > Pull requests and select the Requires a minimum of successful builds and enter the number of builds. Example: If you have  two builds for the pull request, e.g. one continuous integration and one static code analysis build and both should be successful as a prerequisite to merge the build request, enter 2 as the number of builds. jenkins_stash_pr_stash_pull_request_configuration

Install and configure Jenkins Git Plugin

  1. Install the Jenkins Git Plugin via Jenkins > Manage Jenkins > Manage Plugins
  2. Perform all standard configuration so that you can access Stash via SSH and can pull repositories from Stash.

Install and configure Jenkins Pre SCM Buildstep Plugin

  1. Install the Pre SCM Buildstep Plugin via Jenkins > Manage Jenkins > Manage Plugins
  2. Perform all standard configuration so that you can access Stash via SSH and can pull repositories from Stash.

Install and configure Jenkins Stash Notifier Plugin

  1. Install the Jenkins Stash Notifier Plugin via Jenkins > Manage Jenkins > Manage Plugin
  2. Choose Jenkins > Configure System > Stash Notifier and enter the data for your Stash installation. Jenkins Stash Notifier Configuration

Install and configure Jenkins Groovy Plugin

  1. Install the Jenkins Groovy Plugin via Jenkins > Manage Jenkins > Manage Plugins

Create Jenkins job to Build the Pull Request

  1. Add the Git configuration for your Jenkins job as follows
    • Select Git in the Source Code Management section
    • Update the Repository URL to point to your Stash installation.
    • Update Branch Specifier (blank for ‘any’) and Branch to merge to to the branch name which should be build and act as the target branch. If the target branch is master, then the values should be entered in the format as shown in the following picture. Please note the pr prefix in the Branch Specifier (blank for ‘any’) field, which we have used in the URL in the Pull Request Notification Plugin configuration in a previous step.jenkins_stash_pr_jenkins_job_git_configuration
  2. Set the build description with the pull request ID and a link back to the Stash pull request.
    • Select in the Build Environment section the Run buildstep before SCM runs and add an Execute system Groovy script step using the following script:
      def currentBuild = Thread.currentThread().executable
      def PULL_REQUEST_URL = build.buildVariableResolver.resolve('PULL_REQUEST_URL')
      def PULL_REQUEST_ID = build.buildVariableResolver.resolve('PULL_REQUEST_ID')
      def description = "<a href='$PULL_REQUEST_URL'>PR #$PULL_REQUEST_ID</a>"
      currentBuild.setDescription(description)
      
    • This will set add pull request ID in format PR #ID, e.g. PR #1 with a link back to the Stash pull request. Here we use the new capabilities of the Jenkins Git Plugin to pass additional parameters in the commit notification, here PULL_REQUEST_ID and PULL_REQUEST_URL.
  3. Create/update the Stash notification configuration of your Jenkins job(s) as follows
    • Add in the Post-build Actions section the Notify Stash Instance post build action and configure. No further configuration is necessary as we configured the Stash parameters as system wide settings already in step Install and configure Jenkins Stash Notifier Plugin.jenkins_stash_pr_jenkins_job_stash_notifier_configuration
  4. Save job configuration with Save
  5. Optionally, if you have additional jobs which build the same repository/branch combination and which should not be build on pull request notifications, update the Git configuration in these jobs and add the Don’t trigger a build on commit notifications as an additional behaviour as shown on the following example. jenkins_stash_pr_jenkins_job_ignore_notification_configuration

Final Result

Congratulations, all configuration is done!

When everything is setup correctly and you have created your first pull request in Stash you should see a new build triggered in Jenkins along with the ID of the pull request in the build description.

jenkins_stash_pr_jenkins_build_description

Furthermore after the build has finished you should see the build status information updated on the pull request in Stash.

jenkins_stash_pr_stash_build_status

When you click on the 1 build link on the top right then you see more details on the build including a link to the respective Jenkins build.

jenkins_stash_pr_stash_build_status_detailed

 

 

Manage Jobs in Large Scale Jenkins Installations – Archive and Delete Jobs with the Shelve Plugin

Jenkins is a great tool with it comes to continuous integration and other sorts of build automation. There is only one problem when it comes to responsiveness with a larger number of Jobs (in our case ~ 400) managed in Jenkins. When you have a larger number of jobs managed in Jenkins the startup time and also the responsiveness of the UI suffers and starting of Jenkins can easily take 15 minutes or more.

A common recommendation is to reduce the number of jobs. It is certainly a good practice to clean up your Jenkins installation and delete not needed jobs anymore. But if you need to keep your jobs for reporting, auditing or similar purposes you need a different approach. The same is true for jobs which are executed only every now and then but are otherwise idle most of the time.

Here the Jenkins Shelve Plugin come to rescue. The plugin allows you to temporarily shelve not needed jobs into an archive folder. They are then not loaded by Jenkins and improve the startup time significantly. If a job is needed later on you can easily unshelve jobs when they are needed.

But what if a job is really not needed anymore and you want to finally delete it? Up-to-now this was not possible out-of-the box. You basically had two choices. First unshelve a job and the delete the job or second delete the archive directly on the file system. Both is possible but not very user friendly, especially if you want to delete many jobs.

With the upcoming 1.5 release, the Jenkins Shelve Plugin now supports to delete archived jobs directly from the shelved project overview page (JENKINS-11374).

As you can see from the following screenshot you can now select one or more jobs and delete them by clicking on the Delete Project button.

[Update 2014-04-23]: Jenkins Shelve Plugin version 1.5 was just released (2014-04-18) and the delete feature now available.