Blog post

Fully Automated Promotion Pipelines with SonarQube and Artifactory

Fabrice Bellingard photo

Fabrice Bellingard

VP of Products

Date

  • SonarCloud
  • Pipelines
  • Announcement

This blog post is co-authored by Jonathan Roquelaure of JFrog and Fabrice Bellingard of SonarSource, and was originally posted on JFrog blog.


Editor Note: (Oct. 16th): Watch the webinar! The SonarSource/Jfrog joint webinar covering Smart, Metrics-based Release Management is available at the bottom of this blog post (no registration required).


A previous blog post on JFrog website showed how to connect Artifactory and SonarQube to help make better decisions when it comes to deploying or delivering good quality software. With a pretty simple script added to your pipeline, it becomes easy to see in Artifactory, if an artifact passed or failed the quality gate, and decide accordingly, if it can be promoted or not.

Can’t we go a bit further?

Now that we know how to attach quality gate results to artifacts, let’s not just gather this information as metadata in Artifactory. Instead, we want to automatically trigger (or not) a promotion based on this knowledge – like moving or copying artifacts to a location where they can be consumed for the next staging phase. Obviously, every company has its own workflow and should be able to implement its own logic for this automatic promotion.


Also, most real life projects have complex build pipelines, and development teams want feedback as soon as possible (the first or “commit” build” in the pipeline should be as fast as possible – as Jez Humble and Dave Farley say in their book on Continuous Delivery). SonarQube can take some time to analyze a project and provide the quality gate status, and the integration with Artifactory should never block a pipeline; any other potential downstream step should be able to run while SonarQube is processing the analysis report.


So let’s see if we can come up with a non-blocking, customizable and automated solution to make Artifactory and SonarQube work together to help you ship top quality software.

A customizable automated integration

These words ring a bell and we immediately think of webhooks, APIs and user plugins.

SonarQube Webhooks

As one of the many pieces which compose a CI/CD process, SonarQube uses webhooks to notify other services when the processing of an analysis report is complete. The HTTPS call is made regardless of the status of the processing task, and its payload contains a lot of useful information which will be used later on by the Artifactory user plugin to decide what to do for a given artifact. Here is an example of the JSON payload posted by a SonarQube webhook:


The interesting pieces of information in our context are:

  • The taskId – which will be used to identify a given artifact in Artifactory
  • The quality gate status – which is the most important information that should be considered to promote or not the corresponding artifact

In SonarQube, webhooks can be configured per project (in the project settings), or at global level – which is way more convenient when most projects analyzed by SonarQube are also managed in Artifactory.

Artifactory User Plugins

With Artifactory Pro and Enterprise you can easily extend Artifactory’s behavior with your own user plugins written in Groovy. Plugins can implement a wide range of behavior such as executing scheduled tasks (e.g. cleanup), executing  your own logic in response to a specific event (e.g. change response on download, specific security realm,…) and even exposing new API endpoints (e.g. implement specific workflow based on a SonarQube webhook).

{
    "analysedAt": "2016-11-18T10:46:28+0100",
    "project": {
        "key": "org.sonarqube:example",
        "name": "Example"
    },
    "properties": {
    },
    "qualityGate": {
        "conditions": [
            {
                "errorThreshold": "1",
                "metric": "new_security_rating",
                "onLeakPeriod": true,
                "operator": "GREATER_THAN",
                "status": "OK",
                "value": "1"
            },
            {
                "errorThreshold": "1",
                "metric": "new_reliability_rating",
                "onLeakPeriod": true,
                "operator": "GREATER_THAN",
                "status": "ERROR",
                "value": "1"
            },
            ...
        ],
        "name": "SonarQube way",
        "status": "ERROR"
    },
    "serverUrl": "http://localhost:9000",
    "status": "SUCCESS",
    "taskId": "AVh21JS2JepAEhwQ-b3u"
}

In the following snippet:

executions {
   //Expose a new endpoint for sonarqube webhook
   updateSonarTaskStatus(httpMethod: 'POST', users: ["admin"], groups: [], params:[targetRepo: '']) { params, ResourceStreamHandle body ->
       targetRepo = getStringProperty(params, 'targetRepo', true)
       bodyJson = new JsonSlurper().parse(body.inputStream)
	sonarTaskId = bodyJson.taskId
	//Implement your workflow based on SonarQube quality gate result
   }
}

, we are exposing a new endpoint that can be consumed by SonarQube webhooks with the following URL:

http://admin:password@<ARTIFACTORY_URL>:8081/artifactory/api/plugins/execute/updateSonarTaskStatus?params=targetRepo=gradle-staging-local

Jenkins as the glue

In our Jenkins CI server,  we implement a commit build that is responsible for providing quick feedback to committer and (as described in our previous post) establishing the link between a SonarQube analysis (the task Id) and Artifactory build information (and related artifacts).

node() {
   stage 'Build get source'
       git url: 'https://github.com/SonarSource/sonar-scanning-examples.git', branch: 'master'
   stage 'Artifactory configuration'
   // Create an Artifactory server instance
   //(ref:https://www.jfrog.com/confluence/display/RTF/Working+With+Pipeline+Jobs+in+Jenkins)

   def server = Artifactory.server('artifactory_local')

   // Create and set an Artifactory Gradle Build instance:
   def rtGradle = Artifactory.newGradleBuild()
   rtGradle.resolver server: server, repo: 'gradle-dev'
   rtGradle.deployer server: server, repo: 'gradle-dev-local'

   // Set a Gradle Tool defined in Jenkins "Manage":
   rtGradle.tool = 'GRADLE_TOOL'
   rtGradle.usesPlugin = false

stage 'Run Gradle and publish to Artifactory'
   // Run Gradle build with sonarqube and artifactory tasks
   dir('sonarqube-scanner-gradle'){
      def buildInfo = rtGradle.run rootDir: ".", buildFile: 'build.gradle', tasks: "clean sonarqube build artifactoryPublish --stacktrace".toString()

      //get variable from sonar report file (file and path depends on tools and CI-server )
      def ceTaskId = sh(returnStdout: true, script: "cat build/sonar/report-task.txt | grep ceTaskId | cut -f2 -d'='").trim()
      env.SONAR_CETASKID=ceTaskId
      buildInfo.env.capture = true
      buildInfo.env.collect()
      //Publish the build-info to Artifactory:
      server.publishBuildInfo buildInfo
   }
}

The Big picture

Now we have a commit build on jenkins to allow fast feedback on commit (Does my code build in the shared environment? Did I break anything? Can I continue to code?).


Commit Build Workflow:

The logic regarding the results of the SonarQube quality gates is implemented in a second “staging” workflow. Here you can simply promote your build or also trigger external tools for more advanced tests, integration, deployment,…  


Staging Workflow:


WEBINAR: SMART, METRICS-BASED RELEASE MANAGEMENT WITH SONARQUBE AND ARTIFACTORY

SHARE

twitterfacebooklinkedinmail