Last updatedDec 12, 2018

Adding code insights as part of your CI pipeline

We announced the code insights feature as part of Bitbucket Server 5.15. However, this feature doesn't provide any insights itself - it is only an API to surface the insights of other tools. While there are some ready-made integrations available that can be found on the Atlassian Marketplace, it is also possible to create your own integration and run it as part of your normal build.

0. Prerequisites

Code insights basics

If you do not already understand how the code insights feature works, please first have a read of our how-to guide which explains what reports and annotations are, how they are displayed to a user on the pull request, and what kind of data can be displayed.

ESLint

This tutorial will be using the JavaScript static analysis tool ESLint to detect code that doesn't match the JavaScript style guidelines as well as code that may result in errors. While of course the approach that this tutorial describes will work for any static analysis tool, we will assume you are working with a repository that already uses ESLint. If you are starting with an empty repository for this tutorial, then a quick way to get started with a default configuration is to run the following:

1
2
3
npm init -f
npm install eslint
npx eslint --init

It will also assume that you have some JavaScript code in a folder called lib/.

Bamboo

While the general approach of this tutorial will work on any CI system, this tutorial will assume that you have a build configured in Bamboo which will have the repository checked out and ready to run the static analysis and post results back to Bitbucket Server. The tutorial will assume that Bamboo agent is Unix-based and has Curl, Git, Node and Python installed.

1. Create the script file and Bamboo Task

We will build a script which will run as a script task in your Bamboo job. If you use a separate repository for your plan configuration then this is a good place to put this script. Another good option is to put the script in the repository that will be analyzed. For this tutorial, the script will live in the repository being analyzed, and will be called run_insights.sh. Create a script task in your Bamboo job that runs this script.

Action items

  • Create an empty script run_insights.sh in your repository. Remember to make it executable.
  • Create a script task in Bamboo that runs run_insights.sh

2. Run ESLint

The first step the script must perform is to run ESLint and generate a list of violations. Of course the command you run for invoking ESLint may differ slightly to the one below, however for ease of parsing the output this tutorial assumes you will use the --format=json option and output the violation to eslint.out.

You can run ESLint by executing the following

1
2
npm install
npx eslint --format=json -o eslint.out lib/** 

Action Items

  • In run_insights.sh, run ESLint and send output to eslint.out in JSON form.

3. Parse the output with Python

Since parsing the output is a little more complicated, we will use Python instead of bash. Create a Python file containing the following code, or write your own code to convert the ESLint output into a report and annotation format.

This code goes through each error reported in the output file and creates an annotation on the line which the error occurred. For the purposes of the tutorial, lets call this script parse.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
import json, os

severities = {
    0: 'LOW',
    1: 'MEDIUM',
    2: 'HIGH'
}

with open('eslint.out') as eslint_output:
    # The error and warning counts are reported per-file, so lets aggregate them across files
    total_error_count = 0
    total_warning_count = 0
    annotations = []
    for file in json.load(eslint_output):
        total_error_count += file['errorCount']
        total_warning_count += file['warningCount']
        # The path is absolute, but Bitbucket Server requires it to be relative to the git repository
        relativePath = file['filePath'].replace(os.getcwd() + '/', '')
        for message in file['messages']:
            annotations.append({
                'path': relativePath,
                'line': message['line'],
                'message': message['message'],
                'severity': severities[message['severity']]
            })

    with open('report.json', 'w') as report_file:
        report = {
            'title': 'ESLint report',
            'vendor': 'ESLint',
            'logoUrl': 'https://eslint.org/img/logo.svg',
            'data': [
                {
                    'title': 'Error Count',
                    'value': total_error_count            
                },
                {
                    'title': 'Warning Count',
                    'value': total_warning_count
                }
            ]
        }
        # Write the report json to file
        json.dump(report, report_file)
        
    with open('annotations.json', 'w') as annotation_file:
        # Write the annotations json to file
        json.dump({'annotations': annotations}, annotation_file)

Note that the above script creates report.json for the report and annotations.json for the annotations. These files will be used later when doing the REST call to create the report and annotations. For more details on what can be included in the report and annotations JSON, see the REST documentation or the how-to guide.

Run the Python script as the second step in run_insights.sh.

Action Items

  • Create parse.py in your repository
  • Run parse.py in run_insights.sh

4. Set URL parameters

Our script will use curl to create insights in Bitbucket Server, so we need the URL parameters for the endpoints described in the REST documentation. We will do this by creating bash variables in create_insights.sh.

We need variables for

  • The base URL
  • The project key
  • The repository slug
  • The commit hash
  • A chosen report key

Base URL, project key and repository slug

When viewing a repository in the browser, the URL will be in the form <base_url>/projects/<PROJECT_KEY>/repos/<repo_slug>/.... If you would like, you can hard code these variables in the script.

1
2
3
BBS_URL="http://url.to.bitbucket.server.here"
BBS_PROJECT="MY_PROJECT"
BBS_REPO="my_repo"

Another option is to set these variables as Bamboo environment variables.

Commit hash

The commit hash can be obtained by the script by running git rev-parse HEAD. The script should keep track of this value in the form of a bash variable.

1
COMMIT_ID=`git rev-parse HEAD`

Report Key

The report key is a string that represents the analysis that was done. It should be a unique string chosen by the integration and must not clash with report keys from other integrations. We recommend using reverse DNS namespacing or a similar standard to ensure that collision is avoided.

For the purposes of this tutorial, we will use the report key my.example.eslint.report:

1
REPORT_KEY="my.example.eslint.report"

Action Items

Set the following variables in run_insights.sh:

  • BBS_URL
  • BBS_PROJECT
  • BBS_REPO
  • COMMIT_ID
  • REPORT_KEY

5. Set Http Credentials

Now that the report has been parsed and converted into the format required by Bitbucket Server, we need to let Bamboo know the credentials for a user with repository read access. In order to create insights, the user performing the http REST call must be an authenticated Bitbucket Server user with Repository read permission. There are a number of ways to give Bamboo the information required to perform REST calls on behalf of a user. As with the last step, the credential data will be stored in Bamboo as environment variables.

We recommend creating one dedicated user in Bitbucket that can be used for all Bamboo calls. However, if this is not feasible then it is possible to use an existing user that has read access to the repository. Instead of using the user's password directly (which would pose a security risk), we recommend instead that you create a personal access token.

Create a personal access token and set it as a Bamboo environment variable. For the purpose of this tutorial, lets call this variable token_password. In the script, this will be made available as an environmental variable called bitbucket_token_password. With personal access tokens, we are able to perform a REST call without providing the username using bearer authentication, so setting the username environment variable is not required.

Action Items

  • Create bitbucket_token_password environment variable for the Bamboo job

6. Create the insight report

To create the report, do a PUT to the report endpoint: {baseUrl}/rest/insights/latest/projects/{projectKey}/repos/{repositorySlug}/commits/{commitId}/reports/{reportKey}.

The report body was created by parse.py and stored in report.json, and we have variables for the URL parameters and the credentials.

1
2
3
4
5
6
curl \
-H "Content-type: application/json" \
-H "Authorization: Bearer $bitbucket_token_password" \
-X PUT \
-d @report.json \
"$BBS_URL/rest/insights/latest/projects/$BBS_PROJECT/repos/$BBS_REPO/commits/$COMMIT_ID/reports/$REPORT_KEY"

Note that if a different user has already created a report for this commit and report key then the request will be rejected. However if the existing report was created by the same user as this request then the existing report will be replaced by the new report. This can be useful in the cases of rerunning a build.

Action Items

  • Put curl command in run_insights.sh to create a report

7. Create the insight annotations

After creating the report, annotations can be added to the report. To add annotations to a report, do a POST to the annotations endpoint for the given report: {baseUrl}/rest/insights/latest/projects/{projectKey}/repos/{repositorySlug}/commits/{commitId}/reports/{reportKey}/annotations.

The annotations body was created by parse.py and stored in annotations.json, and we have variables for the URL parameters and the credentials.

Note that if annotations already exist on the report, then posting additional annotations will not alter any existing ones. In the case of rerunning a build, this could mean that duplicate annotations get created. Because of this, we recommend deleting all the annotations for a report first by doing a DELETE to {baseUrl}/rest/insights/latest/{projectKey}/repos/{repositorySlug}/commits/{commitId}/reports/{key}/annotations before creating the new annotations.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Delete old annotations from the report (they may not exist but it is better to be safe)
curl \
-H "Authorization: Bearer $bitbucket_token_password" \
-H "X-Atlassian-Token: no-check" \
-X DELETE \
"$BBS_URL/rest/insights/latest/projects/$BBS_PROJECT/repos/$BBS_REPO/commits/$COMMIT_ID/reports/$REPORT_KEY/annotations"

# Create the annotations
curl \
-H "Content-type: application/json" \
-H "Authorization: Bearer $bitbucket_token_password" \
-X POST \
-d @annotations.json \
"$BBS_URL/rest/insights/latest/projects/$BBS_PROJECT/repos/$BBS_REPO/commits/$COMMIT_ID/reports/$REPORT_KEY/annotations"

Action Items

  • Put curl command in run_insights.sh to add annotations to the report

8. Piecing it all together

By this stage you should have completed the following:

  1. Created run_insights.sh to store the script commands
  2. Run a static analysis tool of your choice. In this tutorial we ran ESLint.
  3. Written a parser, parse.py, to convert the output of the static analysis tool into the format required by Bitbucket
  4. Stored the URL variables, BBS_URL, BBS_PROJECT, BBS_REPO, COMMIT_ID, REPORT_KEY
  5. Configured Bamboo to have an environment variable with the personal access token credentials (token_password)
  6. Chosen a report key and created an insight report via REST
  7. Added insight annotations to the report via REST

If the above steps are complete, then your script, run_insights.sh, should look something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
set -e # Make the Bamboo job fail if one of the commands fails

# bitbucket_token_password needs to be set as a Bamboo environment variable.
# If testing this locally, set the value manually
#bitbucket_token_password="OTUwNTIxNTY5MTQwOjdH3JClEHyutj59QTNseDNp2hzt"

# Set up the variables needed for the URL
BBS_URL="http://url.to.bitbucket.server.here"
BBS_PROJECT="MY_PROJECT"
BBS_REPO="my_repo"
COMMIT_ID=`git rev-parse HEAD`
REPORT_KEY="my.example.eslint.report"

# Run the analysis and parse the output
echo "Running ESLint"
npm install
npx eslint --format=json -o eslint.out lib/** || true # Make sure that eslint doesn't make the Bamboo job fail
echo "Done"

echo "Parsing ESLint output" 
python parse.py # This will parse eslint.out and create report.json and annotations.json
echo "Done"

# Create the report or replace the existing one
echo "Creating insight report"
curl \
-H "Content-type: application/json" \
-H "Authorization: Bearer $bitbucket_token_password" \
-X PUT \
-d @report.json \
"$BBS_URL/rest/insights/latest/projects/$BBS_PROJECT/repos/$BBS_REPO/commits/$COMMIT_ID/reports/$REPORT_KEY"
echo "Done"

# Delete old annotations from the report (they may not exist but it is better to be safe)
echo "Deleting any existing annotations"
curl \
-H "Authorization: Bearer $bitbucket_token_password" \
-H "X-Atlassian-Token: no-check" \
-X DELETE \
"$BBS_URL/rest/insights/latest/projects/$BBS_PROJECT/repos/$BBS_REPO/commits/$COMMIT_ID/reports/$REPORT_KEY/annotations"
echo "Done"

# Create the annotations
echo "Adding annotations to report"
curl \
-H "Content-type: application/json" \
-H "Authorization: Bearer $bitbucket_token_password" \
-X POST \
-d @annotations.json \
"$BBS_URL/rest/insights/latest/projects/$BBS_PROJECT/repos/$BBS_REPO/commits/$COMMIT_ID/reports/$REPORT_KEY/annotations"
echo "Done"

And you're done! Commit this script to your repository and ensure that Bamboo is configured correctly to run it as a script task in your normal pull request build. You can view the code insights on a pull request.