Last updated Jul 19, 2018
Improve this page

Performance and scale testing your Data Center app

This page describes the performance and scale testing requirements for Data Center apps in Atlassian Marketplace. For a general overview of Data Center app development, see Developing apps for Atlassian Data Center products.

To aid in our evaluation of your performance testing, you will need to:

  • Detail the method of performance testing used (Framework, Browser Testing, Etc)
  • Outline the actions that users will have with your Data Center App. (User actions),
  • Outline the actions that other systems or services can have with your Data Center App (API Definition),
  • Detail the performance results of each hardware and data complexity scenario for the actions above,
  • Outline the load applied to your application in the performance testing conducted:
  • What were the response times of endpoints based on the actions taken.
  • For each product, how did the response times on each endpoint differ based on the number of nodes in the cluster.

Performance and Scale testing guide

  • If your app does not use a certain product endpoint, you are not required to test it.
  • If your app does use or augment a specific endpoint or URL, then you are required to test it.
  • If your app does background actions after a user call, you may need to detail this in the Data Center App Readiness checklist.
  • Test each of your endpoint queries enough times so that the average of the results is an accurate reflection of the user experience; this reduces the risk of outlier errors.
  • Run the tests on the same hardware deployment in the same testing instance; this reduces variation of hardware or workload on the test results.
  • If you use virtualisation on the local hardware, try not to over subscribe the hardware for the testing; be mindful of your CPU and RAM use staying below the physical limits of the hardware.

Submitting performance results

  • Please provide the raw data that is used to interpret the test results (CSV/JSON is preferred).
  • Please provide the following test results in the format of the chart below.

Latency Performance Results

Please provide the following test results in the format of the chart below.

Table of results

To submit your table of results, follow the below format:

Endpoint Type of call Action Response time without app installed With app installed
/rest/api/2/project/10000/statuses API authenticated-get 42ms 44ms
/issues/?filter=-1 USER authenticated-get 162ms 189ms

Scale testing

The purpose of Scale Testing is to reflect how the customer experience is impacted when operating across multiple nodes. To do this, you will be required to run scale testing on your app.

For many apps and extensions to Atlassian products, there shouldn’t be a significant performance difference between operating on a single node or across many nodes in a Data Center deployment. To demonstrate the performance impacts of operating your app at scale, we require testing your Data Center app in a cluster. This simple test should perform the end user actions and API actions that are present in the Performance testing, as a multi-pass test against a cluster deployment.

You will need:

  • An environment where you can build multiple nodes into a single cluster,
  • The ability to add and remove nodes to your cluster deployment,
  • The response time testing from the Performance Testing stage.

If you are unfamiliar with deploying a Data Center environment, you can use either:

Minimum scale testing requirements

The results of this testing stage will provide you with an understanding of how use of your app is impacted across a Data Center cluster deployment depending on the configuration. At a minimum, you will need to test your app with the following configuration in Data Center:

  • A single node Data Center deployment
  • A two (2) node Data Center deployment (with warm cache)
  • A four (4) node Data Center deployment (with warm cache)

It’s advisable to configure your Data Center deployment using a load balancer to ensure you are deploying in an environment that mirrors a customer deployment. See the below basic diagram of a cluster of Jira Data Center nodes behind a load balancer.

Data Center

Note however, that you will need to perform testing against each individual Jira Data Center node, so you will need to ensure that each node is configured to accept traffic directly to it and that by contacting each node directly you are likely to cause a dialogue to appear on each node as you access it. Jira is likely to give you an error such as this: JIRA Displays an Error Banner about the Base URL, as will each Data Center product in the Atlassian stack. Get to know more about these tips and more as you work through your deployment.

Data Center

Scale testing results submission

Once you have completed running your end user and API requests from your Performance stage, you will need to collate the response times together and submit these as part of your Data Center App Readiness submission.

To submit your results, follow the below format:

Endpoint Type of call Action Response time (single node) Response time (2 nodes) Response time (4 nodes)
/rest/api/2/project/10000/statuses API authenticated-get 42ms 42ms 40ms
/issues/?filter=-1 USER authenticated-get 154ms 167ms 148ms

Testing tools

Atlassian specific performance testing tools are not currently available for all products. Below is a table of currently available tools published by Atlassian which have been used to provide some performance testing results on our products.

Product Testing tools
Bitbucket Data Center, Confluence Data Center The Atlassian Performance Testing Framework (formerly E3) provides Data Center App developers with a framework for performing testing scenarios against different configurations. It is currently a community led project that does not receive active support from Atlassian.
Jira Data Center Jira Performance Testing Framework, is not currently publicly available. You can sign up for the beta in the Atlassian Community.

For generic request testing like this, we encourage you to use leading performance and quality testing products available.

Learn more