Last updatedSep 13, 2019

Performance and scale testing your Data Center app

This page describes the performance and scale testing requirements for Data Center apps in the Atlassian Marketplace. Here, we provide guidelines for configuring an existing test platform to produce the test data we need to evaluate your app. For a general overview of Data Center app development, see Developing apps for Atlassian Data Center products.

Don’t have a test platform yet? We can help you set one up. Go to Data Center App Performance Toolkit User Guide for instructions.

Requirements

  1. You’re required to conduct performance testing of a 1 node DC instance, with and without your app installed.
    • You're required to test the main actions of the base product.
    • You're also required to test any endpoints or interfaces your app provides.
  2. You're required to conduct scale testing of your app. This means testing the same endpoints and interfaces in an additional 2 and 4 node configuration.
  3. With the above testing, you're required to add data to your app that matches that of a typical enterprise installation.
  4. The base dataset should mimic the dataset that ships with the Data Center App Performance Toolkit. See the datasets outlined below.

Dataset requirements

ProductDataset reference
JiraSee the enterprise-scale dataset shape listed on Jira Server Sizing Guide.
ConfluenceSee the large (L) dataset shape listed on Confluence Data Center Load Profiles.
BitbucketSee the large (L) dataset shape listed on Bitbucket Data Center Load Profiles.

Concurrency requirements

We advise using the default workload for all tests. This uses a concurrency of 200, and total user transactions of 54,500. This workload is defined in the main $product.yml configuration.

Typically, this type of workload will require a test host with 16 cores at 3.6GHz and 32GB of RAM.

Additional requirements

To help us evaluate your performance testing, you’ll also need to:

  • Detail the method of performance testing used (Framework, Browser Testing etc).
  • Outline the actions that users will have with your Data Center App (User actions).
  • Outline the actions that other systems or services can have with your Data Center App (API Definition).
  • Detail the performance results of each hardware and data complexity scenario for the actions above.
  • Outline the load applied to your application in the performance testing conducted: - What were the response times of endpoints based on the actions taken. - For each product, how did the response times on each endpoint differ based on the number of nodes in the cluster.

Expected output

You'll need to produce two reports that satisfy the different Requirements:

Performance report

This report should show the response time of each action on a two deployments: one with your app installed, and another without.

Sample performance reports

CSV

ActionWithout app (timing in ms)With app (timing in ms)
view_scrum_board_backlog100105
view_scrum_board100102
.........

Graph

Sample performance graph

Scale report

This report should show the response time of each action on instances with one, two, and four application nodes.

Sample scale reports

CSV

Action1 node (timing in ms)2 nodes (timing in ms)4 nodes (timing in ms)
view_scrum_board_backlog100102110
view_scrum_board500535530
............

Graph

Sample performance graph