Rate this page:
CloudFormation deployment option will be no longer supported starting from January 2024. We recommend using TerraForm deployment. More details could be found in User Guide.
This document walks you through the process of testing your app on Jira Service Management using the Data Center App Performance Toolkit. These instructions focus on producing the required performance and scale benchmarks for your Data Center app:
If your application relays or extends the functionality of Insight (What is Insight?):
Please, make sure you have enabled Insight-specific tests in the jsm.yml
file, by setting True
value next to the insight
variable.
In this document, we cover the use of the Data Center App Performance Toolkit on two types of environments:
Development environment: Jira Service Management Data Center environment for a test run of Data Center App Performance Toolkit and development of app-specific actions. We recommend you use the AWS Quick Start for Jira Data Center with the parameters prescribed here.
Enterprise-scale environment: Jira Service Management Data Center environment used to generate Data Center App Performance Toolkit test results for the Marketplace approval process. Preferably, use the AWS Quick Start for Jira Data Center with the parameters prescribed below. These parameters provision larger, more powerful infrastructure for your Jira Service Management Data Center.
Running the tests in a development environment helps familiarize you with the toolkit. It'll also provide you with a lightweight and less expensive environment for developing. Once you're ready to generate test results for the Marketplace Data Center Apps Approval process, run the toolkit in an enterprise-scale environment.
We recommend that you set up development environment using the AWS Quick Start for Jira Data Center (How to deploy tab). All the instructions on this page are optimized for AWS.
If you are a new user, perform an end-to-end deployment. This involves deploying Jira Service Management into a new ASI:
Navigate to AWS Quick Start for Jira Data Center > How to deploy tab > Deploy into a new ASI link.
If you have already deployed the ASI separately by using the ASI Quick StartASI Quick Start or by deploying another Atlassian product (Jira, Bitbucket, or Confluence Data Center development environment) with ASI, deploy Jira Service Management into your existing ASI:
Navigate to AWS Quick Start for Jira Data Center > How to deploy tab > Deploy into your existing ASI link.
You are responsible for the cost of AWS services used while running this Quick Start reference deployment. This Quick Start doesn't have any additional prices. See Amazon EC2 pricing for more detail.
To reduce costs, we recommend you to keep your deployment up and running only during the performance runs.
AWS Jira Service Management Data Center development environment infrastructure costs about 25 - 50$ per working week depending on such factors like region, instance type, deployment type of DB, and other.
All important parameters are listed and described in this section. For all other remaining parameters, we recommend using the Quick Start defaults.
Jira setup
Parameter | Recommended value |
---|---|
Jira Product | ServiceManagement |
Version | The Data Center App Performance Toolkit officially supports 4.20.26 , 5.4.10 (Long Term Support release) |
Cluster nodes
Parameter | Recommended value |
---|---|
Cluster node instance type | t3.large (we recommend this instance type for its good balance between price and performance in testing environments) |
Maximum number of cluster nodes | 1 |
Minimum number of cluster nodes | 1 |
Cluster node instance volume size | 50 |
Database
Parameter | Recommended value |
---|---|
The database engine to deploy with | PostgresSQL |
The database engine version to use | 11 |
Database instance class | db.t3.medium |
RDS Provisioned IOPS | 1000 |
Master (admin) password | Password1! |
Enable RDS Multi-AZ deployment | false |
Application user database password | Password1! |
Database storage | 200 |
Networking (for new ASI)
Parameter | Recommended value |
---|---|
Trusted IP range | 0.0.0.0/0 (for public access) or your own trusted IP range |
Availability Zones | Select two availability zones in your region |
Permitted IP range | 0.0.0.0/0 (for public access) or your own trusted IP range |
Make instance internet facing | True |
Key Name | The EC2 Key Pair to allow SSH access. See Amazon EC2 Key Pairs for more info. |
Networking (for existing ASI)
Parameter | Recommended value |
---|---|
Make instance internet facing | True |
Permitted IP range | 0.0.0.0/0 (for public access) or your own trusted IP range |
Key Name | The EC2 Key Pair to allow SSH access. See Amazon EC2 Key Pairs for more info. |
After successfully deploying the Jira Service Management Data Center on AWS, configure it as follows:
In the AWS console, go to Services > CloudFormation > Stack > Stack details > Select your stack.
On the Outputs tab, copy the value of the LoadBalancerURL key.
Open LoadBalancerURL in your browser. This will take you to the Jira Service Management setup wizard.
On the Set up application properties page, fill in the following fields:
Then select Next.
On the next page, fill in the Your License Key field in one of the following ways:
Then select Next.
On the Set up administrator account page, fill in the following fields:
Then select Next.
On the Set up email notifications page, configure your email notifications, and then select Finish.
On the first page of the welcome setup select English (United States) language. Other languages are not supported by the toolkit.
After going through the welcome setup, select Create new project to create a new project.
You can load this dataset directly into the database (via a populate_db.sh script), or import it via XML.
To populate the database with SQL:
In the AWS console, go to Services > EC2 > Instances.
On the Description tab, do the following:
Using SSH, connect to the Jira Service Management node via the Bastion instance:
For Linux or MacOS run following commands in terminal (for Windows use Git Bash terminal):
1 2ssh-add path_to_your_private_key_pem export BASTION_IP=bastion_instance_public_ip export NODE_IP=node_private_ip export SSH_OPTS1='-o ServerAliveInterval=60' export SSH_OPTS2='-o ServerAliveCountMax=30' ssh ${SSH_OPTS1} ${SSH_OPTS2} -o "proxycommand ssh -W %h:%p ${SSH_OPTS1} ${SSH_OPTS2} ec2-user@${BASTION_IP}" ec2-user@${NODE_IP}
For more information, go to Connecting your nodes over SSH.
Download the populate_db.sh script and make it executable:
1 2wget https://raw.githubusercontent.com/atlassian/dc-app-performance-toolkit/master/app/util/jira/populate_db.sh && chmod +x populate_db.sh
Review the following Variables section
of the script:
1 2DB_CONFIG="/var/atlassian/application-data/jira/dbconfig.xml" JIRA_DB_NAME="jira" JIRA_DB_USER="postgres" JIRA_DB_PASS="Password1!" # JSM section JIRA_CURRENT_DIR="/opt/atlassian/jira-servicedesk/current" JIRA_SETENV_FILE="${JIRA_CURRENT_DIR}/bin/setenv.sh" JIRA_VERSION_FILE="/media/atl/jira/shared/jira-servicedesk.version"
Run the script:
1 2./populate_db.sh --jsm --small 2>&1 | tee -a populate_db.log
In case of a failure, check the Variables
section and run the script one more time.
We recommend that you only use this method if you are having problems with the populate_db.sh script.
In the AWS console, go to Services > EC2 > Instances.
On the Description tab, do the following:
Using SSH, connect to the Jira Service Management node via the Bastion instance:
For Linux or MacOS run following commands in terminal (for Windows use Git Bash terminal):
1 2ssh-add path_to_your_private_key_pem export BASTION_IP=bastion_instance_public_ip export NODE_IP=node_private_ip export SSH_OPTS1='-o ServerAliveInterval=60' export SSH_OPTS2='-o ServerAliveCountMax=30' ssh ${SSH_OPTS1} ${SSH_OPTS2} -o "proxycommand ssh -W %h:%p ${SSH_OPTS1} ${SSH_OPTS2} ec2-user@${BASTION_IP}" ec2-user@${NODE_IP}
For more information, go to Connecting your nodes over SSH.
Download the xml_backup.zip file corresponding to your Jira Service Management version.
1 2JSM_VERSION=$(sudo su jira -c "cat /media/atl/jira/shared/jira-servicedesk.version") sudo su jira -c "wget https://centaurus-datasets.s3.amazonaws.com/jsm/${JSM_VERSION}/small/xml_backup.zip -O /media/atl/jira/shared/import/xml_backup.zip"
Log in as a user with the Jira System Administrators global permission.
Go to > System > Restore System. from the menu.
Populate the File name field with xml_backup.zip
.
Click Restore and wait until the import is completed.
Using SSH, connect to the Jira Service Management node via the Bastion instance:
For Linux or MacOS run following commands in terminal (for Windows use Git Bash terminal):
1 2ssh-add path_to_your_private_key_pem export BASTION_IP=bastion_instance_public_ip export NODE_IP=node_private_ip export SSH_OPTS1='-o ServerAliveInterval=60' export SSH_OPTS2='-o ServerAliveCountMax=30' ssh ${SSH_OPTS1} ${SSH_OPTS2} -o "proxycommand ssh -W %h:%p ${SSH_OPTS1} ${SSH_OPTS2} ec2-user@${BASTION_IP}" ec2-user@${NODE_IP}
For more information, go to Connecting your nodes over SSH.
Download the upload_attachments.sh script and make it executable:
1 2wget https://raw.githubusercontent.com/atlassian/dc-app-performance-toolkit/master/app/util/jira/upload_attachments.sh && chmod +x upload_attachments.sh
Review the following Variables section
of the script:
1 2# JSM version file location JIRA_VERSION_FILE="/media/atl/jira/shared/jira-servicedesk.version"
Run the script:
1 2./upload_attachments.sh --jsm --small 2>&1 | tee -a upload_attachments.log
For more information, go to Re-indexing Jira.
The re-index time for JSM 4.20.x is about ~30-50 minutes, while for JSM 5.4.x it can take significantly longer at around 110-130 minutes. This increase in re-index time is due to a known issue which affects Jira 5.4.x, and you can find more information about it in this ticket: Re-Index: JSM 5.4.x.
When finished, the Acknowledge button will be available on the re-indexing page.
Make sure English (United States) language is selected as a default language on the > System > General configuration page. Other languages are not supported by the toolkit.
Clone Data Center App Performance Toolkit locally.
Follow the README.md instructions to set up toolkit locally.
Navigate to dc-app-performance-toolkit/app
folder.
Open the jsm.yml
file and fill in the following variables:
application_hostname
: your_dc_jsm_instance_hostname without protocol.application_protocol
: http or https.application_port
: for HTTP - 80, for HTTPS - 443, 8080, 2990 or your instance-specific port.secure
: True or False. Default value is True. Set False to allow insecure connections, e.g. when using self-signed SSL certificate.application_postfix
: set to empty for CloudFormation deployment; e.g., /jira for url like this http://localhost:2990/jira.admin_login
: admin user username.admin_password
: admin user password.load_executor
: executor for load tests. Valid options are jmeter (default) or locust.concurrency_agents
: 1
- number of concurrent JMeter/Locust agents.concurrency_customers
: 1
- number of concurrent JMeter/Locust customers.test_duration
: 5m
- duration of the performance run.ramp-up
: 3s
- amount of time it will take JMeter or Locust to add all test users to test execution.total_actions_per_hour_agents
: 500
- number of total JMeter/Locust actions per hour for agents scenario.total_actions_per_hour_customers
: 1500
- number of total JMeter/Locust actions per hour customers scenario.WEBDRIVER_VISIBLE
: visibility of Chrome browser during selenium execution (False is by default).insight
: True or False. Default value is False. Set True to enable Insight specific tests.In case your application relays or extends the functionality of Insight. Make sure to set True
value next to insight
variable.
Run bzt.
1 2bzt jsm.yml
Review the resulting table in the console log. All JMeter/Locust and Selenium actions should have 95+% success rate.
In case some actions does not have 95+% success rate refer to the following logs in dc-app-performance-toolkit/app/results/jsm/YY-MM-DD-hh-mm-ss
folder:
results_summary.log
: detailed run summaryresults.csv
: aggregated .csv file with all actions and timingsbzt.log
: logs of the Taurus tool executionjmeter.*
: logs of the JMeter tool executionlocust.*
: logs of the Locust tool execution (in case you use Locust as load_executor in jsm.yml)pytest.*
: logs of Pytest-Selenium executionDo not proceed with the next step until you have all actions 95+% success rate. Ask support if above logs analysis did not help.
Data Center App Performance Toolkit has its own set of default test actions for Jira Service Management Data Center: JMeter/Locust and Selenium for load and UI tests respectively.
App-specific action - action (performance test) you have to develop to cover main use cases of your application. Performance test should focus on the common usage of your application and not to cover all possible functionality of your app. For example, application setup screen or other one-time use cases are out of scope of performance testing.
We strongly recommend developing your app-specific actions on the development environment to reduce AWS infrastructure costs.
You can filter your own app-specific issues for your app-specific actions.
JSM_URL/issues/?jql=
and select Advanced
.summary ~ 'AppRequest*'
.dc-app-performance-toolkit/app/jsm.yml
:
custom_dataset_query:
JQL from step 3.Next time when you run toolkit, custom dataset issues will be stored to the dc-app-performance-toolkit/app/datasets/jsm/custom-requests.csv
with columns: request_id
, request_key
, service_desk_id
, project_id
, project_key
.
You develop an app that adds some additional fields to specific types of Jira Service Management requests. In this case, you should develop Selenium app-specific action:
JSM_URL/issues/?jql=
and check if JQL is correct: summary ~ 'AppRequest*'
.dc-app-performance-toolkit/app/jsm.yml
configuration file and set custom_dataset_query: summary ~ 'AppRequest*'
.dc-app-performance-toolkit/app/extension/jsm/extension_ui_agents.py
.dc-app-performance-toolkit/app/extension/jsm/extension_ui_customers.py
.app_specific_action
as specific user uncomment app_specific_user_login
function in code example. Note, that in this case test_1_selenium_custom_action
should follow just before test_2_selenium_agent_z_logout
or test_2_selenium_customer_z_log_out
action.dc-app-performance-toolkit/app/selenium_ui/jsm_ui_agents.py
, review and uncomment the following block of code to make newly created app-specific actions executed:1 2# def test_1_selenium_agent_custom_action(jsm_webdriver, jsm_datasets, jsm_screen_shots): # extension_ui_agents.app_specific_action(jsm_webdriver, jsm_datasets)
dc-app-performance-toolkit/app/selenium_ui/jsm_ui_customers.py
, review and uncomment the following block of code to make newly created app-specific actions executed:1 2# def test_1_selenium_customer_custom_action(jsm_webdriver, jsm_datasets, jsm_screen_shots): # extension_ui_customers.app_specific_action(jsm_webdriver, jsm_datasets)
bzt jsm.yml
command to ensure that all Selenium actions including app_specific_action
are successful.You develop an app that introduces new GET and POST endpoints in Jira Service Management Data Center. In this case, you should develop Locust or JMeter app-specific action.
Locust app-specific action development example
dc-app-performance-toolkit/app/extension/jsm/extension_locust_agents.py
, so that test will call the endpoint with GET request, parse response use these data to call another endpoint with POST request and measure response time.dc-app-performance-toolkit/app/extension/jsm/extension_locust_customers.py
, so that test will call the endpoint with GET request, parse response use these data to call another endpoint with POST request and measure response time.dc-app-performance-toolkit/app/jsm.yml
set load_executor: locust
to make locust
as load executor.agent_standalone_extension
/customer_standalone_extension
. Default value is 0
, which means that agent_standalone_extension
/customer_standalone_extension
action will not be executed. Locust uses actions percentage as relative weights, so if some_action: 10
and standalone_extension: 20
that means that standalone_extension
will be called twice more.agent_standalone_extension
/customer_standalone_extension
weight in accordance with the expected frequency of your app use case compared with other base actions.@run_as_specific_user(username='specific_user_username', password='specific_user_password')
decorator for that.bzt jsm.yml
command to ensure that all Locust actions including app_specific_action
are successful.JMeter app-specific action development example
Check that jsm.yml
file has correct settings of application_hostname
, application_protocol
, application_port
, application_postfix
, etc.
Set desired execution percentage for agent_standalone_extension
and/or customer_standalone_extension
. Default values are 0
, which means that agent_standalone_extension
and customer_standalone_extension
actions will not be executed.
For example, for app-specific action development you could set percentage of agent_standalone_extension
and/or customer_standalone_extension
to 100 and for all other actions to 0 - this way only jmeter_agent_login_and_view_dashboard
and agent_standalone_extension
or jmeter_customer_login_and_view_dashboard
and customer_standalone_extension
actions would be executed.
Navigate to dc-app-performance-toolkit/app
folder and run from virtualenv(as described in dc-app-performance-toolkit/README.md
):
1 2python util/jmeter/start_jmeter_ui.py --app jsm --type agents # or python util/jmeter/start_jmeter_ui.py --app jsm --type customers
Open Agents
/Customers
thread group > actions per login
and navigate to agent_standalone_extension
/customer_standalone_extension
Add GET HTTP Request
: right-click to agent_standalone_extension
/customer_standalone_extension`` >
Add>
Sampler
HTTP Request`, chose method GET and set endpoint in Path.
Add Regular Expression Extractor
: right-click to to newly created HTTP Request
> Add
> Post processor
> Regular Expression Extractor
Add Response Assertion
: right-click to newly created HTTP Request
> Add
> Assertions
> Response Assertion
and add assertion with Contains
, Matches
, Equals
, etc types.
Add POST HTTP Request
: right-click to agent_standalone_extension
/customer_standalone_extension
> Add
> Sampler
HTTP Request
, chose method POST, set endpoint in Path and add Parameters or Body Data if needed.
Right-click on View Results Tree
and enable this controller.
Click Start button and make sure that login_and_view_dashboard
and agent_standalone_extension
/customer_standalone_extension
are successful.
Right-click on View Results Tree
and disable this controller. It is important to disable View Results Tree
controller before full-scale results generation.
Click Save button.
To make agent_standalone_extension
/customer_standalone_extension
executable during toolkit run edit dc-app-performance-toolkit/app/jsm.yml
and set execution percentage of agent_standalone_extension
/customer_standalone_extension
accordingly to your use case frequency.
App-specific tests could be run (if needed) as a specific user. In the agent_standalone_extension
/customer_standalone_extension
uncomment login_as_specific_user
controller. Navigate to the username:password
config element and update values for app_specific_username
and app_specific_password
names with your specific user credentials. Also make sure that you located your app-specific tests between login_as_specific_user
and login_as_default_user_if_specific_user_was_loggedin
controllers.
Run toolkit to ensure that all JMeter actions including agent_standalone_extension
and/or customer_standalone_extension
are successful.
Use or access the following variables in your agent_standalone_extension
action if needed:
${request_id}
- request id being viewed or modified (e.g. 693484)${request_key}
- request key being viewed or modified (e.g. ABC-123)${request_project_id}
- project id being viewed or modified (e.g. 3423)${request_project_key}
- project key being viewed or modified (e.g. ABC)${request_service_desk_id}
- service_desk_id being viewed or modified (e.g. 86)${s_prj_key}
- "small" project (<10k requests per project) key being viewed or modified (e.g. ABC)${s_prj_id}
- "small" project id being viewed or modified (e.g. 123)${s_service_desk_id}
- "small" project service_desk_id being viewed or modified (e.g. 12)${s_prj_total_req}
- "small" project total requests (e.g. 444)${s_prj_all_open_queue_id}
- "small" project "all open" queue id (e.g. 44)${s_created_vs_resolved_id}
- "small" project "created vs resolved" report id (e.g. 45)${s_time_to_resolution_id}
- "small" project "time to resolution" report id (e.g. 46)${m_prj_key}
- "medium" project (>10k and <100k requests per project) key being viewed or modified (e.g. ABC)${m_prj_id}
- "medium" project id being viewed or modified (e.g. 123)${m_service_desk_id}
- "medium" project service_desk_id being viewed or modified (e.g. 12)${m_prj_total_req}
- "medium" project total requests (e.g. 444)${m_prj_all_open_queue_id}
- "medium" project "all open" queue id (e.g. 44)${m_created_vs_resolved_id}
- "medium" project "created vs resolved" report id (e.g. 45)${m_time_to_resolution_id}
- "medium" project "time to resolution" report id (e.g. 46)${username}
- the logged in username (e.g. admin)Use or access the following variables in your customer_standalone_extension
action if needed:
${s_service_desk_id}
- "small" project (<10k requests per project) service_desk_id being viewed or modified (e.g. 12)${rt_project_id}
- project id (e.g. 12)${rt_service_desk_id}
- service_desk_id (e.g. 12)${rt_id}
- request type id for project with project id ${rt_project_id}
and service_desk_id ${rt_service_desk_id}
(e.g. 123)${username}
- the logged in username (e.g. admin)App-specific actions are required. Do not proceed with the next step until you have completed app-specific actions development and got successful results from toolkit run.
After adding your custom app-specific actions, you should now be ready to run the required tests for the Marketplace Data Center Apps Approval process. To do this, you'll need an enterprise-scale environment.
We recommend that you use the AWS Quick Start for Jira Data Center (How to deploy tab) to deploy a Jira Service Management Data Center enterprise-scale environment. This Quick Start will allow you to deploy Jira Service Management Data Center with a new Atlassian Standard Infrastructure (ASI) or into an existing one.
The ASI is a Virtual Private Cloud (VPC) consisting of subnets, NAT gateways, security groups, bastion hosts, and other infrastructure components required by all Atlassian applications, and then deploys Jira Service Management into this new VPC. Deploying Jira Service Management with a new ASI takes around 50 minutes. With an existing one, it'll take around 30 minutes.
If you are a new user, perform an end-to-end deployment. This involves deploying Jira Service Management into a new ASI:
Navigate to AWS Quick Start for Jira Data Center > How to deploy tab > Deploy into a new ASI link.
If you have already deployed the ASI separately by using the ASI Quick StartASI Quick Start or by deploying another Atlassian product (Jira, Bitbucket, or Confluence Data Center development environment) with ASI, deploy Jira Service Management into your existing ASI:
Navigate to AWS Quick Start for Jira Data Center > How to deploy tab > Deploy into your existing ASI link.
You are responsible for the cost of the AWS services used while running this Quick Start reference deployment. There is no additional price for using this Quick Start. For more information, go to aws.amazon.com/pricing.
To reduce costs, we recommend you to keep your deployment up and running only during the performance runs.
AWS Pricing Calculator provides an estimate of usage charges for AWS services based on certain information you provide. Monthly charges will be based on your actual usage of AWS services and may vary from the estimates the Calculator has provided.
*The prices below are approximate and may vary depending on such factors like region, instance type, deployment type of DB, and other.
Stack | Estimated hourly cost ($) |
---|---|
One Node Jira Service Management DC | 0.8 - 1.1 |
Two Nodes Jira Service Management DC | 1.2 - 1.7 |
Four Nodes Jira Service Management DC | 2.0 - 3.0 |
To reduce AWS infrastructure costs you could stop cluster nodes when the cluster is standing idle.
Cluster node might be stopped by using Suspending and Resuming Scaling Processes.
To stop one node within the cluster, follow the instructions below:
Edit
on Advanced configuration
) and add HealthCheck
to the Suspended Processes
. Amazon EC2 Auto Scaling stops marking instances unhealthy as a result of EC2 and Elastic Load Balancing health checks.To return node into a working state follow the instructions:
Edit
on Advanced configuration
) and remove HealthCheck
from Suspended Processes
of Auto Scaling Group.To reduce AWS infrastructure costs database could be stopped when the cluster is standing idle. Keep in mind that database would be automatically started in 7 days.
To stop database:
To start database:
All important parameters are listed and described in this section. For all other remaining parameters, we recommend using the Quick Start defaults.
Jira setup
Parameter | Recommended Value |
---|---|
Jira Product | ServiceManagement |
Version | The Data Center App Performance Toolkit officially supports 4.20.26 , 5.4.10 (Long Term Support release) |
Cluster nodes
Parameter | Recommended Value |
---|---|
Cluster node instance type | m5.2xlarge (This differs from our public recommendation on c4.8xlarge for production instances but is representative for a lot of our Jira Service Management Data Center customers. The Data Center App Performance Toolkit framework is set up for concurrency we expect on this instance size. As such, underprovisioning will likely show a larger performance impact than expected.) |
Maximum number of cluster nodes | 1 |
Minimum number of cluster nodes | 1 |
Cluster node instance volume size | 100 |
Database
Parameter | Recommended Value |
---|---|
The database engine to deploy with | PostgresSQL |
The database engine version to use | 11 |
Database instance class | db.m5.xlarge |
RDS Provisioned IOPS | 1000 |
Master (admin) password | Password1! |
Enable RDS Multi-AZ deployment | false |
Application user database password | Password1! |
Database storage | 200 |
The Master (admin) password will be used later when restoring the SQL database dataset. If password value is not set to default, you'll need to change DB_PASS
value manually in the restore database dump script (later in Preloading your Jira Service Management deployment with an enterprise-scale dataset).
Networking (for new ASI)
Parameter | Recommended Value |
---|---|
Trusted IP range | 0.0.0.0/0 (for public access) or your own trusted IP range |
Availability Zones | Select two availability zones in your region |
Permitted IP range | 0.0.0.0/0 (for public access) or your own trusted IP range |
Make instance internet facing | true |
Key Name | The EC2 Key Pair to allow SSH access. See Amazon EC2 Key Pairs for more info. |
Networking (for existing ASI)
Parameter | Recommended Value |
---|---|
Make instance internet facing | true |
Permitted IP range | 0.0.0.0/0 (for public access) or your own trusted IP range |
Key Name | The EC2 Key Pair to allow SSH access. See Amazon EC2 Key Pairs for more info. |
After successfully deploying Jira Service Management Data Center in AWS, you'll need to configure it:
Data dimensions and values for an enterprise-scale dataset are listed and described in the following table.
Data dimensions | Value for an enterprise-scale dataset |
---|---|
Attachments | ~2 000 000 |
Comments | ~2 000 000 |
Components | ~1 500 |
Custom fields | ~400 |
Organizations | ~300 |
Requests | ~1 000 000 |
Projects | 200 |
Screen schemes | ~500 |
Screens | ~3000 |
Users | ~21 000 |
Workflows | ~700 |
Insight Schemas | ~ 6 |
Insight Object types | ~ 50 |
Insight Schema objects | ~ 1 000 000 |
All the datasets use the standard admin
/admin
credentials.
The following subsections explain each step in greater detail.
You can load this dataset directly into the database (via a populate_db.sh script), or import it via XML.
To populate the database with SQL:
In the AWS console, go to Services > EC2 > Instances.
On the Description tab, do the following:
Using SSH, connect to the Jira Service Management node via the Bastion instance:
For Linux or MacOS run following commands in terminal (for Windows use Git Bash terminal):
1 2ssh-add path_to_your_private_key_pem export BASTION_IP=bastion_instance_public_ip export NODE_IP=node_private_ip export SSH_OPTS1='-o ServerAliveInterval=60' export SSH_OPTS2='-o ServerAliveCountMax=30' ssh ${SSH_OPTS1} ${SSH_OPTS2} -o "proxycommand ssh -W %h:%p ${SSH_OPTS1} ${SSH_OPTS2} ec2-user@${BASTION_IP}" ec2-user@${NODE_IP}
For more information, go to Connecting your nodes over SSH.
Download the populate_db.sh script and make it executable:
1 2wget https://raw.githubusercontent.com/atlassian/dc-app-performance-toolkit/master/app/util/jira/populate_db.sh && chmod +x populate_db.sh
Review the following Variables section
of the script:
1 2DB_CONFIG="/var/atlassian/application-data/jira/dbconfig.xml" JIRA_DB_NAME="jira" JIRA_DB_USER="postgres" JIRA_DB_PASS="Password1!" # JSM section JIRA_CURRENT_DIR="/opt/atlassian/jira-servicedesk/current" JIRA_SETENV_FILE="${JIRA_CURRENT_DIR}/bin/setenv.sh" JIRA_VERSION_FILE="/media/atl/jira/shared/jira-servicedesk.version"
Run the script:
1 2./populate_db.sh --jsm 2>&1 | tee -a populate_db.log
Do not close or interrupt the session. It will take about 40 min to restore SQL database. When SQL restoring is finished, an admin user will have admin
/admin
credentials.
In case of a failure, check the Variables
section and run the script one more time.
We recommend that you only use this method if you are having problems with the populate_db.sh script.
In the AWS console, go to Services > EC2 > Instances.
On the Description tab, do the following:
Using SSH, connect to the Jira Service Management node via the Bastion instance:
For Linux or MacOS run following commands in terminal (for Windows use Git Bash terminal):
1 2ssh-add path_to_your_private_key_pem export BASTION_IP=bastion_instance_public_ip export NODE_IP=node_private_ip export SSH_OPTS1='-o ServerAliveInterval=60' export SSH_OPTS2='-o ServerAliveCountMax=30' ssh ${SSH_OPTS1} ${SSH_OPTS2} -o "proxycommand ssh -W %h:%p ${SSH_OPTS1} ${SSH_OPTS2} ec2-user@${BASTION_IP}" ec2-user@${NODE_IP}
For more information, go to Connecting your nodes over SSH.
Download the xml_backup.zip file corresponding to your Jira Service Management version.
1 2JSM_VERSION=$(sudo su jira -c "cat /media/atl/jira/shared/jira-servicedesk.version") sudo su jira -c "wget https://centaurus-datasets.s3.amazonaws.com/jsm/${JSM_VERSION}/large/xml_backup.zip -O /media/atl/jira/shared/import/xml_backup.zip"
Log in as a user with the Jira System Administrators global permission.
Go to > System > Restore System. from the menu.
Populate the File name field with xml_backup.zip
.
Click Restore and wait until the import is completed.
After Importing the main dataset, you'll now have to pre-load an enterprise-scale set of attachments.
Populate DB and restore attachments scripts could be run in parallel in separate terminal sessions to save time.
Using SSH, connect to the Jira Service Management node via the Bastion instance:
For Linux or MacOS run following commands in terminal (for Windows use Git Bash terminal):
1 2ssh-add path_to_your_private_key_pem export BASTION_IP=bastion_instance_public_ip export NODE_IP=node_private_ip export SSH_OPTS1='-o ServerAliveInterval=60' export SSH_OPTS2='-o ServerAliveCountMax=30' ssh ${SSH_OPTS1} ${SSH_OPTS2} -o "proxycommand ssh -W %h:%p ${SSH_OPTS1} ${SSH_OPTS2} ec2-user@${BASTION_IP}" ec2-user@${NODE_IP}
For more information, go to Connecting your nodes over SSH.
Download the upload_attachments.sh script and make it executable:
1 2wget https://raw.githubusercontent.com/atlassian/dc-app-performance-toolkit/master/app/util/jira/upload_attachments.sh && chmod +x upload_attachments.sh
Review the following Variables section
of the script:
1 2# JSM version file location JIRA_VERSION_FILE="/media/atl/jira/shared/jira-servicedesk.version"
Run the script:
1 2./upload_attachments.sh --jsm 2>&1 | tee -a upload_attachments.log
Do not close or interrupt the session. It will take about two hours to upload attachments to Elastic File Storage (EFS).
For more information, go to Re-indexing Jira.
The re-index time for JSM 4.20.x is about ~30-50 minutes, while for JSM 5.4.x it can take significantly longer at around 110-130 minutes. This increase in re-index time is due to a known issue which affects Jira 5.4.x, and you can find more information about it in this ticket: Re-Index: JSM 5.4.x.
Jira Service Management will be unavailable for some time during the re-indexing process. When finished, the Acknowledge button will be available on the re-indexing page.
Jira Service Management will be unavailable for some time during the index recovery process.
Using SSH, connect to the Jira Service Management node via the Bastion instance:
For Linux or MacOS run following commands in terminal (for Windows use Git Bash terminal):
1 2ssh-add path_to_your_private_key_pem export BASTION_IP=bastion_instance_public_ip export NODE_IP=node_private_ip export SSH_OPTS1='-o ServerAliveInterval=60' export SSH_OPTS2='-o ServerAliveCountMax=30' ssh ${SSH_OPTS1} ${SSH_OPTS2} -o "proxycommand ssh -W %h:%p ${SSH_OPTS1} ${SSH_OPTS2} ec2-user@${BASTION_IP}" ec2-user@${NODE_IP}
Once you're in the node, run command corresponding to your Jira Service Management version:
JSM 5
1 2sudo su -c "du -sh /media/atl/jira/shared/caches/indexesV2/snapshots/IndexSnapshot*" | tail -1
JSM 4
1 2sudo su -c "du -sh /media/atl/jira/shared/export/indexsnapshots/IndexSnapshot*" | tail -1
The snapshot size and name will be shown in the console output.
Please note that the snapshot size must be around 2GB or larger.
After Preloading your Jira Service Management deployment with an enterprise-scale dataset, the admin user will have admin
/admin
credentials.
It's recommended to change default password from UI account page for security reasons.
For generating performance results suitable for Marketplace approval process use dedicated execution environment. This is a separate AWS EC2 instance to run the toolkit from. Running the toolkit from a dedicated instance but not from a local machine eliminates network fluctuations and guarantees stable CPU and memory performance.
jsm.yml
configuration file. Set enterprise-scale Jira Service Management Data Center parametersTrue
next to the insight
variable.Do not push to the fork real application_hostname
, admin_login
and admin_password
values for security reasons.
Instead, set those values directly in .yml
file on execution environment instance.
1 2application_hostname: test_jsm_instance.atlassian.com # Jira Service Management DC hostname without protocol and port e.g. test-jsm.atlassian.com or localhost application_protocol: http # http or https application_port: 80 # 80, 443, 8080, 2990, etc secure: True # set False to allow insecure connections, e.g. when using self-signed SSL certificate application_postfix: # set to empty for CloudFromation deploymente.g. /jira in case of url like http://localhost:2990/jira admin_login: admin admin_password: admin load_executor: jmeter # jmeter and locust are supported. jmeter by default. concurrency_agents: 50 # number of concurrent virtual agents for jmeter or locust scenario concurrency_customers: 150 # number of concurrent virtual customers for jmeter or locust scenario test_duration: 45m ramp-up: 3m # time to spin all concurrent users total_actions_per_hour_agents: 5000 # number of total JMeter/Locust actions per hour total_actions_per_hour_customers: 15000 # number of total JMeter/Locust actions per hour insight: False # Set True to enable Insight specific tests
Push your changes to the forked repository.
Ubuntu Server 22.04 LTS
.c5.2xlarge
30
GiBConnect to the instance using SSH or the AWS Systems Manager Sessions Manager.
1 2ssh -i path_to_pem_file ubuntu@INSTANCE_PUBLIC_IP
Install Docker. Setup manage Docker as a non-root user.
Connect to the AWS EC2 instance and clone forked repository.
At this stage app-specific actions are not needed yet. Use code from master
branch with your jsm.yml
changes.
You'll need to run the toolkit for each test scenario in the next section.
Using the Data Center App Performance Toolkit for Performance and scale testing your Data Center app involves two test scenarios:
Each scenario will involve multiple test runs. The following subsections explain both in greater detail.
This scenario helps to identify basic performance issues without a need to spin up a multi-node Jira Service Management DC. Make sure the app does not have any performance impact when it is not exercised.
To receive performance baseline results without an app installed:
Use SSH to connect to execution environment.
Run toolkit with docker from the execution environment instance:
1 2cd dc-app-performance-toolkit docker run --pull=always --shm-size=4g -v "$PWD:/dc-app-performance-toolkit" atlassian/dcapt jsm.yml
View the following main results of the run in the dc-app-performance-toolkit/app/results/jsm/YY-MM-DD-hh-mm-ss
folder:
results_summary.log
: detailed run summaryresults.csv
: aggregated .csv file with all actions and timingsbzt.log
: logs of the Taurus tool executionjmeter.*
: logs of the JMeter tool executionpytest.*
: logs of Pytest-Selenium executionReview results_summary.log
file under artifacts dir location. Make sure that overall status is OK
before moving to the next steps. For an enterprise-scale environment run, the acceptable success rate for actions is 95% and above.
If you are submitting a Jira Service Management app, you are required to conduct a Lucene Index timing test. This involves conducting a foreground re-index on a single-node Data Center deployment (with your app installed) and a dataset that has 1M issues.
The re-index time for JSM 4.20.x is about ~30-50 minutes, while for JSM 5.4.x it can take significantly longer at around 110-130 minutes. This increase in re-index time is due to a known issue which affects Jira 5.4.x, and you can find more information about it in this ticket: Re-Index: JSM 5.4.x.
If your Amazon RDS DB instance class is lower than db.m5.xlarge
it is required to wait ~2 hours after previous reindex finish before starting a new one.
Benchmark your re-index time with your app installed:
Performance results generation with the app installed:
Run toolkit with docker from the execution environment instance:
1 2cd dc-app-performance-toolkit docker run --pull=always --shm-size=4g -v "$PWD:/dc-app-performance-toolkit" atlassian/dcapt jsm.yml
Review results_summary.log
file under artifacts dir location. Make sure that overall status is OK
before moving to the next steps. For an enterprise-scale environment run, the acceptable success rate for actions is 95% and above.
To generate a performance regression report:
Use SSH to connect to execution environment.
Allow current user (for execution environment default user is ubuntu
) to access Docker generated reports:
1 2sudo chown -R ubuntu:ubuntu /home/ubuntu/dc-app-performance-toolkit/app/results
Install and activate the virtualenv
as described in dc-app-performance-toolkit/README.md
Navigate to the dc-app-performance-toolkit/app/reports_generation
folder.
Edit the performance_profile.yml
file:
Run the following command:
1 2python csv_chart_generator.py performance_profile.yml
In the dc-app-performance-toolkit/app/results/reports/YY-MM-DD-hh-mm-ss
folder, view the .csv
file (with consolidated scenario results), the .png
chart file and performance scenario summary report.
Use scp command to copy report artifacts from execution env to local drive:
1 2export EXEC_ENV_PUBLIC_IP=execution_environment_ec2_instance_public_ip scp -r -i path_to_exec_env_pem ubuntu@$EXEC_ENV_PUBLIC_IP:/home/ubuntu/dc-app-performance-toolkit/app/results/reports ./reports
./reports
folder you will be able to review the action timings with and without your app to see its impact on the performance of the instance. If you see an impact (>20%) on any action timing, we recommend taking a look into the app implementation to understand the root cause of this delta.The purpose of scalability testing is to reflect the impact on the customer experience when operating across multiple nodes. For this, you have to run scale testing on your app.
For many apps and extensions to Atlassian products, there should not be a significant performance difference between operating on a single node or across many nodes in Jira Service Management DC deployment. To demonstrate performance impacts of operating your app at scale, we recommend testing your Jira Service Management DC app in a cluster.
To receive scalability benchmark results for one-node Jira Service Management DC with app-specific actions:
Apply app-specific code changes to a new branch of forked repo.
Use SSH to connect to execution environment.
Pull cloned fork repo branch with app-specific actions.
Run toolkit with docker from the execution environment instance:
1 2cd dc-app-performance-toolkit docker run --pull=always --shm-size=4g -v "$PWD:/dc-app-performance-toolkit" atlassian/dcapt jsm.yml
Review results_summary.log
file under artifacts dir location. Make sure that overall status is OK
before moving to the next steps. For an enterprise-scale environment run, the acceptable success rate for actions is 95% and above.
Before scaling your DC make sure that AWS vCPU limit is not lower than needed number. Use vCPU limits calculator to see current limit. The same article has instructions on how to increase limit if needed.
To receive scalability benchmark results for two-node Jira Service Management DC with app-specific actions:
2
in the Maximum number of cluster nodes and the Minimum number of cluster nodes fields.In case if you got error during update - BastionPrivIp cannot be updated
.
Please use those steps for a workaround:
2
in the Desired capacity, Minimum capacity and Maximum capacity fields.ACTIVE
and application status RUNNING
. To make sure that Jira Service Management index successfully synchronized to the second node.In case if index synchronization is failed by some reason (e.g. application status is MAINTENANCE
) follow those steps:
Run toolkit with docker from the execution environment instance:
1 2cd dc-app-performance-toolkit docker run --pull=always --shm-size=4g -v "$PWD:/dc-app-performance-toolkit" atlassian/dcapt jsm.yml
Review results_summary.log
file under artifacts dir location. Make sure that overall status is OK
before moving to the next steps. For an enterprise-scale environment run, the acceptable success rate for actions is 95% and above.
Before scaling your DC make sure that AWS vCPU limit is not lower than needed number. Use vCPU limits calculator to see current limit. The same article has instructions on how to increase limit if needed.
To receive scalability benchmark results for four-node Jira Service Management DC with app-specific actions:
Scale your Jira Service Management Data Center deployment to 3 nodes as described in Run 4.
Check Index is synchronized to the new node #3 the same way as in Run 4.
Scale your Jira Service Management Data Center deployment to 4 nodes as described in Run 4.
Check Index is synchronized to the new node #4 the same way as in Run 4.
Run toolkit with docker from the execution environment instance:
1 2cd dc-app-performance-toolkit docker run --pull=always --shm-size=4g -v "$PWD:/dc-app-performance-toolkit" atlassian/dcapt jsm.yml
Review results_summary.log
file under artifacts dir location. Make sure that overall status is OK
before moving to the next steps. For an enterprise-scale environment run, the acceptable success rate for actions is 95% and above.
To generate a scalability report:
ubuntu
) to access Docker generated reports:
1 2sudo chown -R ubuntu:ubuntu /home/ubuntu/dc-app-performance-toolkit/app/results
dc-app-performance-toolkit/app/reports_generation
folder.scale_profile.yml
file:
virtualenv
(as described in dc-app-performance-toolkit/README.md
):
1 2python csv_chart_generator.py scale_profile.yml
dc-app-performance-toolkit/app/results/reports/YY-MM-DD-hh-mm-ss
folder, view the .csv
file (with consolidated scenario results), the .png
chart file and summary report.Use scp command to copy report artifacts from execution env to local drive:
1 2export EXEC_ENV_PUBLIC_IP=execution_environment_ec2_instance_public_ip scp -r -i path_to_exec_env_pem ubuntu@$EXEC_ENV_PUBLIC_IP:/home/ubuntu/dc-app-performance-toolkit/app/results/reports ./reports
./reports
folder, you will be able to review action timings on Jira Service Management Data Center with different numbers of nodes. If you see a significant variation in any action timings between configurations, we recommend taking a look into the app implementation to understand the root cause of this delta.After completing all your tests, delete your Jira Service Management Data Center stacks.
Do not forget to attach performance testing results to your ECOHELP ticket.
profile.csv
, profile.png
, profile_summary.log
and profile run result archives. Archives
should contain all raw data created during the run: bzt.log
, selenium/jmeter/locust logs, .csv and .yml files, etc.In case of technical questions, issues or problems with DC Apps Performance Toolkit, contact us for support in the community Slack #data-center-app-performance-toolkit channel.
Rate this page: