Evaluation insights

You can use evaluation insights to learn the potential opportunity size of an evaluation, the evaluations that led to the sale, and how many days has it taken for an evaluation to convert. This information will help you prioritize and target your prospective customers better and help you figure out the ROI of your marketing campaigns.

Evaluation insights are available as part of the Marketplace licenses API reporting. The licenses API contains the following fields to help evaluate the ROI of marketing campaigns and opportunity size:

• evaluationOpportunitySize

• evaluationLicense

• daysToConvertEval

• evaluationStartDate

• evaluationEndDate

• evaluationSaleDate

Get licenses API

API documentation: The response now contains the evaluationOpportunitySize field for each evaluation license and all the evaluation insights fields for a non-evaluation license. The evaluationOpportunitySize field will be always "NA" for a non-evaluation. This data is available when the endpoint is requested with the ?withDataInsights=true query parameter.

Sample API response:

1
2
{
  "_links": {},
  "licenses": [
    {
      "addonLicenseId": "121222",
      "hostLicenseId": "112434",
      "licenseId": "SEN-112434",
      "addonKey": "com.atlassian.marketplace.app",
      "addonName": "Marketplace App",
      "hosting": "Cloud",
      "lastUpdated": "2019-10-01",
      "licenseType": "COMMERCIAL",
      "maintenanceStartDate": "2020-09-22",
      "maintenanceEndDate": "2020-12-18",
      "status": "active",
      "tier": "Subscription",
      "contactDetails": {
        "company": "S.T.A.R. Labs",
        "country": "United States of America",
        "region": "Americas",
        "technicalContact": {
          "email": "barryallen@starlabs.com",
          "name": "Barry Allen"
        }
      },
      "evaluationOpportunitySize": "NA",
      "evaluationLicense": "18762269",
      "daysToConvertEval": "0",
      "evaluationStartDate": "2020-09-22",
      "evaluationEndDate": "2020-11-19",
      "evaluationSaleDate": "2020-11-19"
    }
  ]
}

Export licenses API

API documentation: The CSV export when requested with ?withDataInsights=true contains the above specified evaluation insights fields. The fields other than evaluationOpportunitySize will not be populated for evaluation licenses. All the evaluation insights fields are appended to a non-evaluation license if a suitable mapping is available.

The JSON and CSV files have new fields similar to the get licenses API mentioned above.

• evaluationOpportunitySize

• evaluationLicense

• daysToConvertEval

• evaluationStartDate

• evaluationEndDate

• evaluationSaleDate

Accessing evaluation insights data

In the licenses section under reports, follow these steps:

• Open the filters dropdown

• Change 'Type of data' from "Default" to "With additional insights"

• Select a date range that is less than 90 days

• Apply the filters

• Click the CSV/JSON links at the top of the page to download the CSV/JSON respectively

• Additional columns corresponding to evaluation insights will appear for licenses after Jan 2020

Evaluation opportunity size field values

Here is what each value in the evaluationOpportunitySize field represent:

Things to remember

• Evaluation insights are available for cloud and DC but not for server.

• The data is available approximately 48-72 hrs after an evaluation is purchased or after evaluation is converted to a sale.

• The exporting API has a 90 day limit on the date range for which evaluation insights can be queried. If you're using query parameters startDate and endDate, and the difference in startDate and endDate is greater than > 90 days the response will be Bad request error with the message "Date range must be 90 days or less".

• Evaluation insights are available for evaluations created from January 2020.

• Evaluation insights for cloud evaluations are of higher accuracy as compared to DC evaluations as we have limited insight into DC instances. SEN of a cloud instance remains the same after it is converted, however it changes in DC. In DC, a user need not have to evaluate before a purchase is made.

• Logic to derive opportunity size:

  • When an app is installed by a customer, the number of users in the parent product(Jira/Confluence) as of that date is computed. 

  • If a customer's site has multiple Jira/confluence instances, the opportunity size is the number of users in the largest instance.

  • If an app can be installed on both Jira and Jira service desk(JSD), opportunity size is the number of users in Jira and not JSD.

• Logic to derive evaluation license for a sale:

  • For cloud apps, SEN is the same for evaluation and sale.

  • For DC apps the first match is on hostname. If there is no match, then the email is matched. If there is still no match, the domain is used to find a match. The look back period of the evaluation is 6 months i.e. the evaluation should have been done not greater than 6 months prior to the purchase.

• The parameter daysToConvertEval is the difference between sale date and evaluation end date. When the sale date is after the evaluation end date, this will be a positive value. When the sale date is before the evaluation end date, this will be a negative value.