Atlassian Audit API

Overview

Atlassian Audit (Advanced Auditing for DC customers) is a cross-product feature available in Atlassian DC products (Bitbucket, Confluence, and Jira) which is responsible for storing and retrieving audited events.

Background

At end of 2019 all products had entirely different implementations of auditing, customers use broader event coverage in the products, and that via consolidating to one implementation we could improve the feature set of the products for customers. So began the work to build Atlassian Auditing and thus Advanced Auditing for DC customers.

The products did not all migrate at the same time (Bitbucket migrated in 7.0.0, Confluence in 7.5.0, and in Jira from 8.8.0), and features have continued to be developed and adopted at different rates.

There are no current plans to migrate Bamboo and Crowd, if you'd like to see this, please vote and comment new suggestions on the publicly tracked tickets; Bamboo (BAM-22246), and Crowd (CWD-5941).

Fundamental concepts

What goes into an audit event

The audit events include:

• Who performed the action (author)
• When the action was performed (timestamp)
• What action occurred (category/summary)
• What was affected (affected objects)
• What changed (changed values)
• Where did the author perform action (source/IP)
• How did the author access the system (method)

Permissions and delegated administration

Atlassian Audit - like other parts of the products - has the concept of delegate administration, that is the ability for a user to administer an entity (i.e. project, space, or repository) without necessarily being a global nor system administrator. This enables customers to scale their organisation size by delegating administration duties.

In code, the UI and REST APIs are the same for all users, to enable this there are more complicated permission and visibility checks. The rule of thumb is that users can see events that have an affected object which they can administer.

How to use the API

Internal (Private) APIs

There are SPIs and private REST APIs that are intentionally not documented, these may change at any time (potentially within a bug release) and are generally not suitable for general usage within the rest of the products as they may not have the appropriate permissions management or may not be from the correct source-of-truth.

Public APIs that probably shouldn't be used

• REST - Rebuild the categories and summaries cache - This is an endpoint intended to only be used by administrators and recommended by support if for any reason it becomes broken. It can have a large negative performance impact by scanning through the entire database events table. Ideally this is never needed because the cache should be automatically updated as new events come in. At the time of writing this is only used for the UI so there's no reason for Apps (plugins) to use it.
• REST + OSGi - Anything marked as deprecated - These could be removed at the next major release.
• REST + OSGi - Updating the coverage, excluded events, and retention configuration - Using these could lead to privilege escalation attacks. In addition, users could be confused when, why, and how these settings changed without being informed thus putting them at risk for attacks and/or not meeting required compliance.
• OSGi - AuditRetentionConfigUpdatedEvent - This is only fired on the local node and thus without an additional cluster messaging or invalidation layer can lead to inconsistent results. It could be better to poll against AuditRetentionConfigService (⚠️ be careful; there's currently no caching).
• OSGi - Non-internationalisation compatible APIs - These can make it harder for users to search for audit events as they might only search for one version of the translation and/or description and thus miss other relevant events.

At a next major release we may make these private. If you think there's a missing API or a use-case that hasn't been considered, feel free to create a ticket.

Consume audit events using Java

Out-of-the-box, Advanced Auditing provides two consumers; database and filesystem. The database is used for powering search functionality, and the filesystem is for integration into a long-term storage solution. Both have maximum retention values to prevent performance stability problems. Each AuditConsumer has its own thread and a fixed size (10K, configurable via system properties) queue to buffer the events received. This design choice is to minimise the performance impact caused by a slow consumer.

It's possible to create another consumer by implementing an AuditConsumer OSGi service. Notice that it accepts AuditEntity rather than AuditEvent because there is extra data to be enriched by the consumer if the audit system has not yet filled it.

First, add the Atlassian Audit API as a dependency, in maven this would be done like so;

1
2
<dependency>
    <groupId>com.atlassian.audit</groupId>
    <artifactId>atlassian-audit-api</artifactId>
    <scope>provided</scope> <!-- When the product provides a dependency, bundling it is not required -->
</dependency>

There's multiple ways to export an OSGi service (e.g. using Spring Java configuration, using Spring scanner, using the plugin module descriptors, and using the OSGi namespace in Spring XML, but to give part of an example using Spring Java configuration (as of version 0.6.0 and Spring 5):

1
2
package com.example.audit;

import com.atlassian.audit.entity.AuditEntity;
import java.util.List;
import javax.annotation.Nonnull;

import static java.util.Objects.requireNonNull;

public class ExampleDatabaseAuditConsumer implements AuditConsumer {

    private final ApplicationProperties applicationProperties;
    private long count = 0;

    public ExampleDatabaseAuditConsumer(@Nonnull ApplicationProperties applicationProperties) {
        this.applicationProperties = requireNonNull(applicationProperties);
    }

    @Override
    public void accept(@Nonnull List<AuditEntity> entities) {
        count += entities.size();
        System.out.println("Number of audit events has reached : " + count);
        // Normally the `AuditEntity`s have their missing data enriched and then stored somewhere, ideally with a buffer
        // that drops events for stability and records (potentially via conventional logging) when there are gaps in
        // audit events due to being dropped.
    }

    @Override
    public boolean isEnabled() {
        return Boolean.parseBoolean(applicationProperties.getPropertyValue("com.example.audit.event.counter.enable"));
    }
}

1
2
package com.example.spring.configuration;

import com.atlassian.audit.api.AuditConsumer;
import com.example.audit.ExampleDatabaseAuditConsumer;
import org.osgi.framework.ServiceRegistration;
import org.springframework.beans.factory.FactoryBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import static com.atlassian.plugins.osgi.javaconfig.ExportOptions.as;
import static com.atlassian.plugins.osgi.javaconfig.OsgiServices.exportOsgiService;

@Configuration
public class ExampleOsgiExportConfiguration {
    @Bean
    public FactoryBean<ServiceRegistration> exportExampleAuditConsumer(ExampleDatabaseAuditConsumer exampleAuditConsumer) {
        return exportOsgiService(exampleAuditConsumer, as(AuditConsumer.class));
    }
}

Migrating from the legacy APIs

• Bitbucket
• Developer Community thread

Create an audit event using Java

com.atlassian.audit.api.AuditService is the entry point to record a new AuditEvent

First, add the Atlassian Audit API as a dependency, in maven this would be done like so;

1
2
<dependency>
    <groupId>com.atlassian.audit</groupId>
    <artifactId>atlassian-audit-api</artifactId>
    <scope>provided</scope> <!-- When the product provides a dependency, bundling it is not required -->
</dependency>

Then, import the AuditService OSGi service. There are multiple ways (e.g. using Spring Java configuration, using Spring scanner, using the plugin module descriptors, and using the OSGi namespace in Spring XML), but to give an example using Spring Java configuration:

1
2
package com.example.spring.configuration;

import com.atlassian.audit.api.AuditService;
import org.springframework.beans.factory.FactoryBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import static com.atlassian.plugins.osgi.javaconfig.OsgiServices.importOsgiService;

@Configuration
public class ExampleOsgiImportConfiguration {
    @Bean
    public AuditService auditService() {
        return importOsgiService(AuditService.class);
    }
}

To record an event, call AuditService#audit(AuditEvent). There are three types of information to be specified to construct an AuditEvent:

• AuditType with:
  • Area - An enum representing broad groups of event. Atlassian Auditing allows for use configuration of verbosity per area (see the below point about levels). For third-party apps, this value is always overwritten to ECOSYSTEM. This is to allow turning this down or off without affecting the ability to audit the rest of the system. In the future we could support some mechanism like per app or sub-categories. If you'd like to see this please create a ticket describing the use-case (so we can figure out how to best support this).
  • Category - A group of summaries the event, e.g. "Issue Management". These can be used as filter while searching events.
  • Summary - Describes the exact action, e.g. "Create Issue". These can be used as filter while searching events.
  • Level - Used to indicate the importance and verbosity. It is admin configurable to an individual level or off entirely. The levels are:
    • Base - Most important and least verbose
    • Advanced - Medium importance and verbosity
    • Full - All events and can be very verbose
• Affected objects - Used to identify the entities that were affected, e.g. a user, an issue or project. Projects, spaces, and repositories have to be present in the event for delegate (i.e. project, space, or repository) administrators to see them in delegate audit log UI. These are often directly linked to via the UI. These can be used as filter while searching events. The order of these matters visually, it should be from most to least affected.
• Changed values - Values that changed, typically on one or more of the affected objects, e.g. the issue description. The UI will present the before and after values as a "diff".
• Extra Attributes - Additional information to be recorded.

Putting this all to use, to construct and fire an AuditEvent:

1
2
package com.example.audit;

import com.atlassian.audit.api.AuditService;
import com.atlassian.audit.entity.AuditAttribute;
import com.atlassian.audit.entity.AuditEntity;
import com.atlassian.audit.entity.AuditResource;
import com.atlassian.audit.entity.AuditType;
import com.atlassian.audit.entity.CoverageArea;
import com.atlassian.audit.entity.CoverageLevel;

import static java.util.Objects.requireNonNull;

public class ExampleAuditEventPublisher {

    private static final String I18N_AUTH_CATEGORY_KEY = "com.example.audit.auth.category";
    private static final String I18N_AUTH_SUMMARY_KEY = "com.example.audit.auth.summary";
    
    private final AuditService auditService;

    public ExampleAuditEventPublisher(@Nonnull AuditService auditService) {
        this.auditService = requireNonNull(auditService);
    }

    private void auditEvent() {
        final AuditEvent auditEvent = AuditEvent.builder(
                        AuditType.fromI18nKeys(CoverageArea.SECURITY,
                                        CoverageLevel.ADVANCED,
                                        I18N_AUTH_CATEGORY_KEY,
                                        I18N_AUTH_SUMMARY_KEY)
                                .build())
                .affectedObject(AuditResource
                        .builder("Antoninus Pius", USER)
                        .id("2c9d898c70743a6c0170743b9cf80000")
                        .build())
                .extraAttribute(AuditAttribute
                        .fromI18nKeys("com.atlassian.bi.security.auth.source", "LDAP Server")
                        .build())
                .build();
        auditService.audit(auditEvent);
    }
}

Migrating from the legacy APIs

• Jira
• Bitbucket
• Developer Community thread

Search for audit events using Java

com.atlassian.audit.api.AuditSearchService is the entrypoint to querying audited entities from the database. It's an OSGi service provided by the audit plugin.

First, add the Atlassian Audit API as a dependency, in maven this would be done like so;

1
2
<dependency>
    <groupId>com.atlassian.audit</groupId>
    <artifactId>atlassian-audit-api</artifactId>
    <scope>provided</scope> <!-- When the product provides a dependency, bundling it is not required -->
</dependency>

Then, import the AuditSearchService OSGi service. There are multiple ways (e.g. using Spring Java configuration, using Spring scanner, using the plugin module descriptors, and using the OSGi namespace in Spring XML), but to give an example using Spring Java configuration:

1
2
package com.example.spring.configuration;

import com.atlassian.audit.api.AuditSearchService;
import org.springframework.beans.factory.FactoryBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import static com.atlassian.plugins.osgi.javaconfig.OsgiServices.importOsgiService;

@Configuration
public class ExampleOsgiImportConfiguration {
    @Bean
    public AuditSearchService auditSearchService() {
        return importOsgiService(AuditSearchService.class);
    }
}

To give some examples as how to query the data;

1
2
package com.example.audit;

import com.atlassian.audit.api.AuditEntityCursor;
import com.atlassian.audit.api.AuditQuery;
import com.atlassian.audit.api.AuditSearchService;
import com.atlassian.audit.api.util.pagination.Page;
import com.atlassian.audit.api.util.pagination.PageRequest;
import com.atlassian.audit.entity.AuditEntity;
import com.atlassian.sal.api.user.UserKey;

import javax.annotation.Nonnull;

import java.util.concurrent.TimeoutException;

import static java.util.Objects.requireNonNull;

public class AuditEntitySearchExample {
    private final AuditSearchService auditSearchService;

    public AuditEntitySearchExample(@Nonnull AuditSearchService auditSearchService) {
        this.auditSearchService = requireNonNull(auditSearchService);
    }

    public void search(AuditEntity auditEntity, UserKey userKey) throws TimeoutException {
        final AuditQuery auditQuery = AuditQuery.builder()
                .actions("Delete Issue") // non translated summary/action key
                .categories("Issue Management") // non translated category key
                .resource("PROJECT", "44502") // "PROJECT" like Jira project, and the project/DB ID
                .searchText("space delimited search query")
                .userIds(userKey.getStringValue())
                .minId(auditEntity.getId()) // get all newer events
                .build();
        auditSearchService.count(auditQuery); // Will count using the database

        
        final PageRequest<AuditEntityCursor> pageRequest = new PageRequest.Builder<AuditEntityCursor>()
                .offset(0)
                .limit(100)
                .build();
        final Page<AuditEntity, AuditEntityCursor> result =
                auditSearchService.findBy(auditQuery, pageRequest, 1_000_000); // It's recommended to use a maximum scan
        // limit with free text search ("searchText") as it can both be slow and demanding on the database.

        result.getNextPageRequest(); // Using the next page cursor will help ensure results are contiguous
    }
}

Migrating from the legacy APIs

• Jira
• Bitbucket
• Developer Community thread

Search for audit events using REST

See the /events endpoint in the Atlassian Audit OpenAPI documentation for more information.

Get the coverage and retention configuration using Java

com.atlassian.audit.api.AuditCoverageConfigService and AuditRetentionConfigService are the services that provide the coverage level for a particular coverage area, and getting the audit entity retention period.

First, add the Atlassian Audit API as a dependency, in maven this would be done like so;

1
2
<dependency>
    <groupId>com.atlassian.audit</groupId>
    <artifactId>atlassian-audit-api</artifactId>
    <scope>provided</scope> <!-- When the product provides a dependency, bundling it is not required -->
</dependency>

Then, import the AuditCoverageConfigService and AuditRetentionConfigService OSGi services. There are multiple ways (e.g. using Spring Java configuration, using Spring scanner, using the plugin module descriptors, and using the OSGi namespace in Spring XML), but to give an example using Spring Java configuration:

1
2
package com.example.spring.configuration;

import com.atlassian.audit.api.AuditSearchService;
import org.springframework.beans.factory.FactoryBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import static com.atlassian.plugins.osgi.javaconfig.OsgiServices.importOsgiService;

@Configuration
public class ExampleOsgiImportConfiguration {
    @Bean
    public AuditCoverageConfigService auditCoverageConfigService() {
        return importOsgiService(AuditCoverageConfigService.class);
    }

    @Bean
    public AuditRetentionConfigService auditRetentionConfigService() {
        return importOsgiService(AuditRetentionConfigService.class);
    }
}

To give an example on what getting that data from the services could look like;

1
2
package com.example.audit;

import com.atlassian.audit.api.AuditCoverageConfigService;
import com.atlassian.audit.api.AuditRetentionConfig;
import com.atlassian.audit.api.AuditRetentionConfigService;
import com.atlassian.audit.entity.AuditCoverageConfig;
import com.atlassian.audit.entity.EffectiveCoverageLevel;

import javax.annotation.Nonnull;
import java.time.Period;

import static com.atlassian.audit.entity.CoverageArea.ECOSYSTEM;
import static java.util.Objects.requireNonNull;

public class AuditCoverageAndRetentionExample {
    private final AuditCoverageConfigService auditCoverageConfigService;
    private final AuditRetentionConfigService auditRetentionConfigService;

    public AuditCoverageAndRetentionExample(
            @Nonnull AuditCoverageConfigService auditCoverageConfigService,
            @Nonnull AuditRetentionConfigService auditRetentionConfigService
    ) {
        this.auditCoverageConfigService = requireNonNull(auditCoverageConfigService);
        this.auditRetentionConfigService = requireNonNull(auditRetentionConfigService);
    }

    public void get() {
        final AuditCoverageConfig auditCoverageConfig = auditCoverageConfigService.getConfig();
        final AuditRetentionConfig auditRetentionConfig = auditRetentionConfigService.getConfig();
        final Period minimumTimeToHoldEventsInTheDatabase = auditRetentionConfig.getPeriod();
        final EffectiveCoverageLevel ecosystemCoverage = auditCoverageConfig.getLevelByArea().get(ECOSYSTEM);
    }
}

Get the coverage, retention and excluded events configuration using REST

See the /configuration/coverage, /configuration/retention, /configuration/retention/file, and /configuration/denylist endpoints in the Atlassian Audit OpenAPI documentation for more information.

Extending the frontend

The audit web-resource context is reserved for when the Atlassian Audit UI is presented to a user, feel free to reuse it to load more resource.

There's no web-item, web-section, nor web-panel location (read: extension points) as CSE (Client-Side Extensions) wasn't yet broadly available in all the supported products. CSE is now more available, and it's possible to add traditional web-panel locations above and below the UI. If you'd like to see this, please create a ticket along with the anticipated end-user use-case (so we can figure out how to best support you).

Javadoc links

• Javadoc
• AuditConsumer
• AuditEntity
• AuditEvent
• AuditRetentionConfigService
• AuditCoverageConfigService
• AuditSearchService
• AuditService
• AuditType