Preliminary Plugin Guide to JIRA Archiver

Note for customer and plugin vendors: We did some initial investigation on Archiving in 2014. Since then we've been investing in; a) JIRA Data Center's robustness and b) Performance for all large scale JIRA instances. We don't currently have archiving on our roadmap and will be re-evaluating our plans for Archiving later in 2015. Questions or feedback on the SPI can be left as comments on this page but are not likely to get replies until we re-open our work on archiving. The information on this page should be taken as a draft outline of how Atlassian may plan to develop a plugin SPI for a future JIRA Archiver solution. The final solution and the SPI does not yet exist.

Background

Currently JIRA Archiver has no way to handle any plugin data beyond Custom Fields and the support for Custom Fields is limited in many ways, e.g. the GreenHopper sprint field might bring data across pointing to a sprint, but that sprint will not exist in the Archive.

The JIRA Archiver plugin cannot solve this problem directly as it has, and cannot have, any idea of what data is stored by plugins or what the meaning and relationships with in that data are.  It cannot tell what data refers to what projects or issues or even if the data is related to a project.  GreenHopper for example is not organised around Projects at all, but rather uses JQL to define rapid boards etc.

In the light of this it is necessary to provide a mechanism to allow plugins to actively participate in the archiving process.

Archiving Overview

In order to bring the JIRA Archiver to market in a reasonable time frame, it leverages a number of technologies already present in JIRA.   The main two of these are:

  • System Export, aka backup, to XML and
  • Project Import.

System Export

A standard system export contains all JIRA's basic configuration and transactional data, with one  omission of the issue attachments.  It also contains the backup of any plugin data stored in Property sets or in Active Objects tables.

Project Import

Project import supports importing a project's issues along with their comments and change history from the system backup of one instance into another instance.  JIRA Archiver extends this capability to also import the configuration that supports this project and also to import only a selection of the project's issues.

Additional functions of JIRA Archiver

JIRA Archiver also supports removing the archived issues from the source server at the end of the archiving process, updating the configuration of a project during subsequent archiving events and keeping an audit of successful, redirecting of browse issue and query requests and keeping an audit of successful and incomplete archiving events.

Archiver Plugin SPI

This SPI does not provide an explicit way for plugins to move data stored in random places like the file system or their own database or in a cardboard box in the laundry.  They can just do this in one handler or the other by implementing some mechanism such as a rest service to drag data across as we do for attachments.

In this document we refer to the Archiver Plugin SPI or just the SPI as we are mostly describing a set of interfaces that plugins will need to implement to fully participate in Archiving.  There will however be both SPI and API points described and provided.

The Archiving process proceeds in a number of steps and this SPI allows a plugin to participate in any one of these steps.  The basic flow is:

  1. The Source system is backed up to XML
  2. The XML is sent to the Target system
  3. The target system processes the XML extracting information about the project and issues in the backup and mapping configuration items, e.g. issue type ids from the source system to the values on the target system.
  4. The target system processes the XML again splitting it into a number of files each one or more entity types, e.g.<jiraissue>, <customfield> or <jiraaction>
  5. The target system creates the project, if required.
  6. The target system then creates the configuration data required from the split up XML files.  The files are processed in such an order that the dependencies between data can be built as the files are processed.
  7. The target system then creates the issues and the issue related data such as comments and change history.
  8. The source system then, after user confirmation, deletes the project and issues from the source system

There are two basic mechanisms that the plugin developer needs to interact with:

  1. Each data entity type (table) can be processed by the plugin, if it chooses to provide a handler for that entity type.
  2. There is a collection of Mappers that the plugin can use and contibute to to work out how data is mapped between Ids in the source and ids in the target system.

Additionally for AO data the plugin can specify the order in which the AO entities should be imported, so that they can control the rebuilding of dependencies between data.

Interface com.atlassian.jira.imports.project.handler.ImportEntityHandler

Defines a handler class that will be able to perform some operation given an entity name and the entities attributes.

Method Summary

  • endDocument() – void.

    • Provides the implementation an opportunity to perform some action when the document is finished being read.

  • handleEntity(java.lang.String entityName, java.util.Map<java.lang.String,java.lang.String> attributes) – void.

    • This is the main method to implement when using this ImportEntityHandler. This method will provide the entity name and a complete map of attribute key/value pairs.

  • startDocument() – void.

    • Provides the implementation an opportunity to perform some action when the document is starting to be read.

  • setArchiveImportMapper(ArchiveImportMapper projectImportMapper)  – void.

    • A setter for the ArchiveImportMapper.  This will be injected when the instance is created.

  • setProjectImportResults(ProjectImportResults projectImportResults)   – void.

    • A setter for the ProjectImportResults.  This will be injected when the instance is created.

  • setBackupSystemInformation(BackupSystemInformation backupSystemInformation)   – void.

    • A setter for the BackupSystemInformation.  This will be injected when the instance is created.

  • setBackupProject(BackupProject backupProject)   – void.

    • A setter for the BackupProject.  This will be injected when the instance is created.

Plugins should define one or more implementation of this interface to process any data they wish to persist into the database, either PropertySet data or Active Objects data and also to observer any data they wish to observe from any other JIRA entities.

This can probably be split to be either an AO or a JIRA entity handler.  AO entity handlers should implement a .handle(AOModelObject object) method where each entity associated with the handler in the plugin.xml has a specific handle method which will be passed a correctly typed model object, instead of just a map of untyped stuff.

 

Class com.atlassian.jira.imports.project.mapper.AbstractMapper

Plugins should provide a concrete implementation of this class when they need to map old values from the source system to new values in the destination and/or to flag values as required.  They can, of course, extend it to provide any additional data they required to help in completing the archival process. A general purpose mapper, SimpleProjectImportIdMapperImpl, is provided for use in trivial cases.

Instances of the mapper are available by calling the getPluggableMapper(String mapperKey) on the ArchiveImportMapper.

Method Summary

  • clearMappedValues()  – void.

    •  This will clear any mapped data that may have been entered into the mappers.

  • flagValueAsRequired(java.lang.String oldId) – protected void.

    •  This is an internal method for use by Mappers extending AbstractMapper and should not be called from other classes.

  • getAllMappedIds() – java.util.Collection<java.lang.String>.

    • Returns a Collection of all the new IDs that are mapped to.

  • getDisplayName(java.lang.String oldId)    – java.lang.String.

    • Returns a display name for the given old ID.

  • getKey(java.lang.String oldId) – java.lang.String.

    • Returns the registered "key" for the given old ID, or null if none is registered.

  • getMappedId(java.lang.String oldId) – java.lang.String.

    • Retrieves a String that corresponds to the id in the target JIRA system, null if none has been mapped.

  • getRegisteredOldIds()  – java.util.Collection<java.lang.String>.

    • Returns a collection of ID's as String objects identifying all objects of the appropriate type found in the import file.

  • getRequiredOldIds()– java.util.Collection<java.lang.String>.

    • Returns a collection of ID's as String objects identifying objects from the import file that are required for the import.

  • getValuesFromImport() – java.util.Collection<IdKeyPair>.

    • Deprecated. Use getRegisteredOldIds()

  • mapValue(java.lang.String oldId, java.lang.String newId) – void.

    • This method maps a value from the backup system to a valid value in the target system.

  • registerOldValue(java.lang.String oldId, java.lang.String oldKey) – protected void.

    • This is an internal method for use by Mappers extending AbstractMapper and should not be called from other classes.

Interface com.atlassian.jira.imports.project.handler.PreImportHandler

Defines a handler class that will be called after the project object is created but before any configuration or issue data is imported. Plugins can use this handler to examine the state of the target system before the import begins.

Method Summary

  • run() – void.
    • Provides the implementation an opportunity to perform some action before importing commences.
  • setArchiveImportMapper(ArchiveImportMapper projectImportMapper) – void. 
    • A setter for the ArchiveImportMapper.  This will be injected when the instance is created.
  • setProjectImportResults(ProjectImportResults projectImportResults) – void. 
    • A setter for the ProjectImportResults.  This will be injected when the instance is created.
  • setBackupSystemInformation(BackupSystemInformation backupSystemInformation) – void. 
    • A setter for the BackupSystemInformation.  This will be injected when the instance is created.
  • setBackupProject(BackupProject backupProject) – void.
    • A setter for the BackupProject.  This will be injected when the instance is created.

Interface com.atlassian.jira.imports.project.handler.PostImportHandler

Defines a handler class that will be called after the all data is imported on the destination system.

Method Summary

  • run() – void.
    • Provides the implementation an opportunity to perform some action after importing completes.
  • setArchiveImportMapper(ArchiveImportMapper projectImportMapper) – void. 
    • A setter for the ArchiveImportMapper.  This will be injected when the instance is created.
  • setProjectImportResults(ProjectImportResults projectImportResults) – void. 
    • A setter for the ProjectImportResults.  This will be injected when the instance is created.
  • setBackupSystemInformation(BackupSystemInformation backupSystemInformation) – void. 
    • A setter for the BackupSystemInformation.  This will be injected when the instance is created.
  • setBackupProject(BackupProject backupProject) – void. 
    • A setter for the BackupProject.  This will be injected when the instance is created.

Interface com.atlassian.jira.imports.project.handler.DeleteHandler

Defines a handler class that will be called on the source system after the import is complete and before the issues and project are deleted on the source. Plugins should delete any issue and project related data that has been archived.

Method Summary

  • cleanUp(List<Project> projects, List<String> issueKeys) – void.

    • Provides the implementation an opportunity to clean archived data from the source system.

  • cleanUp(List<Project> projects) – void.

    • Provides the implementation an opportunity to clean archived data from the source system.

Plugin custom field types

Interface com.atlassian.jira.imports.project.customfield.ProjectImportableCustomField

Any plugins that provide custom field types that need to be migrated need to implement the ProjectImportableCustomField interface. This interface requires the class to provide a ProjectCustomFieldImporter that will look after any data translation required during the archiving process.

Method Summary

  • getProjectImporter() – ProjectCustomFieldImporter. 

    • Returns the object that will perform the actual project import functions for the custom field type.

Interface com.atlassian.jira.imports.project.customfield.ProjectCustomFieldImporter

Plugins also need to supply or use a current JIRA implementation of this interface to actually map the imported data as required.

Method Summary

  • canMapImportValue(ProjectImportMapper projectImportMapper, ExternalCustomFieldValue customFieldValue, FieldConfig fieldConfig, I18nHelper i18n) – MessageSet.

    • The custom field needs to determine if the provided custom field value, in the context of the config and project import mapper, is a valid value that can be imported.

  • getMappedImportValue(ProjectImportMapper projectImportMapper, ExternalCustomFieldValue customFieldValue, FieldConfig fieldConfig) – ProjectCustomFieldImporter.MappedCustomFieldValue.

    • The custom field needs to determine what the "mapped" value will be for the provided custom field value and return this new string representation of the value.

Plugin XML

Example

<?xml version="1.0" encoding="UTF-8"?>
<atlassian-plugin >
    <!-- Need to process the templates before their dependent fields -->
    <archive-entity-handler key="my-entityhandler-fields" sequence="30">
        <description>Persists templates </description>
        <class>com.atlassian.jira.plugins.issueTemplates.archiving.TemplatePersister</class>
        <entity type="AO">com.atlassian.jira.plugins.issueTemplates.model.Template</entity>
    </archive-entity-handler>
    <archive-entity-handler key="my-entityhandler-templates" sequence="31">
        <description>Persists templates fields </description>
        <class>com.atlassian.jira.plugins.issueTemplates.archiving.FieldPersister</class>
        <entity type="AO">com.atlassian.jira.plugins.issueTemplates.model.TemplateField</entity>
    </archive-entity-handler>
    <archive-entity-mapper key="my-template-mapper">
        <description>Maps old to new template Ids.</description>
        <class>com.atlassian.jira.plugins.issueTemplates.archiving.TemplateMapper</class>
    </archive-entity-mapper>
    <archive-removal-handler key="my-removal-handler">
        <description>Removes templates from the source</description>
        <class>com.atlassian.jira.plugins.issueTemplates.archiving.Cleaner</class>
    </archive-removal-handler>
</atlassian-plugin>

Contents of the Archive Entity Handler Definition

Element

Description

<archive-entity-handler>

This block defines a handler for processing database entities.

<archive-entity-handler>

Attribute: key.

The key of this handler definition.  Must be unique within the plugin.

<archive-entity-handler>

Attribute: sequence.

For AO entities, this defines the order in which the entity handled by this handler will be processed, relative to other entities handled by this plugin.

<description>

A description of this handler

<class>

The implementing class.  This must implement com.atlassian.jira.imports.project.handler.ImportEntityHandler

<entity>

The entity to handle. For JIRA entities this is the name from entities.xml, for AO entities it is the name defined in the plugin xml.

Contents of the Archive Pre-Import Handler Definition

Element

Description

<archive-entity-handler>

This block defines a handler that will be called before the data is imported

<archive-entity-handler>

Attribute: key.

The key of this handler definition.  Must be unique within the plugin.

<description>

A description of this handler

<class>

The implementing class.  This must implement com.atlassian.jira.imports.project.handler.PreProcessHandler

Contents of the Archive Post-Import Handler Definition

Element

Description

<archive-entity-handler>

This block defines a handler that will be called after the data is imported

<archive-entity-handler>

Attribute: key.

The key of this handler definition.  Must be unique within the plugin.

<description>

A description of this handler

<class>

The implementing class.  This must implement com.atlassian.jira.imports.project.handler.PostProcessHandler

Contents of the Archive Entity Mapper Definition

Element

Description

<archive-entity-handler>

This block defines a mapper to be used to map old ids to new ids.

<archive-entity-handler>

Attribute: key.

The key of this handler definition.  Must be unique within the plugin.

<description>

A description of this handler

<class>

The implementing class.  This must extend com.atlassian.jira.imports.project.mapper.AbstractMapper

Contents of the Archive Removal Handler Definition

Element

Description

<archive-entity-handler>

This block defines a handler to clean up data on the source server after an archive.

<archive-entity-handler>

Attribute: key.

The key of this handler definition.  Must be unique within the plugin.

<description>

A description of this handler

<class>

The implementing class.  This must implement com.atlassian.jira.imports.project.handler.DeleteHandler

Notes for plugin developers

  • Active Object and Property set data is imported after JIRA Configuration data and JIRA Issues (the "jiraissue" entity), but before any custom field data.
  • Property set data is pre-munged into a joined format, joining <property-entry> and <property-????> data, i.e. an <OSPropertyString> entry will also have the entityName and propertyKey attributes.
  • run() – void.

    • Provides the implementation an opportunity to perform some action before importing commences.

  • setArchiveImportMapper(ArchiveImportMapper projectImportMapper) – void.

    • A setter for the ArchiveImportMapper.  This will be injected when the instance is created.

  • setProjectImportResults(ProjectImportResults projectImportResults) – void.

    • A setter for the ProjectImportResults.  This will be injected when the instance is created.

  • setBackupSystemInformation(BackupSystemInformation backupSystemInformation) – void.

    • A setter for the BackupSystemInformation.  This will be injected when the instance is created.

  • setBackupProject(BackupProject backupProject) – void.

    • A setter for the BackupProject.  This will be injected when the instance is created.

Was this page helpful?

Have a question about this article?

See questions about this article

Powered by Confluence and Scroll Viewport