Atlassian is introducing a new and simplified way to move app data around, designed to simplify, enable more code re-use by app partners, and allow for faster app migrations.

App Migration Building Blocks splits the concept of migrations into two parts:

Exporters are responsible for getting data out of an app (e.g. a Forge Remote) and sending it to Atlassian.
Importers are responsible for getting data from Atlassian and loading it into an app.

Exporters and Importers form the Building Blocks which customers will be able to put together to support different types of data movement. An Exporter or Importer implementation can be re-used for multiple types of data movement, as long as they support the same export contract (i.e. data shape).

Use cases for Building Blocks

Experimental use case: Simplified Data Center to Cloud Migrations

Atlassian provides a standardised Data Center Exporter.
You implement a compatible Importer for your Forge app.
Status: Currently in Experiment for Jira. Confluence not supported yet. Allows for hybrid migrations where your Importer only covers a subset of the app data, and the classic Forge migration moves the rest.

Experimental use case: Cloud to Cloud Migrations

You implement a compatible Exporter and Importer for your Forge app.
Atlassian sets up a flow between them to migrate between cloud sites.
You choose if you send all data (for easy cross-region moves, and re-use of the Importer built for Data Center to Cloud), or just a token enabling data copying within your Forge Remote.
Status: Currently in Experiment. Currently, Atlassian staff need to initiate the transfer manually (for high-value customer migrations). There are future plans to allow App Partners to provide configuration information about their building blocks in the Forge Manifest, and to build a UI to allow site admins to initiate and approve the app data migration. There is also a proposal to allow configuration of Atlassian-provided support for importing and exporting from Forge Storage.

Proposed future use case: Backup and Restore

Data from your Exporter is saved to a file that the customer can download for a backup.
Later, the customer can re-upload the file and send it to the Importer to restore data.
Status: Proposed but not yet implemented.

App Data Flows and App Data Movements

Unlike classic migrations, Building Blocks migrations have two separate concepts:

An App Data Flow, which is a potentially long-lived connection between an app data source (for example, your app in a particular Data Center deployment) and an app data destination (for example, an installation of your Forge app).
An App Data Movement, which represents an explicit action to move data at a point in time.

Because there can be multiple Movements in the same Flow, some data can be moved using a snapshot (first Movement) and diff (subsequent Movements in the same Flow). This can be extended further so that a snapshot created for one destination can be reused (and supplemented with diffs) when exporting to a second destination, allowing for data to be exported only once during a migration programme. This means that even for the first Movement of a Flow, the importer may receive a snapshot and deltas.

Event Files, Binary Files and File Lists

The basic unit of data flow in Building Blocks is an Event File. Event Files use JSONL, with a one-line header, followed by data records. Different types of data are split into different Event Files. Event Files typically contain 'upsert' (insert or update) records, and 'delete' records, allowing them to support both snapshots and subsequent diffs.

Event Files can reference 'Binary Files'. These are typically used to represent assets such as images or on-disk files. There is still a record in the Event File representing the upsert (or delete) of the file, but the upsert would refer to a Binary File by ID for the actual contents of the file.

The platform creates a JSONL file containing a list of all Event Files (filtered by File Group Label, which is introduced later). This file also contains metadata about the Event Files, so an Importer can decide whether or not it needs to read it. Note that most types of Event File accumulate across App Data Movements within the Flow, so the File List can contain Event Files that may have already been imported in a previous Movement. It is up to Importers to decide if they will always restart the import on every movement, or if they'll keep track of what's already been imported (by Event File ID) and skip those Event Files.

File Group Labels

Every Event File is labelled with a 'File Group Label'. When all Event Files for the same File Group Label (for the current Movement) have been exported, the Exporter notifies the platform, and the platform notifies the Importers. This means that Importers can start processing one File Group while the Exporters are still processing another File Group. When Importers are notified of a File Group being complete, they receive a File List filtered by the File Group Label with the event.