P2 to Forge Migrations, for Connect developers

As a Connect app developer, you might be curious about the new server to Forge migration path that Atlassian is introducing. This new path retains many elements from the existing Connect migration process, which has been extensively tested and widely used by our customers. However, it also incorporates some aspects of Forge's design and functionality.

The areas that required some changes to support Forge migrations are the migration listeners and the way you interact with the migration platform. We will cover them in detail in the subsequent sections.

New migration listeners

Starting from version 1.5.2 the library atlassian-app-cloud-migration-listener will bring a new type of listener with DiscoverableForgeListener.

AppCloudForgeMigrationGateway

Compared to the existing gateway we use for Connect (AppCloudMigrationGateway), this one have some simplified method parameters due to the lack of transferId in them. This is thanks to the fact that now the gateway is transfer-aware, so you don't need to inform it.

If your app requires transferId for an operation, you can retrieve it using the getTransferId() method.

Use events instead of webhooks

Because Forge is a serverless platform, webhooks aren't as convenient as they are in Connect. Instead, Forge uses events and method invocations.

The same events as before will be produced, but you will need to subscribe to them through the manifest file.

Here's an example:

1
2
modules:
  trigger:
    - key: test-trigger
      function: migration-start-funct
      events:
        - avi:ecosystem.migration:triggered:listener
        - avi:ecosystem.migration:uploaded:app_data

And here's a complete list of events:

• avi:ecosystem.migration:triggered:listener
• avi:ecosystem.migration:uploaded:app_data
• avi:ecosystem.migration:requested:transfer_cancellation
• avi:ecosystem.migration:errored:listener
• avi:ecosystem.migration:completed:export_phase
• avi:ecosystem.migration:settled:transfer

Whenever there's an event, the subscribed functions will be invoked with two arguments: the event and the context.

Here's an example of an event:

1
2
{
  "eventType": "avi:ecosystem.migration:uploaded:app_data",
  "transferId": "86f5ea78-8608-5479-9ee7-15b0bb777f6e",
  "migrationDetails": {
    "migrationId": "731ad5f9-f258-4a18-94da-7a579ca1c8ad",
    "migrationScopeId": "7ed3ea6c-02dc-459c-ba49-9befb4e23104",
    "createdAt": 1716273864561,
    "cloudUrl": "https://your-site.atlassian.net",
    "name": "Migrating all the accounting projects"
  },
  "key": "29e3b203-1c7c-4ec1-a323-f94efd009baa",
  "label": "file-9",
  "messageId": "11c5a6f2-e844-4c81-aaf1-b7aa33e082ff",
  "selfGenerated": false,
  "context": {
    "cloudId": "0c7a3eb6-f72e-428e-bc0b-f7e261d1d85f",
    "moduleKey": "test-trigger"
  },
  "contextToken": "..."
}

And here's an example of a context:

1
2
{
  "installContext": "ari:cloud:jira::site/0c7a3eb6-f72e-428e-bc0b-f7e261d1d85f"
}

Use Forge modules instead of the migration platform REST APIs

With Forge, you use modules to interact with the app migration platform instead of REST APIs.

All you need to do is to import @forge/migrations module (starting from version 1.0.0) and have access to the following features:

1
2
interface MigrationAdapter {
    getAppDataList(transferId: string): Promise<AppDataListResponse>;
    getAppDataPayload(key: string): Promise<APIResponse>;
    getContainers(transferId: string, containerType: string): DefaultQueryBuilder;
    getMappingById(transferId: string, namespace: string, keys: Array<string>): Promise<MappingResponse>;
    getMappings(transferId: string, namespace: string): DefaultQueryBuilder;
    messageProcessed(transferId: string, messageId: string);
}

Workflow differences between Forge and Connect migrations

In terms of triggering app migrations, nothing changes. Your app migration listeners will still get a call on onStartAppMigration() once the product migrations has completed.

However, once the data export is done, we require an explicit call to completeExport() on the gateway so the progress reporting can start. If the method doesn't get called, the platform will assume all the data has been exported after 15 minutes of inactivity.

Also, there's no longer cloud feedback for the server to listen to. This is so that we have predictable exports that can later be replayed (feature not available yet).

That's all there is for server-side changes, but there's one more difference on the cloud (Forge) side. For every uploaded data (event avi:ecosystem.migration:uploaded:app_data), your forge function will need to inform us that the data got processed. That's done by invoking messageProcessed(transferId: string, messageId: string) from the @forge/migrations module. The transferId and messageId are provided in the event payload. If the message isn't flagged as processed within 15 minutes we'll assume it timed-out and that will impact the transfer progress.

Please note that there's no limit on the number of operations to export data. In case Forge compute limitations are reached, one possible way to overcome this is by spreading the workload across multiple exports. That will also help customers to observe migration progress with a greater granularity.

Forge custom fields vs Connect custom fields migration

Migrating Forge custom fields is mostly the same as Connect custom fields refer to our guide here.

The difference is that we have added support to migrate custom field types for Forge. To specify a custom field type you want to migrate use null for the name e.g.

1
2
@Override
public Map<ServerAppCustomField, String> getSupportedCustomFieldMappings() {
    HashMap<ServerAppCustomField, String> serverToForgeCFMap = new HashMap<>();
    serverToForgeCFMap.put(new ServerAppCustomField(null, "cf-server-type"), "forge-cf-type");
    return serverToForgeCFMap;
}

In this example we will migrate all custom fields of type cf-server-type and recreate it as forge-cf-type in the cloud site.

Best practices for long running migrations

Your entire migration should not fit into a single app data export because this will cause a lack of progress reporting and force your migration to fit in a 15-minute window. Instead we recommend splitting up your migration into smaller chunks.

An example may look like this:

1
2
public void exportAllEntityProperties(AppCloudForgeMigrationGateway gateway) {
    List<List<Long>> chunkedIssueIds = entityPropertyApi.fetchIssueIdsChunked(CHUNK_SIZE);
    chunkedIssueIds.forEach(issueIds -> exportEntityPropertiesForIssues(gateway, issueIds));
}

private void exportEntityPropertiesForIssues(AppCloudForgeMigrationGateway gateway, List<Long> issueIds) {
    try (OutputStream os = gateway.createAppData()) {
        List<EntityProperty> ep = entityPropertyApi.fetch(issueIds);
        os.write(gson.toJson(ep).getBytes(StandardCharsets.UTF_8));
    } catch (IOException e) { ... }
}

There is a 1:1 mapping between the app data export and the messageProcessed call.

Sometimes you may have post migration tasks that can take a long time, a similar example to above may look like this:

1
2
private void startPostMigrationTaskForPages(AppCloudForgeMigrationGateway gateway) {
    List<List<Long>> chunkedPageIds = spaceApi.fetchPageIdsChunked(CHUNK_SIZE);
    chunkedPageIds.forEach(pageIds -> sendPageIdsToForgeApp(gateway, pageIds));
}

private void sendPageIdsToForgeApp(AppCloudForgeMigrationGateway gateway, List<Long> pageIds) {
    try (OutputStream os = gateway.createAppData()) {
        os.write(pageIds.toString().getBytes(StandardCharsets.UTF_8));
    } catch (IOException e) { ... }
}

Your export step may take a while to complete so you can make use of the addLog() method to inform customers of the progress e.g. addLog('Uploaded 1000 entities').

What are the Limitations?

In your migration listener, you can still export as many entities as you want, but each App Data can't be bigger than 16 mega bytes.

Your migration code running on Forge functions will also be subjected the same limits as any other Forge function.