EPM runs a series of tasks that populate data into the database. This data is what makes up everything that can be seen in the EPM Dashboard. We can think of these tasks as building blocks that make up a workflow. A task can sometimes run in isolation and other times be coupled to a different task. Meaning, that the data output of a task can have dependencies on another task.
Keeping each task in mind, a Workflow is a grouping and ordering of tasks with dependencies kept in mind. A workflow is created in expectation that it:
Gets invoked on a scheduled basis.
Invokes each task within the workflow in a defined order.
Ensures that the end state expected once all tasks are execute is achieved.
Ensures an alert is triggered when something unexpected happens so that EPM engineers can act swiftly to avoid data corruption and inconsistency.
Submits metrics pertaining to success/failure, duration and metadata for each task.
Today, EPM contains one Workflow comprised of multiple tasks. This workflow is called the Component Workflow.
Component Workflow Tasks
The Component Workflow consists of a few different tasks as seen below.
Service Loader
Service Loader workflow is responsible for discovering all services existing within Atlassian. Today, the logic aggregates service and service metadata from the following sources:
Asset Attribution
Service Central
Service Discovery
Component Discovery
EPM supports contribution of Component Discovery. Anyone interested in having EPM discover a certain component can contribute code for this.
Component Discovery task traverses through all supported Component types in EPM, invokes the logic to query and discover instances of a Component and stores them in the database along with their owning, parent service.
Responsibility Sync
All Standards consist of one-to-many responsibilities. These responsibilities get applied to Component Instances discovered via Component Discovery Task.
The Responsibility Sync task goes through each responsibility in the system, evaluates which instances are in-scope for the responsibility and creates an explicit relationship in the system. This is called a Component Instance Responsibility.
Attestation Sync
One way to fulfill a responsibility is via Service Owner Attestation.
Some Service owners are able to validate they are fulfilling a specific responsibility by attesting to it on the UI. When a new instance of a Component is discovered and an attestation already exists for its corresponding responsibility, this task goes and propagates the attestation to newly discovered component instances.
Assessments Sync
Another means to fulfill a responsibility is via Assessments.
Asset readiness enables evaluation of assets (component instances) with respect to a set of rules in order to determine if that rule set is being met or not. These rules equate to responsibilities in EPM and thus, can be validated via assessments. This task queries all assessments from asset readiness, checks whether instances in EPM have matching identities and evaluate if they fulfill specific responsibilities.
Component Adoption Sync
The third way to fulfill a responsibility is by adopting a different Component that takes care of the responsibility.
This task traverses through each component instance, checks corresponding responsibilities and determines if a service adopts a whitelisted Component type that meets the responsibility.
User Loader
In order to support a User view in EPM, we need to maintain a mapping of User to Services they are a part of.
This workflow discovers this mapping and persists it in the database for the View layer to leverage when rendering UI views.
Delete Stale Data
Component Instances can be ephemeral. Therefore, if an existing instance in the database no longer exists, there needs to be a mechanism to delete those outdated instances from EPM data source.
This task goes and marks outdated instances as stale in the database. Stale instances do not get queried for.