Last updated Aug 13, 2024

How to Integrate with DROID Read Through

Please reach out to #help-tcs, as we’re not quite ready for self-service.

DROID - External Ingestion vs Read Through Comparison

Consider read-through integration if:

  • transformation (via Transformer Service) is not required,
  • the data-set contains infrequently accessed records,
  • the ‘warm/cached data-set’ will fit into system memory (ballpark <10GB)
  • either:
    • the data is immutable, or
    • changes infrequently + your service can handle invalidation load
  • tier-1 service level is acceptable, and
  • your backing-store service:
    • tier-1,
    • has an on-call roster
    • is present in multiple regions
    • can provide a HTTP endpoint which fits our contract

If you require <20ms latency for cache-misses (your backing store is too slow), or your ‘active/frequently-accessed record set’ exceeds 10GB, consider other integration paths.

For a comprehensive operational checklist for Read Through, please refer to the DROID Operational Readiness Checklist.

We’ll need the following:

  • ASAP Audience - e.g. your-service-name
  • record type - a descriptive label for the data
  • allowed issuers - list of services which require access
    • if invalidation events are required, this cannot be ‘allow-all’
  • baseUrl - The URL prefix/endpoint to be queried for this record-type
    • This can either be a single DNS alias using latency-based routing, or configured explicitly for each TCS environment.
    • Beware of cross-region latency between TCS and your service - to see the regions TCS exists in, check
1
2
# from https://stash.atlassian.com/projects/ES/repos/tlt-services/browse/tenant-context-service/src/main/resources/application.yml#67
third-party-services.metadata:
  config-registry:
    asapAudience: config-registry
    allowedIssuers:
      - "micros-group/tcs-ref-client"
    recordType: CONFIGREGISTRY
    baseUrl: https://config-registry.staging.atl-paas.net/api/v1/namespaces

Granting access to consumers

Access Control List

DROID uses ASAP keys for authentication. For more information on how to set up ASAP keys, see here.

Consumers can raise a pull-request adding their ASAP issuer to the access control list, for each of your record types. We ask that consumers discuss their requirements with the read-through providers either prior or on the PR.

Cache Size and Expiry

Cache limits are defined for each record-type. When a cache is full, records will be evicted. While these values can be tuned later based on cache-hit ratios, it’s helpful to understand:

  • how many records/keys would you expect the cache to contain?
  • what is the average / maximum size of a record?
  • is computing a 404 / empty response expensive? is caching a 404-response desirable?
  • is a stale cache risky/harmful? does a short TTL (time-to-live/expiry) help, or do you need to configure invalidation events with a longer (e.g. several hours) TTL?

TCS has separate caches for 200 (Found), 404 (Not Found) and any exceptions (other codes). Both size and expiry are independently configurable.

Connection Limits

A connection limit (between TCS and each backing-store) is configurable by record-type. This limit exists to prevent 'noisy-neighbour' problems harming TCS, and to protect the backing-store service.

The limit is configured as x requests in-flight per-record-type, is tracked independently by each EC2 instance. Typically, each TCS region has 4-8 nodes per-region (usually 4). TCS is currently deployed in six regions. For example, with the default limit of ‘16 parallel requests’, a maximum of 16 (requests) * 4 (ec2 instances) * 6 (regions) = 360 requests could be made to the backing-store service.

It’s possible to configure.

If the limit is exceeded, TCS will reject (HTTP 429) requests for the record-type until the # of connections falls below the limit. Consumers should expect and handle this response.

API Contract

When a cache-miss for a key occurs, DROID will make a HTTP GET request to your backing service. For more information on this API contract, see here.

Cache Invalidation (Optional)

Content in the DROID read-through cache can be invalidated by sending events via StreamHub. For more information on this, see here.

Rate this page: