If you work for a big company, you know all the big ALM tools. Fortunately, the experience fills out a résumé nicely. Unfortunately, you have to deal with all of them at the same time. I’ve seen it happen for any number of reasons: outsourcing, acquisitions, restructuring. Whatever the history, the variety of tools can cause communication silos. According to Melvin Conway, those silos doom the software you produce to be siloed. If you thought the only alternative was a lot of custom development using a hodge-podge of languages and protocols, then you may have missed a new generation of middleware designed specifically for ALM integration.
Middleware for ALM
Although I’ve seen many problems with mixed tooling in big companies, I’ve also been witness to how easy it can be to solve the headaches with a new breed of ALM middleware. For each of these products, I’ve seen production usage involving 1000s of users with 1000s of objects. What’s common to these ALM middleware products is hub-and-spoke architecture. The hub performs the work of mapping between fields. For enumerated values like workflow status, there are submappings. The hub has filters that scope the data synchronization. Between mappings and filters, it is possible to replicate most data structures. Things can get complicated when one product has a flat list structure but the target has a tree structure. The hub has rules for deciding the direction of flow and how to detect and resolve conflicts. Unlike point-to-point integrations, the hub-and-spoke architecture is suitable for multiple scenarios:
-
Migrate between different tools.
-
Migrate between instances of the same tool, usually for consolidation.
-
Federation of different tools, usually to cross organizational boundaries.
-
Federation of instances of the same tool, usually to cross geographic boundaries.
-
And combinations of the above.
The spokes are adapters that convert each ALM product’s API into a consistent internal representation. While all connect to JIRA, each product has a different set of adapters. Despite the architectural similarities, each middleware vendor has a different business model. Pricing can be a function of one or more of the following:
-
Users across the integrated ALM products.
-
Connectors by ALM product.
-
Number of instances of ALM products.
-
Who you buy from. Some ALM vendors have become resellers of ALM middleware.
All of the options provide a user interface for creating and maintaining the mappings and integration flows.
This is a young market. The 2 following vendors seem to be the front-runners. The list is not meant to be exhaustive.
Tasktop
If you are a Java developer using Eclipse, then you may already know about Eclipse Mylyn. Originally Mylyn was a way to improve the Eclipse user experience by hiding information in context of a task. In time, Mylyn became the de-facto standard for accessing tasks from ALM tools from inside Eclipse. The Tasktop company formed around commercializing Mylyn as a product, now known as Tasktop Dev.
Building on the existing ecosystem of Mylyn connectors, Tasktop created a server-to-server middleware product, Tasktop Sync. In addition to leveraging the Eclipse de facto standard task interface, Tasktop Sync is an adapter for the Open Services for Lifecycle Collaboration specification.
Most recently, Tasktop has released Tasktop Data. Tasktop Data aggregate ALM data from multiple systems for reporting. This is an excellent separation of concerns. It avoids the problem of overloading one “source of truth” system with synchronization from everything else just for the purpose of aggregate reports.
Tasktop products are not currently SaaS. If, for example, you wanted to synchronize GitHub Issues and JIRA Cloud, then you’ll have to provision and maintain your own machine to run Tasktop Sync. Even if as a VM in the cloud, that may be more cost and effort than you want for synchronizing 2 cloud products. That caveat might not matter much since many big companies still rely on behind-the-firewall development tools.
OpsHub
While OpsHub is a newer company, they beat Tasktop to market with server-to-server integration. OpsHub Integration Manager (OIM) was publicly available before Tasktop Sync. Although OpsHub makes no appeals to open source and open standards, that helps them keep laser focused on building out an impressive list of both commercial and open-source ALM systems. The from-scratch approach has also helped OIM have broader support for the variation of “things” in ALM. With Tasktop’s background in “task-focused” Mylyn, it has stayed closer to the kinds of workitems you find in JIRA. In addition to JIRA Issues, OIM has reached for things like tests, builds, and changesets.
OpsHub has on-premise, customer cloud, and SaaS hosting options. When I was last informed of OpsHub’s pricing model, it seemed to scale down to team level better than Tasktop. Those aspects may make OpsHub a better choice for a single team looking to solve their own integration problems inside a big company.
Useful not magical
While ALM middleware does help reassemble information from the tool silos, the middleware is still subject to the CAP theorem. That means you should have realistic expectations about what “synchronization” means. Namely, don’t expect every change to propagate immediately. These ALM middleware products can be configured to synchronize on very short cycles. However, the integrated ALM products are typically the bottleneck. In my experience, 5 minute synchronization intervals are reasonable. Faster is possible with fine-tuning of filters and mappings to the bare essentials. If you can get away with lightweight linking instead sending all the data, so much the better. Sometimes just a textual reference is sufficient. Longer may be required when synchronization includes elaborate descriptions, images, and/or file attachments.
While I have seen these products deliver value in many situations, there are still simple scenarios where custom coding is the best option. All of these products are built on APIs. You can use them too. But when you are comparing custom coding to these ALM middleware products, don’t forget how much these products have invested in the non-functional capabilities you’re going to need for large scale implementations.