Powerful new search and property storage APIs in Confluence 5.7

February 11th 2015 Steve Lancashire in Confluence

With the release of Confluence 5.7, developers get two new and exciting tools to help them build Confluence add-ons. The REST API has been extended with advanced search capabilities that we're calling CQL (Confluence Query Language) providing great new ways to find the content you're interested in. In addition, content properties now support indexing and querying using CQL - especially handy for Connect add-ons that need to store and query their data in Confluence. I'll walk you through these new features with a simple use case.

So what does CQL allow me to do?

CQL is an approachable and natural language for querying content in Confluence. Announced at Atlascamp 2014, it's now available in Confluence 5.7. Let's start with a simple example to see some of the fundamental concepts. Let's say I wanted to find all blog posts on developer.atlassian.com created by me that mention CQL, visually we could represent this query as:

"CQL Venn diagram"

CQL gives us a language to represent this query. A simple CQL expression is made up of a field, an operator and a value, taking one of the circles in the diagram above we could find all blogposts, with:

type = blogpost

If you've used JQL, JIRA's query language for querying issues, you'll immediately recognise CQL's similar syntax, but breaking this down:

  • type is the field
  • = is the (equals) operator
  • blogpost is the value

And by passing this as the cql query parameter to the REST API we can retrieve all blogposts from Confluence, for example:

developer.atlassian.com/rest/api/content/search?cql=type=blogpost

Simple expressions can be combined with boolean keywords, (AND, OR, NOT). Continuing our example above, to further narrow our search from the blue circle of all blogposts, to the intersection of the three we can combine two more expressions:

creator.fullname ~ "Steve Lancashire"

and

text ~ CQL

These combine to form our full query to fetch all blog posts created by me that contain the text CQL:

type = blogpost and creator.fullname ~ "Steve Lancashire" and text ~ CQL

If we then want to find the most recent blogs we can add an ORDER BY statement to the end of the query to get the blog posts listed in order of creation date, for instance:

type = blogpost and creator.fullname ~ "Steve Lancashire" and text ~ CQL order by created DESC

More details about the available fields operators and functions can be found in the advanced searching documentation.

But how do I know if an instance of Confluence supports CQL?

We know add-on developers are always trying to support as many versions of Confluence as possible from the one code base. To make it easier to achieve this, and take advantage of CQL when it's available, we've added a capability, confluence-content-search-api, to the capability service. To determine if CQL search is available on a given instance of Confluence you can query the capabilities API, it is published under:

/rest/capabilities

Example response

    links: {
        self: "https://localhost:8080/wiki/rest/capabilities"
    },
    application: "Confluence",
    buildDate: "2014-12-16T13:00:00Z",
    capabilities: {
        ...
        confluence-content-search-api: "https://localhost:8080/wiki/rest/api/content/search",
    }
}

or from within a add-on use the CapabilityService

CapabilityService capabilityService;
...
capabilityService.getHostApplication().hasCapability("confluence-content-search-api");

The CQL search language forms part of the a public REST API, and as a result it conforms to the Atlassian API policy with guarantees about its stability across versions of Confluence.

What about content properties?

Content properties allow an add-on to store data, as key / value pairs, against a piece of content in Confluence, be it a page, blog, comment or attachment. So say we had an add-on that implemented a formal approval for a page in Confluence, allowing a reviewer to click an "Approve" button to sign off on a document. Prior to content properties there were other ways a P2 add-on could implement this, like creating an Active Objects table and storing the data in that, and then keeping it in sync when that page was moved or deleted.

As Connect add-ons run outside the Confluence server, they do not have access to Confluence's storage Java APIs. Prior to content properties, a Connect add-on would need to provide their own datastore, calling back to their server when rendering the add-on. Content properties provide an easy way to store data against content in Confluence using the REST API, facilitating performant "static" Connect add-ons and removing the need for a Connect add-on to have their own backend server. For instance:

PUT http://myserver:8080/Confluence/rest/api/content/{id}/property/approval-plugin-data 

{
    "approver":"mjones",
    "approval-date":"2015-01-25"
}

or a P2 add-on using the ContentPropertyService in java:

ContentPropertyService contentPropertyService;
ContentService contentService;
...
Content content = contentService.find().withId(ContentId.valueOf("1234")).fetchOneOrNull();
JsonContentProperty property = JsonContentProperty.builder()
    .content(content)
    .key("approval-plugin-data")
    .value(new JsonString("{\"approver\":\"mjones\", \"approval-date\":\"2015-01-24\"}")).build();
contentPropertyService.create(property);

You can learn more about indexing content properties and the range of data types supported by reading the documenation.

Putting them together

We've seen how CQL can be used for querying for content in Confluence, and how content properties can be used to store and retrieve data against pieces of content in Confluence. Wouldn't it be great if, having used content properties for storage, we could refer to them in CQL as filters? Fetching content that has a particular value for a content property. In Confluence 5.7 with the combination of CQL and content property indexing, add-on developers can query the properties they've added to content. To continue our example, we might want to show all blog posts that have been approved in the last week. By adding an index schema module to the content property, the property will be indexed and automatically exposed in CQL. We create a content property index schema in the Atlassian plugin descriptor:

<content-property-index-schema key="myindex-schema-module-key">
    <key property-key="approval-plugin-data">
        <extract path="approver" type="string" />
        <extract path="approval-date" type="date" />
    </key>
</content-property-index-schema>

This extracts the approver value and the approval date into Confluence's Lucene index. Using the REST API and CQL we can retrieve all blogposts that have been approved in the last week with the query:

type = blogpost and content.property[approval-plugin-data].approval-date > now(-7d)

We'd love to hear from you!

So now you've seen a little of what CQL and content properties can do for your add-on, and how combining them can boost the power you get out of the Confluence Platform when building your add-on. Let us know in the comments how you're using these new APIs and any enhancements you'd like to see to them. Stay tuned for more information on how to further extend CQL with concepts from your add-on.