Add a field to CQL

This tutorial shows you how to add a field to the Confluence Query Language (CQL) that is accessible via REST API or the Confluence advanced search UI. For more detailed documentation about CQL see Advanced searching using CQL and CQL field module.

In this tutorial we will create a plugin that adds property "status" for a page. The property can take one of 3 possible values: PENDING, REVIEW, and COMPLETE, which are available for use in CQL.

The tutorial consists of 4 main plugin modules:

1. A listener which, when a page is created, will set the status to Pending.
2. An extractor which will extract the property value and put it into the search index (content index from Confluence 7.9) to make it searchable.
3. A CQL field handler that translates CQL expression containing the field into a v2 search query.
4. A REST resource that provides data for autocompletion of the field in the advanced search UI.

After completion you should be able to execute a CQL query containing the newly created field via the REST API and filter the contents using the field in the Confluence advanced search UI.

Before you begin

To complete this tutorial, you'll need to be familiar with:

• The atlassian-sdk
• Basic unix command line utilities: rm, ls, curl
• The basics of Java development - classes, interfaces, methods, how to use the compiler, and so on
• The basics of Atlassian plugin development - Confluence plugin modules, Atlassian Spring Scanner

Source code

You can find the source code for this tutorial on Atlassian Bitbucket.

To clone the repository, run the following command:

1
2
git clone https://bitbucket.org/atlassian_tutorial/confluence-query-lang-add-field-tutorial.git

Alternatively, you can download the source as a ZIP archive.

Key elements

Field mapping

Mapping is a schema that defines the properties of a field in the index. These properties may contain the data type of each field and how fields are going to be analyzed and indexed. You can define a mapping of an index by implementing FieldMappingsProvider.

1
2
public class StatusFields implements FieldMappingsProvider {

    public static final StringFieldMapping STATUS = StringFieldMapping.builder("status").store(true).build();

    @Override
    public Collection<FieldMapping> getFieldMappings() {
        return List.of(STATUS);
    }
}

Mappings must be registered in the system by adding a configuration into the atlassian-plugin.xml config file.

1
2
<field-mappings-provider key="statusFieldMappings"
     index="CONTENT"
     class="com.atlassian.confluence.plugins.cql.tutorial.impl.StatusFields" />

The PageCreatedListener event listener

The PageCreatedListener is activated when a Confluence page is created and the listener uses ContentPropertyManager#setStringProperty to create a custom property status with the value 'PENDING'.

1
2
@Component
public class PageCreatedListener {
    ...
    @EventListener
    public void pageCreated(PageCreateEvent pageCreateEvent) {
        ContentEntityObject page = pageCreateEvent.getContent();
        contentPropertyManager.setStringProperty(page, StatusFields.SEARCH_FIELD_NAME.getName(), PageStatus.PENDING.name());
    }
    ...
}

We annotate the listener with @Component so Confluence will create a singleton at runtime. It registers itself with Confluence after creation. For more detailed documentation, see Event Listener Module.

1
2
@Component
public class PageCreatedListener {
    ...
    @PostConstruct
    public void register() {
        eventPublisher.register(this);
    }
    ...
}

Status property extractor

The StatusPropertyExtractor accesses ContentPropertyManager to retrieve the property if the given content is a page. It also creates a collection of fields (in this case only a single field) to be added into a search index document corresponding to the page.

1
2
public class StatusPropertyExtractor implements Extractor2 {
    ...
    @Autowired
    public StatusPropertyExtractor(@ComponentImport ContentPropertyManager contentPropertyManager) {
        this.contentPropertyManager = requireNonNull(contentPropertyManager);
    }
 
    public Collection<FieldDescriptor> extractFields(Object searchable) {
        if (searchable instanceof Page) {
            Page page = (Page) searchable;
            String value = contentPropertyManager.getStringProperty(page, CONTENT_PROPERTY_NAME);
 
            if (value == null || value.isEmpty()) {
                value = PageStatus.PENDING.name();
            }
 
            return Collections.singletonList(StatusFields.STATUS.createField(value));
        }
        return Collections.emptyList();
    }
}

The extractor must be registered in the system. This is done by adding a configuration into the atlassian-plugin.xml config file. For more detailed documentation, see Extractor2 Module.

1
2
<extractor2 name="Status Property Extractor" key="statusPropertyExtractor"
    class="com.atlassian.confluence.plugins.cql.tutorial.impl.StatusPropertyExtractor" priority="1000">
    <description>Extracts status property from page and adds it to the content search index</description>
</extractor2>

CQL field handler

We extend the BaseFieldHandler which provides much of the default implementation required and is the class recommended as a base for all CQL FieldHandler implementations. The class must also implement at least one typed FieldHandler interface, EqualityFieldHandler in our case, which will allow support for =, !=, IN, and NOT IN CQL operators for querying the status of a page. For a full list of available field types, see CQL field module.

The EqualityFieldHandler interface requires that we implement two build methods. One will support SetExpressions, “status IN (‘PENDING’, ‘REVIEW’)”, and the other will support EqualityExpressions, “status = ‘PENDING’”.

These methods usually look quite similar, and we provide a number of helper functions via the CQL API for performing common operations found in these build method implementations.

1. validateSupportedOp – is used to check whether the operator used in the CQL statement that is being executed is supported by our FieldHandler. The first argument is ExpressionData.getOperator method that contains the operator passed by the user and the second argument is a set of the operations we have decided to support. validateSupportedOp performs the validation for us by throwing an exception with a meaningful error message to the user when they attempt to execute a CQL statement with an unsupported operator.
2. wrapV2Search – is used to combine a V2SeachQuery with the ExpressionData object provided to the build method to return a V2SearchQueryWrapper object, which is expected as the return type of the build method. This V2SearchQueryWrapper object contains all the necessary information to execute the CQL to retrieve the required data from the index.
3. joinSingleValueQueries – is used to create a composite V2SeachQuery based on the given values.

1
2
public class StatusFieldHandler extends BaseFieldHandler implements EqualityFieldHandler<String, V2SearchQueryWrapper> {
    private static final String CQL_FIELD_NAME = "status";
 
    public StatusFieldHandler() {
        super(CQL_FIELD_NAME);
    }
 
    @Override
    public V2SearchQueryWrapper build(SetExpressionData setExpressionData, Iterable<String> values) {
        validateSupportedOp(setExpressionData.getOperator(), ImmutableSet.of(IN, NOT_IN));
        SearchQuery query = joinSingleValueQueries(values, this::createEqualityQuery);
        return wrapV2Search(query, setExpressionData);
    }
 
    @Override
    public V2SearchQueryWrapper build(EqualityExpressionData equalityExpressionData, String value) {
        validateSupportedOp(equalityExpressionData.getOperator(), ImmutableSet.of(EQUALS, NOT_EQUALS));
        return wrapV2Search(createEqualityQuery(value), equalityExpressionData);
    }
 
    private TermQuery createEqualityQuery(String value) {
        return new TermQuery(StatusPropertyExtractorService.SEARCH_FIELD_NAME, value);
    }
}

To add a field, we need to add the following to the atlassian-plugin.xml file. The cql-query-field declaration is what defines a new CQL field.

1
2
<cql-query-field fieldName="status"
    key="status-field" name="Content Status Field"
    class="com.atlassian.confluence.plugins.cql.tutorial.impl.StatusFieldHandler">
    <ui-support value-type="string" default-operator="=" i18n-key="cql.field.status" data-uri="/rest/status-field/status-values"/>
</cql-query-field>

REST resource to support UI

The ui-support declaration allows the new field to appear in the UI. For this example, we implement a Status field using the generic string UI value type backed by a data-uri, which provides the user with a set of values to select from in the front end.

data-uri points to a simple REST resource and responds with a set of status values. For more detailed documentation, see Rest Module.

1
2
<rest key="status-values" path="/status-field" version="none">
  <description>Provides status values to the front end to power CQL status field UI support</description>
</rest>

The StatusValueResource is a REST resource implementation that simply returns a list of different statuses.

1
2
@Path("/")
@Consumes(value = MediaType.APPLICATION_JSON)
@Produces(value = MediaType.APPLICATION_JSON)
public class StatusValueResource {
    @Path("/status-values")
    @GET
    public Map<String, List<Map<String, String>>> getStatusValues() {
        Map<String, List<Map<String, String>>> response = new HashMap<>();
        response.put("suggestedResults", Collections.emptyList());
        response.put("searchResults", getSearchResults());
        return response;
    }
 
    private static List<Map<String, String>> getSearchResults() {
        return Stream.of(PageStatus.values())
                .map(x -> ImmutableMap.of("id", x.name(), "text", x.humanName()))
                .collect(toList());
    }
}

The response from the /rest/status-field/status-values endpoint is:

1
2
{"suggestedResults":[],"searchResults":[{"id":"PENDING","text":"Pending"},{"id":"REVIEW","text":"Review"},{"id":"COMPLETE","text":"Complete"}]}

See how it works

Start a local Confluence instance

1. Make sure you have saved all your code changes to this point.

2. Open a Terminal and navigate to the plugin root folder (where the pom.xml file is stored).

3. Run the following command:

   1
   2
   atlas-run
   

   This command builds your plugin code, starts a Confluence instance, and installs your plugin. This may take a while. When the process is complete, you’ll see many status lines on your screen concluding with something like this:

   1
   2
   

[INFO] Confluence started successfully in 71s at http://localhost:1990/confluence [INFO] Type CTRL-D to shutdown gracefully [INFO] Type CTRL-C to exit

1
2

4. In your browser, go to local Confluence instance at http://localhost:1990/confluence login as admin/admin.
5. Create and/or modify a couple of pages. 

### Check Rest API

Run the following two commands and compare the result. 

1. Select pages with the status 'Pending'.

curl -u admin:admin -G "http://localhost:1990/confluence/rest/api/content/search"
--data-urlencode "cql=type=page and status=PENDING"

1
2

2. Select pages with the status 'Complete'.

curl -u admin:admin -G "http://localhost:1990/confluence/rest/api/content/search"
--data-urlencode "cql=type=page and status=COMPLETE"

1
2

### Check Advanced Search UI

1.  In your browser, go to your local Confluence instance at http://localhost:1990/confluence and log in as admin / admin.

2. Navigate to the Confluence search page (http://localhost:1990/confluence/dosearchsite.action), which is backed by CQL.

3. Click Add a filter. Your new field will appear in the list of CQL fields available to filter the search.

*Screenshot: Confluence advanced search page showing the newly added Status CQL field.*
<img src="/server/confluence/images/status-field-pending.png" width="700" height="420" />

## Next steps

Learn more about extending Confluence's search capabilities with these tutorials:

-   [Using the v2 Search API](/server/confluence/v2-search-api-tutorial)  
-   [Create a search decorator](/server/confluence/search-decorator-module-tutorial) 
-   [Create a search extractor](/server/confluence/extractor2-module-tutorial)
-   [Create an attachment text extractor](/server/confluence/attachment-text-extractor-module-tutorial)