Adding a field to CQL

Overview

This tutorial shows you how to add a field to the Confluence Query Language (CQL). For more detailed reference documentation, see CQL field module.

Before you begin

To complete this tutorial, you must already understand:

1. The basics of Java development: classes, interfaces, methods, how to use the compiler, and so on. 
2. How to create an Atlassian plugin project using the Atlassian plugin SDK.

Plugin source

We encourage you to work through this tutorial. If you want to skip ahead or check your work when you are done, you can find the plugin source code on Atlassian Bitbucket.

To clone the repository, run the following command:

1

git clone https://bitbucket.org/atlassian_tutorial/confluence-query-lang-add-field-tutorial.git

Alternatively, you can download the source as a ZIP archive.

Step 1. Create the plugin project and prune the skeleton

In this step, you'll generate skeleton code for your plugin. Because you won't need skeleton files, you also delete them in this step. 

1. Open a Terminal and navigate to directory where you would like to keep your plugin code.

2. To create a plugin skeleton, run the following command:

   1

   atlas-create-confluence-plugin

   The atlas- commands are part of the Atlassian Plugin SDK, and automate some of the work of plugin development for you.  

3. To identify your plugin, enter the following information.

   group-id	com.example.plugins.tutorial.confluence
   artifact-id	tutorial-cql-add-field
   version	1.0-SNAPSHOT
   package	com.example.plugins.tutorial.confluence

4. Confirm your entries when prompted.
   The SDK creates your project skeleton and puts it in a tutorial-cql-add-field directory. 

5. Navigate to the tutorial-cql-add-field directory created in the previous step.

6. Delete the test directories.

   1
   2

   rm -rf ./src/test/java
   rm -rf ./src/test/resources/

7. Delete the unneeded Java class files.

   1

   rm -rf ./src/main/java/com/example/plugins/tutorial/confluence/*

8. Import the project into your favorite IDE.

Step 2. Add the field declaration to your atlassian-plugin.xml

1. Add a dependency on the SPI to the pom.xml:

   1
   2
   3
   4
   5
   6

   <dependency>
       <groupId>com.atlassian.querylang</groupId>
       <artifactId>confluence-query-lang-spi</artifactId>
       <version>2.0.0</version>
       <scope>provided</scope>
   </dependency>

2. To add a field, we need to add the following to atlassian-plugin.xml file:

   1
   2
   3
   4
   5

   <cql-query-field fieldName="status"
                    key="status-field" name="Content Status Field"
                    class="com.example.plugins.tutorial.confluence.impl.StatusFieldHandler">
       <ui-support value-type="string" default-operator="=" i18n-key="cql.field.status" data-uri="/rest/status-field/status-values"/>
   </cql-query-field>

   The cql-query-field declaration is what defines a new CQL field.

   Attribute	Purpose
   fieldName	Name of the field that will be used in CQL statements.
   key	Unique ID of this CQL field declaration in the plugin.
   name	A human readable name for the field.
   class	The implementation of the CQL field. This should extend the  1 com.atlassian.querylang.fields.BaseFieldHandler class

   The ui-support declaration allows the new field to appear in the UI.

   Attribute	Purpose
   value-type	Type of values that can be stored against this field (see supported UI value types on CQL field module page).
   default-operator	CQL operator used in conjunction with the contents of the UI field when it's converted to a statement.
   i18n-key	The i18n key in the i18n properties file for the text that will be displayed as a label for the field in the UI.
   data-uri	REST endpoint that provides field values.

   For this example, we implement a Status field using the generic string UI value type backed by a data-uri, which provides the user with a set of values to select from in the front end.

   data-uri points to a simple REST resource and responds with a set of status values.

   1
   2
   3

   <rest key="status-values" path="/status-field" version="none">
       <description> Provides status values to the front end to power CQL status field UI support</description>
   </rest>

3. Create a new rest package under src/main/java/com/example/plugins/tutorial/confluence, and then create a new class that will provide available status values.

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   27
   28
   29
   30
   31
   32
   33
   34
   35
   36
   37
   38
   39
   40
   41
   42
   43
   44
   45
   46
   47
   48
   49
   50
   51
   52
   53
   54
   55
   56
   57
   58
   59
   60
   61
   62
   63

   ``` java
   package com.example.plugins.tutorial.confluence.rest;
   
   import com.google.common.collect.Lists;
   import com.google.common.collect.Maps;
   
   import javax.ws.rs.Consumes;
   import javax.ws.rs.GET;
   import javax.ws.rs.Path;
   import javax.ws.rs.Produces;
   import javax.ws.rs.core.MediaType;
   import java.util.HashMap;
   import java.util.List;
   import java.util.Map;
   
   @Path("/")
   @Consumes(value = MediaType.APPLICATION_JSON)
   @Produces(value = MediaType.APPLICATION_JSON)
   public class StatusValueResource {
   
       @Path("/status-values")
       @GET
       public Map<String, List<Map<String, String>>> getStatusValues() {
           Map<String, List<Map<String, String>>> response = new HashMap<>();
           response.put("suggestedResults", Lists.newArrayList());
           response.put("searchResults", getSearchResultsList());
           return response;
       }
   
       private static List<Map<String, String>> getSearchResultsList() {
           List<Map<String, String>> searchResults = Lists.newArrayList();
           for (String statusValue : Lists.newArrayList("pending", "review", "complete")) {
               Map<String, String> result = Maps.newHashMap();
               result.put("id", statusValue);
               result.put("text", statusValue.substring(0, 1).toUpperCase() + statusValue.substring(1));
               searchResults.add(result);
           }
           return searchResults;
       }
   }
   ```
   
   The response from the `/rest/status-field/status-values` endpoint is:
   
   ``` javascript
   {
     "suggestedResults":[],
     "searchResults":[
       {
         "id":"pending",
         "text":"Pending"
       },
       {
         "id":"review",
         "text":"Review"
       },
       {
         "id":"complete",
         "text":"Complete"
       }
     ]
   }
   ```

4. Add the i18n field name to i18n properties file:

1

cql.field.status=Status

Step 3. Add an EventListener and an Extractor

This step is not specifically related to the process of adding fields to CQL, but the field requires it to function correctly.

1. Create a NewPageListener that will listen for page create events and set the default status of pending as a content property on the page. Setting a content property to a piece of content can be done in a variety of ways, such as via a REST resource, but, for simplicity, we use the EventListener.

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   27
   28
   29
   30
   31
   32
   33
   34
   35
   36
   37
   38
   39
   40
   41
   42

   ``` java
   package com.example.plugins.tutorial.confluence.impl;
   
   import com.atlassian.confluence.core.ContentEntityObject;
   import com.atlassian.confluence.core.ContentPropertyManager;
   import com.atlassian.confluence.event.events.content.page.PageCreateEvent;
   import com.atlassian.event.api.EventListener;
   import com.atlassian.event.api.EventPublisher;
   import org.springframework.beans.factory.DisposableBean;
   import com.atlassian.plugin.spring.scanner.annotation.imports.ComponentImport;
   import com.atlassian.plugin.spring.scanner.annotation.imports.ConfluenceImport;
   
   import javax.inject.Inject;
   import javax.inject.Named;
   
   @Named
   public class NewPageListener implements DisposableBean {
   
       public static final String DEFAULT_PAGE_STATUS = "pending";
   
       private ContentPropertyManager contentPropertyManager;
   
       @Inject
       public NewPageListener(@ComponentImport ContentPropertyManager contentPropertyManager,
                              @ConfluenceImport EventPublisher eventPublisher) {
           this.contentPropertyManager = contentPropertyManager;
           this.eventPublisher = eventPublisher;
           eventPublisher.register(this);
       }
   
       @EventListener
       public void pageCreated(PageCreateEvent pageCreateEvent) {
           ContentEntityObject page = pageCreateEvent.getContent();
           contentPropertyManager.setStringProperty(page, StatusPropertyExtractor.INDEXABLE_PROPERTY, DEFAULT_PAGE_STATUS);
       }
   
       @Override
       public void destroy() throws Exception {
           eventPublisher.unregister(this);
       }
   }
   ```

2. Create a new StatusPropertyExtractor class that implements Extractor interface. It will run after the EventListener (that is, after a page is created). The class will extract the status information from a content property, and then add the information to the index allowing it to be searched. 

Here is an Extractor implementation used in the example repository and adapted from the  Extractor module documentation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

package com.example.plugins.tutorial.confluence.impl;

import com.atlassian.bonnie.Searchable;
import com.atlassian.bonnie.search.Extractor;
import com.atlassian.confluence.core.ContentEntityObject;
import com.atlassian.confluence.core.ContentPropertyManager;
import com.atlassian.plugin.spring.scanner.annotation.component.Scanned;
import com.atlassian.plugin.spring.scanner.annotation.imports.ComponentImport;
import com.opensymphony.util.TextUtils;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.StringField;

import javax.inject.Inject;

@Scanned
public class StatusPropertyExtractor implements Extractor {
    public static final String INDEXABLE_PROPERTY = "status";

    private ContentPropertyManager contentPropertyManager;

    @Inject
    public StatusPropertyExtractor(@ComponentImport ContentPropertyManager contentPropertyManager) {
        this.contentPropertyManager = contentPropertyManager;
    }

    public void addFields(Document document, StringBuffer defaultSearchableText, Searchable searchable) {
        if (searchable instanceof ContentEntityObject) {
            ContentEntityObject contentEntityObject = (ContentEntityObject) searchable;
            String value = contentPropertyManager.getStringProperty(contentEntityObject, INDEXABLE_PROPERTY);

            if (TextUtils.stringSet(value)) {
                defaultSearchableText.append(value);
                document.add(new StringField(INDEXABLE_PROPERTY, value, Field.Store.YES));
            }
        }
    }

}

To make this Extractor work, add few more lines in atlassian-plugin.xml file.

1
2
3
4

<extractor name="Status Property Extractor" key="statusPropertyExtractor"
           class="com.example.plugins.tutorial.confluence.impl.StatusPropertyExtractor" priority="1000">
    <description>Extracts certain keys from a page's metadata and adds them to the search index.</description>
</extractor>

Step 4. Add a FieldHandler

1. To complete the addition of new CQL field, provide the implementation of the class that is referenced in cql-query-field declaration. 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

package com.example.plugins.tutorial.confluence.impl;

import com.atlassian.confluence.plugins.cql.spi.v2searchhelpers.V2SearchQueryWrapper;
import com.atlassian.confluence.search.v2.SearchQuery;
import com.atlassian.confluence.search.v2.query.TermQuery;
import com.atlassian.querylang.fields.BaseFieldHandler;
import com.atlassian.querylang.fields.EqualityFieldHandler;
import com.atlassian.querylang.fields.expressiondata.EqualityExpressionData;
import com.atlassian.querylang.fields.expressiondata.SetExpressionData;
import com.google.common.base.Function;

import static com.atlassian.confluence.plugins.cql.spi.v2searchhelpers.V2FieldHandlerHelper.joinSingleValueQueries;
import static com.atlassian.confluence.plugins.cql.spi.v2searchhelpers.V2FieldHandlerHelper.wrapV2Search;
import static com.atlassian.querylang.fields.expressiondata.EqualityExpressionData.Operator.EQUALS;
import static com.atlassian.querylang.fields.expressiondata.EqualityExpressionData.Operator.NOT_EQUALS;
import static com.atlassian.querylang.fields.expressiondata.SetExpressionData.Operator.IN;
import static com.atlassian.querylang.fields.expressiondata.SetExpressionData.Operator.NOT_IN;
import static com.google.common.collect.Sets.newHashSet;

public class StatusFieldHandler extends BaseFieldHandler implements EqualityFieldHandler<String, V2SearchQueryWrapper> {

    private static final String FIELD_NAME = "status";

    public StatusFieldHandler() {
        super(FIELD_NAME);
    }

    @Override
    public V2SearchQueryWrapper build(SetExpressionData setExpressionData, Iterable<String> values) {
        validateSupportedOp(setExpressionData.getOperator(), newHashSet(IN, NOT_IN));
        SearchQuery query = joinSingleValueQueries(values, (Function<String, TermQuery>) this::createEqualityQuery);
        return wrapV2Search(query, setExpressionData);
    }

    @Override
    public V2SearchQueryWrapper build(EqualityExpressionData equalityExpressionData, String value) {
        validateSupportedOp(equalityExpressionData.getOperator(), newHashSet(EQUALS, NOT_EQUALS));
        return wrapV2Search(createEqualityQuery(value), equalityExpressionData);
    }

    private TermQuery createEqualityQuery(String value) {
        return new TermQuery(StatusPropertyExtractor.INDEXABLE_PROPERTY, value);
    }
}

We extend the BaseFieldHandler that provides much of the default implementation required and is the class recommended as a base for all CQL FieldHandler implementations. The class must also implement at least one typed FieldHandler interface, EqualityFieldHandler in our case, which will allow for support of =, !=, IN, and NOT IN CQL operators for querying the status of a page.

For a full list of available field types, see CQL field module.

The EqualityFieldHandler interface requires that we implement two build methods. One will support SetExpressions, "status IN ('pending', 'review')", and the other will support EqualityExpressions, "status = 'pending'".

These methods usually look quite similar, and we provide a number of helper functions via the CQL SPI for performing common operations found in these build method implementations.

• validateSupportedOp – is used to check whether the operator used in the CQL statement that is being executed is supported by our FieldHandler. The first argument is ExpressionData.getOperator method that contains operator passed by user and the second argument is a set of the operations we have decided to support. validateSupportedOp performs the validation for us by throwing an exception with a meaningful error message to the user when they attempt to execute a CQL statement with an unsupported operator. We recommend using this helper to let existing FieldHandlers keep working (that is, provide the appropriate response in the event) in case we add new operators to CQL. 
• wrapV2Search – Used to combine a V2SeachQuery with the ExpressionData object provided to the build method to return a V2SearchQueryWrapper object, which is expected as the return type of the build method. This V2SearchQueryWrapper object contains all the necessary information to execute the CQL and handles negation based on the ExpressionData operator to retrieve the required data from the index.
• joinSingleValueQueries – A helper method used in SetExpressionData build method implementations. It allows a single V2SearchQuery to be transformed into a single QueryObject that can then be passed to wrapV2Search.

Step 5. Build, install, and run the plugin

1. Make sure you have saved all your code changes to this point.
2. Open a Terminal and navigate to the plugin root folder (where the pom.xml file is stored).

3. Run the following command:

   1

   atlas-run

   This command builds your plugin code, starts a Confluence instance, and installs your plugin. This may take a while. When the process is complete, you'll see many status lines on your screen concluding with something like this:

   1
   2
   3

   [INFO] Confluence started successfully in 71s at http://localhost:1990/confluence
   [INFO] Type CTRL-D to shutdown gracefully
   [INFO] Type CTRL-C to exit

4. In your browser, go to local Confluence instance.

5. Log in with admin / admin.
6. Navigate to the Confluence search page (http://localhost:1990/confluence/dosearchsite.action), which is backed by CQL.
7. Click Add a filter. Your new field will appear in the list of CQL fields available to filter the search. 

Screenshot: Confluence Search page showing the newly added Status CQL field.