Skip to end of metadata
Go to start of metadata

Available:

JQL is available in JIRA 4.0 and later.

 

The introduction of advanced searching (JQL) significantly enhances JIRA's searching functionality. One of the extension points that JQL provides to developers are JQL functions. Functions provide a way for values within a JQL query to be calculated at runtime. They are simple to write and can be surprisingly powerful. For example, consider the issueKey clause in JQL. It matches an issue with a particular issue key. This in itself is not very useful, but when combined with a function that returns all of a user's watched issues (watchedIssues), it provides a way to find all the issues that the current user is watching (issuekey in watchedIssues()).

JQL functions can only provide values to a query; most importantly, they cannot be used to process the results. For example, it is not possible to write a JQL function that will calculate the total time remaining from all issues returned from a search. A consequence of this is that functions can only be used with JQL clauses that already exist. The only way implement new JQL clauses is to implement a new searchable custom field. While this gives more control to the plugin developer, it is much more complicated.

JQL functions can take arguments. These arguments must take the form of simple string values. For example, fixVersion in releasedVersions('JIRA') contains a function call to releasedVersions to find all the released versions in the JIRA project. Making the arguments simple strings means that JQL lists and other JQL functions cannot be used as arguments. For example, it is not possible to do something like myFunction(currentUser()).

A JQL function is an implementation of the JqlFunction interface that is registered in JIRA as a jql-function plugin. The registered JqlFunction will only be instantiated once per jql-function plugin. All queries that use the function will share the single instance. Consequently, a function can be called by multiple threads at the same time and as such must be thread-safe.

Try the introductory tutorial first

Icon

If you like, you can start with our introductory tutorial to JQL function plugins and then return to this guide for a more detailed example.

In the following guide we will be stepping through the implementation of a new JQL function called roleMembers. This function returns the users that are members of a particular JIRA project role. The first argument is the name of the role whose members we are trying to find. It is compulsory. Any other arguments name the projects whose roles should be checked. When no project is specified, all projects that the searcher can see are queried. For example, a call to roleMembers(trole, tproj) will find all the users in the role trole for the project tproj. On the other hand, a call to roleMembers('testrole') returns all the users in the role testrole across all projects that the searcher can see.

The function has a number of limitations that need to be addressed before it can be put into production. These limitations will be noted as we progress through implementing the function below. The plugin is available here if you want to follow along. The function is implemented in the com.atlassian.example.jira.jqlfunc.RoleFunction class.

On this page:

JqlFunction.init Method

The JqlFunction.init method is called by JIRA to tell the JqlFunction about its associated JqlFunctionModuleDescriptor. This object represents JIRA's view of the JqlFunction and can be used to find plugin resources. The init method is only called once and is guaranteed to be called before the function is actually used by JIRA. In our example we simply store the JqlFunctionModuleDescriptor in a variable so that we can use it later to access our internationalised messages.

The observant may have noted that we store the JqlFunctionModuleDescriptor in a volatile variable. We do this to ensure that our JQL function is thread-safe. While the init method will only be called once, we need to make the variable volatile to guarantee visibility to the many threads that will need to read it.

JqlFunction.getFunctionName Method

The JqlFunction.getFunctionName method returns the name that can be used in JQL to invoke the function. In this case we will simply return the constant string roleMembers.

JIRA must get the same name each time it calls getFunctionName. Importantly, this means that the function name cannot be translated. The function name does not have to be in English, however it must be in the same language for every user in JIRA irrespective of their language settings.

The function name should also be unique across all instances of JIRA where it is expected to run. Having two JQL functions of the same name in JIRA will produce confusing results. JIRA will only register the first function for use in JIRA and will simply ignore any others of the same name. The plugin that JIRA determines to be first is somewhat arbitrary and may result in different JQL functions of the same name being registered on each start up. The moral of the story: try very hard to make your function names unique.

JqlFunction.getMinimumNumberOfExpectedArguments Method

The JqlFunction.getMinimumNumberOfExpectedArguments basically returns the smallest number of arguments that the function may accept. In this case, our function can take 1 or more arguments so we will be returning 1.

The value returned from this method must be consistent across method invocations.

JqlFunction.isList Method

The JqlFunction.isList should return true if the function returns a list, or false if it returns a scalar. The main difference is that a list type can be used with the IN and NOT IN operators while a scalar type can be used with =, !=, <, >, <=, >=, IS and IS NOT. For our function it makes sense to say assignee IN roleMembers(Administrators) so we will be returning true. Here is the code:

The value returned from this method must be constant. It cannot change based on the parameters or the function's result. The function must either always return a list or must always return a scalar.

The easiest way to work out whether the function should return a list or not is to simply consider where it is going to be used. If the function makes sense with the IN or NOT IN operators, then function returns a list and needs to return true for this method. This will normally be the case when the function logically returns more than one value (e.g. releasedVersons(), membersOf()) . On the other hand if the function should be used with =, !=, <, >, <=, >=, IS and IS NOT then it should return false. This will normally be the case when a function logically returns one value (e.g. now(), currentUser()).

JqlFunction.getDataType Method

The JqlFunction.getDataType method is called to determine the type of data the function returns. The value tells JIRA which JQL clauses the function can be expected to work with. For example, returning JiraDataTypes.VERSION indicates that the function should only be used with clauses that work with JIRA versions. You may return JiraDataTypes.ALL if you wish the function to be available across all JQL conditions. Here we are returning JIRA users (via their names) so we will return JiraDataTypes.USER.

Again the value returned must be consistent across all invocations of this method.

JqlFunction.validate Method

The JqlFunction.validate method is called by JIRA when the function needs to be validated. The job of this method is to check the arguments to the function to ensure that it is being used correctly. Here is the interface:


The most important argument is the FunctionOperand. It contains all of the functions arguments as given by the FunctionOperand.getArgs method. All JQL function arguments come in as Strings and it is up to the function to interpret them correctly. The searcher is the user that the function should be validated for, that is, the user that any security checks should be performed for. The TerminalClause is JIRA's representation of the JQL condition we are validating for. For functions it represents a JQL condition of the form name operator function(arg1, arg2, ..., argn). The name, operator and function can be returned by calling TerminalClause.getName, TerminalClause.getOperator and TerminalClause.getOperand respectively. The value returned from getOperand() will be the FunctionOperand that is passed to this method. This method is only called when the passed arguments are relevant to the JQL function, that is, the validation does not need to check if the FunctionOperand has the correct function name.

The validate method must always return a MessageSet as its result. A null return is not allowed. A MessageSet is an object that contains all of the errors and warnings that occur during validation. All messages in the MessageSet need to be translated with respect to the passed searching user. An empty MessageSet indicates that no errors have occurred. A MessageSet with errors indicates that the JQL is invalid and should not be allowed to run. The returned messages will be displayed to the user so that any problems may be rectified. A MessageSet with warnings indicates that the JQL may have problems but that it can still be run. Any warning messages will be displayed above the results.

Functions need to respect JIRA security. A function should not return references to JIRA objects (e.g. projects, issues) that they are not allowed to see. Further, a function should not leak information about JIRA objects that the searcher does not have permission to use. For example, a function should not differentiate between a project not existing and a project that the user has no permission to see. A function that behaves badly will not cause JQL to expose issues that the searcher is not allowed to see (since JQL does permission checks when it runs the filter), though it does open up an attack vector for information disclosure.

Only one instance of each JQL function is created. This means that your function can (and probably will) be called by two threads at the same time. To accommodate this, your function must be thread-safe or unexpected behaviour can result.

In the validation of the roleMembers the implementation needs to:

  1. Check that we have at least one argument.
  2. Check that the passed project role is valid.
  3. Check that any passed projects are valid.

The implementation is listed here:

The first thing the validation checks is that the function is supplied at least one argument (lines 7-11). If not, then we add an error message to the MessageSet and return immediately as the role is compulsory. Note that we use the module descriptor that we stored away in the JIRA:JqlFunction.init method to get access to the plugin's I18nHelper to help with the translations.

The next call ensures that the passed role name is actually valid (lines 14-15). It does this by calling the private validateRole method. The validateRole method is not currently production ready for two main reasons. Firstly, the lookup is not very forgiving as the user must enter in a role name exactly, including case, as it appears in JIRA. In a production version of this function, the lookup should be made case-insensitive. It may also be useful to try looking up the role by ID if the name lookup fails. This is is useful for queries that are generated programmatically.

The second problem is security. The implementation currently allows the searcher to enter in any valid role. There may be a case for restricting access to roles. It really depends on the usage, however, it is important to realise that by implementing this function we are giving users a way to find all the members of a particular role even when they do not have administrator access.

The next step in the validate is to check the correctness and applicability of any project arguments (lines 18-24). We do this by calling the internal validateProject method. This method checks that the project exists and that the searcher has permission to browse the project, that is, view issues in the project. The project lookup first tries to find the project by name, and then by project ID if that fails. This implementation would also need some tweaking before it could be used in production as the user must enter in the project name exactly as it appears in JIRA. We need to make the project name lookup case-insensitive. It would also be nice to try the project key if the name lookup fails. It may even be nicer to try the project key lookup first.

The implementation of this method must be thread-safe. The dependencies we use are thread-safe and are stored in final or volatile variables to ensure visibility. All method state is kept local to ensure that it is not visible to other threads.

Note that when the function is valid we actually return an empty MessageSet. A null MessageSet is never returned.

JqlFunction.getValues Method

The JqlFunction.getValues method is called by JIRA when it needs to execute the function so that it can perform a query.

The FunctionOperand and the TerminalClause are as described previously in the JIRA:validate method. The new argument here is the QueryCreationContext. This object contains the variables that may be necessary when executing the function. The QueryCreationContext.getUser method returns the user that is running the search and as such should be used to perform any security checks that may be necessary. The QueryCreationContext.isSecurityOverriden method indicates whether or not this function should actually perform security checks. When it returns true, the function should assume that the searcher has permission to see everything in JIRA. When it returns false, the function should perform regular JIRA security checks and make sure it only returns things that the searcher has permission to see. This parameter is used by JIRA in certain administrative operations where finding all issues is important.

The JQL function returns a list of QuerylLiteral. A QueryLiteral represents either a String, Long or EMPTY value. These three represent JQL's distinguishable types. The type of the QueryLiteral is determined at construction time and cannot be changed. Construct it with no value and it will represent EMPTY, construct it with a String and it represents a String, or construct it with a Long and it represents a Long.

Most JQL clauses will treat each type differently. For example, let's consider the affectsVersion clause. When passed a Long QueryLiteral, it will look for all issues with an Affects Version of the specified ID. This is useful when a function would need to identify a particular version exactly. Where possible we suggest that functions try to return IDs so that query results are unambiguous. When passed a String QueryLiteral, the affectsVersion clause will run one of two searches depending upon the value in the QueryLiteral:

  1. If version(s) with the name given in the QueryLiteral exist, then return all issues with the specified Affects Version(s). This may return empty results.
  2. If the value given in the QueryLiteral can be parsed into a version ID and that version exists, then return all issues that have an Affects Version of the parsed ID. This may return empty results.
  3. Return empty results.

JQL functions may return String QueryLiterals, however, the result of the query will be dependent on the lookup procedure of the JQL clause it is used with. Finally, the EMPTY QueryLiteral will make the affectsVersion condition look for all issues that have no Affects Version set.

The function always returns a list of QueryLiteral objects. It is even valid for a scalar function (i.e. a function whose JIRA:isList method returns false) to return multiple QueryLiteral objects. In such a situation it is the JQL clause the function is being used with that decides what this means. All of the core JIRA JQL clauses simply treat such a situation as an OR between each of the returned values. The function must return an empty list of QueryLiteral objects (not an empty QueryLiteral) to indicate an error. Importantly, the function can never return a null list.

The JqlFunction.getValues method may be called with arguments that would not pass the JqlFunction.validate method. Under this situation it is important that the function not throw an error, as JQL is designed to try and run invalid queries where possible. The function should run if possible, or otherwise, return an empty list. The only thing the function can assume is that the FunctionOperand argument is meant to be executed by the function.

Only one instance of each JQL function is created. This means that your function can (and probably will) be called by two threads at the same time. To accommodate this, your function must be thread-safe or unexpected behaviour can result.

The JqlFunction.getValues method must execute quickly. Keep in mind, that your function will be executed each time the query is run. If your function takes 10 seconds to run, then the entire JQL query will take at least 10 seconds. Functions also need to perform well under concurrent load. Keep synchronization (locking) down to a minimum. The simplest way to do this is to keep all the functions' calculation state on the stack and out of member variables.

Let's implement this method for the roleMembers function. In our implementation we will need to:

  1. Check that at least one argument is passed.
  2. Find the role passed in as an argument.
  3. Lookup the projects we are going to query. This may come from the arguments, or may be all the projects that the user has permission to see.
  4. Find all the users in the role for the projects we looked up.
  5. Turn the users into QueryLiteral objects and return them.

This is implemented here:

The first check is to ensure that the user has entered in the role name (lines 5-7). If they have not, then the function simply returns an empty list since it cannot continue.

The next check is that the role the user has specified actually exists (lines 10-15). If it does not then the function simply returns an empty list. It is important that the JIRA:JqlFunction.validate and JqlFunction.getValues methods use the same logic when looking up the role to ensure consistency between the two methods. Because of this, the implementation suffers from the same issues with role lookup that were outlined in the section on validation. Namely, the lookup is not user friendly and may actually be a security hole.

Next we find the users in the role for the specified projects (lines 17-27). There are two situations to consider here. When the user has specified some projects, we need to limit the search to those projects. We do this in the getUsersForProjects method. The project lookup in this method simply ignores project arguments that do not match any current projects. It also ignores any projects that the searcher does not have permission to see, that is, unless the QueryCreationContext tells the function to ignore such query permission checks. On the other hand, if no projects are specified then we look for all projects that the passed user can see, or alternatively, all the projects in JIRA if the QueryCreationContext is configured to ignore security. This is implemented in the getUsersForAllProjects method. To keep things consistent with validation we look up projects using the project name and project ID. In production it would be better to provide a more user friendly lookup as outlined in the section on implementing the JIRA:JqlFunction.validate method.

Next the implementation converts the users into their equivalent QueryLiteral representations so they can be returned (line 30-34). Since users are uniquely identified by the user name (bad JIRA) we will return one String QueryLiteral per user. Note that if the function finds no users then an empty list is returned and not null.

The implementation of this method must be thread-safe. The dependencies we use are thread-safe and are stored in final or volatile variables to ensure visibility. All method state is kept local to ensure that it is not visible to other threads.

Function sanitisation (Optional)

To make our function truly production ready, we have to also implement ClauseSanitisingJqlFunction. A saved JQL search (filter) can be shared with multiple users. While this functionality is very useful, it also allows information to be leaked. For example, let's say you have a filter that contains assignee in roleMembers(Administrators, Proj) and you share the filter with Janice who cannot see Proj. The search will not return any results, however, Janice will know that a project called Proj exists even though she does not have permission to see it. A JQL function that can expose sensitive information (that is, a function that does security checks) should also implement the optional ClauseSanitisingJqlFunction interface. The interface has one method:

This method takes a searcher and a FunctionOperand and returns an equivalent operand that hides any privileged information the passed searcher should not see. The usage of the FunctionOperand is outlined in the discussion of the JIRA:validate method. The returned function is what the passed searcher will see when trying to load the filter.

It is important that the FunctionOperand that is returned from sanitisation is equivalent to the passed operand. If this is not the case, then it is possible for two people running the exact same filter to be actually running two different searches.

The roleMembers function should check that the passed user can see all the project arguments. If all the passed projects are visible, the we can simply return the FunctionOperand as passed. On the other hand, if some of the projects are not visible then the sanitiser should return a new FunctionOperand that has any names replaced with project IDs. This is not a perfect solution as we still leak the fact that a project exists that the searcher cannot see, however, we no longer leak the project name. The implementation does this:

The first check makes sure that the function has project arguments (lines 6-9). When the function has no project arguments it can return the input operand unchanged as there is nothing to sanitise. The function then loops across all the arguments and replaces projects that the user cannot see with their equivalent ID (lines 14-26). Finally, the method returns either a new equivalent FunctionOperand if any of the argument values have changed, or the original FunctionOperand if no arguments have changed (lines 28-35).

Another way to implement this sanitisation would be to simply remove the projects that the user is not able to see and return a new function. This implementation is broken as it could actually change what the query does if the last project was deleted from the function call. In this case, the function call would go from "finding all the users in a role in the specified projects" to "finding all the users in a role in all projects".

As already noted, the roleMembers function should probably perform some permissions checks on the project role argument. If such a check was implemented, then the sanitiser would probably have to change to reflect any logic here.

The plugin module descriptor

Once we have implemented the function we need to create a plugin descriptor that points JIRA at our function. The roleMembers function has the following XML descriptor:

The module type for a JQL plugin is jql-function. There is no other JQL function specific configuration needed here. The rest of the descriptor is standard Atlassian plugin configuration and is documented elsewhere.

Now the plugin can be built and packaged into a plugin JAR. Once this plugin is installed in JIRA it will become available for use in JQL queries (e.g. assignee in membersOf(Administrators)). The function will even show up in the JQL autocomplete.

Important Points

  • Your function will be executed when the query is run. Make sure your function runs quickly even under concurrent load.
  • Only one instance of a function is created. This instance is shared by JQL queries that use the function. This means that a JQL function may be called concurrently by different threads. As a result, your JQL function must be thread-safe.
  • Ensure that you take notice of the QueryCreationContext.isSecurityOverriden when running the function.
  • JQL functions need to respect JIRA security. If a function does not respect JIRA security, then it becomes an attack vector for information disclosure.
RELATED TOPICS

Plugin Tutorial - Adding a JQL Function to JIRA

  • No labels