Test Collection and Reporting
Bamboo 3.1 and later
TestCollationServiceAPIs its possible to write your own Tasks that parse files or logs that contain test results and have the results display in Bamboo.
Parsing Test Results
Example test report format
To make this tutorial simple we have developed a file format for a fictional testing framework called My Unit. In MyUnit, each test result is written to a new line and each field is separated by the pipe character (
The test result line in the MyUnit format has the following fields:
- Suite - the name of the suite of tests that the test belongs to
- Test Name - the name of the test
- Duration - time the test has taken to run in seconds
- State - SUCCESS or FAIL
Heres what the typical output looks like from MyUnit:
In this example we assume that MyUnit test files use the
.result file extension.
Parsing Test results
TestCollationService supports parsing custom test formats defined by either a custom
TestReportCollector (file based) or
TestReportProvider (non-file based) implementation. Since MyUnit writes its files out to the file system this example implements the
collateTestResults method on
TestCollationService accepts Ant-style file glob pattern matching. This makes it possible to select single files or files with certain names that match an expression. For example the pattern
/foo/bar.result will only select the file
/foo/bar.result relative to the working directory of the server. For selection of multiple
.result files in the same directory one could use the pattern
When a file has been found that matches the configured expression the
collect method is called on the
TestReportCollector and the results are then added to the build result.
Be aware that if multiple matches are found, the same instance of the
TestReportCollector will be used on multiple threads to reduce the parsing time, so it is important that your implementation is stateless.
Files that do not match the extensions in the
getSupportedFileExtensions method of the
TestReportCollector are automatically excluded from the pattern matching. This is so the user does not specify a pattern such as "/*" (which selects all files recursively in the current directory) and attempt to parse them as test results.
Using a TestReportCollector in a Task
All that needs to be done now is to create a new
TaskType that injects the
TestCollationService in the constructor and have it use the
MyUnitTestReportCollector to collate the test results for the user configured file pattern.