Mundorévès

Cada día me veo en un mundo al revés

Confluence plugin integration testing

I have been working on the Approvals Workflow Plugin (AWP) since 2007 and just recently released version 2.

Because of the nature of the application, it appeals to enterprises, and they're not that keen on upgrading that often, therefore I have to support several versions of Confluence, and that can be challenging.

For the new version of the plugin, I started using web integration tests to run tests which I used to do manually.

The supported versions of Confluence are from 2.6 to 2.10, so although I have managed to have only one binary, I still have to test on all the supported versions on every release.

In addition to the 80+ unit tests I already have, there are the manual tests that require creating pages, approving documents, adding labels, changing permissions, etc.

But now that I created the Web Tests and included them into my Bamboo plans, every time I commit changes, the tests run in just a few minutes on 2.8.2, 2.9.1 and 2.10.1, which I used to do only when I was about to release.

Although I am not able to run all the tests against all the versions I need, it reduces the amount of manual testing required.

There is not too much documentation on the subject, so I started by looking at the chart-plugin test and still took me a while to get it going, so for the community, here is what I have learnt.

UPDATE: Customware has put together this neat documentation

POM Changes

The dependency is already included in the confluence-plugin-base so as long as you created your plugin using the plugin archetype you are all set.

To start, you only have to make a couple of changes in your xml.pom (here's mines, which you could use a reference).

First you need to define some properties:

<properties>
    <confluence.version>2.10.1</confluence.version>
    <atlassian.product.test-lib.version>1.4.2</atlassian.product.test-lib.version>
    <atlassian.product.data.version>2.10</atlassian.product.data.version>
    <confluence.plugin.bundled>true</confluence.plugin.bundled>
    <confluence.jars.to.remove.regex.list>${basedir}/test/resources/jar-files-to-remove-from-bundle.txt</confluence.jars.to.remove.regex.list>
</properties>
Property Description Notes
confluence.version The version of Confluence on which you are going to be compiling and running the tests. I was only able to run against 2.8.x, 2.9.x and 2.10.x. There are missing libraries on earlier versions.
In theory you also have to define atlassian.product.version but I've found that confluence.version is enough
atlassian.product.test-lib.version test lib version I am currently using version 1.4.2 of the confluence-functestrpc-plugin. The latest stable version (2.1) uses a newer version of JWebUnit and changed all the packages. I tried upgrading but had several problems running it.
atlassian.product.data.version data set version You can omit defining this one if you are testing against 2.8 or 2.9, but you must set it to 2.10 if running on 2.10
confluence.jars.to.remove.regex.list the name says it all The file has to be included as part of the test resorces, see below. Mines contains the rexep of my plugin: ^approvalsworkflow-.*.jar$ This important when rerunning the tests.

Then you need to define your test resources, where you need to store that jar-files-to-remove-from-bundle.txt file and any other files you need for your testing (more on this below).

<build>
    ...
    <testSourceDirectory>test</testSourceDirectory>
    <testResources>
        <testResource>
            <directory>test/resources</directory>
            <includes>
                <include>**/*.*</include>
            </includes>
        </testResource>
    </testResources>                          
    ...
</build>

You can look at my pom.xml.

The abstract web test case

You can start by looking at ChartMacroTestCase.java for a set of simple tests. Something to be noted is that the Web tests' package names must start with 'it.'.

My tests were more complex because:

  • Some of tests deal with content permissions
  • Some require the Metadata Plugin to be installed
  • The plugin has some macros which are disabled by default.
  • The tests require workflows to be loaded. (workflows are a connection of macros, which contain approvals and triggers)

I had to create an abstract web test class, to provide some common services. Some of them can be useful for other applications and also can give you some ideas.

public class AbstractAwpWebTestCase extends AbstractConfluencePluginWebTestCase {
    protected void setUp() throws Exception {
        super.setUp();
        SpaceHelper spaceHelper;
        spaceHelper = getSpaceHelper();
        spaceHelper.setKey(SPACE_KEY);
        spaceHelper.setName("AWP Test Space");
        spaceHelper.setDescription("AWP Test Space");
        assertTrue(spaceHelper.create());

        configureSpacePermissions();

        enableMacro( "workflow");
        enableMacro("include-segment");
        enableMacro("send-email");
        configurePlugin();
        configureUsers();
        loadRequiredPlugins();
    }
    ...
}

Setting space permissions

The first part of setUp() only creates test space. However, because some of my testing deals with permissions, I had to configure space permissions.

What is done through the UI like this:


It is done in the web test like this:

private void configureSpacePermissions() throws Exception {
    gotoPage("/spaces/editspacepermissions.action?key=" + SPACE_KEY);
    setFormElement("groupsToAdd",GROUP_CONFLUENCEUSERS);
    submit("groupsToAddButton");

    gotoPage("/spaces/editspacepermissions.action?key=" + SPACE_KEY);
    checkCheckbox("confluence_checkbox_viewspace_group_confluence-users");
    checkCheckbox("confluence_checkbox_editspace_group_confluence-users");
    checkCheckbox("confluence_checkbox_exportpage_group_confluence-users");

    checkCheckbox("confluence_checkbox_removepage_group_confluence-users");
    checkCheckbox("confluence_checkbox_editblog_group_confluence-users");
    checkCheckbox("confluence_checkbox_comment_group_confluence-users");
    checkCheckbox("confluence_checkbox_createattachment_group_confluence-users");
    submit();
}

You can use firebug or look at the HTML source to identify the form element names.

Enabling plugin's modules

Then I need to enable some of the plugin's modules that are disabled by default upon installation.

So again, what is done through the UI like this:

It is done in the web test like this:

protected void enableMacro(String name) throws Exception {
    gotoPage("/admin/plugins.action?mode=enable&moduleKey=com.comalatech.workflow%3A" + name);
    assertEquals(200,getDialog().getResponse().getResponseCode());
}

Adding users

The Confluence web test library provide some helpers which use the SOAP API for some operations.

Because of my test cases, I need to create some users:

(view code)

Loading other plugins

My tests require the Metadata Plugin so I copied the jar to the test resources directory and this is how it gets loaded:

private void loadRequiredPlugins() throws Exception {
    ConfluenceWebTester confluenceWebTester = getConfluenceWebTester();
    File plugin = new File(SystemUtils.JAVA_IO_TMPDIR,"metadata-2.1.0.jar");
    InputStream in = null;
    OutputStream out = null;
    try {
        in = new BufferedInputStream(getClass().getClassLoader().getResourceAsStream("metadata-2.1.0.jar"));
        out = new BufferedOutputStream(new FileOutputStream(plugin));
        IOUtils.copy(in, out);
    } finally {
        IOUtils.closeQuietly(out);
        IOUtils.closeQuietly(in);
    }
    confluenceWebTester.setConfluencePluginJar(plugin);
    confluenceWebTester.setConfluencePluginName("Metadata Plugin");
    installPlugin();
}

Loading test pages

I need some predefined workflows for my tests. The workflows are plain wiki markup. The workflows are stored also in the test resources directory, and loading them is just a matter of creating a page using the helper classes:

(view code)

There are other methods I created, which include workflow-specific operations (i.e. approving pages) and assertion (i.e. asserting approvals availability and status) .

The test cases

Developing the first tests was very time consuming, as it takes a while to get confluence running, but it was worth it, as the new ones were really easy.

Here's an example of a test that simulates what this demo does:

(view workflow)
(view code)

Running the tests and adding to Bamboo

Running the tests is just a matter of:

mvn clean integration-test

I also added the tests to the Trunk Plan which compiles and tests on 2.8.2.

Running the tests take a few minutes because some of them depend on queued tasks on Confluence.

I also created two other plans, one for 2.9.2 and another for 2.10.1.

(Note that for 2.10 I had to define atlassian.product.data.version).

The tests on 2.9.2 and 2.10.1 are fired when the Trunk build is completed:

Conclusion

I found difficult getting the integration testing going. Besides having to dig into code because of lack of documentation, running the tests can take a while.

However, as everything invested in repeatable testing, it was worth the effort. The benefit is not only for checking changes but also for verifying new versions of confluence. i.e. running the tests on 3.0-m3 was just a matter of:

mvn  -Dconfluence.version=3.0-m3 -Dmaven.test.unit.skip=true clean integration-test;

References

Labels

 
  1. Jan 30, 2009

    Anonymous says:

    You deserve a medal for this post. Would it be too much to ask for you to add th...

    You deserve a medal for this post. Would it be too much to ask for you to add this to confluence.atlassian.com somewhere? I think people would find this info very helpful.

  2. Feb 19, 2009

    Anonymous says:

    Thanks!! You've saved me a LOAD of work... Yes it's a shame that the doc is so p...

    Thanks!! You've saved me a LOAD of work... Yes it's a shame that the doc is so poor