Login | Register
My pages Projects Community openCollabNet

Popper extends JUnit to allow you to specify theories, general statements about your code's behavior that may be true over infinite sets of input values. For a detailed description of why you might want to do this, see our paper. If you just want to try it out and get started, here's the place.

First, if you haven't already, download the latest Popper. Popper should work with any version of JUnit between 4.0 and 4.3, and Java 5 and above.

Put Popper and JUnit on your classpath.

Now, let's start with a simple test class for a type that performs arithmetic on numbers of dollars:

public class DollarTest {
   @Test public void multiplyByAnInteger() {
     assertThat(new Dollar(5).times(2).getAmount(), is(10));
   }
}

This demonstrates the syntax of JUnit 4 tests written with Popper's assertThat syntax, which is lovingly stolen from Hamcrest. If the @Test syntax is new to you, you may want to quickly get up to speed on JUnit 4. If the convenient assertThat assertion syntax is new, you may want to get up to speed on Hamcrest. (You'll need to add some imports. Eclipse can take care of most of them for you, but here's the whole list)

Go ahead and get the test above to pass. I especially suggest that you cheat by always returning 10 from getAmount.

Now, we'll see how Popper extends JUnit to allow the specification and verification of general properties. First, give Dollar test a new superclass:

import net.saff.theories.api.TheoryContainer;

public class DollarTest extends TheoryContainer {
...

Next, add a theory to the test class:

  @Theory public void multiplyIsInverseOfDivide(int amount, int m) {
    assumeThat(m, not(0));
    assertThat(new Dollar(amount).times(m).divideBy(m).getAmount(),
        is(amount));
  }

You'll have to add a divideBy method to Dollar in order to get the class to compile. This theory expresses a general truth about currency multiplication and division: one is the inverse of the other. By adding two parameters to the method, and adding the assumption that m is not zero, you've specified that this theory should hold true for any value of amount and all non-zero values of m.

Go ahead and run the tests, and you'll notice two things:

  • Your multiplyByAnInteger test passes as before.
  • multiplyIsInverseOfDivide fails, with the message "Never found parameters that satisfied method."

The Popper runner does not automatically provide potential values for a theory. It can take a long time between being able to assert a general principle, and actually making it work with any conceivable input. Because of this, Popper expects to only run theories on a set of accepted data points, which allows you to incrementally build up a set of inputs for which you know the theory passes, until you feel confident that the theory is universally true.

By default, Popper assumes that all public fields of a test class are accepted data points. For example, add the following constants to DollarTest:

  public static int TWO = 2;
  public static int FIVE = 5;
  public static int TEN = 10;

Now the theory will probably fail, telling you which inputs caused the failure. Fix your implementations of times and divideBy, and the theory should pass.

To customize how Popper finds accepted data points, you can use parameter suppliers. One parameter supplier comes with Popper by default, which allows you to specify inline a set of acceptable integer inputs to a theory:

  @Theory public void multiplyIsInverseOfDivideWithInlineDataPoints(
    @TestedOn(ints={0, 5, 10}) int amount, 
    @TestedOn(ints={0, 1, 2})  int m
  ) {
    assumeThat(m, not(0));
    assertThat(new Dollar(amount).times(m).divideBy(m).getAmount(),
        is(amount));
  }

This theory ignores the public fields of DollarTest, and tests with every combination of 0, 5, and 10 for amount, and 0, 1, and 2 for m. Notice that even though 0 is tried as a value for m, it does not trigger an arithmetic exception or failure: values that violate assumptions are quietly ignored.

You can extend Popper by providing your own parameter supplier. For example, here is a supplier for values between two integers, and an annotation that references it:

  @Retention(RetentionPolicy.RUNTIME)
  @ParametersSuppliedBy(BetweenSupplier.class)
  public @interface Between {
    int first();
    int last();
  }

  public static class BetweenSupplier extends ParameterSupplier {
    @Override
    public List getValues(Object test, ParameterSignature sig) {
      Between annotation = (Between) sig.getSupplierAnnotation();
      
      ArrayList list = new ArrayList();
      for (int i = annotation.first(); i <= annotation.last(); i++)
        list.add(i);
      return list;
    }
  }

With this supplier defined, you can accept data points as ranges:

  @Theory public void multiplyIsInverseOfDivideWithInlineDataPoints(
    @Between(first=-100, last=100) int amount, 
    @Between(first=-100, last=100) int m
  ) {
    assumeThat(m, not(0));
    assertThat(new Dollar(amount).times(m).divideBy(m).getAmount(),
        is(amount));
  }

The paper refers to a capacity for automatically generating stubs to pass as object parameters to theories. This work is ongoing, so we've separated it from Popper, and put it in a new project. Look there for further updates.

We're very interested in feedback, by individual e-mail, on our mailing lists, or by posting issues.