Limiting integration tests to a time constraint

4 Replies

I have recently had the need to restrict a test to run within a certain time limit. If the test does not complete within a specific time constraint then the test should fail. The approach that I took was to run the function under test in its own thread and use a CountDownLatch to specify the duration in which the process should complete within. The code for this is as follows.

public class TimeConstrainedIntegrationTest {

    private static final int RUN_ONCE = 1;

    @Test
    public void shouldVerifyGoogleIsAlive() throws Exception {
        CountDownLatch doneSignal = new CountDownLatch(RUN_ONCE);

        new Thread(new VerifyGoogleIsAliveWorker(doneSignal)).start();

        boolean finishedInTime = doneSignal.await(10, TimeUnit.SECONDS);
        assertTrue(finishedInTime);
    }

    private class VerifyGoogleIsAliveWorker implements Runnable {
        private final CountDownLatch doneSignal;

        VerifyGoogleIsAliveWorker(CountDownLatch doneSignal) {
            this.doneSignal = doneSignal;
        }

        public void run() {
            try {
                try {
                    verifyGoogleIsAlive();
                } catch (IOException e) {
                    throw new InterruptedException();
                }

                doneSignal.countDown();
            } catch (InterruptedException ex) {
            }
        }

        private void verifyGoogleIsAlive() throws IOException {
            WebClient webClient = new WebClient();
            Page page = webClient.getPage("http://google.com");
            assertTrue(StringUtils.contains(page.getTitleText(), "Google"));
        }
    }
}

In the code above I assert that Google is alive by checking the title text for http://google.com is in fact Google. Essentially I am asserting expected behaviour, which is what you would typically do. I have encapsulated this within the verifyGoogleIsAlive() method above.

To make sure that the test runs within a specific time, the verifyGoogleIsAlive() method is called within the run method of a class that implements Runnable. In this case I have called the class VerifyGoogleIsAliveWorker. The test case shouldVerifyGoogleIsAlive() should be quite self-explanatory. First a CountDownLatch is instantiated. Then a new VerifyGoogleIsAliveWorker thread is started. The CountDownLatch then awaits for the VerifyGoogleIsAliveWorker thread to run its course and complete.

Of interest though is that the awaits() method on the CountDownLatch allows you to specify the length of time in which the CountDownLatch should wait before ending the thread. If the process doesn’t complete within the specified time, then the awaits() method will return false, and therefore fail the assertion that the test will run within a specific time. If on the other hand the process completes within the allocated time frame then the test will pass.

The above concurrency pattern should be useful in making sure that your tests run within a specific time frame.

4 thoughts on “Limiting integration tests to a time constraint”

  2. Nick

    Thanks Kevin. In the above example you could get away with using the timeout parameter in the annotation, but I needed more of a finer scope for which to time constrain a part of my test as oppose to the whole test case. I probably should have provided more context to what I was trying to do. :)

    Reply ↓
  3. Conrad Benham

    Good stuff! I’m wondering if JUnitPerf would be sufficient for your needs (http://clarkware.com/software/JUnitPerf.html). This library decorates your tests and allows you to define periods in which the test will fail. Generally, this approach will mitigate the boilerplate code required to ensure the test executes within the required period.

    Also, be careful when raising assertions within a thread, that you don’t lose the failure. In this case you’re sitting pretty as the countdown doesn’t occur until after the assertion.

    Consider the following code:

    public class LostAssertionSpikeTest {
    @Test
    public void testFailingAssertionMadeInThreadIsLost() throws Exception {
    Runnable r = new Runnable() {
    public void run() {
    Assert.fail(“Fails in the thread (thread stack is dumped), but test still passes.”);
    }
    };
    Thread t = new Thread(r);
    t.start();
    Thread.sleep(2000);
    }
    }

    In this trivial case, the test will pass even though the assertion within the test failed. As I said, it’s not an issue in your case but it is something for new players to be mindful of.

    Reply ↓
  4. Alex Wei

    How about using Future? I know you have to catch exceptions instead of having a method that returns a boolean. However, it can be used to cancel the unwanted task.

    Reply ↓