Home » Java » How do you assert that a certain exception is thrown in JUnit 4 tests?

How do you assert that a certain exception is thrown in JUnit 4 tests?

Posted by: admin November 2, 2017 Leave a comment

Questions:

How can I use JUnit4 idiomatically to test that some code throws an exception?

While I can certainly do something like this:

@Test
public void testFooThrowsIndexOutOfBoundsException() {
  boolean thrown = false;

  try {
    foo.doStuff();
  } catch (IndexOutOfBoundsException e) {
    thrown = true;
  }

  assertTrue(thrown);
}

I recall that there is an annotation or an Assert.xyz or something that is far less kludgy and far more in-the-spirit of JUnit for these sorts of situations.

Answers:

JUnit 4 has support for this:

@Test(expected = IndexOutOfBoundsException.class)
public void testIndexOutOfBoundsException() {
    ArrayList emptyList = new ArrayList();
    Object o = emptyList.get(0);
}

Questions:
Answers:

Edit Now that JUnit5 has released, the best option would be to use Assertions.assertThrows() (see my other answer).

If you haven’t migrated to JUnit 5, but can use JUnit 4.7, you can use the ExpectedException Rule:

public class FooTest {
  @Rule
  public final ExpectedException exception = ExpectedException.none();

  @Test
  public void doStuffThrowsIndexOutOfBoundsException() {
    Foo foo = new Foo();

    exception.expect(IndexOutOfBoundsException.class);
    foo.doStuff();
  }
}

This is much better than @Test(expected=IndexOutOfBoundsException.class) because the test will fail if IndexOutOfBoundsException is thrown before foo.doStuff()

See this article for details

Questions:
Answers:

Be careful using expected exception, because it only asserts that the method threw that exception, not a particular line of code in the test.

I tend to use this for testing parameter validation, because such methods are usually very simple, but more complex tests might better be served with:

try {
    methodThatShouldThrow();
    fail( "My method didn't throw when I expected it to" );
} catch (MyException expectedException) {
}

Apply judgement.

Questions:
Answers:

As answered before, there are many ways of dealing with exceptions in JUnit. But with Java 8 there is another one: using Lambda Expressions. With Lambda Expressions we can achieve a syntax like this:

@Test
public void verifiesTypeAndMessage() {
    assertThrown(new DummyService()::someMethod)
            .isInstanceOf(RuntimeException.class)
            .hasMessage("Runtime exception occurred")
            .hasMessageStartingWith("Runtime")
            .hasMessageEndingWith("occurred")
            .hasMessageContaining("exception")
            .hasNoCause();
}

assertThrown accepts a functional interface, whose instances can be created with lambda expressions, method references, or constructor references. assertThrown accepting that interface will expect and be ready to handle an exception.

This is relatively simple yet powerful technique.

Have a look at this blog post describing this technique: http://blog.codeleak.pl/2014/07/junit-testing-exception-with-java-8-and-lambda-expressions.html

The source code can be found here: https://github.com/kolorobot/unit-testing-demo/tree/master/src/test/java/com/github/kolorobot/exceptions/java8

Disclosure: I am the author of the blog and the project.

Questions:
Answers:

in junit, there are three ways to test exception.

  • use the optional ‘expected’ attribute of Test annonation

    @Test(expected = IndexOutOfBoundsException.class)
    public void testFooThrowsIndexOutOfBoundsException() {
        foo.doStuff();
    }
    
  • use the ExpectedException rule

    public class XxxTest {
        @Rule
        public ExpectedException thrown = ExpectedException.none();
    
        @Test
        public void testFooThrowsIndexOutOfBoundsException() {
            thrown.expect(IndexOutOfBoundsException.class)
            //you can test the exception message like
            thrown.expectMessage("expected messages");
            foo.doStuff();
        }
    }
    
  • finally, you also can use the classic try/catch way widely used under junit 3 framework

    @Test
    public void testFooThrowsIndexOutOfBoundsException() {
        try {
            foo.doStuff();
            fail("expected exception was not occured.");
        } catch(IndexOutOfBoundsException e) {
            //if execution reaches here, 
            //it indicates this exception was occured.
            //so we need not handle it.
        }
    }
    
  • so

    • the 1st way used when you only want test the type of exception
    • the 2nd and 3rd way used when you want test exception message further
    • if you use junit 3, then the 3rd one is preferred.
  • for more info, you can read this document for details.

Questions:
Answers:

tl;dr

  • pre-JDK8 : I will recommend the old good trycatch block.

  • post-JDK8 : Use AssertJ or custom lambdas to assert exceptional behaviour.

the long story

It is possible to write yourself a do it yourself trycatch block or use the JUnit tools (@Test(expected = ...) or the @Rule ExpectedException JUnit rule feature).

But these way are not so elegant and don’t mix well readability wise with other tools.

  1. The trycatch block you have to write the block around the tested behavior, and write the assertion in the catch block, that may be fine but many find taht this style interrupts the reading flow of a test. Also you need to write an Assert.fail at the end of the try block otherwise the test may miss one side of the assertions ; PMD, findbugs or Sonar will spot such issues.

  2. The @Test(expected = ...) feature is interesting as you can write less code and then writing this test is supposedly less prone to coding errors. But ths approach is lacking a some areas.

    • If the test needs to check additional things on the exception like the cause or the message (good exception messages are really important, having a precise exception type may not be enough).
    • Also as the expectation is placed around in the method, depending on how the tested code is written then the wrong part of the test code can throw the exception, leading to false positive test and I m not sure that PMD, findbugs or Sonar will give hints on such code.

      @Test(expected = WantedException.class)
      public void call2_should_throw_a_WantedException__not_call1() {
          // init tested
          tested.call1(); // may throw a WantedException
      
          // call to be actually tested
          tested.call2(); // the call that is supposed to raise an exception
      }
      
  3. The ExpectedException rule is also an attempt to fix the previous caveats, but it feels a bit awkward to use as it uses an expectation style, EasyMock users knows very well this style. It might be convenient for some, but if you follow Behaviour Driven Development (BDD) or Arrange Act Assert (AAA) principles the ExpectedException rule won’t fit in those writing style. Aside of that it may suffer from the same issue as the as the @Test way, depending where you place the expectation.

    @Rule ExpectedException thrown = ExpectedException.none()
    
    @Test
    public void call2_should_throw_a_WantedException__not_call1() {
        // expectations
        thrown.expect(WantedException.class);
        thrown.expectMessage("boom");
    
        // init tested
        tested.call1(); // may throw a WantedException
    
        // call to be actually tested
        tested.call2(); // the call that is supposed to raise an exception
    }
    

    Even the expected exception is placed before the test statement, it breaks your reading flow if the tests follow BDD or AAA.

    Also see this comment issue on JUnit of the author of ExpectedException.

So these above options have all their load of caveats, and clearly not immune to coder errors.

  1. There’s a project I became aware after creating this answer that looks promising, it’s catch-exception.

    As the description of the project says, it let a coder write in a fluent line of code catching the exception and offer this exception for later assertion. And you can use any assertion library like Hamcrest or AssertJ.

    A rapid example taken from the home page :

    // given: an empty list
    List myList = new ArrayList();
    
    // when: we try to get the first element of the list
    when(myList).get(1);
    
    // then: we expect an IndexOutOfBoundsException
    then(caughtException())
            .isInstanceOf(IndexOutOfBoundsException.class)
            .hasMessage("Index: 1, Size: 0") 
            .hasNoCause();
    

    As you can see the code is really straightforward, you catch the exception on a specific line, the then API is an alias that will use AssertJ APIs (similar to using assertThat(ex).hasNoCause()...). At some point the project relied on FEST-Assert the ancestor of AssertJ. EDIT: It seems the project is brewing a Java 8 Lambdas support.

    Currently this library has two shortcomings :

    • At the time of this writing it is noteworthy to say this library is based on Mockito 1.x as it creates a mock of the tested object behind the scene. As Mockito is still not updated this library cannot work with final classes or final methods. And even if it was based on mockito 2 in the current version, this would require to declare a global mock maker (inline-mock-maker), something that may not what you want, as this mockmaker has different drawbacks that the regular mockmaker.

    • It requires yet another test dependency.

    These issues won’t apply once the library will support lambdas, however the functionality will be duplicated by AssertJ toolset.

    Taking all into account if you don’t want to use the catch-exception tool, I will recommend the old good way of the trycatch block, at least up to the JDK7. And for JDK 8 users you might prefer to use AssertJ as it offers may more than just asserting exceptions.

  2. With the JDK8, lambdas enter the test scene, and they have proved to be an interesting way to assert exceptional behaviour. AssertJ has been updated to provide a nice fluent API to assert exceptional behaviour.

    And a sample test with AssertJ :

    @Test
    public void test_exception_approach_1() {
        ...
        assertThatExceptionOfType(IOException.class)
                .isThrownBy(() -> someBadIOOperation())
                .withMessage("boom!"); 
    }
    
    @Test
    public void test_exception_approach_2() {
        ...
        assertThatThrownBy(() -> someBadIOOperation())
                .isInstanceOf(Exception.class)
                .hasMessageContaining("boom");
    }
    
    @Test
    public void test_exception_approach_3() {
        ...
        // when
        Throwable thrown = catchThrowable(() -> someBadIOOperation());
    
        // then
        assertThat(thrown).isInstanceOf(Exception.class)
                          .hasMessageContaining("boom");
    }
    
  3. With a near complete rewrite of JUnit 5, assertions have been improved a bit, they may prove interesting as an out of the box way to assert properly exception. But really the assertion API is still a bit poor, there’s nothing outside assertThrows.

    @Test
    @DisplayName("throws EmptyStackException when peeked")
    void throwsExceptionWhenPeeked() {
        Throwable t = assertThrows(EmptyStackException.class, () -> stack.peek());
    
        Assertions.assertEquals("...", t.getMessage());
    }
    

    As you noticed assertEquals is still returning void, and as such doesn’t allow chaining assertions like AssertJ.

    Also if you remember name clash with Matcher or Assert, be prepared to meet the same clash with Assertions.

I’d like to conclude that today (2017-03-03) AssertJ‘s ease of use, discoverable API, rapid pace of development and as a de facto test dependency is the best solution with JDK8 regardless of the test framework (JUnit or not), prior JDKs should instead rely on trycatch blocks even if they feel clunky.

This answer has been copied from another question that don’t have the same visibility, I am the same author.

Questions:
Answers:

BDD Style Solution: JUnit 4 + Catch Exception + AssertJ

@Test
public void testFooThrowsIndexOutOfBoundsException() {

    when(foo).doStuff();

    then(caughtException()).isInstanceOf(IndexOutOfBoundsException.class);

}

Source code

Dependencies

eu.codearte.catch-exception:catch-exception:1.3.3

Questions:
Answers:

To solve the same problem I did set up a small project:
http://code.google.com/p/catch-exception/

Using this little helper you would write

verifyException(foo, IndexOutOfBoundsException.class).doStuff();

This is less verbose than the ExpectedException rule of JUnit 4.7.
In comparison to the solution provided by skaffman, you can specify in which line of code you expect the exception. I hope this helps.

Questions:
Answers:

How about this: Catch a very general exception, make sure it makes it out of the catch block, then assert that the class of the exception is what you expect it to be. This assert will fail if a) the exception is of the wrong type (eg. if you got a Null Pointer instead) and b) the exception wasn’t ever thrown.

public void testFooThrowsIndexOutOfBoundsException() {
  Throwable e = null;

  try {
    foo.doStuff();
  } catch (Throwable ex) {
    e = ex;
  }

  assertTrue(e instanceof IndexOutOfBoundsException);
}

Questions:
Answers:

Using an AssertJ assertion, which can be used alongside JUnit:

import static org.assertj.core.api.Assertions.*;

@Test
public void testFooThrowsIndexOutOfBoundsException() {
  Foo foo = new Foo();

  assertThatThrownBy(() -> foo.doStuff())
        .isInstanceOf(IndexOutOfBoundsException.class);
}

It’s better than @Test(expected=IndexOutOfBoundsException.class) because it guarantees the expected line in the test threw the exception and lets you check more details about the exception, such as message, easier:

assertThatThrownBy(() ->
       {
         throw new Exception("boom!");
       })
    .isInstanceOf(Exception.class)
    .hasMessageContaining("boom");

Maven/Gradle instructions here.

Questions:
Answers:

You can also do this:

@Test
public void testFooThrowsIndexOutOfBoundsException() {
    try {
        foo.doStuff();
        assert false;
    } catch (IndexOutOfBoundsException e) {
        assert true;
    }
}

Questions:
Answers:

Update: JUnit5 has an improvement for exceptions testing: assertThrows.

following example is from: Junit 5 User Guide

 @Test
void exceptionTesting() {
    Throwable exception = assertThrows(IllegalArgumentException.class, () -> 
    {
        throw new IllegalArgumentException("a message");
    });
    assertEquals("a message", exception.getMessage());
}

Original asnwer using JUnit 4.

There are several ways to test that an exception is thrown. I have also discussed the below options in my post How to write great unit tests with JUnit

Set the expected parameter @Test(expected = FileNotFoundException.class).

@Test(expected = FileNotFoundException.class) 
public void testReadFile() { 
    myClass.readFile("test.txt");
}

Using try catch

public void testReadFile() { 
    try {
        myClass.readFile("test.txt");
        fail("Expected a FileNotFoundException to be thrown");
    } catch (FileNotFoundException e) {
        assertThat(e.getMessage(), is("The file test.txt does not exist!"));
    }

}

Testing with ExpectedException Rule.

@Rule
public ExpectedException thrown = ExpectedException.none();

@Test
public void testReadFile() throws FileNotFoundException {

    thrown.expect(FileNotFoundException.class);
    thrown.expectMessage(startsWith("The file test.txt"));
    myClass.readFile("test.txt");
}

You could read more about exceptions testing in JUnit4 wiki for Exception testing and bad.robot – Expecting Exceptions JUnit Rule.

Questions:
Answers:

IMHO, the best way to check for exceptions in JUnit is the try/catch/fail/assert pattern:

// this try block should be as small as possible,
// as you want to make sure you only catch exceptions from your code
try {
    sut.doThing();
    fail(); // fail if this does not throw any exception
} catch(MyException e) { // only catch the exception you expect,
                         // otherwise you may catch an exception for a dependency unexpectedly
    // a strong assertion on the message, 
    // in case the exception comes from anywhere an unexpected line of code,
    // especially important if your checking IllegalArgumentExceptions
    assertEquals("the message I get", e.getMessage()); 
}

The assertTrue might be a bit strong for some people, so assertThat(e.getMessage(), containsString("the message"); might be preferable.

Questions:
Answers:

JUnit 5 Solution

@Test
void testFooThrowsIndexOutOfBoundsException() {    
  Throwable exception = expectThrows( IndexOutOfBoundsException.class, foo::doStuff );

  assertEquals( "some message", exception.getMessage() );
}

More Infos about JUnit 5 on http://junit.org/junit5/docs/current/user-guide/#writing-tests-assertions

Questions:
Answers:

I tried many of the methods here, but they were either complicated or didn’t quite meet my requirements. In fact, one can write a helper method quite simply:

public class ExceptionAssertions {
    public static void assertException(BlastContainer blastContainer ) {
        boolean caughtException = false;
        try {
            blastContainer.test();
        } catch( Exception e ) {
            caughtException = true;
        }
        if( !caughtException ) {
            throw new AssertionFailedError("exception expected to be thrown, but was not");
        }
    }
    public static interface BlastContainer {
        public void test() throws Exception;
    }
}

Use it like this:

assertException(new BlastContainer() {
    @Override
    public void test() throws Exception {
        doSomethingThatShouldExceptHere();
    }
});

Zero dependencies: no need for mockito, no need powermock; and works just fine with final classes.

Questions:
Answers:

JUnit has built-in support for this, with an “expected” attribute

Questions:
Answers:

In my case I always get RuntimeException from db, but messages differ. And exception need to be handled respectively. Here is how I tested it:

@Test
public void testThrowsExceptionWhenWrongSku() {

    // Given
    String articleSimpleSku = "999-999";
    int amountOfTransactions = 1;
    Exception exception = null;

    // When
    try {
        createNInboundTransactionsForSku(amountOfTransactions, articleSimpleSku);
    } catch (RuntimeException e) {
        exception = e;
    }

    // Then
    shouldValidateThrowsExceptionWithMessage(exception, MESSAGE_NON_EXISTENT_SKU);
}

private void shouldValidateThrowsExceptionWithMessage(final Exception e, final String message) {
    assertNotNull(e);
    assertTrue(e.getMessage().contains(message));
}

Questions:
Answers:

Java 8 solution

If you would like a solution which:

  • Utilizes Java 8 lambdas
  • Does not depend on any JUnit magic
  • Asserts that a specific exception is thrown by a specific piece of code
  • Yields the actual exception so that you can further examine it

Here is a utility function that I wrote:

public final <T extends Throwable> T expectException( Class<T> exceptionClass, Runnable runnable )
{
    try
    {
        runnable.run();
    }
    catch( Throwable throwable )
    {
        if( throwable instanceof AssertionError && throwable.getCause() != null )
            throwable = throwable.getCause(); //allows "assert x != null : new IllegalArgumentException();"
        assert exceptionClass.isInstance( throwable ) : throwable; //exception of the wrong kind was thrown.
        assert throwable.getClass() == exceptionClass : throwable; //exception thrown was a subclass, but not the exact class, expected.
        @SuppressWarnings( "unchecked" )
        T result = (T)throwable;
        return result;
    }
    assert false; //expected exception was not thrown.
    return null; //to keep the compiler happy.
}

(taken from my blog)

Use it as follows:

@Test
public void testThrows()
{
    RuntimeException e = expectException( RuntimeException.class, () -> 
        {
            throw new RuntimeException( "fail!" );
        } );
    assert e.getMessage().equals( "fail!" );
}

Questions:
Answers:

We can use an assertion fail after the method that must return an exception:

try{
   methodThatThrowMyException();
   Assert.fail("MyException is not thrown !");
} catch (MyException ex) {              
} catch (Exception ex){
    Assert.fail("An exception other than MyException is thrown !");
}

Questions:
Answers:

Just make a Matcher that can be turned off and on, like this:

public class ExceptionMatcher extends BaseMatcher<Throwable> {
    private boolean active = true;
    private Class<? extends Throwable> throwable;

    public ExceptionMatcher(Class<? extends Throwable> throwable) {
        this.throwable = throwable;
    }

    public void on() {
        this.active = true;
    }

    public void off() {
        this.active = false;
    }

    @Override
    public boolean matches(Object object) {
        return active && throwable.isAssignableFrom(object.getClass());
    }

    @Override
    public void describeTo(Description description) {
        description.appendText("not the covered exception type");
    }
}

To use it:

add public ExpectedException exception = ExpectedException.none();,
then:

ExceptionMatcher exMatch = new ExceptionMatcher(MyException.class);
exception.expect(exMatch);
someObject.somethingThatThrowsMyException();
exMatch.off();

Questions:
Answers:

In JUnit 4 or later you can test the exceptions as follows

@Rule
public ExpectedException exceptions = ExpectedException.none();

this provides a lot of features which can be used to improve our JUnit tests.
If you see the below example I am testing 3 things on the exception.

  1. The Type of exception thrown
  2. The exception Message
  3. The cause of the exception

public class MyTest {

    @Rule
    public ExpectedException exceptions = ExpectedException.none();

    ClassUnderTest classUnderTest;

    @Before
    public void setUp() throws Exception {
        classUnderTest = new ClassUnderTest();
    }

    @Test
    public void testAppleisSweetAndRed() throws Exception {

        exceptions.expect(Exception.class);
        exceptions.expectMessage("this is the exception message");
        exceptions.expectCause(Matchers.<Throwable>equalTo(exceptionCause));

        classUnderTest.methodUnderTest("param1", "param2");
    }

}

Questions:
Answers:

Additionally to what NamShubWriter has said, make sure that:

  • The ExpectedException instance is public (Related Question)
  • The ExpectedException isn’t instantiated in say, the @Before method. This post clearly explains all the intricacies of JUnit’s order of execution.

Do not do this:

@Rule    
public ExpectedException expectedException;

@Before
public void setup()
{
    expectedException = ExpectedException.none();
}

Finally, this blog post clearly illustrates how to assert that a certain exception is thrown.

Questions:
Answers:

Take for example, you want to write Junit for below mentioned code fragment

public int divideByZeroDemo(int a,int b){

    return a/b;
}

public void exceptionWithMessage(String [] arr){

    throw new ArrayIndexOutOfBoundsException("Array is out of bound");
}

The above code is to test for some unknown exception that may occur and the below one is to assert some exception with custom message.

 @Rule
public ExpectedException exception=ExpectedException.none();

private Demo demo;
@Before
public void setup(){

    demo=new Demo();
}
@Test(expected=ArithmeticException.class)
public void testIfItThrowsAnyException() {

    demo.divideByZeroDemo(5, 0);

}

@Test
public void testExceptionWithMessage(){


    exception.expectMessage("Array is out of bound");
    exception.expect(ArrayIndexOutOfBoundsException.class);
    demo.exceptionWithMessage(new String[]{"This","is","a","demo"});
}

Questions:
Answers:

With Java 8 you can create a method taking a code to check and expected exception as parameters:

private void expectException(Runnable r, Class<?> clazz) { 
    try {
      r.run();
      fail("Expected: " + clazz.getSimpleName() + " but not thrown");
    } catch (Exception e) {
      if (!clazz.isInstance(e)) fail("Expected: " + clazz.getSimpleName() + " but " + e.getClass().getSimpleName() + " found", e);
    }
  }

and then inside your test:

expectException(() -> list.sublist(0, 2).get(2), IndexOutOfBoundsException.class);

Benefits:

  • not relying on any library
  • localised check – more precise and allows to have multiple assertions like this within one test if needed
  • easy to use
Questions:
Answers:

Now that JUnit 5 has released, the best option is to use Assertions.assertThrows() (see
the Junit 5 User Guide).

Here is an example that verifies an exception is thrown, and uses Truth to make assertions on the exception message:

public class FooTest {
  @Test
  public void doStuffThrowsIndexOutOfBoundsException() {
    Foo foo = new Foo();

    IndexOutOfBoundsException e = assertThrows(
        IndexOutOfBoundsException.class, foo::doStuff);

    assertThat(e).hasMessageThat().contains("woops!");
  }
}

The advantages over the approaches in the other answers are:

  1. Built into JUnit
  2. You get a useful exception message if the code in the lambda doesn’t throw an exception, and a stacktrace if it throws a different exception
  3. Concise
  4. Allows your tests to follow Arrange-Act-Assert
  5. You can precisely indicate what code you are expecting to throw the exception
  6. You don’t need to list the expected exception in the throws clause
  7. You can use the assertion framework of your choice to make assertions about the caught exception

A similar method will be added to org.junit Assert in JUnit 4.13.

Questions:
Answers:

My solution using Java 8 lambdas:

public static <T extends Throwable> T assertThrows(Class<T> expected, ThrowingRunnable action) throws Throwable {
    try {
        action.run();
        Assert.fail("Did not throw expected " + expected.getSimpleName());
        return null; // never actually
    } catch (Throwable actual) {
        if (!expected.isAssignableFrom(actual.getClass())) { // runtime '!(actual instanceof expected)'
            System.err.println("Threw " + actual.getClass().getSimpleName() 
                               + ", which is not a subtype of expected " 
                               + expected.getSimpleName());
            throw actual; // throw the unexpected Throwable for maximum transparency
        } else {
            return (T) actual; // return the expected Throwable for further examination
        }
    }
}

You have to define a FunctionalInterface, because Runnable doesn’t declare the required throws.

@FunctionalInterface
public interface ThrowingRunnable {
    void run() throws Throwable;
}

The method can be used as follows:

class CustomException extends Exception {
    public final String message;
    public CustomException(final String message) { this.message = message;}
}
CustomException e = assertThrows(CustomException.class, () -> {
    throw new CustomException("Lorem Ipsum");
});
assertEquals("Lorem Ipsum", e.message);

Questions:
Answers:

I wanted to comment with my solution to this problem, which avoided needing any of the exception related JUnit code.

I used assertTrue(boolean) combined with try/catch to look for my expected exception to be thrown. Here’s an example:

public void testConstructor() {
    boolean expectedExceptionThrown;
    try {
        // Call constructor with bad arguments
        double a = 1;
        double b = 2;
        double c = a + b; // In my example, this is an invalid option for c
        new Triangle(a, b, c);
        expectedExceptionThrown = false; // because it successfully constructed the object
    }
    catch(IllegalArgumentException e) {
        expectedExceptionThrown = true; // because I'm in this catch block
    }
    catch(Exception e) {
        expectedExceptionThrown = false; // because it threw an exception but not the one expected
    }
    assertTrue(expectedExceptionThrown);
}