ScalaTest User Guide

Getting started

Selecting testing styles

Defining base classes

Writing your first test

Using assertions

Tagging your tests

Running your tests

Sharing fixtures

Sharing tests

Using matchers

Testing with mock objects

Property-based testing

Using Selenium

Other goodies

Philosophy and design

Migrating to 2.0

Using assertions

ScalaTest makes three assertions available by default in any style trait. You can use:

  • assert for general assertions;
  • assertResult to differentiate expected from actual values;
  • intercept to ensure a bit of code throws an expected exception.

To get moving quickly in ScalaTest, learn and use these three assertions. Later if you prefer you can switch to the more expressive matchers DSL.

ScalaTest's assertions are defined in trait Assertions, which is extended by Suite, the supertrait to all style traits. Trait Assertions also provides:

  • fail to fail a test unconditionally;
  • assume to cancel a test;
  • withClue to add more information about a failure.

All of these constructs are described below.

The assert macro

In any Scala program, you can write assertions by invoking assert and passing in a Boolean expression, such as:

val left = 2
val right = 1
assert(left == right)

If the passed expression is true, assert will return normally. If false, Scala's assert will complete abruptly with an AssertionError. This behavior is provided by the assert method defined in object Predef, whose members are implicitly imported into every Scala source file.

ScalaTest defines another assert method that hides the one in Predef. It behaves the same, except that if false is passed it throws TestFailedException instead of AssertionError. Why? Because unlike AssertionError, TestFailedException carries information about exactly which item in the stack trace represents the line of test code that failed, which can help users more quickly find an offending line of code in a failing test.

If you pass the previous Boolean expression, left == right to assert in a ScalaTest test, a failure will be reported that includes reporting the left and right values. (This is possible because ScalaTest's assert is implemented as a macro.)

For example, given the same code as above but using ScalaTest assertions:

import org.scalatest.Assertions._
val left = 2
val right = 1
assert(left == right)

The detail message in the thrown TestFailedException from this assert will be: "2 did not equal 1".

You can provide an alternate error message by providing a String as a second argument to assert, like this:

val attempted = 2
assert(attempted == 1, "Execution was attempted " + left + " times instead of 1 time")

Using this form of assert, the failure report will be more specific to your problem domain, thereby helping you debug the problem. This Assertions trait also mixes in the TripleEquals, which gives you a === operator that allows you to customize Equality and perform equality checks with a numeric Tolerance. You can also use === to enforce type constraints at compile time with sibling traits TypeCheckedTripleEquals and ConversionCheckedTripleEquals.

Expected results

Although ScalaTest's assert macro provides a natural, readable extension to Scala's assert mechanism that provides good error messages, as the operands become lengthy, the code becomes less readable. In addition, the error messages generated for == and === comparisons don't distinguish between actual and expected values. The operands are just called left and right, because if one were named expected and the other actual, it would be difficult for people to remember which was which. To help with these limitations of assertions, Assertions includes a method called assertResult that can be used as an alternative to assert. To use assertResult, you place the expected value in parentheses after assertResult, followed by curly braces containing code that should result in the expected value. For example:

val a = 5
val b = 2
assertResult(2) {
  a - b
}

In this case, the expected value is 2, and the code being tested is a - b. This assertion will fail, and the detail message in the TestFailedException will read, "Expected 2, but got 3."

Forcing failures

If you just need the test to fail, you can write:

fail()

Or, if you want the test to fail with a message, write:

fail("I've got a bad feeling about this")

Intercepted exceptions

Sometimes you need to test whether a method throws an expected exception under certain circumstances, such as when invalid arguments are passed to the method. You can do this in the JUnit 3 style, like this:

val s = "hi"
try {
  s.charAt(-1)
  fail()
}
catch {
  case _: IndexOutOfBoundsException => // Expected, so continue
}

If charAt throws IndexOutOfBoundsException as expected, control will transfer to the catch case, which does nothing. If, however, charAt fails to throw an exception, the next statement, fail(), will be run. The fail method always completes abruptly with a TestFailedException, thereby signaling a failed test.

To make this common use case easier to express and read, ScalaTest provides an intercept method. You use it like this:

val s = "hi"
intercept[IndexOutOfBoundsException] {
  s.charAt(-1)
}

This code behaves much like the previous example. If charAt throws an instance of IndexOutOfBoundsException, intercept will return that exception. But if charAt completes normally, or throws a different exception, intercept will complete abruptly with a TestFailedException. intercept returns the caught exception so that you can inspect it further if you wish, for example, to ensure that data contained inside the exception has the expected values.

Assumptions

Trait Assertions also provides methods that allow you to cancel a test. You would cancel a test if a resource required by the test was unavailable. For example, if a test requires an external database to be online, and it isn't, the test could be canceled to indicate it was unable to run because of the missing database. Such a test assumes a database is available, and you can use the assume method to indicate this at the beginning of the test, like this:

assume(database.isAvailable)

For each overloaded assert method, trait Assertions provides an overloaded assume method with an identical signature and behavior, except the assume methods throw TestCanceledException whereas the assert methods throw TestFailedException. As with assert, assume hides a Scala method in Predef that performs a similar function, but throws AssertionError. And just as you can with assert, you can optionally provide a clue string, or use === to get a more detailed error message. Here are some examples:

assume(database.isAvailable, "The database was down again")
assume(database.getAllUsers.count == 9)

Forcing cancelations

For each overloaded fail method, there's a corresponding cancel method with an identical signature and behavior, except the cancel methods throw TestCanceledException whereas the fail methods throw TestFailedException. Thus if you just need to cancel a test, you can write:

cancel()

If you want to cancel the test with a message, just place the message in the parentheses:

cancel("Can't run the test because no internet connection was found")

Getting a clue

If you want more information that is provided by default by the methods if this trait, you can supply a "clue" string in one of several ways. The extra information (or "clues") you provide will be included in the detail message of the thrown exception. Both assert and assertResult provide a way for a clue to be included directly, intercept does not. Here's an example of clues provided directly in assert:

assert(1 + 1 === 3, "this is a clue")

and in assertResult:

assertResult(3, "this is a clue") { 1 + 1 }

The exceptions thrown by the previous two statements will include the clue string, "this is a clue", in the exception's detail message. To get the same clue in the detail message of an exception thrown by a failed intercept call requires using withClue:

withClue("this is a clue") {
  intercept[IndexOutOfBoundsException] {
    "hi".charAt(-1)
  }
}

The withClue method will only prepend the clue string to the detail message of exception types that mix in the ModifiableMessage trait. See the documentation for ModifiableMessage for more information. If you wish to place a clue string after a block of code, see the documentation for AppendedClues.

Next, learn about tagging your tests.

ScalaTest is brought to you by Bill Venners, with contributions from several other folks. It is sponsored by Artima, Inc.
ScalaTest is free, open-source software released under the Apache 2.0 license.

Copyright © 2009-2013 Artima, Inc. All Rights Reserved.

artima