Using matchers
(See also Matchers quick reference.)
ScalaTest provides a domain specific language (DSL) for expressing assertions in tests
using the word should or must. If you prefer the word should, you mix in ShouldMatchers, like this:
import org.scalatest.FlatSpec
import org.scalatest.matchers.ShouldMatchers
class ExampleSpec extends FlatSpec with ShouldMatchers { ...
If you prefer
must, you mix in
MustMatchers:
import org.scalatest.FlatSpec
import org.scalatest.matchers.MustMatchers
class ExampleSpec extends FlatSpec with MustMatchers { ...
You can alternatively import the members of either trait, a technique particularly useful when you want to try out matcher expressions in the
Scala interpeter. Here's an example where the members of ShouldMatchers are imported:
import org.scalatest.FlatSpec
import org.scalatest.matchers.ShouldMatchers._
class ExampleSpec extends FlatSpec { // Can use should matchers here ...
Here is a table of contents for this page:
Checking equality
If you mix ShouldMatchers into
a suite class (or import its members), you can write an equality assertion in that suite like this:
result should equal (3)
Here result is a variable, and can be of any type. If the object is an
Int with the value 3, execution will continue (i.e., the expression will result
in the unit value, ()). Otherwise, a TestFailedException
will be thrown with a detail message that explains the problem, such as "7 did not equal 3".
This TestFailedException will cause the test to fail.
The left should equal (right) syntax works by calling == on the left
value, passing in the right value, on every type except arrays. If both left and right are arrays, deep
will be invoked on both left and right before comparing them with ==. Thus, even though this expression
will yield false, because Array's equals method compares object identity:
Array(1, 2) == Array(1, 2)
The following expression will not result in a TestFailedException, because ScalaTest compares
the two arrays structurally, taking into consideration the equality of the array's contents:
Array(1, 2) should equal (Array(1, 2))
If you ever do want to verify that two arrays are actually the same object (have the same identity), you can use the
be theSameInstanceAs syntax, described below.
Checking size and length
You can check the size or length of just about any type of object for which it
would make sense. Here's how checking for length looks:
result should have length (3)
Size is similar:
result should have size (10)
The length syntax can be used with any object that has a field or method named length
or a method named getLength. Similarly, the size syntax can be used with any
object that has a field or method named size or a method named getSize.
The type of a length or size field, or return type of a method, must be either Int
or Long. Any such method must take no parameters. (The Scala compiler will ensure at compile time that
the object on which should is being invoked has the appropriate structure.)
Checking strings
You can check for whether a string starts with, ends with, or includes a substring like this:
string should startWith ("Hello")
string should endWith ("world")
string should include ("seven")
You can check for whether a string starts with, ends with, or includes a regular expression, like this:
string should startWith regex ("Hel*o")
string should endWith regex ("wo.ld")
string should include regex ("wo.ld")
And you can check whether a string fully matches a regular expression, like this:
string should fullyMatch regex ("""(-)?(\d+)(\.\d*)?""")
The regular expression passed following the regex token can be either a String
or a scala.util.matching.Regex.
Greater and less than
You can check whether any type that is, or can be implicitly converted to,
an Ordered[T] is greater than, less than, greater than or equal, or less
than or equal to a value of type T. The syntax is:
one should be < (7)
one should be > (0)
one should be <= (7)
one should be >= (0)
Checking equality with be ===
An alternate way to check for equality of two objects is to use be with
===. Here's an example:
result should be === (3)
Here result is a variable, and can be of any type. If the object is an
Int with the value 3, execution will continue (i.e., the expression will result
in the unit value, ()). Otherwise, a TestFailedException
will be thrown with a detail message that explains the problem, such as "7 was not equal to 3".
This TestFailedException will cause the test to fail.
The left should be === (right) syntax works by calling == on the left
value, passing in the right value, on every type except arrays. If both left and right are arrays, deep
will be invoked on both left and right before comparing them with ==. Thus, even though this expression
will yield false, because Array's equals method compares object identity:
Array(1, 2) == Array(1, 2)
The following expression will not result in a TestFailedException, because ScalaTest compares
the two arrays structurally, taking into consideration the equality of the array's contents:
Array(1, 2) should be === (Array(1, 2))
If you ever do want to verify that two arrays are actually the same object (have the same identity), you can use the
be theSameInstanceAs syntax, described below.
Checking Boolean properties with be
If an object has a method that takes no parameters and returns boolean, you can check
it by placing a Symbol (after be) that specifies the name
of the method (excluding an optional prefix of "is"). A symbol literal
in Scala begins with a tick mark and ends at the first non-identifier character. Thus,
'empty results in a Symbol object at runtime, as does
'defined and 'file. Here's an example:
emptySet should be ('empty)
Given this code, ScalaTest will use reflection to look on the object referenced from
emptySet for a method that takes no parameters and results in
Boolean,
with either the name
empty or
isEmpty. If found, it will invoke
that method. If the method returns
true, execution will continue. But if it returns
false, a
TestFailedException will be thrown that will contain a detail message, such as:
Set(1, 2, 3) was not empty
This be syntax can be used with any type. If the object does
not have an appropriately named predicate method, you'll get a TestFailedException
at runtime with a detail message that explains the problem.
(For the details on how a field or method is selected during this
process, see the documentation for BeWord.)
If you think it reads better, you can optionally put a or an after
be. For example, java.io.File has two predicate methods,
isFile and isDirectory. Thus with a File object
named temp, you could write:
temp should be a ('file)
Or, given java.awt.event.KeyEvent has a method isActionKey that takes
no arguments and returns Boolean, you could assert that a KeyEvent is
an action key with:
keyEvent should be an ('actionKey)
If you prefer to check Boolean properties in a type-safe manner, you can use a BePropertyMatcher.
This would allow you to write expressions such as:
emptySet should be (empty)
temp should be a (file)
keyEvent should be an (actionKey)
These expressions would fail to compile if should is used on an inappropriate type, as determined
by the type parameter of the BePropertyMatcher being used. (For example, file in this example
would likely be of type BePropertyMatcher[java.io.File]. If used with an appropriate type, such an expression will compile
and at run time the Boolean property method or field will be accessed directly; i.e., no reflection will be used.
See the documentation for BePropertyMatcher for more information.
Creating custom BeMatchers
If you want to create a new way of using
be, which doesn't map to an actual property on the
type you care about, you can create a
BeMatcher. You could use this, for example, to create
BeMatcher[Int]
called
odd, which would match any odd
Int, and
even, which would match
any even
Int.
Given this pair of
BeMatchers, you could check whether an
Int was odd or even with expressions like:
num should be (odd)
num should not be (even)
For more information, see the documentation for
BeMatcher.
Checking object identity
If you need to check that two references refer to the exact same object, you can write:
ref1 should be theSameInstanceAs (ref2)
Checking numbers against a range
To check whether a floating point number has a value that exactly matches another, you
can use should equal:
sevenDotOh should equal (7.0)
Often, however, you may want to check whether a floating point number is within a
range. You can do that using be and plusOrMinus, like this:
sevenDotOh should be (6.9 plusOrMinus 0.2)
This expression will cause a TestFailedException to be thrown if the floating point
value, sevenDotOh is outside the range 6.7 to 7.1.
You can also use plusOrMinus with integral types, for example:
seven should be (6 plusOrMinus 2)
Traversables, iterables, sets, sequences, and maps
You can use some of the syntax shown previously with Iterable and its
subtypes. For example, you can check whether an Iterable is empty,
like this:
iterable should be ('empty)
You can check the length of an Seq (Array, List, etc.),
like this:
array should have length (3)
list should have length (9)
You can check the size of any Traversable, like this:
map should have size (20)
set should have size (90)
In addition, you can check whether an Iterable contains a particular
element, like this:
iterable should contain ("five")
You can also check whether a Map contains a particular key, or value, like this:
map should contain key (1)
map should contain value ("Howdy")
Java collections and maps
You can use similar syntax on Java collections (java.util.Collection) and maps (java.util.Map).
For example, you can check whether a Java Collection or Map is empty,
like this:
javaCollection should be ('empty)
javaMap should be ('empty)
Even though Java's List type doesn't actually have a length or getLength method,
you can nevertheless check the length of a Java List (java.util.List) like this:
javaList should have length (9)
You can check the size of any Java Collection or Map, like this:
javaMap should have size (20)
javaSet should have size (90)
In addition, you can check whether a Java Collection contains a particular
element, like this:
javaCollection should contain ("five")
One difference to note between the syntax supported on Java collections and that of Scala
iterables is that you can't use contain (...) syntax with a Java Map.
Java differs from Scala in that its Map is not a subtype of its Collection type.
If you want to check that a Java Map contains a specific key/value pair, the best approach is
to invoke entrySet on the Java Map and check that entry set for the appropriate
element (a java.util.Map.Entry) using contain (...).
Despite this difference, the other (more commonly used) map matcher syntax works just fine on Java Maps.
You can, for example, check whether a Java Map contains a particular key, or value, like this:
javaMap should contain key (1)
javaMap should contain value ("Howdy")
Be as an equality comparison
All uses of be other than those shown previously perform an equality comparison. In other words, they work
the same as equals. This redundance between be and equals exists because it enables syntax
that sometimes sounds more natural. For example, instead of writing:
result should equal (null)
You can write:
result should be (null)
(Hopefully you won't write that too much given null is error prone, and Option
is usually a better, well, option.)
Here are some other examples of be used for equality comparison:
sum should be (7.0)
boring should be (false)
fun should be (true)
list should be (Nil)
option should be (None)
option should be (Some(1))
As with equal, using be on two arrays results in deep being called on both arrays prior to
calling equal. As a result,
the following expression would not throw a TestFailedException:
Array(1, 2) should be (Array(1, 2))
Because be is used in several ways in ScalaTest matcher syntax, just as it is used in many ways in English, one
potential point of confusion in the event of a failure is determining whether be was being used as an equality comparison or
in some other way, such as a property assertion. To make it more obvious when be is being used for equality, the failure
messages generated for those equality checks will include the word equal in them. For example, if this expression fails with a
TestFailedException:
option should be (Some(1))
The detail message in that TestFailedException will include the words "equal to" to signify be
was in this case being used for equality comparison:
Some(2) was not equal to Some(1)
Being negative
If you wish to check the opposite of some condition, you can simply insert not in the expression.
Here are a few examples:
result should not be (null)
sum should not be <= (10)
mylist should not equal (yourList)
string should not startWith ("Hello")
Logical expressions with and and or
You can also combine matcher expressions with and and/or or, however,
you must place parentheses or curly braces around the and or or expression. For example,
this and-expression would not compile, because the parentheses are missing:
map should contain key ("two") and not contain value (7)
Instead, you need to write:
map should (contain key ("two") and not contain value (7))
Here are some more examples:
number should (be > (0) and be <= (10))
option should (equal (Some(List(1, 2, 3))) or be (None))
string should (
equal ("fee") or
equal ("fie") or
equal ("foe") or
equal ("fum")
)
Two differences exist between expressions composed of these and and or operators and the expressions you can write
on regular Booleans using its && and || operators. First, expressions with and
and or do not short-circuit. The following contrived expression, for example, would print "hello, world!":
"yellow" should (equal ("blue") and equal { println("hello, world!"); "green" })
In other words, the entire and or or expression is always evaluated, so you'll see any side effects
of the right-hand side even if evaluating
only the left-hand side is enough to determine the ultimate result of the larger expression. Failure messages produced by these
expressions will "short-circuit," however,
mentioning only the left-hand side if that's enough to determine the result of the entire expression. This "short-circuiting" behavior
of failure messages is intended
to make it easier and quicker for you to ascertain which part of the expression caused the failure. The failure message for the previous
expression, for example, would be:
"yellow" did not equal "blue"
Most likely this lack of short-circuiting would rarely be noticeable, because evaluating the right hand side will usually not
involve a side effect. One situation where it might show up, however, is if you attempt to and a null check on a variable with an expression
that uses the variable, like this:
map should (not be (null) and contain key ("ouch"))
If map is null, the test will indeed fail, but with a NullPointerException, not a
TestFailedException. Here, the NullPointerException is the visible right-hand side effect. To get a
TestFailedException, you would need to check each assertion separately:
map should not be (null)
map should contain key ("ouch")
If map is null in this case, the null check in the first expression will fail with
a TestFailedException, and the second expression will never be executed.
The other difference with Boolean operators is that although && has a higher precedence than ||,
and and or
have the same precedence. Thus although the Boolean expression (a || b && c) will evaluate the && expression
before the || expression, like (a || (b && c)), the following expression:
traversable should (contain (7) or contain (8) and have size (9))
Will evaluate left to right, as:
traversable should ((contain (7) or contain (8)) and have size (9))
If you really want the and part to be evaluated first, you'll need to put in parentheses, like this:
traversable should (contain (7) or (contain (8) and have size (9)))
Working with Options
ScalaTest matchers has no special support for Options, but you can
work with them quite easily using syntax shown previously. For example, if you wish to check
whether an option is None, you can write any of:
option should equal (None)
option should be (None)
option should not be ('defined)
option should be ('empty)
If you wish to check an option is defined, and holds a specific value, you can write either of:
option should equal (Some("hi"))
option should be (Some("hi"))
If you only wish to check that an option is defined, but don't care what it's value is, you can write:
option should be ('defined)
If you mix in (or import the members of) OptionValues,
you can write one statement that indicates you believe an option should be defined and then say something else about its value. Here's an example:
import org.scalatest.OptionValues._
option.value should be < (7)
Checking arbitrary properties with have
Using have, you can check properties of any type, where a property is an attribute of any
object that can be retrieved either by a public field, method, or JavaBean-style get
or is method, like this:
book should have (
'title ("Programming in Scala"),
'author (List("Odersky", "Spoon", "Venners")),
'pubYear (2008)
)
This expression will use reflection to ensure the title, author, and pubYear properties of object book
are equal to the specified values. For example, it will ensure that book has either a public Java field or method
named title, or a public method named getTitle, that when invoked (or accessed in the field case) results
in a the string "Programming in Scala". If all specified properties exist and have their expected values, respectively,
execution will continue. If one or more of the properties either does not exist, or exists but results in an unexpected value,
a TestFailedException will be thrown that explains the problem. (For the details on how a field or method is selected during this
process, see the documentation for HavePropertyMatcherGenerator.)
When you use this syntax, you must place one or more property values in parentheses after have, seperated by commas, where a property
value is a symbol indicating the name of the property followed by the expected value in parentheses. The only exceptions to this rule is the syntax
for checking size and length shown previously, which does not require parentheses. If you forget and put parentheses in, however, everything will
still work as you'd expect. Thus instead of writing:
array should have length (3)
set should have size (90)
You can alternatively, write:
array should have (length (3))
set should have (size (90))
If a property has a value different from the specified expected value, a TestFailedError will be thrown
with a detail message that explains the problem. For example, if you assert the following on
a book whose title is Moby Dick:
book should have ('title ("A Tale of Two Cities"))
You'll get a TestFailedException with this detail message:
The title property had value "Moby Dick", instead of its expected value "A Tale of Two Cities",
on object Book("Moby Dick", "Melville", 1851)
If you prefer to check properties in a type-safe manner, you can use a HavePropertyMatcher.
This would allow you to write expressions such as:
book should have (
title ("Programming in Scala"),
author (List("Odersky", "Spoon", "Venners")),
pubYear (2008)
)
These expressions would fail to compile if should is used on an inappropriate type, as determined
by the type parameter of the HavePropertyMatcher being used. (For example, title in this example
might be of type HavePropertyMatcher[org.publiclibrary.Book]. If used with an appropriate type, such an expression will compile
and at run time the property method or field will be accessed directly; i.e., no reflection will be used.
See the documentation for HavePropertyMatcher for more information.
Using custom matchers
If none of the built-in matcher syntax (or options shown so far for extending the syntax) satisfy a particular need you have, you can create
custom Matchers that allow
you to place your own syntax directly after should. For example, class java.io.File has a method exists, which
indicates whether a file of a certain path and name exists. Because the exists method takes no parameters and returns Boolean,
you can call it using be with a symbol or BePropertyMatcher, yielding assertions like:
file should be ('exists)
file should be (inExistance)
Although these expressions will achieve your goal of throwing a TestFailedException if the file does not exist, they don't produce
the most readable code because the English is either incorrect or awkward. In this case, you might want to create a
custom Matcher[java.io.File]
named exist, which you could then use to write expressions like:
file should exist
file should not (exist)
file should (exist and have ('name ("temp.txt")))
Note that when you use custom Matchers, you will need to put parentheses around the custom matcher in more cases than with
the built-in syntax. For example you will often need the parentheses after not, as shown above. (There's no penalty for
always surrounding custom matchers with parentheses, and if you ever leave them off when they are needed, you'll get a compiler error.)
One good way to organize custom matchers is to place them inside one or more
traits that you can then mix into the suites or specs that need them. Here's an example:
trait CustomMatchers {
class FileExistsMatcher extends Matcher[java.io.File] {
def apply(left: java.io.File) = {
val fileOrDir = if (left.isFile) "file" else "directory"
val failureMessageSuffix =
fileOrDir + " named " + left.getName + " did not exist"
val negatedFailureMessageSuffix =
fileOrDir + " named " + left.getName + " existed"
MatchResult(
left.exists,
"The " + failureMessageSuffix,
"The " + negatedFailureMessageSuffix,
"the " + failureMessageSuffix,
"the " + negatedFailureMessageSuffix
)
}
}
val exist = new FileExistsMatcher
}
object CustomMatchers extends CustomMatchers
Note: the CustomMatchers companion object exists to make it easy to bring the
matchers defined in this trait into scope via importing, instead of mixing in the trait. The ability
to import them is useful, for example, when you want to use the matchers defined in a trait in the Scala interpreter console.
This trait contains one matcher class, FileExistsMatcher, and a val named exist that refers to
an instance of FileExistsMatcher. Because the class extends Matcher[java.io.File],
the compiler will only allow it be used to match against instances of java.io.File. A matcher must declare an
apply method that takes the type decared in Matcher's type parameter, in this case java.io.File.
The apply method will return a MatchResult whose matches field will indicate whether the match succeeded.
The failureMessage field will provide a programmer-friendly error message indicating, in the event of a match failure, what caused
the match to fail.
The FileExistsMatcher matcher in this example determines success by calling exists on the passed java.io.File. It
does this in the first argument passed to the MatchResult factory method:
left.exists,
In other words, if the file exists, this matcher matches.
The next argument to MatchResult's factory method produces the failure message string:
"The " + failureMessageSuffix,
If the passed java.io.File is a file (not a directory) and has the name temp.txt, for example, the failure
message would be:
The file named temp.txt did not exist
For more information on the fields in a MatchResult, including the subsequent three fields that follow the failure message,
please see the documentation for MatchResult.
Given the CustomMatchers trait as defined above, you can use the exist syntax in any suite or spec in
which you mix in the trait:
class ExampleSpec extends Spec with ShouldMatchers with CustomMatchers {
describe("A temp file") {
it("should be created and deleted") {
val tempFile = java.io.File.createTempFile("delete", "me")
try {
tempFile should exist
}
finally {
tempFile.delete()
}
tempFile should not (exist)
}
}
}
Note that when you use custom Matchers, you will need to put parentheses around the custom matcher when if follows not,
as shown in the last assertion above: tempFile should not (exist).
Other ways to create matchers
There are other ways to create new matchers besides defining one as shown above. For example, you would normally check to ensure
an option is defined like this:
Some("hi") should be ('defined)
If you wanted to get rid of the tick mark, you could simply define defined like this:
val defined = 'defined
Now you can check that an option is defined without the tick mark:
Some("hi") should be (defined)
Perhaps after using that for a while, you realize you're tired of typing the parentheses. You could
get rid of them with another one-liner:
val beDefined = be (defined)
Now you can check that an option is defined without the tick mark or the parentheses:
Some("hi") should beDefined
You can also use ScalaTest matchers' logical operators to combine existing matchers into new ones, like this:
val beWithinTolerance = be >= 0 and be <= 10
Now you could check that a number is within the tolerance (in this case, between 0 and 10, inclusive), like this:
num should beWithinTolerance
When defining a full blown matcher, one shorthand is to use one of the factory methods in Matcher's companion
object. For example, instead of writing this:
val beOdd =
new Matcher[Int] {
def apply(left: Int) =
MatchResult(
left % 2 == 1,
left + " was not odd",
left + " was odd"
)
}
You could alternately write this:
val beOdd =
Matcher { (left: Int) =>
MatchResult(
left % 2 == 1,
left + " was not odd",
left + " was odd"
)
}
Either way you define the beOdd matcher, you could use it like this:
3 should beOdd
4 should not (beOdd)
You can also compose matchers. If for some odd reason, you wanted a Matcher[String] that
checked whether a string, when converted to an Int,
was odd, you could make one by composing beOdd with
a function that converts a string to an Int, like this:
val beOddAsInt = beOdd compose { (s: String) => s.toInt }
Now you have a Matcher[String] whose apply method first
invokes the converter function to convert the passed string to an Int,
then passes the resulting Int to beOdd. Thus, you could use
beOddAsInt like this:
"3" should beOddAsInt
"4" should not (beOddAsInt)
You can also define a method that produces a matcher using matcher
composition and a passed parameter. For example, here's how you could create a Matcher[File] from a
Matcher[String]:
import java.io.File
def endWithExtension(ext: String) = endWith(ext) compose { (f: File) => f.getPath }
Every time you call the above endWithExtension method, you'll get a new Matcher[File]. Here's an example:
new File("output.txt") should endWithExtension("txt")
Checking for expected exceptions
Sometimes you need to test whether a method throws an expected exception under certain circumstances, such
as when invalid arguments are passed to the method. With ShouldMatchers mixed in, you can
check for an expected exception like this:
evaluating { s.charAt(-1) } should produce [IndexOutOfBoundsException]
If charAt throws an instance of StringIndexOutOfBoundsException,
this expression will result in that exception. But if charAt completes normally, or throws a different
exception, this expression will complete abruptly with a TestFailedException.
This expression returns the caught exception so that you can inspect it further if you wish, for
example, to ensure that data contained inside the exception has the expected values. Here's an
example:
val thrown = evaluating { s.charAt(-1) } should produce [IndexOutOfBoundsException]
thrown.getMessage should equal ("String index out of range: -1")
Those pesky parens
Perhaps the most tricky part of writing assertions using ScalaTest matchers is remembering
when you need or don't need parentheses, but bearing in mind a few simple rules should help.
It is also reassuring to know that if you ever leave off a set of parentheses when they are
required, your code will not compile. Thus the compiler will help you remember when you need the parens.
That said, the rules are:
1. Although you don't always need them, it is recommended style to always put parentheses
around right-hand values, such as the 7 in num should equal (7):
result should equal (4)
array should have length (3)
book should have (
'title ("Programming in Scala"),
'author (List("Odersky", "Spoon", "Venners")),
'pubYear (2008)
)
option should be ('defined)
catMap should (contain key (9) and contain value ("lives"))
keyEvent should be an ('actionKey)
javaSet should have size (90)
2. Except for length and size, you must always put parentheses around
the list of one or more property values following a have:
file should (exist and have ('name ("temp.txt")))
book should have (
title ("Programming in Scala"),
author (List("Odersky", "Spoon", "Venners")),
pubYear (2008)
)
javaList should have length (9) // parens optional for length and size
3. You must always put parentheses around and and or expressions, as in:
catMap should (contain key (9) and contain value ("lives"))
number should (equal (2) or equal (4) or equal (8))
4. Although you don't always need them, it is recommended style to always put parentheses
around custom Matchers when they appear directly after not:
file should exist
file should not (exist)
file should (exist and have ('name ("temp.txt")))
file should (not (exist) and have ('name ("temp.txt"))
file should (have ('name ("temp.txt") or exist)
file should (have ('name ("temp.txt") or not (exist))
That's it. With a bit of practice it should become natural to you, and the compiler will always be there to tell you if you
forget a set of needed parentheses.
Next, learn about testing with mock objects.
