Using matchers
ScalaTest provides a domain specific language (DSL) for expressing assertions in tests
using the word should
. Just mix in should.Matchers
, like this:
import org.scalatest.flatspec._
import org.scalatest.matchers.should._
class ExampleSpec extends AnyFlatSpec with Matchers { ...
You can alternatively import the members of the trait, a technique particularly useful when you want to try out matcher expressions in the
Scala interpreter. Here's an example where the members of should.Matchers
are imported:
import org.scalatest.flatspec._
import org.scalatest.matchers.should.Matchers._
class ExampleSpec extends AnyFlatSpec { // Can use matchers here ...
Here is a table of contents for this page:
If you mix should.Matchers
into
a suite class, you can write an equality assertion in that suite like this:
result should equal (3)
Here result
is a variable, and can be of any type. If the object is an
Int
with the value 3, execution will continue (i.e., the expression will result
in the unit value, ()
). Otherwise, a TestFailedException
will be thrown with a detail message that explains the problem, such as "7 did not equal 3"
.
This TestFailedException
will cause the test to fail.
Trait must.Matchers
is an alternative to should.Matchers
that provides the exact same meaning, syntax, and behavior as should.Matchers
, but
uses the verb must
instead of should
. The two traits differ only in the English semantics of the verb: should
is informal,
making the code feel like conversation between the writer and the reader; must
is more formal, making the code feel more like a written specification.
Checking equality with matchers
ScalaTest matchers provides five different ways to check equality, each designed to address a different need. They are:
result should equal (3)
result should === (3)
result should be (3)
result shouldEqual 3
result shouldBe 3
The “left
should
equal
(right)
” syntax requires an
org.scalactic.Equality[L]
to be provided (either implicitly or explicitly), where
L
is the left-hand type on which should
is invoked. In the "left
should
equal
(right)
" case,
for example, L
is the type of left
. Thus if left
is type Int
, the "left
should
equal
(right)
"
statement would require an Equality[Int]
.
By default, an implicit Equality[T]
instance is available for any type T
, in which equality is implemented
by simply invoking ==
on the left
value, passing in the right
value, with special treatment for arrays. If either left
or right
is an array, deep
will be invoked on it before comparing with ==. Thus, the following expression
will yield false, because Array
's equals
method compares object identity:
Array(1, 2) == Array(1, 2)
The next expression will by default not result in a TestFailedException
, because default Equality[Array[Int]]
compares
the two arrays structurally, taking into consideration the equality of the array's contents:
Array(1, 2) should equal (Array(1, 2))
If you ever do want to verify that two arrays are actually the same object (have the same identity), you can use the
be theSameInstanceAs
syntax, described below.
You can customize the meaning of equality for a type when using "should
equal
," "should
===
,"
or shouldEqual
syntax by defining implicit Equality
instances that will be used instead of default Equality
.
You might do this to normalize types before comparing them with ==
, for instance, or to avoid calling the ==
method entirely,
such as if you want to compare Double
s with a tolerance.
For an example, see the main documentation of trait Equality
.
You can always supply implicit parameters explicitly, but in the case of implicit parameters of type Equality[T]
, ScalaUtils provides a
simple "explictly" DSL. For example, here's how you could explicitly supply an Equality[String]
instance that normalizes both left and right
sides (which must be strings), by transforming them to lowercase:
scala> import org.scalatest.matchers.should.Matchers._
import org.scalatest.matchers.should.Matchers._
scala> import org.scalactic.Explicitly._
import org.scalactic.Explicitly._
scala> import org.scalactic.StringNormalizations._
import org.scalactic.StringNormalizations._
scala> "Hi" should equal ("hi") (after being lowerCased)
The after
being
lowerCased
expression results in an Equality[String]
, which is then passed
explicitly as the second curried parameter to equal
. For more information on the explictly DSL, see the main documentation
for trait Explicitly
.
The "should
be
" and shouldBe
syntax do not take an Equality[T]
and can therefore not be customized.
They always use the default approach to equality described above. As a result, "should
be
" and shouldBe
will
likely be the fastest-compiling matcher syntax for equality comparisons, since the compiler need not search for
an implicit Equality[T]
each time.
The should
===
syntax (and its complement, should
!==
) can be used to enforce type
constraints at compile-time between the left and right sides of the equality comparison. Here's an example:
scala> import org.scalatest.matchers.should.Matchers._
import org.scalatest.matchers.should.Matchers._
scala> import org.scalactic.TypeCheckedTripleEquals._
import org.scalactic.TypeCheckedTripleEquals._
scala> Some(2) should === (2)
<console>:17: error: types Some[Int] and Int do not adhere to the equality constraint
selected for the === and !== operators; the missing implicit parameter is of
type org.scalactic.Constraint[Some[Int],Int]
Some(2) should === (2)
^
By default, the "Some(2)
should
===
(2)
" statement would fail at runtime. By mixing in
the equality constraints provided by TypeCheckedTripleEquals
, however, the statement fails to compile. For more information
and examples, see the main documentation for trait TypeCheckedTripleEquals
.
Checking size and length
You can check the size or length of any type of object for which it
makes sense. Here's how checking for length looks:
result should have length 3
Size is similar:
result should have size 10
The length
syntax can be used with String
, Array
, any scala.collection.GenSeq
,
any java.util.List
, and any type T
for which an implicit Length[T]
type class is
available in scope.
Similarly, the size
syntax can be used with Array
, any scala.collection.GenTraversable
,
any java.util.Collection
, any java.util.Map
, and any type T
for which an implicit Size[T]
type class is
available in scope. You can enable the length
or size
syntax for your own arbitrary types, therefore,
by defining Length
or Size
type
classes for those types.
In addition, the length
syntax can be used with any object that has a field or method named length
or a method named getLength
. Similarly, the size
syntax can be used with any
object that has a field or method named size
or a method named getSize
.
The type of a length
or size
field, or return type of a method, must be either Int
or Long
. Any such method must take no parameters. (The Scala compiler will ensure at compile time that
the object on which should
is being invoked has the appropriate structure.)
Checking strings
You can check for whether a string starts with, ends with, or includes a substring like this:
string should startWith ("Hello")
string should endWith ("world")
string should include ("seven")
You can check for whether a string starts with, ends with, or includes a regular expression, like this:
string should startWith regex "Hel*o"
string should endWith regex "wo.ld"
string should include regex "wo.ld"
And you can check whether a string fully matches a regular expression, like this:
string should fullyMatch regex """(-)?(\d+)(\.\d*)?"""
The regular expression passed following the regex
token can be either a String
or a scala.util.matching.Regex
.
With the startWith
, endWith
, include
, and fullyMatch
tokens can also be used with an optional specification of required groups, like this:
"abbccxxx" should startWith regex ("a(b*)(c*)" withGroups ("bb", "cc"))
"xxxabbcc" should endWith regex ("a(b*)(c*)" withGroups ("bb", "cc"))
"xxxabbccxxx" should include regex ("a(b*)(c*)" withGroups ("bb", "cc"))
"abbcc" should fullyMatch regex ("a(b*)(c*)" withGroups ("bb", "cc"))
Greater and less than
You can check whether any type for which an implicit Ordering[T]
is available
is greater than, less than, greater than or equal, or less
than or equal to a value of type T
. The syntax is:
one should be < 7
one should be > 0
one should be <= 7
one should be >= 0
Checking Boolean
properties with be
If an object has a method that takes no parameters and returns boolean, you can check
it by placing a Symbol
(after be
) that specifies the name
of the method (excluding an optional prefix of "is
"). A symbol literal
in Scala begins with a tick mark and ends at the first non-identifier character. Thus,
'traversableAgain
results in a Symbol
object at runtime, as does
'completed
and 'file
. Here's an example:
iter shouldBe 'traversableAgain
Given this code, ScalaTest will use reflection to look on the object referenced from
emptySet
for a method that takes no parameters and results in Boolean
,
with either the name empty
or isEmpty
. If found, it will invoke
that method. If the method returns true
, execution will continue. But if it returns
false
, a TestFailedException
will be thrown that will contain a detail message, such as:
non-empty iterator was not traversableAgain
This be
syntax can be used with any reference (AnyRef
) type. If the object does
not have an appropriately named predicate method, you'll get a TestFailedException
at runtime with a detailed message that explains the problem.
(For the details on how a field or method is selected during this
process, see the documentation for BeWord
.)
If you think it reads better, you can optionally put a
or an
after
be
. For example, java.io.File
has two predicate methods,
isFile
and isDirectory
. Thus with a File
object
named temp
, you could write:
temp should be a 'file
Or, given java.awt.event.KeyEvent
has a method isActionKey
that takes
no arguments and returns Boolean
, you could assert that a KeyEvent
is
an action key with:
keyEvent should be an 'actionKey
If you prefer to check Boolean
properties in a type-safe manner, you can use a BePropertyMatcher
.
This would allow you to write expressions such as:
xs shouldBe traversableAgain
temp should be a file
keyEvent should be an actionKey
These expressions would fail to compile if should
is used on an inappropriate type, as determined
by the type parameter of the BePropertyMatcher
being used. (For example, file
in this example
would likely be of type BePropertyMatcher[java.io.File]
. If used with an appropriate type, such an expression will compile
and at run time the Boolean
property method or field will be accessed directly; i.e., no reflection will be used.
See the documentation for BePropertyMatcher
for more information.
Using custom BeMatchers
If you want to create a new way of using be
, which doesn't map to an actual property on the
type you care about, you can create a BeMatcher
. You could use this, for example, to create BeMatcher[Int]
called odd
, which would match any odd Int
, and even
, which would match
any even Int
.
Given this pair of BeMatcher
s, you could check whether an Int
was odd or even with expressions like:
num shouldBe odd
num should not be even
For more information, see the documentation for BeMatcher
.
Checking object identity
If you need to check that two references refer to the exact same object, you can write:
ref1 should be theSameInstanceAs ref2
Checking an object's class
If you need to check that an object is an instance of a particular class or trait, you can supply the type to
“be
a
” or “be
an
”:
result1 shouldBe a [Tiger]
result1 should not be an [Orangutan]
Because type parameters are erased on the JVM, we recommend you insert an underscore for any type parameters
when using this syntax. Both of the following test only that the result is an instance of List[_]
, because at
runtime the type parameter has been erased:
result shouldBe a [List[_]]
result shouldBe a [List[Fruit]]
Checking numbers against a range
Often you may want to check whether a number is within a
range. You can do that using the +-
operator, like this:
sevenDotOh should equal (6.9 +- 0.2)
sevenDotOh should === (6.9 +- 0.2)
sevenDotOh should be (6.9 +- 0.2)
sevenDotOh shouldEqual 6.9 +- 0.2
sevenDotOh shouldBe 6.9 +- 0.2
Any of these expressions will cause a TestFailedException
to be thrown if the floating point
value, sevenDotOh
is outside the range 6.7
to 7.1
.
You can use +-
with any type T
for which an implicit Numeric[T]
exists, such as integral types:
seven should equal (6 +- 2)
seven should === (6 +- 2)
seven should be (6 +- 2)
seven shouldEqual 6 +- 2
seven shouldBe 6 +- 2
Checking for emptiness
You can check whether an object is "empty", like this:
traversable shouldBe empty
javaMap should not be empty
The empty
token can be used with any type L
for which an implicit Emptiness[L]
exists.
The Emptiness
companion object provides implicits for GenTraversable[E]
, java.util.Collection[E]
,
java.util.Map[K, V]
, String
, Array[E]
, and Option[E]
. In addition, the
Emptiness
companion object provides structural implicits for types that declare an isEmpty
method that
returns a Boolean
. Here are some examples:
scala> import org.scalatest.matchers.should.Matchers._
import org.scalatest.matchers.should.Matchers._
scala> List.empty shouldBe empty
scala> None shouldBe empty
scala> Some(1) should not be empty
scala> "" shouldBe empty
scala> new java.util.HashMap[Int, Int] shouldBe empty
scala> new { def isEmpty = true} shouldBe empty
scala> Array(1, 2, 3) should not be empty
Working with "containers"
You can check whether a collection contains a particular element like this:
traversable should contain ("five")
The contain
syntax shown above can be used with any type C
that has a "containing" nature, evidenced by
an implicit org.scalatest.enablers.Containing[L]
, where L
is left-hand type on
which should
is invoked. In the Containing
companion object, implicits are provided for types GenTraversable[E]
, java.util.Collection[E]
,
java.util.Map[K, V]
, String
, Array[E]
, and Option[E]
.
Here are some examples:
scala> import org.scalatest.matchers.should.Matchers._
import org.scalatest.matchers.should.Matchers._
scala> List(1, 2, 3) should contain (2)
scala> Map('a' -> 1, 'b' -> 2, 'c' -> 3) should contain ('b' -> 2)
scala> Set(1, 2, 3) should contain (2)
scala> Array(1, 2, 3) should contain (2)
scala> "123" should contain ('2')
scala> Some(2) should contain (2)
ScalaTest's implicit methods that provide the Containing[L]
type classes require an Equality[E]
, where
E
is an element type. For example, to obtain a Containing[Array[Int]]
you must supply an Equality[Int]
,
either implicitly or explicitly. The contain
syntax uses this Equality[E]
to determine containership.
Thus if you want to change how containership is determined for an element type E
, place an implicit Equality[E]
in scope or use the explicitly DSL. Although the implicit parameter required for the contain
syntax is of type Containing[L]
,
implicit conversions are provided in the Containing
companion object from Equality[E]
to the various
types of containers of E
. Here's an example:
scala> import org.scalatest.matchers.should.Matchers._
import org.scalatest.matchers.should.Matchers._
scala> List("Hi", "Di", "Ho") should contain ("ho")
org.scalatest.exceptions.TestFailedException: List(Hi, Di, Ho) did not contain element "ho"
at ...
scala> import org.scalactic.Explicitly._
import org.scalactic.Explicitly._
scala> import org.scalactic.StringNormalizations._
import org.scalactic.StringNormalizations._
scala> (List("Hi", "Di", "Ho") should contain ("ho")) (after being lowerCased)
Note that when you use the explicitly DSL with contain
you need to wrap the entire
contain
expression in parentheses, as shown here.
(List("Hi", "Di", "Ho") should contain ("ho")) (after being lowerCased)
^ ^
In addition to determining whether an object contains another object, you can use contain
to
make other determinations.
For example, the contain
oneOf
syntax ensures that one and only one of the specified elements are
contained in the containing object:
List(1, 2, 3, 4, 5) should contain oneOf (5, 7, 9)
Some(7) should contain oneOf (5, 7, 9)
"howdy" should contain oneOf ('a', 'b', 'c', 'd')
Note that if multiple specified elements appear in the containing object, oneOf
will fail:
scala> List(1, 2, 3) should contain oneOf (2, 3, 4)
org.scalatest.exceptions.TestFailedException: List(1, 2, 3) did not contain one of (2, 3, 4)
at ...
If you really want to ensure one or more of the specified elements are contained in the containing object,
use atLeastOneOf
, described below, instead of oneOf
. Keep in mind, oneOf
means "exactly one of."
Note also that with any contain
syntax, you can place custom implicit Equality[E]
instances in scope
to customize how containership is determined, or use the explicitly DSL. Here's an example:
(Array("Doe", "Ray", "Me") should contain oneOf ("X", "RAY", "BEAM")) (after being lowerCased)
The contain
noneOf
syntax does the opposite of oneOf
: it ensures none of the specified elements
are contained in the containing object:
List(1, 2, 3, 4, 5) should contain noneOf (7, 8, 9)
Some(0) should contain noneOf (7, 8, 9)
"12345" should contain noneOf ('7', '8', '9')
Working with "aggregations"
As mentioned, the "contain
," "contain
oneOf
," and "contain
noneOf
" syntax requires a
Containing[L]
be provided, where L
is the left-hand type. Other contain
syntax, which
will be described in this section, requires an Aggregating[L]
be provided, where again L
is the left-hand type.
(An Aggregating[L]
instance defines the "aggregating nature" of a type L
.)
The reason, essentially, is that contain
syntax that makes sense for Option
is enabled by
Containing[L]
, whereas syntax that does not make sense for Option
is enabled
by Aggregating[L]
. For example, it doesn't make sense to assert that an Option[Int]
contains all of a set of integers, as it
could only ever contain one of them. But this does make sense for a type such as List[Int]
that can aggregate zero to many integers.
The Aggregating
companion object provides implicit instances of Aggregating[L]
for types GenTraversable[E]
, java.util.Collection[E]
,
java.util.Map[K, V]
, String
, Array[E]
. Note that these are the same types as are supported with
Containing
, but with Option[E]
missing.
Here are some examples:
The contain
atLeastOneOf
syntax, for example, works for any type L
for which an Aggregating[L]
exists. It ensures
that at least one of (i.e., one or more of) the specified objects are contained in the containing object:
List(1, 2, 3) should contain atLeastOneOf (2, 3, 4)
Array(1, 2, 3) should contain atLeastOneOf (3, 4, 5)
"abc" should contain atLeastOneOf ('c', 'a', 't')
Similar to Containing[L]
, the implicit methods that provide the Aggregating[L]
instances require an Equality[E]
, where
E
is an element type. For example, to obtain a Aggregating[Vector[String]]
you must supply an Equality[String]
,
either implicitly or explicitly. The contain
syntax uses this Equality[E]
to determine containership.
Thus if you want to change how containership is determined for an element type E
, place an implicit Equality[E]
in scope or use the explicitly DSL. Although the implicit parameter required for the contain
syntax is of type Aggregating[L]
,
implicit conversions are provided in the Aggregating
companion object from Equality[E]
to the various
types of aggregations of E
. Here's an example:
(Vector(" A", "B ") should contain atLeastOneOf ("a ", "b", "c")) (after being lowerCased and trimmed)
The "contain
atMostOneOf
" syntax lets you specify a set of objects at most one of which should be contained in the containing object:
List(1, 2, 3, 4, 5) should contain atMostOneOf (5, 6, 7)
The "contain
allOf
" syntax lets you specify a set of objects that should all be contained in the containing object:
List(1, 2, 3, 4, 5) should contain allOf (2, 3, 5)
The "contain
only
" syntax lets you assert that the containing object contains only the specified objects, though it may
contain more than one of each:
List(1, 2, 3, 2, 1) should contain only (1, 2, 3)
The "contain
theSameElementsAs
" and "contain
theSameElementsInOrderAs
syntax differ from the others
in that the right hand side is a GenTraversable[_]
rather than a varargs of Any
. (Note: in a future release,
these will likely be widened to accept any type R
for which an Aggregating[R]
exists.)
The "contain
theSameElementsAs
" syntax lets you assert that two aggregations contain the same objects:
List(1, 2, 2, 3, 3, 3) should contain theSameElementsAs Vector(3, 2, 3, 1, 2, 3)
The number of times any family of equal objects appears must also be the same in both the left and right aggregations.
The specified objects may appear multiple times, but must appear in the order they appear in the right-hand list. For example, if
the last 3 element is left out of the right-hand list in the previous example, the expression would fail because the left side
has three 3's and the right hand side has only two:
List(1, 2, 2, 3, 3, 3) should contain theSameElementsAs Vector(3, 2, 3, 1, 2)
org.scalatest.exceptions.TestFailedException: List(1, 2, 2, 3, 3, 3) did not contain the same
elements as Vector(3, 2, 3, 1, 2)
at ...
Working with "sequences"
The rest of the contain
syntax, which
will be described in this section, requires a Sequencing[L]
be provided, where again L
is the left-hand type.
(A Sequencing[L]
instance defines the "sequencing nature" of a type L
.)
The reason, essentially, is that contain
syntax that implies an "order" of elements makes sense only for types that place elements in a sequence.
For example, it doesn't make sense to assert that a Map[String, Int]
or Set[Int]
contains all of a set of integers in a particular
order, as these types don't necessarily define an order for their elements. But this does make sense for a type such as Seq[Int]
that does define
an order for its elements.
The Sequencing
companion object provides implicit instances of Sequencing[L]
for types GenSeq[E]
, java.util.List[E]
,
String
, and Array[E]
.
Here are some examples:
Similar to Containing[L]
, the implicit methods that provide the Aggregating[L]
instances require an Equality[E]
, where
E
is an element type. For example, to obtain a Aggregating[Vector[String]]
you must supply an Equality[String]
,
either implicitly or explicitly. The contain
syntax uses this Equality[E]
to determine containership.
Thus if you want to change how containership is determined for an element type E
, place an implicit Equality[E]
in scope or use the explicitly DSL. Although the implicit parameter required for the contain
syntax is of type Aggregating[L]
,
implicit conversions are provided in the Aggregating
companion object from Equality[E]
to the various
types of aggregations of E
. Here's an example:
The "contain
inOrderOnly
" syntax lets you assert that the containing object contains only the specified objects, in order.
The specified objects may appear multiple times, but must appear in the order they appear in the right-hand list. Here's an example:
List(1, 2, 2, 3, 3, 3) should contain inOrderOnly (1, 2, 3)
The "contain
inOrder
" syntax lets you assert that the containing object contains only the specified objects in order, like
inOrderOnly
, but allows other objects to appear in the left-hand aggregation as well:
contain more than one of each:
List(0, 1, 2, 2, 99, 3, 3, 3, 5) should contain inOrder (1, 2, 3)
Note that "order" in inOrder
, inOrderOnly
, and theSameElementsInOrderAs
(described below)
in the Aggregation[L]
instances built-in to ScalaTest is defined as "iteration order".
Lastly, the "contain
theSameElementsInOrderAs
" syntax lets you assert that two aggregations contain
the same exact elements in the same (iteration) order:
List(1, 2, 3) should contain theSameElementsInOrderAs collection.mutable.TreeSet(3, 2, 1)
The previous assertion succeeds because the iteration order of aTreeSet
is the natural
ordering of its elements, which in this case is 1, 2, 3. An iterator obtained from the left-hand List
will produce the same elements
in the same order.
Working with "sortables"
You can also ask whether the elements of "sortable" objects (such as Array
s, Java List
s, and GenSeq
s)
are in sorted order, like this:
List(1, 2, 3) shouldBe sorted
Working with iterators
Although it seems desirable to provide similar matcher syntax for Scala and Java iterators to that provided for sequences like
Seq
s, Array
, and java.util.List
, the
ephemeral nature of iterators makes this problematic. Some syntax (such as should
contain
) is relatively straightforward to
support on iterators, but other syntax (such
as, for example, Inspector
expressions on nested iterators) is not. Rather
than allowing inconsistencies between sequences and iterators in the API, we chose to not support any such syntax directly on iterators:
scala> val it = List(1, 2, 3).iterator
it: Iterator[Int] = non-empty iterator
scala> it should contain (2)
<console>:15: error: could not find implicit value for parameter
typeClass1: org.scalatest.enablers.Containing[Iterator[Int]]
it should contain (2)
^
Instead, you will need to convert your iterators to a sequence explicitly before using them in matcher expressions:
scala> it.toStream should contain (2)
We recommend you convert (Scala or Java) iterators to Stream
s, as shown in the previous example, so that you can
continue to reap any potential benefits provided by the laziness of the underlying iterator.
Inspector shorthands
You can use the Inspectors
syntax with matchers as well as assertions. If you have a multi-dimensional collection, such as a
list of lists, using Inspectors
is your best option:
val yss =
List(
List(1, 2, 3),
List(1, 2, 3),
List(1, 2, 3)
)
forAll (yss) { ys =>
forAll (ys) { y => y should be > 0 }
}
For assertions on one-dimensional collections, however, matchers provides "inspector shorthands." Instead of writing:
val xs = List(1, 2, 3)
forAll (xs) { x => x should be < 10 }
You can write:
all (xs) should be < 10
The previous statement asserts that all elements of the xs
list should be less than 10.
All of the inspectors have shorthands in matchers. Here is the full list:
all
- succeeds if the assertion holds true for every elementatLeast
- succeeds if the assertion holds true for at least the specified number of elementsatMost
- succeeds if the assertion holds true for at most the specified number of elementsbetween
- succeeds if the assertion holds true for between the specified minimum and maximum number of elements, inclusiveevery
- same as all
, but lists all failing elements if it fails (whereas all
just reports the first failing element)exactly
- succeeds if the assertion holds true for exactly the specified number of elements
Here are some examples:
scala> import org.scalatest.matchers.should.Matchers._
import org.scalatest.matchers.should.Matchers._
scala> val xs = List(1, 2, 3, 4, 5)
xs: List[Int] = List(1, 2, 3, 4, 5)
scala> all (xs) should be > 0
scala> atMost(2, xs) should be >= 4
scala> atLeast(3, xs) should be < 5
scala> between(2, 3, xs) should (be > 1 and be < 5)
scala> exactly (2, xs) should be <= 2
scala> every (xs) should be < 10
scala> // And one that fails...
scala> exactly (2, xs) shouldEqual 2
org.scalatest.exceptions.TestFailedException: 'exactly(2)' inspection failed, because only 1 element
satisfied the assertion block at index 1:
at index 0, 1 did not equal 2,
at index 2, 3 did not equal 2,
at index 3, 4 did not equal 2,
at index 4, 5 did not equal 2
in List(1, 2, 3, 4, 5)
at ...
Note: in the current release, the type of object used with inspector shorthands must be GenTraversable
, but this will likely be widened to
include Java collections and iterators for future release.
Single-element collections
To assert both that a collection contains just one "lone" element as well as something else about that element, you can use
the loneElement
syntax. For example, if a Set[Int]
should contain just one element, an Int
less than or equal to 10, you could write:
set.loneElement should be <= 10
You can invoke loneElement
on any type T
for which an implicit Collecting[E, T]
is available, where E
is the type returned by the loneElement
invocation.
Java collections and maps
You can use similar syntax on Java collections (java.util.Collection
) and maps (java.util.Map
).
For example, you can check whether a Java Collection
or Map
is empty
,
like this:
javaCollection should be ('empty)
javaMap should be ('empty)
Even though Java's List
type doesn't actually have a length
or getLength
method,
you can nevertheless check the length of a Java List
(java.util.List
) like this:
javaList should have length 9
You can check the size of any Java Collection
or Map
, like this:
javaMap should have size 20
javaSet should have size 90
In addition, you can check whether a Java Collection
contains a particular
element, like this:
javaCollection should contain ("five")
One difference to note between the syntax supported on Java and Scala collections is that
in Java, Map
is not a subtype of Collection
, and does not
actually define an element type. You can ask a Java Map
for an "entry set"
via the entrySet
method, which will return the Map
's key/value pairs
wrapped in a set of java.util.Map.Entry
, but a Map
is not actually
a collection of Entry
. To make Java Map
s easier to work with, however,
ScalaTest matchers allows you to treat a Java Map
as a collection of Entry
,
and defines a convenience implementation of java.util.Map.Entry
in
org.scalatest.Entry
. Here's how you use it:
javaMap should contain (Entry(2, 3))
javaMap should contain oneOf (Entry(2, 3), Entry(3, 4))
You can also just check whether a Java Map
contains a particular key, or value, like this:
javaMap should contain key 1
javaMap should contain value "Howdy"
String
s and Array
s as collections
You can also use all the syntax described above for Scala and Java collections on Array
s and
String
s. Here are some examples:
scala> import org.scalatest._
import org.scalatest._
scala> import matchers.should.Matchers._
import matchers.should.Matchers._
scala> atLeast (2, Array(1, 2, 3)) should be > 1
scala> atMost (2, "halloo") shouldBe 'o'
scala> Array(1, 2, 3) shouldBe sorted
scala> "abcdefg" shouldBe sorted
scala> Array(1, 2, 3) should contain atMostOneOf (3, 4, 5)
scala> "abc" should contain atMostOneOf ('c', 'd', 'e')
be
as an equality comparison
All uses of be
other than those shown previously perform an equality comparison. They work
the same as equal
when it is used with default equality. This redundancy between be
and equals
exists in part
because it enables syntax that sometimes sounds more natural. For example, instead of writing:
result should equal (null)
You can write:
result should be (null)
(Hopefully you won't write that too much given null
is error prone, and Option
is usually a better, well, option.)
Here are some other examples of be
used for equality comparison:
sum should be (7.0)
boring should be (false)
fun should be (true)
list should be (Nil)
option should be (None)
option should be (Some(1))
As with equal
used with default equality, using be
on arrays results in deep
being called on both arrays prior to
calling equal
. As a result,
the following expression would not throw a TestFailedException
:
Array(1, 2) should be (Array(1, 2))
Because be
is used in several ways in ScalaTest matcher syntax, just as it is used in many ways in English, one
potential point of confusion in the event of a failure is determining whether be
was being used as an equality comparison or
in some other way, such as a property assertion. To make it more obvious when be
is being used for equality, the failure
messages generated for those equality checks will include the word equal
in them. For example, if this expression fails with a
TestFailedException
:
option should be (Some(1))
The detail message in that TestFailedException
will include the words "equal to"
to signify be
was in this case being used for equality comparison:
Some(2) was not equal to Some(1)
Being negative
If you wish to check the opposite of some condition, you can simply insert not
in the expression.
Here are a few examples:
result should not be (null)
sum should not be <= (10)
mylist should not equal (yourList)
string should not startWith ("Hello")
Checking that a snippet of code does not compile
Often when creating libraries you may wish to ensure that certain arrangements of code that
represent potential “user errors” do not compile, so that your library is more error resistant.
ScalaTest should.Matchers
trait includes the following syntax for that purpose:
"val a: String = 1" shouldNot compile
If you want to ensure that a snippet of code does not compile because of a type error (as opposed
to a syntax error), use:
"val a: String = 1" shouldNot typeCheck
Note that the shouldNot
typeCheck
syntax will only succeed if the given snippet of code does not
compile because of a type error. A syntax error will still result in a thrown TestFailedException
.
If you want to state that a snippet of code does compile, you can make that
more obvious with:
"val a: Int = 1" should compile
Although the previous three constructs are implemented with macros that determine at compile time whether
the snippet of code represented by the string does or does not compile, errors
are reported as test failures at runtime.
Logical expressions with and
and or
You can also combine matcher expressions with and
and/or or
, however,
you must place parentheses or curly braces around the and
or or
expression. For example,
this and
-expression would not compile, because the parentheses are missing:
map should contain key ("two") and not contain value (7)
Instead, you need to write:
map should (contain key ("two") and not contain value (7))
Here are some more examples:
number should (be > (0) and be <= (10))
option should (equal (Some(List(1, 2, 3))) or be (None))
string should (
equal ("fee") or
equal ("fie") or
equal ("foe") or
equal ("fum")
)
Two differences exist between expressions composed of these and
and or
operators and the expressions you can write
on regular Boolean
s using its &&
and ||
operators. First, expressions with and
and or
do not short-circuit. The following contrived expression, for example, would print "hello, world!"
:
"yellow" should (equal ("blue") and equal { println("hello, world!"); "green" })
In other words, the entire and
or or
expression is always evaluated, so you'll see any side effects
of the right-hand side even if evaluating
only the left-hand side is enough to determine the ultimate result of the larger expression. Failure messages produced by these
expressions will "short-circuit," however,
mentioning only the left-hand side if that's enough to determine the result of the entire expression. This "short-circuiting" behavior
of failure messages is intended
to make it easier and quicker for you to ascertain which part of the expression caused the failure. The failure message for the previous
expression, for example, would be:
"yellow" did not equal "blue"
Most likely this lack of short-circuiting would rarely be noticeable, because evaluating the right hand side will usually not
involve a side effect. One situation where it might show up, however, is if you attempt to and
a null
check on a variable with an expression
that uses the variable, like this:
map should (not be (null) and contain key ("ouch"))
If map
is null
, the test will indeed fail, but with a NullPointerException
, not a
TestFailedException
. Here, the NullPointerException
is the visible right-hand side effect. To get a
TestFailedException
, you would need to check each assertion separately:
map should not be (null)
map should contain key ("ouch")
If map
is null
in this case, the null
check in the first expression will fail with
a TestFailedException
, and the second expression will never be executed.
The other difference with Boolean
operators is that although &&
has a higher precedence than ||
,
and
and or
have the same precedence. Thus although the Boolean
expression (a || b && c)
will evaluate the &&
expression
before the ||
expression, like (a || (b && c))
, the following expression:
traversable should (contain (7) or contain (8) and have size (9))
Will evaluate left to right, as:
traversable should ((contain (7) or contain (8)) and have size (9))
If you really want the and
part to be evaluated first, you'll need to put in parentheses, like this:
traversable should (contain (7) or (contain (8) and have size (9)))
Working with Option
s
You can work with options using ScalaTest's equality, empty
,
defined
, and contain
syntax.
For example, if you wish to check
whether an option is None
, you can write any of:
option shouldEqual None
option shouldBe None
option should === (None)
option shouldBe empty
If you wish to check an option is defined, and holds a specific value, you can write any of:
option shouldEqual Some("hi")
option shouldBe Some("hi")
option should === (Some("hi"))
If you only wish to check that an option is defined, but don't care what it's value is, you can write:
option shouldBe defined
If you mix in (or import the members of) OptionValues
,
you can write one statement that indicates you believe an option should be defined and then say something else about its value. Here's an example:
import org.scalatest.OptionValues._
option.value should be < (7)
As mentioned previously, you can use also use ScalaTest's contain
, contain oneOf
, and
contain noneOf
syntax with options:
Some(2) should contain (2)
Some(7) should contain oneOf (5, 7, 9)
Some(0) should contain noneOf (7, 8, 9)
Checking arbitrary properties with have
Using have
, you can check properties of any type, where a property is an attribute of any
object that can be retrieved either by a public field, method, or JavaBean-style get
or is
method, like this:
book should have (
'title ("Programming in Scala"),
'author (List("Odersky", "Spoon", "Venners")),
'pubYear (2008)
)
This expression will use reflection to ensure the title
, author
, and pubYear
properties of object book
are equal to the specified values. For example, it will ensure that book
has either a public Java field or method
named title
, or a public method named getTitle
, that when invoked (or accessed in the field case) results
in the string "Programming in Scala"
. If all specified properties exist and have their expected values, respectively,
execution will continue. If one or more of the properties either does not exist, or exists but results in an unexpected value,
a TestFailedException
will be thrown that explains the problem. (For the details on how a field or method is selected during this
process, see the documentation for HavePropertyMatcherGenerator
.)
When you use this syntax, you must place one or more property values in parentheses after have
, separated by commas, where a property
value is a symbol indicating the name of the property followed by the expected value in parentheses. The only exceptions to this rule is the syntax
for checking size and length shown previously, which does not require parentheses. If you forget and put parentheses in, however, everything will
still work as you'd expect. Thus instead of writing:
array should have length (3)
set should have size (90)
You can alternatively, write:
array should have (length (3))
set should have (size (90))
If a property has a value different from the specified expected value, a TestFailedError
will be thrown
with a detailed message that explains the problem. For example, if you assert the following on
a book
whose title is Moby Dick
:
book should have ('title ("A Tale of Two Cities"))
You'll get a TestFailedException
with this detail message:
The title property had value "Moby Dick", instead of its expected value "A Tale of Two Cities",
on object Book("Moby Dick", "Melville", 1851)
If you prefer to check properties in a type-safe manner, you can use a HavePropertyMatcher
.
This would allow you to write expressions such as:
book should have (
title ("Programming in Scala"),
author (List("Odersky", "Spoon", "Venners")),
pubYear (2008)
)
These expressions would fail to compile if should
is used on an inappropriate type, as determined
by the type parameter of the HavePropertyMatcher
being used. (For example, title
in this example
might be of type HavePropertyMatcher[org.publiclibrary.Book]
. If used with an appropriate type, such an expression will compile
and at run time the property method or field will be accessed directly; i.e., no reflection will be used.
See the documentation for HavePropertyMatcher
for more information.
Using length
and size
with HavePropertyMatcher
s
If you want to use length
or size
syntax with your own custom HavePropertyMatcher
s, you
can do so, but you must write (of [“the type”])
afterwords. For example, you could write:
book should have (
title ("A Tale of Two Cities"),
length (220) (of [Book]),
author ("Dickens")
)
Prior to ScalaTest 2.0, “length
(22)
” yielded a HavePropertyMatcher[Any, Int]
that used reflection to dynamically look
for a length
field or getLength
method. In ScalaTest 2.0, “length
(22)
” yields a
MatcherFactory1[Any, Length]
, so it is no longer a HavePropertyMatcher
. The (of [<type>])
syntax converts the
the MatcherFactory1[Any, Length]
to a HavePropertyMatcher[<type>, Int]
.
Checking that an expression matches a pattern
ScalaTest's Inside
trait allows you to make assertions after a pattern match.
Here's an example:
case class Name(first: String, middle: String, last: String)
val name = Name("Jane", "Q", "Programmer")
inside(name) { case Name(first, _, _) =>
first should startWith ("S")
}
You can use inside
to just ensure a pattern is matched, without making any further assertions, but a better
alternative for that kind of assertion is matchPattern
. The matchPattern
syntax allows you
to express that you expect a value to match a particular pattern, no more and no less:
name should matchPattern { case Name("Sarah", _, _) => }
Using custom matchers
If none of the built-in matcher syntax (or options shown so far for extending the syntax) satisfies a particular need you have, you can create
custom Matcher
s that allow
you to place your own syntax directly after should
. For example, although you can ensure that a java.io.File
has a name
that ends with a particular extension like this:
file.getName should endWith (".txt")
You might prefer
to create a custom Matcher[java.io.File]
named endWithExtension
, so you could write expressions like:
file should endWithExtension ("txt")
file should not endWithExtension "txt"
file should (exist and endWithExtension ("txt"))
One good way to organize custom matchers is to place them inside one or more
traits that you can then mix into the suites that need them. Here's an example:
import org.scalatest._
import matchers._
trait CustomMatchers {
class FileEndsWithExtensionMatcher(expectedExtension: String) extends Matcher[java.io.File] {
def apply(left: java.io.File) = {
val name = left.getName
MatchResult(
name.endsWith(expectedExtension),
s"""File $name did not end with extension "$expectedExtension"""",
s"""File $name ended with extension "$expectedExtension""""
)
}
}
def endWithExtension(expectedExtension: String) = new FileEndsWithExtensionMatcher(expectedExtension)
}
object CustomMatchers extends CustomMatchers
Note: the CustomMatchers
companion object exists to make it easy to bring the
matchers defined in this trait into scope via importing, instead of mixing in the trait. The ability
to import them is useful, for example, when you want to use the matchers defined in a trait in the Scala interpreter console.
This trait contains one matcher class, FileEndsWithExtensionMatcher
, and a def
named endWithExtension
that returns a new
instance of FileEndsWithExtensionMatcher
. Because the class extends Matcher[java.io.File]
,
the compiler will only allow it be used to match against instances of java.io.File
. A matcher must declare an
apply
method that takes the type decared in Matcher
's type parameter, in this case java.io.File
.
The apply method will return a MatchResult
whose matches
field will indicate whether the match succeeded.
The failureMessage
field will provide a programmer-friendly error message indicating, in the event of a match failure, what caused
the match to fail.
The FileEndsWithExtensionMatcher
matcher in this example determines success by determining if the passed java.io.File
ends with
the desired extension. It does this in the first argument passed to the MatchResult
factory method:
name.endsWith(expectedExtension)
In other words, if the file name has the expected extension, this matcher matches.
The next argument to MatchResult
's factory method produces the failure message string:
s"""File $name did not end with extension "$expectedExtension"""",
For example, consider this matcher expression:
import org.scalatest._
import matchers.should.Matchers._
import java.io.File
import CustomMatchers._
new File("essay.text") should endWithExtension ("txt")
Because the passed java.io.File
has the name essay.text
, but the expected extension is "txt"
, the failure
message would be:
File essay.text did not have extension "txt"
For more information on the fields in a MatchResult
, including the subsequent field (or fields) that follow the failure message,
please see the documentation for MatchResult
.
Creating dynamic matchers
There are other ways to create new matchers besides defining one as shown above. For example, you might check that a file is hidden like this:
new File("secret.txt") should be ('hidden)
If you wanted to get rid of the tick mark, you could simply define hidden
like this:
val hidden = 'hidden
Now you can check that a file is hidden without the tick mark:
new File("secret.txt") should be (hidden)
You could get rid of the parens by using shouldBe
:
new File("secret.txt") shouldBe hidden
Creating matchers using logical operators
You can also use ScalaTest matchers' logical operators to combine existing matchers into new ones, like this:
val beWithinTolerance = be >= 0 and be <= 10
Now you could check that a number is within the tolerance (in this case, between 0 and 10, inclusive), like this:
num should beWithinTolerance
When defining a full blown matcher, one shorthand is to use one of the factory methods in Matcher
's companion
object. For example, instead of writing this:
val beOdd =
new Matcher[Int] {
def apply(left: Int) =
MatchResult(
left % 2 == 1,
left + " was not odd",
left + " was odd"
)
}
You could alternately write this:
val beOdd =
Matcher { (left: Int) =>
MatchResult(
left % 2 == 1,
left + " was not odd",
left + " was odd"
)
}
Either way you define the beOdd
matcher, you could use it like this:
3 should beOdd
4 should not (beOdd)
Composing matchers
You can also compose matchers. For example, the endWithExtension
matcher from the example above
can be more easily created by composing a function with the existing endWith
matcher:
scala> import org.scalatest._
import org.scalatest._
scala> import matchers.should.Matchers._
import matchers.should.Matchers._
scala> import java.io.File
import java.io.File
scala> def endWithExtension(ext: String) = endWith(ext) compose { (f: File) => f.getPath }
endWithExtension: (ext: String)org.scalatest.matchers.Matcher[java.io.File]
Now you have a Matcher[File]
whose apply
method first
invokes the converter function to convert the passed File
to a String
,
then passes the resulting String
to endWith
. Thus, you could use this version
endWithExtension
like the previous one:
scala> new File("output.txt") should endWithExtension("txt")
In addition, by composing twice, you can modify the type of both sides of a match statement
with the same function, like this:
scala> val f = be > (_: Int)
f: Int => org.scalatest.matchers.Matcher[Int] = <function1>
scala> val g = (_: String).toInt
g: String => Int = <function1>
scala> val beAsIntsGreaterThan = (f compose g) andThen (_ compose g)
beAsIntsGreaterThan: String => org.scalatest.matchers.Matcher[String] = <function1>
scala> "8" should beAsIntsGreaterThan ("7")
At this point, however, the error message for the beAsIntsGreaterThan
gives no hint that the Int
s being compared were parsed from String
s:
scala> "7" should beAsIntsGreaterThan ("8")
org.scalatest.exceptions.TestFailedException: 7 was not greater than 8
To modify error message, you can use trait MatcherProducers
, which
also provides a composeTwice
method that performs the compose
...
andThen
... compose
operation:
scala> import matchers._
import matchers._
scala> import MatcherProducers._
import MatcherProducers._
scala> val beAsIntsGreaterThan = f composeTwice g // means: (f compose g) andThen (_ compose g)
beAsIntsGreaterThan: String => org.scalatest.matchers.Matcher[String] = <function1>
scala> "8" should beAsIntsGreaterThan ("7")
Of course, the error messages is still the same:
scala> "7" should beAsIntsGreaterThan ("8")
org.scalatest.exceptions.TestFailedException: 7 was not greater than 8
To modify the error messages, you can use mapResult
from MatcherProducers
. Here's an example:
scala> val beAsIntsGreaterThan =
f composeTwice g mapResult { mr =>
mr.copy(
failureMessageArgs =
mr.failureMessageArgs.map((LazyArg(_) { "\"" + _.toString + "\".toInt"})),
negatedFailureMessageArgs =
mr.negatedFailureMessageArgs.map((LazyArg(_) { "\"" + _.toString + "\".toInt"})),
midSentenceFailureMessageArgs =
mr.midSentenceFailureMessageArgs.map((LazyArg(_) { "\"" + _.toString + "\".toInt"})),
midSentenceNegatedFailureMessageArgs =
mr.midSentenceNegatedFailureMessageArgs.map((LazyArg(_) { "\"" + _.toString + "\".toInt"}))
)
}
beAsIntsGreaterThan: String => org.scalatest.matchers.Matcher[String] = <function1>
The mapResult
method takes a function that accepts a MatchResult
and produces a new
MatchResult
, which can contain modified arguments and modified error messages. In this example,
the error messages are being modified by wrapping the old arguments in LazyArg
instances that lazily apply the given prettification functions to the toString
result of the old args.
Now the error message is clearer:
scala> "7" should beAsIntsGreaterThan ("8")
org.scalatest.exceptions.TestFailedException: "7".toInt was not greater than "8".toInt
Checking for expected exceptions
Sometimes you need to test whether a method throws an expected exception under certain circumstances, such
as when invalid arguments are passed to the method. With should.Matchers
mixed in, you can
check for an expected exception like this:
an [IndexOutOfBoundsException] should be thrownBy s.charAt(-1)
If charAt
throws an instance of StringIndexOutOfBoundsException
,
this expression will result in that exception. But if charAt
completes normally, or throws a different
exception, this expression will complete abruptly with a TestFailedException
.
If you need to further inspect an expected exception, you can capture it using this syntax:
val thrown = the [IndexOutOfBoundsException] thrownBy s.charAt(-1)
This expression returns the caught exception so that you can inspect it further if you wish, for
example, to ensure that data contained inside the exception has the expected values. Here's an
example:
thrown.getMessage should equal ("String index out of range: -1")
If you prefer you can also capture and inspect an expected exception in one statement, like this:
the [ArithmeticException] thrownBy 1 / 0 should have message "/ by zero"
the [IndexOutOfBoundsException] thrownBy {
s.charAt(-1)
} should have message "String index out of range: -1"
You can also state that no exception should be thrown by some code, like this:
noException should be thrownBy 0 / 1
Those pesky parens
Perhaps the most tricky part of writing assertions using ScalaTest matchers is remembering
when you need or don't need parentheses, but bearing in mind a few simple rules should help.
It is also reassuring to know that if you ever leave off a set of parentheses when they are
required, your code will not compile. Thus the compiler will help you remember when you need the parens.
That said, the rules are:
1. Although you don't always need them, you can choose to always put parentheses
around right-hand values, such as the 7
in num should equal (7)
:
result should equal (4)
array should have length (3)
book should have (
'title ("Programming in Scala"),
'author (List("Odersky", "Spoon", "Venners")),
'pubYear (2008)
)
option should be ('defined)
catMap should (contain key (9) and contain value ("lives"))
keyEvent should be an ('actionKey)
javaSet should have size (90)
2. Except for length
and size
, you must always put parentheses around
the list of one or more property values following a have
:
file should (exist and have ('name ("temp.txt")))
book should have (
title ("Programming in Scala"),
author (List("Odersky", "Spoon", "Venners")),
pubYear (2008)
)
javaList should have length (9) // parens optional for length and size
3. You must always put parentheses around and
and or
expressions, as in:
catMap should (contain key (9) and contain value ("lives"))
number should (equal (2) or equal (4) or equal (8))
That's it. With a bit of practice it should become natural to you, and the compiler will always be there to tell you if you
forget a set of needed parentheses.
Next, learn about testing with mock objects.