Generator-driven property checks
To use generator-driven property checks, you must mix in trait org.scalatestplus.scalacheck.ScalaCheckPropertyChecks
(previously known as org.scalatest.props.PropertyChecks
). If you are also using table-driven property checks, you can mix in
trait org.scalatestplus.scalacheck.ScalaCheckPropertyChecks
,
which extends both ScalaCheckDrivenPropertyChecks
and
TableDrivenPropertyChecks
.
Generator-driven property checks uses ScalaCheck and
ScalaTest + ScalaCheck, so you must also include both jars on your classpath when
you compile and run your tests.
Trait ScalaCheckDrivenPropertyChecks
contains forAll
methods that provide various ways to check properties using
generated data.
It also contains a wherever
method that can be used to indicate a property need only hold whenever
some condition is true.
For an example of trait ScalaCheckDrivenPropertyChecks
in action, imagine you want to test this Fraction
class:
class Fraction(n: Int, d: Int) {
require(d != 0)
require(d != Integer.MIN_VALUE)
require(n != Integer.MIN_VALUE)
val numer = if (d < 0) -1 * n else n
val denom = d.abs
override def toString = numer + " / " + denom
}
To test the behavior of Fraction
, you could mix in or import the members of ScalaCheckDrivenPropertyChecks
(and should.Matchers
) and check a property using a forAll
method, like this:
forAll { (n: Int, d: Int) =>
whenever (d != 0 && d != Integer.MIN_VALUE
&& n != Integer.MIN_VALUE) {
val f = new Fraction(n, d)
if (n < 0 && d < 0 || n > 0 && d > 0)
f.numer should be > 0
else if (n != 0)
f.numer should be < 0
else
f.numer should be === 0
f.denom should be > 0
}
}
Trait ScalaCheckDrivenPropertyChecks
provides overloaded forAll
methods
that allow you to check properties using the data provided by a ScalaCheck generator. The simplest form
of forAll
method takes two parameter lists, the second of which is implicit. The first parameter list
is a "property" function with one to six parameters. An implicit Arbitrary
generator and Shrink
object needs to be supplied for
The forAll
method will pass each row of data to
each parameter type. ScalaCheck provides many implicit Arbitrary
generators for common types such as
Int
, String
, List[Float]
, etc., in its org.scalacheck.Arbitrary
companion
object. So long as you use types for which ScalaCheck already provides implicit Arbitrary
generators, you needn't
worry about them. Same for Shrink
objects, which are provided by ScalaCheck's org.scalacheck.Shrink
companion
object. Most often you can simply pass a property function to forAll
, and the compiler will grab the implicit
values provided by ScalaCheck.
The forAll
methods use the supplied Arbitrary
generators to generate example
arguments and pass them to the property function, and
generate a GeneratorDrivenPropertyCheckFailedException
if the function
completes abruptly for any exception that would normally cause a test to
fail in ScalaTest other than DiscardedEvaluationException
. An
DiscardedEvaluationException
,
which is thrown by the whenever
method (defined in trait Whenever
, which this trait extends) to indicate
a condition required by the property function is not met by a row
of passed data, will simply cause forAll
to discard that row of data.
Supplying argument names
You can optionally specify string names for the arguments passed to a property function, which will be used
in any error message when describing the argument values that caused the failure. To supply the names, place them in a comma separated list
in parentheses after forAll
before the property function (a curried form of forAll
). Here's
an example:
forAll ("a", "b") { (a: String, b: String) =>
a.length + b.length should equal ((a + b).length + 1)
}
When this fails, you'll see an error message that includes this:
Occurred when passed generated values (
a = "",
b = ""
)
When you don't supply argument names, the error message will say arg0
, arg1
, etc..
For example, this property check:
forAll { (a: String, b: String) =>
a.length + b.length should equal ((a + b).length + 1)
}
Will fail with an error message that includes:
Occurred when passed generated values (
arg0 = "",
arg1 = ""
)
Supplying generators
ScalaCheck provides a nice library of compositors that makes it easy to create your own custom generators. If you
want to supply custom generators to a property check, place them in parentheses after forAll
, before
the property check function (a curried form of forAll
).
For example, to create a generator of even integers between (and including) -2000 and 2000, you could write this:
import org.scalacheck.Gen
val evenInts = for (n <- Gen.choose(-1000, 1000)) yield 2 * n
Given this generator, you could use it on a property check like this:
forAll (evenInts) { (n) => n % 2 should equal (0) }
Custom generators are necessary when you want to pass data types not supported by ScalaCheck's arbitrary generators,
but are also useful when some of the values in the full range for the passed types are not valid. For such values you
would use a whenever
clause. In the Fraction
class shown above, neither the passed numerator or
denominator can be Integer.MIN_VALUE
, and the passed denominator cannot be zero. This shows up in the
whenever
clause like this:
whenever (d != 0 && d != Integer.MIN_VALUE
&& n != Integer.MIN_VALUE) { ...
You could in addition define generators for the numerator and denominator that only produce valid values, like this:
val validNumers =
for (n <- Gen.choose(Integer.MIN_VALUE + 1, Integer.MAX_VALUE)) yield n
val validDenoms =
for (d <- validNumers if d != 0) yield d
You could then use them in the property check like this:
forAll (validNumers, validDenoms) { (n: Int, d: Int) =>
whenever (d != 0 && d != Integer.MIN_VALUE
&& n != Integer.MIN_VALUE) {
val f = new Fraction(n, d)
if (n < 0 && d < 0 || n > 0 && d > 0)
f.numer should be > 0
else if (n != 0)
f.numer should be < 0
else
f.numer should be === 0
f.denom should be > 0
}
}
Note that even if you are use generators that don't produce the invalid values, you still need the
whenever
clause. The reason is that once a property fails, ScalaCheck will try and shrink
the values to the smallest values that still cause the property to fail. During this shrinking process ScalaCheck
may pass invalid values. The whenever
clause is still needed to guard against those values. (The
whenever
clause also clarifies to readers of the code exactly what the property is in a succinct
way, without requiring that they find and understand the generator definitions.)
Supplying both generators and argument names
If you want to supply both generators and named arguments, you can do so by providing a list of (<generator>, <name>)
pairs
in parentheses after forAll
, before the property function. Here's an example:
forAll ((validNumers, "n"), (validDenoms, "d")) { (n: Int, d: Int) =>
whenever (d != 0 && d != Integer.MIN_VALUE
&& n != Integer.MIN_VALUE) {
val f = new Fraction(n, d)
if (n < 0 && d < 0 || n > 0 && d > 0)
f.numer should be > 0
else if (n != 0)
f.numer should be < 0
else
f.numer should be === 0
f.denom should be > 0
}
}
Were this property check to fail, it would mention the names n and d in the error message, like this:
Occurred when passed generated values (
n = 17,
d = 21
)
Property check configuration
The property checks performed by the forAll
methods of this trait can be flexibly configured via the services
provided by supertrait Configuration
. The five configuration parameters for property checks along with their
default values and meanings are described in the following table:
Configuration Parameter
|
Default Value
|
Meaning
|
minSuccessful
|
10
|
the minimum number of successful property evaluations required for the property to pass (Note that this is different from ScalaCheck's default of 100.)
|
maxDiscarded
|
50
|
the maximum number of discarded property evaluations allowed during a property check
|
minSize
|
0
|
the minimum size parameter to provide to ScalaCheck, which it will use when generating objects for which size matters (such as strings or lists)
|
maxSize
|
100
|
the maximum size parameter to provide to ScalaCheck, which it will use when generating objects for which size matters (such as strings or lists)
|
workers
|
1
|
specifies the number of worker threads to use during property evaluation
|
The forAll
methods of trait GeneratorDrivenPropertyChecks
each take a PropertyCheckConfiguration
object as an implicit parameter. This object provides values for each of the five configuration parameters. Trait Configuration
provides an implicit val
named generatorDrivenConfig
with each configuration parameter set to its default value.
If you want to set one or more configuration parameters to a different value for all property checks in a suite you can override this
val (or hide it, for example, if you are importing the members of the GeneratorDrivenPropertyChecks
companion object rather
than mixing in the trait.) For example, if
you want all parameters at their defaults except for minSize
and maxSize
, you can override
generatorDrivenConfig
, like this:
implicit override val generatorDrivenConfig =
PropertyCheckConfiguration(minSize = 10, maxSize = 20)
Or, if hide it by declaring a variable of the same name in whatever scope you want the changed values to be in effect:
implicit val generatorDrivenConfig =
PropertyCheckConfiguration(minSize = 10, maxSize = 20)
In addition to taking a PropertyCheckConfiguration
object as an implicit parameter, the forAll
methods of trait
GeneratorDrivenPropertyChecks
also take a variable length argument list of PropertyCheckConfigParam
objects that you can use to override the values provided by the implicit PropertyCheckConfiguration
for a single forAll
invocation. For example, if you want to set minSuccessful
to 500 for just one particular forAll
invocation,
you can do so like this:
forAll (minSuccessful(500)) { (n: Int, d: Int) => ...
This invocation of forAll
will use 500 for minSuccessful
and whatever values are specified by the
implicitly passed PropertyCheckConfiguration
object for the other configuration parameters.
If you want to set multiple configuration parameters in this way, just list them separated by commas:
forAll (minSuccessful(500), maxDiscarded(300)) { (n: Int, d: Int) => ...
If you are using an overloaded form of forAll
that already takes an initial parameter list, just
add the configuration parameters after the list of generators, names, or generator/name pairs, as in:
forAll ("n", "d", minSuccessful(500), maxDiscarded(300)) {
(n: Int, d: Int) => ...
forAll (validNumers, validDenoms, minSuccessful(500), maxDiscarded(300)) {
(n: Int, d: Int) => ...
forAll ((validNumers, "n"), (validDenoms, "d"), minSuccessful(500), maxDiscarded(300)) {
(n: Int, d: Int) => ...
For more information, see the documentation for trait Configuration
,
a supertrait of GeneratorDrivenPropertyChecks
.
Next, learn about ScalaTest's philosophy and design.