ScalaTest/Scalactic 2.2.0 Release Notes

ScalaTest/Scalactic 2.2.0 includes the enhancements and deprecations listed below. No source code using ScalaTest/ScalaUtils 2.1.x should break, but you will likely need to do a clean build to upgrade.

For information on how to include ScalaTest in your project, see the download page. For information on how to use Scalactic in your production code, see its download page.

Overview

Introducing the Scalactic library

In 2.2.0, org.scalautils has been renamed to org.scalactic (rhymes with "galactic")'>). All existing code using org.scalautils directly will continue to work, but will receive a deprecation warning. Please change scalautils to scalactic when convenient, because after a lengthy deprecation period, org.scalautils will be removed.

ScalaUtils started out as a small library carved out of ScalaTest that made sense for production code as well as tests. It has since been growing more mature as a production-code library focused on quality, and deserved a more distinctive name. The name "scalactic" comes from this 2009 scala-internals discussion. I proposed it as a word to mean Scala-like, like "Pythonic" for Python. It never caught on for that, so instead I thought it might work well as the new name for ScalaUtils. Also, because Scalactic precedes ScalaTest alphabetically, its documentation shows up at the top of the combined Scaladoc instead of the bottom, where ScalaUtils has less prominently appeared since 2.0.

Somewhat ironically, that 2009 scala-internals thread was a debate about Paul Phillip's addition of a times method to RichInt, which would make the imperative looping syntax, 5 times println, supported by the Scala standard library by default. Martin Odersky indicated he felt the syntax was not Scala-like, and asked, "what's the analogue of 'Pythonic' in Scala?" By suggesting scalactic I was implying that times on Int was not "scalactic," yet since its initial release of ScalaUtils, that very syntax has been available via its TimesOnInt trait. Since ScalaUtils is now called Scalactic, TimesOnInt is now part of Scalactic. Perhaps what's "scalactic" about this is that the 5 times println syntax is available from a third-party library rather than by default from the standard library.

Note that although org.scalautils package name in your source code will continue to work during the deprecation period, there will be no artifacts with an org.scalautils group ID released for 2.2.0. If you were using ScalaUtils only through ScalaTest, then your build will continue to work as before. But if you were using ScalaUtils standalone in your production code, you may see an error like: 

[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     ::          UNRESOLVED DEPENDENCIES         ::
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     :: org.scalautils#scalautils_2.10;2.2.0: not found
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::

If so, you'll need to change "scalautils" to "scalactic" in your build. For example, in an sbt build, you would need to change: 

libraryDependencies += "org.scalautils" % "scalautils" % "2.1.7" % "test"

to: 

libraryDependencies += "org.scalactic" % "scalactic" % "2.2.0" % "test"

Enhanced Assertions error messages

For ScalaTest 2.2.0, we have enhanced the macro that produces error messages from the assert methods of trait Assertions. Here are some examples:

scala> import org.scalatest._
import org.scalatest._

scala> import Assertions._
import Assertions._

scala> val a = 1
a: Int = 1

scala> val b = 2
b: Int = 2

scala> val c = 3
c: Int = 3

scala> val d = 4
d: Int = 4

scala> val num = 1.0
num: Double = 1.0

scala> assert(a == b || c >= d)
org.scalatest.exceptions.TestFailedException: 1 did not equal 2, and 3 was not greater than or equal to 4
	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

scala> val xs = List(1, 2, 3)
xs: List[Int] = List(1, 2, 3)

scala> assert(xs.exists(_ == 4))
org.scalatest.exceptions.TestFailedException: List(1, 2, 3) did not contain 4
	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

scala> assert("hello".startsWith("h") && "goodbye".endsWith("y"))
org.scalatest.exceptions.TestFailedException: "hello" started with "h", but "goodbye" did not end with "y"
	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

scala> assert(num.isInstanceOf[Int])
org.scalatest.exceptions.TestFailedException: 1.0 was not instance of scala.Int
	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

scala> assert(Some(2).isEmpty)
org.scalatest.exceptions.TestFailedException: Some(2) was not empty
	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

The macro works by recognizing patterns in the AST of the expression passed to assert and, for a finite set of common expressions, giving an error message that an equivalent ScalaTest matcher expression would give. For expressions that are not recognized, the macro currently prints out a string representation of the (desugared) AST and adds "was false". Here are some examples of error messages for unrecognized expressions: 

scala> assert(None.isDefined)
org.scalatest.exceptions.TestFailedException: scala.None.isDefined was false
	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

scala> assert(xs.exists(i => i > 10))
org.scalatest.exceptions.TestFailedException: xs.exists(((i: Int) => i.>(10))) was false
	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
        ...

In the future we hope to improve the default error messages by showing values at the leaf nodes, and to the extent possible, showing a representation of the original AST before it was desugared. Getting back to the original AST before desugaring is difficult (if not impossible) using macros as they are currently defined, but hopefully this use case will help motivate some improvements in that direction to Scala macros.

Note that if a clue string is supplied in the assertion, it will be appended to the macro-generated error message: 

scala> val p = true
p: Boolean = true

scala> val q = false
q: Boolean = false

scala> assert(p == q, s", though now that I think of it, $p has never equaled $q!")
org.scalatest.exceptions.TestFailedException: true did not equal false, though now that I
          think of it, true has never equaled false!
	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
        ...

Lastly, the assume methods received the same enhancement as the assert methods. Compared to assert, you'll get the same error message given the same expression from assume, but via a thrown TestCanceledException instead of a TestFailedException. Here's an example: 

scala> assume(a + 1 == b && c != d - 1)
org.scalatest.exceptions.TestCanceledException: 2 equaled 2, but 3 equaled 3
	at org.scalatest.Assertions$class.newTestCanceledException(Assertions.scala:433)
        ...

DiagrammedAssertions

Building on work that Peter Niederwieser has done in Spock and Expecty, ScalaTest 2.2.0 introduces trait DiagrammedAssertions. This trait extends >Assertions and overrides its assert methods, modifying the default macro to give error messages that show the original line of code and a value for each part of the expression. Here are some examples:

scala> import DiagrammedAssertions._
import DiagrammedAssertions._

scala> assert(a == b || c >= d) 
org.scalatest.exceptions.TestFailedException: 

assert(a == b || c >= d)
       | |  | |  | |  |
       1 |  2 |  3 |  4
         |    |    false
         |    false
         false

	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

scala> assert(xs.exists(_ == 4))
org.scalatest.exceptions.TestFailedException: 

assert(xs.exists(_ == 4))
       |  |
       |  false
       List(1, 2, 3)

	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

scala> assert("hello".startsWith("h") && "goodbye".endsWith("y"))
org.scalatest.exceptions.TestFailedException: 

assert("hello".startsWith("h") && "goodbye".endsWith("y"))
       |       |          |    |  |         |        |
       "hello" true       "h"  |  "goodbye" false    "y"
                               false

	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

scala> assert(num.isInstanceOf[Int])
org.scalatest.exceptions.TestFailedException: 

assert(num.isInstanceOf[Int])
       |   |
       1.0 false

	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

scala> assert(Some(2).isEmpty)
org.scalatest.exceptions.TestFailedException: 

assert(Some(2).isEmpty)
       |    |  |
       |    2  false
       Some(2)

	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

scala> assert(None.isDefined)
org.scalatest.exceptions.TestFailedException: 

assert(None.isDefined)
       |    |
       None false

	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

scala> assert(xs.exists(i => i > 10))
org.scalatest.exceptions.TestFailedException: 

assert(xs.exists(i => i > 10))
       |  |
       |  false
       List(1, 2, 3)

	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

If the expression passed to assert spans more than one line, DiagrammedAssertions falls back to the default style of error message, since drawing a diagram would be difficult. Here's an example showing how DiagrammedAssertions will treat a multi-line assertion (i.e., you don't get a diagram): 

scala> assert("hello".startsWith("h") &&
     |   "goodbye".endsWith("y"))
org.scalatest.exceptions.TestFailedException: "hello" started with "h", but "goodbye" did not end with "y"
	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

Also, since an expression diagram essentially represents multi-line ascii art, if a clue string is provided, it appears above the diagram, not after it. It will often also show up in the diagram: 

scala> assert(None.isDefined, "Don't do this at home")
org.scalatest.exceptions.TestFailedException: Don't do this at home

assert(None.isDefined, "Don't do this at home")
       |    |
       None false

	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

scala> assert(None.isDefined,
     |   "Don't do this at home")
org.scalatest.exceptions.TestFailedException: Don't do this at home

assert(None.isDefined,
       |    |
       None false

	at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:422)
	...

Scalactic Requirements

Scalactic includes a new trait that was not in ScalaUtils called Requirements. This trait applies macros to the task of pre-condition checking through methods named require, requireState, and requireNonNull. These three methods aim to improve error messages provided when a pre-condition check fails at runtime in production code. Although it is recommended practice to supply helpful error messages when doing pre-condition checks, often people don't. Instead of this: 

scala> val length = 5
length: Int = 5

scala> val idx = 6
idx: Int = 6

scala> require(idx >= 0 && idx <= length, "index, " + idx + ", was less than zero or greater than or equal to length, " + length)
java.lang.IllegalArgumentException: requirement failed: index, 6, was less than zero or greater than or equal to length, 5
	at scala.Predef$.require(Predef.scala:233)
	...

People write simply: 

scala> require(idx >= 0 && idx <= length)
java.lang.IllegalArgumentException: requirement failed
	at scala.Predef$.require(Predef.scala:221)
	...

Note that the detail message of the IllegalArgumentException thrown by the previous line of code is simply, "requirement failed". Such messages often end up in a log file or bug report, where a better error message can save you time in debugging the problem. By importing the members of Requirements (or mixing in its companion trait), you'll get a more helpful error message extracted by a macro, whether or not a clue message is provided: 

scala> import org.scalactic._
import org.scalactic._

scala> import Requirements._
import Requirements._

scala> require(idx >= 0 && idx <= length)
java.lang.IllegalArgumentException: 6 was greater than or equal to 0, but 6 was not less than or equal to 5
	at org.scalactic.Requirements$RequirementsHelper.macroRequire(Requirements.scala:56)
	...

scala> require(idx >= 0 && idx <= length, "(hopefully that helps)")
java.lang.IllegalArgumentException: 6 was greater than or equal to 0, but 6 was not less than or equal to 5 (hopefully that helps)
	at org.scalactic.Requirements$RequirementsHelper.macroRequire(Requirements.scala:56)
	...

The requireState method provides identical error messages to require, but throws IllegalStateException instead of IllegalArgumentException: 

scala> val connectionOpen = false
connectionOpen: Boolean = false

scala> requireState(connectionOpen)
java.lang.IllegalStateException: connectionOpen was false
	at org.scalactic.Requirements$RequirementsHelper.macroRequireState(Requirements.scala:71)
	...

Thus, whereas the require methods throw the Java platform's standard exception indicating a passed argument violated a precondition, IllegalArgumentException, the requireState methods throw the standard exception indicating an object's method was invoked when the object was in an inappropriate state for that method, IllegalStateException.

The requireNonNull method takes one or more variables as arguments and throws NullPointerException with an error messages that includes the variable names if any are null. Here's an example: 

scala> val e: String = null
e: String = null

scala> val f: java.util.Date = null
f: java.util.Date = null

scala> requireNonNull(a, b, c, d, e, f)
java.lang.NullPointerException: e and f were null
	at org.scalactic.Requirements$RequirementsHelper.macroRequireNonNull(Requirements.scala:101)
	...

Although trait Requirements can help you debug problems that occur in production, bear in mind that a much better alternative is to make it impossible for such events to occur at all. Use the type system to ensure that all pre-conditions are met so that the compiler can find broken pre-conditions and point them out with compiler error messages. When this is not possible or practical, however, trait Requirements is helpful.

Scalactic Snapshots

Scalactic also includes a new trait named Snapshots that was not part of ScalaUtils. Snapshots provides a snap method that takes one or more arguments and results in a SnapshotSeq, whose toString lists the names and values of each argument. Its intended use case is to help you write debug and log messages that give a "snapshot")'> of program state. Here's an example:

scala> import Snapshots._
import Snapshots._

scala> snap(a, b, c, d, e, f)
res3: org.scalactic.SnapshotSeq = a was 1, b was 2, c was 3, d was 4, e was null, f was null

SnapshotSeq offers a lines method that places each variable name/value pair on its own line:

scala> snap(a, b, c, d, e, f).lines
res4: String = 
a was 1
b was 2
c was 3
d was 4
e was null
f was null

Or, because a SnapshotSeq is a IndexedSeq[Snapshot], you can process it just like any other Seq, for example: 

scala> snap(a, b, c, d, e, f).mkString("Wow! ", ", and ", ". That's so awesome!")
res6: String = Wow! a was 1, and b was 2, and c was 3, and d was 4, and e was null, and f was null. That's so awesome!

New matchers and assertions

Added a matchPattern matcher that uses a macro to enforce intended usage. ScalaTest's Inside trait has long provided an inside construct that allows you to perform assertions on values obtained via a pattern match. Here's an example:

scala> import org.scalatest._
import org.scalatest._

scala> import Matchers._
import Matchers._

scala> import Inside._
import Inside._

scala> case class Name(first: String, middle: String, last: String)
defined class Name

scala> val name = Name("Jane", "Q", "Programmer")
name: Name = Name(Jane,Q,Programmer)

scala> inside(name) { case Name(first, _, _) =>
     |   first should startWith ("S")
     | }
org.scalatest.exceptions.TestFailedException: "Jane" did not start with substring "S",
inside Name(Jane,Q,Programmer)
	at org.scalatest.MatchersHelper$.newTestFailedException(MatchersHelper.scala:160)
	...

This inside construct can also be used simply to ensure a value matches a given pattern, like this: 

scala> inside(name) { case Name("Sandra", _, _) => }
org.scalatest.exceptions.TestFailedException: The partial function passed as the second parameter
    to inside was not defined at the value passed as the first parameter to inside, which was: Name(Jane,Q,Programmer)
	at org.scalatest.Inside$class.inside(Inside.scala:126)
	...

However, the programmer intent in the previous code is less obvious--did they just want to ensure the pattern matched or did they forget to add an assertion? The new matchPattern syntax is designed to make this more obvious for users of >Matchers. Here's what it looks like: 

scala> name should matchPattern { case Name("Sandra", _, _) => }
org.scalatest.exceptions.TestFailedException: Name(Jane,Q,Programmer) did not match the given pattern
	at org.scalatest.MatchersHelper$.newTestFailedException(MatchersHelper.scala:160)
	...

The reason matchPattern was not added earlier was that until the advent of macros, there was no way to prevent this kind of usage: 

name should matchPattern { case Name(first, _, _) =>
  first should startWith ("S")
}

The above use case is what inside is for. To make user code consistent, matchPattern uses a macro to ensure only a pattern is provided--no code is allowed to the right of the rocket. Here's what you'll see at compile-time if you accidentally attempt this: 

scala> name should matchPattern { case Name(first, _, _) =>
     |   first should startWith ("S")
     | }
<console>:21: error: No code is allowed to the right of rocket symbols (=>) in a partial function
    passed to matchPattern, because matchPattern is intended only for ensuring that an expression matches a
    pattern. If you want to make further assertions after a successful pattern match, use org.scalatest.Inside instead.
                first should startWith ("S")
                      ^

Added should compile and shouldNot typeCheck syntax to Matchers and assertCompiles and assertDoesNotCompile methods to Assertions:

  • should compile and assertCompiles: fails if the given snippet of code does not compile for any reason, else succeeds
  • shouldNot compile and assertDoesNotCompile: succeeds the given snippet of code does not compile for any reason, else fails
  • shouldNot typeCheck and assertTypeError: succeeds only if the given snippet of code fails to compile because of a type error; fails if the code either compiles or fails to compile because a syntax (not type) error

TestRegistration trait

Added trait TestRegistration (and fixture.TestRegistration), which offers methods registerTest and registerIgnoredTest. ScalaTests's style traits that register tests as functions--FunSuite, FunSpec, FeatureSpec, FlatSpec, FreeSpec, PropSpec, and WordSpec--now extend TestRegistration. This provides a more generic way to define tests programmatically, such as this use case from the >Discipline project. Currently trait Discipline will only work with FunSuite. With this enhancement to ScalaTest Discipline can be made to work with any style in which test are functions registered at construction time.

Another use case for TestRegistration is to make a type error to mix traits that facilitate registration of shared tests into a style that does not support test registration. For example, the AllBrowsersPerSuite and AllBrowsersPerTest traits of the ScalaTest + Play library only work if mixed into a trait that support registration of tests as functions. When ScalaTest + Play is released against 2.2.0, TestRegistration will be added as a self-type to AllBrowsersPerSuite and AllBrowsersPerTest to make it a compiler error if someone attempts to mix them into a style that does not support test registration.

Other enhancements

  • Moved Page out of WebBrowser to become a sibling in package >>org.scalatest.selenium, to make it easier to create reusabled Page classes.
  • Added recover and recoverWith methods to Or, modeled after similar methods in scala.util.Try.
  • Loosened type restrictions on HTML 5 input elements in the WebBrowser trait's Selenium DSL to better support browsers that do not fully support the new elements. (Contributed by Matt Hughes.)
  • Allow multiple -o and -e parameters passed from sbt to ScalaTest's Framework implementation, so that you can have a -oS, for example, in your testOptions in the build file and override it, say, with test-only ... -- -oF.
  • Allow multiple -l and -n parameters passed to Runner. The driving use case for this change is to make it easier to write sbt code that will include or exclude different tags in different situations.
  • Broadened where ScalaTest looks for tag annotations on a class. Previously it only looked on the class itself. Now it will look at all superclasses, and will discover any tag annotations there that are also annotated with @Inherited. Annotated all the tag annotations in the org.scalatest.tags method with @Inherited so they could be used in this way. Did not make org.scalatest.Ignore @Inherited.
  • Made NoArg thread safe. Originally it was not synchronized because its intended use is by one thread. Although this behavior was described in its Scaladoc documentation, this was changed in 2.2.0 to make it more bulletproof.
  • Made NoArg thread safe. Originally it was not synchronized because its intended use is by one thread. Although this behavior was described in its Scaladoc documentation, this was changed in 2.2.0 to make it more bulletproof.
  • Added ScalaTestVersion to the org.scalatest package object and ScalacticVersion to the org.scalactic package object that return a string giving the version number.
  • Note that like ScalaTest 2.1.7, 2.2.0 was compiled without the no-specialization flag. See the 2.1.7 release notes for details.

Bug fixes

  • Ensured the thread sbt users to call done on a Runner instance in ScalaTest's Framework implementation waits until the ServerSocket thread has received an event signalling the run is completed.
  • Ensured the -o configuration will be used when running sbt in fork mode.

Visit ScalaTest Release Notes for links to the release notes of all previous versions, or step back in time by visiting the release notes for the previous version.

ScalaTest is brought to you by Bill Venners and Artima.
ScalaTest is free, open-source software released under the Apache 2.0 license.

If your company loves ScalaTest, please consider sponsoring the project.

Copyright © 2009-2024 Artima, Inc. All Rights Reserved.

artima