ScalaTest 2.0 Detailed History

For historians, this page preserves the relevant sections of the individual release notes for major 2.0 milestones and release candidates. For a quicker to read and more understandable overview, see the Highlights section of the 2.0 release notes.

2.0.RC3 Highlights:

Changed update method in style traits to note, Updater to Notifier, Updating to Notifying.
Made minor formatting improvements to HTML reporter for markup text.
Improved the error messages when an attempt is made to open a scope inside a test.

2.0.RC2 Highlights:

Fixed a race condition in Waiter (contributed by Nick Wade)
Fixed a bug in the new Framework API implementation in which TestWildcardSelectors weren't being handled, which will be needed for sbt 0.13.1.
Completed Scaladoc for 2.0 final release
Changed the name of the public field from left to leftSide in Equalizer and")' CheckingEqualizer so that it")' doesn't conflict with a left method added by an implicit conversion in scalaz.
Added pretty toStrings to the Equality's")' produced by TolerantNumerics
Added three missing lifecycle methods to SuiteMixin, to")' enable the stackable trait pattern for those methods
Implemented and documented the scan methods on Every.
Enhanced assume to use similar macros as assert for better error messages in Assertions
Changed the formatting behavior of the assert and assume methods of Assertions when a string "clue" message")' is provided to match that of AppendedClues

2.0.RC1 Highlights:

Added simple assert macros, made TripleEquals (whose === and !== operators return Boolean) and deprecated the Legacy variants whose === and !== operator return Option[String]
Require at least two arguments in contain oneOf, allOf, noneOf, atLeastOneOf, atMostOneOf, inOrderOnly, and inOrder
Added an implicit language.postfixOps to SpanSugar
Added scala-reflect library to ScalaTest pom, now that we're using assert macros
Added a trap method to Assertions to make it easier to work with assertions and matcher expressions in the Scala interpreter

2.0.M8 Highlights:

ScalaTest/ScalaUtils 2.0.M8 mostly contains Scaladoc documentation, but also contains some adjustments to Or in in ScalaUtils, the addition of a OneArgTest factory method, and one important bug fix. Here is the list of changes in 2.0.M8 compared to 2.0.M7:

Changed BeforeAndAfterAll so that it no longer blocks waiting for")' nested suites to complete before executing afterAll. This approach was taken originally to ensure that afterAll ran on the same thread as beforeAll, and worked fine with ScalaTest prior to adding support for the new sbt Framework API. With the new Framework API, ScalaTest by default will allow sbt to run nested suites on its thread pool rather than making a separate thread pool when parallel execution is enabled in an sbt build. This caused a deadlock when nested suites were used with an sbt build with BeforeAndAfterAll, so in 2.0.M8, we have removed the guarantee that afterAll will execute on the same thread as beforeAll. This removes the potential for deadlock, but means that users will need to synchronize access to any shared data in beforeAll and afterAll.
Added a OneArgTest factory method to support the use")' case of needing to do further fixture set and teardown work with a FixtureParam.
Added an orElse method to Or that takes a by-name.
Made getOrElse method of Or take a by-name.
Added asOr method to Or and")' asEvery method to Every to provide an easy")' way to widen more specific sub-type types to the supertypes.
Moved the Interval,")' Timeout,")' and PatienceConfigParam types out of")' trait PatienceConfiguration and put them instead in its companion object. This way their names won't be in scope in tests unless imported, and won't conflict with like-named classes, such as the Interval from Joda Time.
Refactored Or such that all methods that accumulate are actually")' on an Accumulatable class available through an implicit")' conversion defined in trait Validations, which ensures that")' the accumulated result type is the least upper bound of the two input accumulated types.
Dropped the old transform method on Or in favor of a")' new one similar in spirit to the transform method on scala.util.Try.

2.0.M7 Highlights:

ScalaTest 2.0.M7 mostly contains Scaladoc documentation, but also contains one new feature in ScalaUtils: Or, an "either with attitude," and")' Every, an ordered data structure that can have one-to-many elements,")' but never be empty. An Or can be either Good or")' Bad. If the Bad type is an Every, you can")' use various means to accumulate errors during data validations. I (Bill Venners) will attempt to get a blog post introducing Or out within the next few days. In the meantime, your best bet if you want to know more is to scan the tests in OrSpec, EverySpec, and ValidationsSpec. And feel free to ask questions on scalatest-users. Here is the list of changes in 2.0.M7 compared to 2.0.M6:

Changed TestPendingException from a case class back")' into a class.
Use an actual Composifier class in the implicit conversion provided by MatcherProducers.
Removed AMatcher and AnMatcher in the org.scalatest.matchers package. Although included in 2.0.M6, these turned out to be inconsistent with the rest of the matcher enhancements, thus incomplete. We will bring them back 2.0 once they are fully finished.
Changed the name of TypeConstraint to Constraint in the org.scalautils package.
Removed the undocumented legacyEqual matcher syntax from Matchers.
Obey the sbt.log.noformat in both the old and new sbt Framework API implementation.
Gave the remaining matchers that lacked them pretty toString implementations.

2.0.M6 Highlights:

ScalaUtils

The first big change is ScalaTest 2.0.M6 includes a package named org.scalautils.")' This package is focused on aspects of equality, normalization, and tolerance that may make sense to use in your production code as well as your tests. The org.scalautils package is included in the ScalaTest jar, but is also available separately in a ScalaUtils jar. The idea is you can add a dependency on ScalaUtils in your production classpath, so you can use it in your production code without having ScalaTest on the classpath. You should never include ScalaTest in your production classpath, because when ScalaTest doesn't like something it tends to throw a TestFailedException. ScalaUtils will always be released alongside ScalaTest, with version numbers that match exactly. So when ScalaTest 2.1 is released, for example, ScalaUtils 2.1 will also be released.

Equality

ScalaUtils 2.0.M6 allows you to customize what equality means for a type T by defining an Equality[T] typeclass. This can be used with the === operator in")' production code and in tests, with assertions, the should equal, shouldEqual, and the new-in-2.0.M6 should === matcher syntax.

ScalaUtils defines a default Equality that will be used by default for any")' type T for which a more specific implicit Equality[T] is not available. The default equality will first call the deep method on an array on either the left or right side (or both sides), then use == to determine equality. This default is compatible with ScalaTest's existing === operator and equal and be matchers, so existing test code will continue to work as before.

Equality constraints

ScalaUtils 2.0.M6 also provides a way to get a compile-time type error if two types being compared with === don't adhere to a tunable TypeConstraint. By default you can compare any type with any type (which means")' all existing uses of ScalaTest's === continue to compile and work as before). But you can use traits such as TypeCheckedTripleEquals,")' ConversionCheckedTripleEquals,")' and TraversableEqualityConstraints to get a")' compile error when comparing, for example, an Option[String] with a String for equality. (Equality and TypeConstraint are inspired in part by the Equal typeclass in scalaz.)

Tolerance

ScalaUtils 2.0.M6 also includes a Tolerance trait that facilitates checking tolerances of numeric types with a +- symbolic operator. The plusOrMinus operator used with matchers previously in ScalaTest has been deprecated in 2.0 in favor of the +- symbol, which is also now more broadly usable. In addition to:")'

result should be (2.0 +- 0.1)

You can now also use the syntax with should equal, shouldEqual, and shouldBe syntax:

result should equal (2.0 +- 0.1)
result shouldEqual 2.0 +- 0.1
result shouldBe 2.0 +- 0.1
result should === (2.0 +- 0.1)

Or just with plain-old ===, in an assertion in tests or just as part of a Boolean expression in production code:

// In tests:
assert(result === (3.0 +- 0.2))

// In production code:
if (result === (2.0 +- 0.1)) "close enough" else "sorry"

Normalization

ScalaUtils 2.0.M6 also contains a Normalization trait that allows you to")' define strategies for normalizing types. The NormalizingEquality trait")' allows you to define an Equality[T] in terms of a normalization of that type T by first normalizing the left and, if also a T, the right hand side, then using a specified "after-normalization Equality" to compare the normalized values.

Explicitly

ScalaUtils 2.0.M6 also includes an Explicitly DSL for defining an")' implicit Equality parameter explicitly. If you want to customize equality for a type in general, you would usually want to place an implicit Equality[T] for that type in scope (or in T's companion object). That implicit equality definition will then be picked up and used when that type is compared for equality with the equal matcher in tests and with === in both tests and production code. If you just want to use a custom equality for a single comparison, however, you may prefer to pass it explicitly. For example, if you have an implicit Equality[String] in scope, you can force a comparison to use the default equality with this syntax:

// In production code:
if ((result === "hello")(decided by defaultEquality)) true else false

// In tests:
result should equal ("hello") (decided by defaultEquality)

The explicitly DSL also provides support for specifying a one-off equality that is based on a normalization. For example, ScalaUtils has a StringNormalizations trait that provides trimmed and lowerCased methods that return Normalization[String]s that normalize by trimming and lower-casing, respectively. If you bring those into scope by mixing in or importing the members of StringNormalizations, you could use the explicitly DSL like this:")'

// In production code:
if ((result === "hello")(after being lowerCased)) true else false

// In tests:
result should equal ("hello") (after being lowerCased and trimmed)

ScalaUtils 2.0.M6 also includes a few other items, such as Catcher, an extractor factory;")' Prettifier")' and PrettyMethods, for defining alternate toString implementations;")' and TimesOnInt, a counting loop moved from org.scalatest.")'

Inspectors

Back in ScalaTest proper (i.e. not ScalaUtils), ScalaTest 2.0.M6 includes an Inspectors trait that allows you to make assertions about (to")' inspect) collections, including deeply nested collections:

forAll - succeeds if the assertion holds true for every element
forAtLeast - succeeds if the assertion holds true for at least the specified number of elements
forAtMost - succeeds if the assertion holds true for at most the specified number of elements
forBetween - succeeds if the assertion holds true for between the specified minimum and maximum number of elements, inclusive
forEvery - same as forAll, but lists all failing elements if it fails (whereas forAll just reports the first failing element)
forExactly - succeeds if the assertion holds true for exactly the specified number of elements

We worked hard to craft error messages that will help you understand what went wrong when an inspection against a nested collection fails. For example, given this nested collection:

val xss =
  List(
    List(1, 2, 3),
    List(1, 2, 3),
    List(1, 2, 3)
   )

And this inspection:

forAll (xss) { xs =>
  forAll (xs) { y => y should be < 2 }
}

Your error message will be:

TestFailedException: forAll failed, because:
  at index 0, forAll failed, because:
    at index 1, 2 was not less than 2 (<console>:20)
  in List(1, 2, 3) (<console>:20)
in List(List(1, 2, 3), List(1, 2, 3), List(1, 2, 3))
     at ...

Inspector shorthands

If you are using Matchers, you can use "inspector shorthands" for inspecting top-level (i.e., non-nested) collections:")'

all - succeeds if the assertion holds true for every element
atLeast - succeeds if the assertion holds true for at least the specified number of elements
atMost - succeeds if the assertion holds true for at most the specified number of elements
between - succeeds if the assertion holds true for between the specified minimum and maximum number of elements, inclusive
every - same as forAll, but lists all failing elements if it fails (whereas forAll just reports the first failing element)
exactly - succeeds if the assertion holds true for exactly the specified number of elements

Here are some examples:

all (xs) should be > 0
atMost (2, xs) should be >= 4
atLeast (3, xs) should be < 5
between (2, 3, xs) should (be > 1 and be < 5)
exactly (2, xs) should be <= 2
every (xs) should be < 10

org.scalatest.Matchers

Speaking of matchers, in ScalaTest 2.0.M6 ShouldMatchers and MustMatchers, both members of package org.scalatest.matchers have been deprecated in favor of Matchers, which resides in package org.scalatest. (MustMatchers was")'")' actually already deprecated in ScalaTest 1.9.2.) For folks using ShouldMatchers, getting rid of the deprecation warning should be as simple as replacing org.scalatest.matchers.ShouldMatchers with org.scalatest.Matchers. For folks using MustMatchers, however, it will unfortnately be slightly more trouble, because the new Matchers trait only supports should not must. So you will need to search and replace your uses of must with should. MustMatchers and must will continue to work for a good long deprecation period, but eventually it will be removed to make way for must possibly coming back later to serve a different purpose. Apologies for this rather large deprecation.

Contain matchers

The contain matcher syntax has been greatly enhanced in 2.0.M6, in part inspired by similar syntax in specs2. In addition to simple assertions about containership, like:

xs should contain ("hi")

You can also make assertions such as:

xs should contain oneOf (1, 2, 3)
xs should contain noneOf (1, 2, 3)
xs should contain atLeastOneOf (1, 2, 3)
xs should contain allOf (1, 2, 3)
xs should contain only (1, 2, 3)
xs should contain inOrderOnly (1, 2, 3)
xs should contain inOrder (1, 2, 3)
xs should contain theSameElementsAs List(1, 2, 3)
xs should contain theSameElementsInOrderAs List(1, 2, 3)

In the new contain syntax, "containership" is determined by Equality, so you can customize how containership is determined for a type T by via an implicit Equality[T] or the explicitly DSL. Here's an example:

(result should contain oneOf ("hi", "he", "ho")) (after being trimmed)

Implicit conversions => implicit parameters

ScalaTest 2.0.M6 includes a major refactor of how matchers are implemented. Since ScalaTest 1.0 matchers have been implemented primarily with implicit conversions, and as of 2.0.M6 they are mostly implemented with implicit parameters instead. This will make it simpler to enable the syntax on your own types, because you will be able to enable syntax by defining the typeclass "enablers" for that syntax for your own types.

For example, the contain (<value>) syntax is enabled for type T by an implicit Containing[T]. ScalaTest 2.0.M6 provides implementations for")' GenTraversable, java.util.Collection, java.util.Map, Array, Option, and String in the Containing companion object. These enable the contain syntax")' to be used on more types than in previous versions of ScalaTest, such as strings and options:

Some("hi") should contain ("hi")
"hello" should contain ('e')

You can enable the contain (<value>) syntax for one of your own types, U, simply by providing an implicit Containing[U] for that type. The enabling implicit parameter traits are gathered in the org.scalatest.enablers package.")'

ScalaTest 2.0 matchers enable syntax based on the left-hand type, i.e., the type on which should is invoked. As a result, many expressions that resulted in a matcher in ScalaTest 1.x result instead in a matcher factory in ScalaTest 2.0. The factory can produce a matcher given an implicit enabler. For example, in ScalaTest 1.x, the expression have length 3 produced a Matcher[Any] that threw an exception if the left-hand object did not have a length or getLength method. In ScalaTest 2.0, it returns a MatcherFactory1[Any, Length]:

scala> have length 3
res7: org.scalatest.matchers.MatcherFactory1[Any,org.scalatest.enablers.Length] = have length 3

This matcher factory can produce a matcher for Any type T given an implicit Length[T]. If no Length[T] is available for the left-hand type, you'll get a compiler error:

scala> 2.0 should have length 3
<console>:21: error: could not find implicit value for parameter len: org.scalatest.enablers.Length[Double]
              2.0 should have length 3
                              ^

Because the Length companion object provides an implicit Length[String], the compiler will be happy when the type to the left of should is a String:

scala> "2.0" should have length 3

Despite the major reorganization of matchers implementation, existing matchers user code should continue to work fine in all but very rare cases.")'

Other new matcher syntax

ScalaTest 2.0.M6 also includes other new matcher syntax, for the most part enabled with implicit parameters, including:

option shouldBe defined
partialFunction should be definedAt 8
file should exist
otherFile shouldNot exist
file shouldBe readable
file shouldBe writable
"abbccxxx" should startWith regex ("a(b*)(c*)" withGroups ("bb", "cc"))
result1 shouldBe a [String]
tummy should not be empty
list shouldBe sorted
val ex = the [ArithmeticException] thrownBy { 1 / 0 }
ex should have message "/ by zero"
noException should be thrownBy { "hi" }
an [ArithmeticException] should be thrownBy { 1 / 0 }
the [StringIndexOutOfBoundsException] thrownBy {
  "hi".charAt(-1)
} should have message "String index out of range: -1"
javaMap should contain oneOf (Entry(2, 3), Entry(3, 4))

The last example illustrates that you can now use contain syntax on Java Maps using java.util.Entry instances. ScalaTest provides an implementation, org.scalatest.Entry, to make this more convenient.")'

Lone elements and `Try` values

ScalaTest 2.0.M6 also includes a LoneElement trait, that adds a loneElement method to types with a “Collecting” nature")' classes. The loneElement method will throw TestFailedException if the collection on which it is invoked contains other than a single element. You can use it like this:

set.loneElement should be <= 10

ScalaTest 2.0.M6 also includes a TryValues trait, that offers syntax that allows you to state that you expect")' a Try to be a success or a failure, and something else about the contained value. Here are some examples:

try1.success.value shouldEqual 3
try2.failure.exception shouldBe an [ArithmeticException]

Improved matcher performance and composition

In 2.0.M5, MatchResult has undergone a")' backwards-compatible refactor that will both improve runtime performance, enable better Matcher composition, and")' eventually (though not yet in 2.0) allow objects involved in assertion failures to be inspected in IDEs.

Profiling of ScalaTest 1.x tests that used Matchers indicated that a significant amount of time was spent constructing error message strings for MatchResults, even though the vast majority of those strings ultimately where never used in a failure message. This was especially noticeable when testing with very large collections, because calling toString on very large collections creates very large strings, which in turn creates even larger error message strings.

ScalaTest's error messages were created by combining localized "raw" strings with arguments. For example, for 22 should be < 20, the raw string would be "{0} was not less than {1}" and the arguments Array(22, 21). To eliminate the unnecessary string creations MatchResult now carries an args field containing those arguments along with the raw strings. The actual final strings, such as 22 was not less than 21, are computed from the raw strings and args only if and when they are needed.

ScalaTest Matchers, being Function1s, were always composable via the standard methods in Scala for function composition: compose and andThen. For example, in ScalaTest 1.x you could transform a matcher for one type into a matcher for another with compose. Here's an example:

scala> import org.scalatest._
import org.scalatest._

scala> import Matchers._
import Matchers._

scala> import java.io.File
import java.io.File

scala> def endWithExtension(ext: String) = endWith(ext) compose { (f: File) => f.getPath }
endWithExtension: (ext: String)org.scalatest.matchers.Matcher[java.io.File]

scala> new File("output.txt") should endWithExtension("txt")

In addition, by composing twice, you could in ScalaTest 1.x modify the type of both sides of a match statement with the same function, like this:

scala> val f = be > (_: Int)
f: Int => org.scalatest.matchers.Matcher[Int] = <function1>

scala>   val g = (_: String).toInt
g: String => Int = <function1>

scala> val beAsIntsGreaterThan = (f compose g) andThen (_ compose g)
beAsIntsGreaterThan: String => org.scalatest.matchers.Matcher[String] = <function1>

scala> "8" should beAsIntsGreaterThan ("7")

What you could not do in ScalaTest 1.x is modify the error messages when you performed matcher composition. For example, the error message for the beAsIntsGreaterThan gives no hint that the Ints being compared were parsed from Strings:

scala> "7" should beAsIntsGreaterThan ("8")
org.scalatest.exceptions.TestFailedException: 7 was not greater than 8

This inability to modify error messages when composing matchers was first brought to my attention in an offhand tweet by Nat Pryce several years ago. Although no other user ever complained about it, I ultimatedly decided to address it once MatchResults were computing their error messages lazily, since the original arguments would still be available, waiting patiently to have their toString called. I placed the functionality in a trait named MatcherProducers.")' Since we needed implicit conversions anyway, I also added a shorthand for composing twice:

scala> import matchers._
import matchers._

scala> import MatcherProducers._
import MatcherProducers._

scala> val beAsIntsGreaterThan = f composeTwice g // means: (f compose g) andThen (_ compose g)
beAsIntsGreaterThan: String => org.scalatest.matchers.Matcher[String] = <function1>

scala> "8" should beAsIntsGreaterThan ("7")

Of course, the error messages are still the original ones:

scala> "7" should beAsIntsGreaterThan ("8")
org.scalatest.exceptions.TestFailedException: 7 was not greater than 8

To modify the error messages, you can use mapResult from MatcherProducers. Here's an example:

scala> val beAsIntsGreaterThan = f composeTwice g mapResult { mr =>
     |   mr.copy(
     |     failureMessageArgs = mr.failureMessageArgs.map((LazyArg(_) { _.toString + ".toInt"})),
     |     negatedFailureMessageArgs = mr.negatedFailureMessageArgs.map((LazyArg(_) { _.toString + ".toInt"})),
     |     midSentenceFailureMessageArgs =
     |       mr.midSentenceFailureMessageArgs.map((LazyArg(_) { _.toString + ".toInt"})),
     |     midSentenceNegatedFailureMessageArgs = 
     |       mr.midSentenceNegatedFailureMessageArgs.map((LazyArg(_) { _.toString + ".toInt"}))
     |   )
     |  }
beAsIntsGreaterThan: String => org.scalatest.matchers.Matcher[String] = <function1>

Now the error message is clearer:

scala> "7" should beAsIntsGreaterThan ("8")
org.scalatest.exceptions.TestFailedException: 7.toInt was not greater than 8.toInt

Voila!

`Outcome`s and `OutcomeOf`

One breaking change in 2.0.M6 is that the withFixture life-cycle methods, and the test functions passed to them, now return an Outcome. Because they previously")' returned Unit, this will likely require users to insert an = before the open curly brace, or in some cases, :Outcome =. That will most often be all that's required, but if you were catching exceptions in a withFixture implementation, you will likely need to change your catch clauses to a corresponding pattern match on Outcome types.

Also added OutcomeOf, deprecating FailureOf, to transform")'")' the outcome of evaluating an expression to an Outcome. For example, thrown TestFailedExceptions are transformed into a")' Failed")' outcome; thrown TestPendingExceptions are transformed into a Pending outcome;")'")' the lack of a thrown exception is transformed into a Succeeded outcome.")'

Other enhancements in 2.0.M6

Note: We had to drop support of 2.9 recently because implicit lookup of contravariant type classes did not behave well, and this was required to keep all existing ScalaTest matcher code compiling. If you're on Scala 2.9, please continue with ScalaTest 1.9.2 or ScalaTest 2.0.M5b until the day comes when you can upgrade to Scala 2.10 and ScalaTest 2.0.M6.

Added Checkpoints, which allows you to accumulate and report multiple assertions in a single test
Added Informing, a trait that contains the info method, which was previously provided via an implicit that has been removed in 2.0.M6
Added Documenting, a trait that contains the markup method, which was previously provided via an implicit that has been removed in 2.0.M6
Added Sequential, a combinator for nested suites that you want executed completely sequentially
Added Stepwise, a combinator for nested suites that you want executed one after another, but with parallel execution still enabled for the individual nested suites
Added TryValues for making assertions when you expect a Try to be a Success
Added ScalaFutures to help write tests involving Scala futures
In org.scalatest.tags, added new tags CPU, Disk, Network, and Slow
Refactored all style traits into the original name as a class and the trait with name with Like appended, for speedier compiles. For example: FlatSpecLike, FunSpecLike, etc. (Done in ScalaTest 1.9.2 also.)
Enhanced both sbt and ScalaTest with the new Framework API. sbt included this enhancement in 0.13. As of 2.0.M6, ScalaTest's implementation is in org.scalatest.tools.Framework.")'
Added support for pending scopes and a ScopePending event.
Added trait BeforeAndAfterEachTestData and deprecated the beforeEach and afterEach")' methods of trait BeforeAndAfterEach take take a config map.
Added BeforeAndAfterAllConfigMap and deprecated the beforeAll and afterAll methods")' of trait BeforeAndAfterAll that take a config map.
Added TestDataFixture for easy access to test data, such as test name and tags, inside tests.
Enhanced GUI reporter to support new events and show multiple indented lines nicely.
Added support for they (in addition to it) to FunSpec and FlatSpec.
Added a toString to style classes that makes them look prettier in the interpreter.
Added MatcherFactor1 through 9.
Added NoArg to support a rare use case (motivated by Akka's TestKit).
Added DiscoveryStarting and DiscoveryCompleted events.
Added a non-blocking whenCompleted method to Status, enabling callbacks to be registered with a Status.
Enhanced the GUI reporter to show a spinning icon during discovery.
Added support for "reminders," a summary of failed and/or canceled tests that can be optionally printed out at the end of a run in string (stdout, stderr, file) reporters.
Enabled discovery to be used if only -z <test name wildcard> or -t <test name>
Enhanced -s to take a glob in addition to a complete, fully-qualified suite class name.
Added the ability send "alert" and "update" messages immediately from tests (rather than being recorded and sent with the test completion event like "info" and "markup" text). Added AlertProvided, Alerting, and Alerter (containing the alert method you can call from tests) and UpdateProvided, Updating, and Updater (containing the update method you can call from tests.
Added a "slow poke" detector, which will emit AlertProvided events periodically listing")' tests that have been running longer than a specified time limit.
Added a cancelAfter method to Timeouts.
Added Catcher.
Added the ability to run previously failed and/or canceled tests in Runner.
Added a payload to TestCanceledException.
Added a rerunner to TestCanceled.
Added test-canceled reports to the view menu in the GUI reporter, and made rerun work with canceled tests.
Added structural type support for Length, Size, Definition, Emptiness, Readability, and Writability enablers
Added Prettifier and PrettyMethods to org.scalautils.
Added support of Retries.
Added suiteId to NameInfo
Enhanced have length and have size error messages to include the actual length or size.
Refactored helper classes from Matchers into the org.scalatest.words package, and moved members of verbs package to words. These were "implementation" classes anyway, so likely no one referred to them in user code, but just in case, left deprecated type aliases in verbs.
Changed JUnitXmlReporter to output files based on suiteId, instead of the fully qualified name of the class.
Changed the behavior of BeforeAndAfterAll such that it only calls beforeAll and afterAll in the outer")' instance of a OneInstancePerTest or ParallelTestExecution.
Changed BeforeAndAfterAll and BeforeAndAfterAllConfigMap to")'")' take expectedTestCount into consideration whether to invoke beforeAll and afterAll.
Enhanced Suite.diffStrings to display brackets only when there is difference in the passed strings.
Use Ordering instead of Ordered for be </>/<=/>=.
Removed "WrappedArray" in error messages.
Changed withPayloads to infer and return the same type as the passed in function; changed payload: Any parameter to => Any to be lazy.
To make thread dumps more clear, gave useful names to ScalaTest threads: The main thread is now ScalaTest-main. The run thread is ScalaTest-run. The dispatcher thread is ScalaTest-dispatcher. Thread-pool threads for parallel execution are ScalaTest-N. And main or thread-pool threads running test suites are renamed to ScalaTest-running-<name of suite>, then changed back again once the suite has completed.

Bug fixes in 2.0.M6

Fixed an osgi.bnd issue.
Fixed argument parsing bug when parsing two -h arguments.
Fixed a bug in JUnitXmlReporter when it xmlified failure messages.
Fixed a stack depth problem when WordSpec and fixture.WordSpec's scope")'")' is used with after word.
Fixed HTMLReporter's race condition for suites that mixed in ParallelTestExecution by ensuring SuiteCompleted is fired")'")' after the suite is actually completed.

2.0.M5b Highlights:

Added an HTML reporter that gives a one-page view of the results of one test run. The HTML report includes a summary with a pie chart at the top left and a table of suites below that. Each row of the table gives the summary of one suite. By default the table is sorted to show failing suites, if any, at the top. You can resort the table by clicking on the column names, and view the detailed report of a suite by clicking on the suite name in the table. The detailed report will appear in the right half of the page. You can click on a link at the bottom of the suite report to open it in its own tab for printing. Here's an example HTML report generated from ScalaTest's run of itself. You can request an HTML report via the -h command of Runner.
Added a Status return type to the four “run methods” of trait Suite: run, runNestedSuites, runTests, and runTest. The Status can be queried to determine whether all the tests and/or nested suites started by the “run method” have completed and, if completed, whether any failure occurred. Created four subclasses of this trait for convenience: objects FailedStatus and SucceededStatus, useful when the “run” has already completed (such as when it was run on the same thread), class StatefulStatus, useful when a “run” has not yet completed (such as when multiple threads are involved), and CompositeStatus, useful for composing multiple Status objects into one Status. Status was added primarily to enable BeforeAndAfterAll to wait until any parallel events started by invoking run complete before executing the “after all” code. This will break any subclass that overrode one of the four run methods, but those are also already broken because of the change to the method's arguments. To fix it, you'll need to declare Status as the return type of your overloaded method, and return a Status.
Enhanced BeforeAndAfterAll so that it uses the Status object returned by run to wait until any parallel execution of tests and nested suites has completed before executing the “after all” code.
Dropped implicit conversions from structural types that enabled "have length/size" to be used on any object with an appropriate structure (such as a getSize method). In its place created Length and Size type class traits. This will be a breaking change for anyone who was relying on those structural implicits. This should affect very little code, but if it hits you, you'll need to create a type class as described in the Scaladoc for Length and Size.
Deprecated given/when/then/and methods of trait GivenWhenThen in favor of capitalized forms, Given/When/Then/And, because then has been deprecated as an identifier in Scala 2.10.
Bumped matcher collection types from Traversable to GenTraversable, Seq to GenSeq, etc., to make it easier to test with parallel collections.
Added traits Spec and fixture.Spec to serve as the new style trait in which tests are methods.
Deprecated trait Suite as a style trait.
Added trait SuiteMixin to serve as the new base class for stackable traits that can be mixed into Suite, and deprecated trait AbstractSuite.
Added the ScalaTest Selenium DSL.
Added trait TimesOnInt, which allows you to repeatedly execute a block of code an integer number of times via syntax such as, 3 times println.
Added the ability to dynamically tag tests, which facilitates features such as test name wildcards and rerunning previously failed tests; Dynamic tagging enables selection of tests even in nested suites that can't be instantiated directly.
Added to Runner a way to specify tests to run by full or partial ("wildcard")') name.
Enabled on-the-fly sorting of events (with a timeout) during parallel runs.
Added the ability to cancel tests if a needed resource is not available.
Added ability to tag all tests in a suite class by annotating the class with a tag annotation. For example, can ignore all tests in a suite class with @Ignore.
Added a @DoNotDiscover annotation that prevents discovery of an otherwise discoverable Suite class.
Simplified Suite's run method signatures, to make them more pleasant to override.
Enabled pretty output for test methods defined in backticks; Deprecated old-style test method names in Suite in favor of `test: ...` style.
DSL-ified Suite's execute method, including treating a given test name as a wildcard.
Reworked much of ScalaTest's extensive Scaladoc documentation to make code examples simpler, clearer, ensuring only the best, recommended style is shown. Added "Recommended Usage" callout boxes, tables, and text to ensure it is easy to understand the intended purpose of each of the various traits and features.
Added unformatted mode to stdout, stderr, and file reporters for helping debug problematic parallel test runs.
Added TestData trait that offers information about a test, some of which was already being passed into withFixture. In addition to the config map and test name, TestData also offers test text, scopes, and tags. In addition to being mixed into NoArgTest and OneArgTest, which are passed to withFixture, new forms of beforeEach and afterEach were added to BeforeAndafterEach that take a TestData (and the old forms that took only a config map were deprecated). This gives beforeEach and afterEach methods to the same information enjoyed by withFixture methods.

Enhancements in 2.0.M5b:

Added traits Spec and fixture.Spec to serve as the new style trait in which tests are methods.
Deprecated trait Suite as a style trait.
Added trait SuiteMixin to serve as the new base class for stackable traits that can be mixed into Suite, and deprecated trait AbstractSuite.
Added trait TimesOnInt, which allows you to repeatedly execute a block of code an integer number of times via syntax such as, 3 times println.
Added the org.scalatest.selenium package, which provides the ScalaTest Selenium DSL.
Added unformatted mode to stdout, stderr, and file reporters for helping debug problematic parallel test runs.
Added case class Args, to hold the arguments passed into Suite's run, runNestedSuites, runTests, and runTest methods. The purpose of Args is to simplify the signatures of these four "run")' lifecycle methods, making the signatures easier to remember and the methods more pleasant to override.
Added chosenStyles to Args, to hold the (possibly empty) set of styles chosen for the project. Previously this was passed around via the config map.
Added an overloaded run method to AbstractSuite that takes an optional test name string and a Args. Deprecated the old (and now overloaded) run method that takes seven args and added a final implementation of it to AbstractSuite that calls the new, two-arg form of run.
Changed runNestedSuites method so instead of taking six parameters it takes a single Args parameter. This will be a breaking change to any subtrait or subclass that overrode or invoked this method. See the breaking changes section for info on how to fix it.
Changed runTests method so instead of taking an optional test name string plus six parameters it takes the optional test name string and a single Args parameter. This will be a breaking change to any subtrait or subclass that overrode or invoked this method. See the breaking changes section for info on how to fix it.
Changed runTest method so instead of taking a test name string and six parameters it takes the test name string and a single Args parameter. This will be a breaking change to any subtrait or subclass that overrode or invoked this method. See the breaking changes section for info on how to fix it.
Narrowed the result types of newInstance in OneInstancePerTest and ParallelTestExecution to include the trait type. I.e., previously the result type of newInstance was just Suite. Now in OneInstancePerTest the result type is Suite with OneInstancePerTest, and in ParallelTestExecution is Suite with ParallelTestExecution.
Added several overloaded assume methods to trait Assertions. For symmetry, each assume method corresponds to an overloaded assert method with an identical signature. The assume methods behave similarly to the corresponding assert methods, except instead of throwing TestFailedException the assume methods throw TestCanceledException.
Added several overloaded cancel methods to trait Assertions. For symmetry, each cancel method corresponds to an overloaded fail method with an identical signature. The cancel methods behave similarly to the corresponding fail methods, except instead of throwing TestFailedException the cancel methods throw TestCanceledException.
Added a TestCanceled event to indicate a test has been canceled.
Added a TestCanceledException that indicates a test has been canceled. (This is thrown by the cancel and assume methods added in this release to trait Assertions.)
Added ScopeOpened and ScopeClosed events to indicate "scopes," such as the scope of a describe clause in a FunSpec. Previously a "scope opened" event was indiciated via an InfoProvided events with IndentedText.
Added a MarkupProvided event to indicate some markup text has been provided.
Added a markup() method to style traits in which tests are functions (I.e., the same style traits that already offered an info() method.) to allow markup text to be sent to the reporter from those traits. Added trait Documenter, the result type of markup.
Enhanced stdout, stderr, and file reporters to display markup text.
Added a testText field to all test events (TestStarting, TestIgnored, TestPending, TestCanceled, TestFailed, and TestSucceeded), primarily for integration with IntelliJ IDEA. The "test text" is either the complete test name or a suffix of it. In a FunSpec, for example, the test text will be the string passed to the it method, whereas the full test name will be the test text prefixed by the strings passed to enclosing describe method calls.
Added a location API for better integration with tools, especially IDEs. The location API consists of a sealed family of classes and objects, consisting of abstract base class Location, three case subclasses LineInFile, TopOfMethod, and TopOfClass, and singleton object SeeStackDepthException. (For the time being there's also a ToDoLocation we're using to mark places that aren't yet firing proper locations, but this will be removed before 2.0.final is released.)
Added location, an Option[Location] field, to class Event.
Gave Ordinal nicer toString output, for example: "Ordinal(99, 1, 0)".
Added a boolean excludeNestedSuites field to the primary constructor of class Filter. This field supports rerunning selected (such as previously failed) tests.
Added class DynaTags to support the dynamic tagging of tests. The dynamic tagging feature (new in 2.0) can be used, for example, to select a subset of tests to run (such as tests that previously failed), by tagging them with "Selected" and only including tests tagged "Selected")'.
Added a dynaTags field to the primary constructor of class Filter. This field supports the dynamic tagging of tests.
Added the suiteId lifecycle method to trait AbstractSuite and its subtraits. This new lifecycle method is intended to provide a unique ID for each suite that executes in a suite of tests, so that tests can be dynamically tagged. (A dynamic tag identifies tests by suiteId and testName.)
Added a suiteId to relevant events: TestStarting, TestSucceeded, TestFailed, TestFailed, TestPending, TestCanceled, SuiteStarting, SuiteCompleted, SuiteAborted. This enables tests run previously to be dynamically tagged in future runs (because a dynamic tag requires both the testName and suiteId).
Deprecated the previous two apply methods on Filter, and added two new ones that take an additional suiteId parameter.
Added the rerunner lifecycle method to AbstractSuite and its subtraits, which provides an optional fully qualified name of a suite (with a public, no-arg constructor) that can be used create to rerun its tests. If class has a public, no-arg constructor, then that class's fully qualified name can be returned from rerunner. If it does not have a public, no-arg constructor, but rather is created as a nested suite of a class that does have a public, no-arg constructor, then the nested class can return from rerunner the fully qualified name of that outer, nesting class. To rerun a test in the nested class, ScalaTest can create the nesting class via its public, no-arg constructor, and run it. The nesting class will create the actual class in which the test resides, and run it as a nested suite, thereby rerunning the test.
Added a socket reporter. This can be selected by passing a -k argument to Runner.
Added to the summary output of "string" reporters—i.e., the stdout, stderr, and file reporters—how many tests were canceled during a run.
Added decodedTestName, an Option[String], to test-related events. If defined, it means that the test was named using Scala's backtick notation, such as a test method in a Suite named `test: an empty stack should complain on pop`. The real test name will be encoded as test$colon$u0020an$u0020empty$u0020stack$u0020should$u0020complain$u0020on$u0020pop, whereas the decoded name will be test: an empty stack should complain on pop. This enables test names to be defined with backtick notation and still be presented to users in the more human-friendly, decoded form. If the decoded name would be identical to the encoded name (which will be the case if the name was not expressed in backticks and therefore never encoded in the first place), decodedTestName will be None. The events to which decodedTestName was added are: TestStarting, TestSucceeded, TestFailed, TestIgnored, TestPending, TestCanceled.
Added decodedSuiteName, an Option[String], to suite-related events. If defined, it means that the suite was named using Scala's backtick notation, such as `Stack Spec`. The real suite name will, by default, be encoded as Stack$u0020Spec, whereas the decoded name will be Stack Spec. This enables suite names to be defined with backtick notation and still be presented to users in the more human-friendly, decoded form. If the decoded name would be identical to the encoded name (which will be the case if the name was not expressed in backticks and therefore never encoded in the first place), decodedSuiteName will be None. The events to which decodedSuiteName was added are: TestStarting, TestSucceeded, TestFailed, TestIgnored, TestPending, TestCanceled, SuiteStarting, SuiteCompleted, SuiteAborted.
Added a DoNotDiscover annotation that can be used to tell Runner not to discover a suite class that it would otherwise discover.
Added a Finders annotation that specifies one or more Finders that can be used to determine selected tests from an AST in IDEs.
Added a DoNotDiscover annotation to prevent Suite classes that would otherwise be discoverable (because they extend Suite and have a public, no-arg constructor) from being discovered by Runner.
Added a -i command line argument to Runner to provide a way to specify a suite by suite ID. This complements the -s command, which allows you to identify a suite by fully qualified name. The -i command was added primarily to allow IDEs and other tools to rerun failed (or otherwise selected) tests.
Added a -t to Runner, a way to specify a specific test name. A -t clauses must follow -s or -i. All -t's immediately following a -s or -i will be considered tests of the suite specified by that -s or -i.
Added a -z argument to Runner, a means to specify test names via a wildcard: any test whose string name includes the string specified with -z will be selected. -z clauses must follow -s or -i clauses. All -z's immediately following a -s or -i will be considered wildcards selecting tests of the suite by that -s or -i. (Note: This approach to wildcard test names was suggested by Mathias Doenitz.)
Changed the result type of nestedSuites from List to immutable.IndexedSeq. (Under the covers we currently return a Vector.) When ScalaTest 1.0 was initially released, Vector did not yet exist in the Scala library and List was considered by many to be the default Seq type to use. In practice, it was discovered that it is too easy to use List (and other LinearSeq's) in a non-performant way. In 2.0 we wanted to take the opportunity to make the potentially breaking change of replacing List result types with immutable.IndexedSeq. This potentially breaking change is expected to affect very little, if any, user code.
Added a new overloaded apply method to Distributor that takes a Args, and deprecated the old form. During the deprecation period, the old form will call the new form with default args. The purpose of this change was to make more information available to Distributor implementations.
Added a recordedEvents field to test completion events: TestPending, TestSucceeded, TestFailed, and TestCanceled. Added sealed trait RecordedEvent, which extends Event and has two subtraits: InfoProvided and MarkupProvided. Prior to 2.0, InfoProvided events were recorded during tests and played back after, so that they could appear after the test text in the standard out reporter, and in a color that matched that of the test text. The color of the test text (green if passed, red if failed, yellow if pending, etc.) could not be known until after the test had completed. By sending recorded events along with the test completion event, rather than playing them back afterwords, it is easier for Reporters to figure out when the last event for a test has been received. Enhanced filtering so that recorded events would be filtered out if the user asked for that.
Changed a primary/auxiliary constructor pair to a primary constructor with a default parameter in Tracker.
Added default values for tagsToInclude and tagsToExclude in Filter.
Updated the JUnit XML Reporter so that it no longer uses ordinals in determining which events belonged to which suites. It now uses suiteId.
Added the ability to sort events locally when tests are run in parallel via ParallelTestExecution, with a timeout if something takes too long. This balances the desire to have sorted output while you are watching with the desire to see stuff happening while you are watching. Added sortingTimeout to ParallelTestExecution, the default of which is to use the timeout specified to Runner via -T (or 15 seconds if no -T given), but you can override the method. Added trait DistributedTestSorter to support this featuren, and a field holding one in Args
Added the ability to specify a timeout for sorting during parallel runs. The -T takes a number of seconds, and this is used for test sorting. The value specified with -T plus one second is used for suite sorting.
Added the ability to sort events locally when suites are run in parallel. This is disabled by default, and can be enabled with -PS argument to Runner.
Made the suiteId of DiscoverySuite unique in a run.
Ensure info() invocations in fixture.Suite are recorded and reported in the completion event.
Modified OneInstancePerTest so that it calls super.runTests which emits events for scopes (such as describe clauses in FunSpec).
Similarly, modified ParallelTestExecution so that it calls super.runTests which emits events for scopes (such as describe clauses in FunSpec).
Added duration to TestPending event.
Changed the overloaded apply factory methods in Filter to one apply method with default args.
Deprecated the old form of the public constructor in Filter, in favor of just using the apply method in the Filter companion object.
Adding type aliases in org.scalatest package object for ShouldMatchers and MustMatchers to save an import when using matchers.
Added new-style test method names (in backticks, starting with "test: ") in Suite and deprecated the old-style names.
Changed ConfigMapFixture so that it calls withFixture(test.toNoArgTest(...)) in its withFixture(OneArgTest) implementation. Previously it was just calling the test function directly. This enhancement enables trait stacking.
Added expectResult to Assertions, and deprecated expect. expect will be used for a different purpose in a future version of ScalaTest.
Did a major overhaul of the ScalaTest Scaladoc documentation. Most examples were rewritten to be simpler and clearer, ensuring they only show the best, recommended style. Callout boxes, tables, and text were added to ensure it is easy to understand the recommended usage of the various tools. Also aded a table of command-line arguments for Runner.
Enable tagging of all tests in a suite by annotating the suite class with a tag annotation. One common use case for this is to annotate a suite class with @Ignore to ignore all tests in the suite.
Hacked up a way to get ScalaTest's annotations (@DoNotDiscover, @Ignore, @Style, @TagAnnotation, and @WrapWith), which must be implemented in Java, to appear in ScalaTest's Scaladoc. This was done by pretending they are traits that extend java.lang.annotation.Annotation, and explaining at the top of the documentation that they are really Java @interfaces.
Renamed Matchers in package org.scalatest.matchers to ClassicMatchers and deprecated Matchers.

Bug fixes in 2.0.M5b:

Report an aborted suite as a failed test when running with JUnitRunner.
Print should/can/must from WordSpec and fixture.WordSpec in the standard out reporter for failed, canceled, and ignored tests.
Include a skipped attribute in the JUnit XML output for ignored, pending, and canceled tests.