ScalaTest User Guide

Writing your first test

Using assertions

Tagging your tests

Running your tests

Sharing fixtures

Sharing tests

Using matchers

Testing with mock objects

Tests as specifications

Property-based testing

Philosophy and design

Selecting a style

ScalaTest Release Notes

Changes in 1.6.1 (for Scala 2.9.x)

1.6.1 includes all the enhancements listed in the ScalaTest 1.5 release notes, plus a few other enhancements relative to 1.5:

  • Reworked some of the Scaladoc documentation, particularly the documentation on how to use shared fixtures to make clearer the intent and tradeoffs of each way of doing shared fixtures
  • Added the BeforeAndAfter trait and deprecated BeforeAndAfterEachFunctions and BeforeAndAfterAllFunctions. The BeforeAndAfterEach/AllFunctions traits were added in ScalaTest 1.3. They were deprecated for two reasons. One is that the trait names were too long. The other was that BeforeAndAfterEachFunctions has a beforeEach method that takes one parameter, and so does BeforeAndAfterEach. If these two traits were mixed together there was an (extremely unlikely) chance you could accidentally invoke the wrong beforeEach. BeforeAndAfter replaces BeforeAndAfterEachFunctions, with before replacing beforeEach. If you used BeforeAndAfterEachFunctions, you can get rid of the deprecation warning by changing your code to use BeforeAndAfter instead. If you used BeforeAndAfterAllFunctions, you can get rid of the deprecation warning by changing your code to use BeforeAndAfterAll instead.
  • Deprecated the MultipleFixtureXXX traits in the fixture package. The MultipleFixtureXXX traits were deprecated because because using explicit conversions to implement multiple fixtures yields more readable code than using implicit conversions. You can continue to use the implicit conversion approach if you wish, of course. To get rid of the deprecation warning, change "MultipleFixtureXXX" to "FixtureXXX with ConfigMapFixture".

Changes in 1.5.1 (for Scala 2.8.1)

Version 1.5.1 has the same features as version 1.6.1, but ScalaTest works on Scala 2.8.1+ whereas ScalaTest 1.6.1 works on Scala 2.9.0+. See the release notes for ScalaTest 1.6.1 for information on the enhancements this release contains on top of ScalaTest 1.5.

Changes in 1.5 (for Scala 2.8.1)

  • Changed the method signature of whenExecuting in EasyMockSugar so that it takes a single var arg of mock objects instead of one mock object followed by a second var arg of optional mock objects. The original goal was to make it a compiler error to not pass no mock objects. But a user who had a list of mock objects (say, named mocks) wanted to say whenExecuting(mocks: _*), and could not. I decided it was better to make that situation easy and just allow the error of passing no mocks to when executing to become a runtime error. Since this method will most likely be called inside test code, that error would simply result in a test failure. This change breaks binary compatibility, but not source code, unless someone was using named parameters in an invocation of whenExecuting, which is unlikely). So a simple recompile should suffice.
  • Added failedCodeFileName and failedCodeLineNumber methods to the StackDepth trait. These are implemented already, so should be source (and even binary) compatible, unless you have a method of that name already in a custom exception you made that extends StackDepth. Likely no one does.
  • Added failureOf method in a FailureOf trait. This method executes a passed block of code. If no exception is thrown it returns a None. Else if it is one of the exceptions that normally causes a test to fail in ScalaTest (all but some Error types), it returns that exception wrapped in a Some.
  • Added TableFor1 through TableFor22, which host tables of data on which property checks can be performed. Added a TableDrivenPropertyChecks trait that provides convenient factory methods for creating tables and checking properties against them. Fleshed out a PropertyCheckFailedException hierarchy, adding PropertyCheckFailedException ,TableDrivenPropertyCheckFailedException, and GeneratorDrivenPropertyCheckFailedException (deprecated PropertyTestFailedException in favor of the name GeneratorDrivenPropertyCheckFailedException).
  • Enhanced the standard out/error reporters, and the reporters used with sbt, so they show truncated stacktraces if available even if the exception contains or is a "cause" exception. Previously if a cause existed, full stack traces were always displayed.
  • Fixed tiny little display bug which caused "duplicate test name" to show up twice in the error message when duplicate tests were discovered in a FunSuite or FixtureFunSuite.
  • Made Iterable and Collection into Traversable in ScalaTest matchers. This was the final step in bringing matchers up to date with Scala 2.8. In the process, some classes that implement the matchers DSL were renamed. This will be a breaking change only if someone did something that used those explicit names, which is highly unlikely. For example, the name of implicit conversion convertIterableMatcherToArrayMatcher was changed to convertTraversableMatcherToArrayMatcher. Most people would be using that implicitly, so the name change won't break code. Anyone who just used the ScalaTest matchers DSL, and did not attempt to explicitly use its internals, will only need to recompile. In the process, ScalaTest's matchers became more general. Where before they were only working with Iterables, they now they work with Traversables.
  • Dropped the Suite self type from Checkers, and made a Checkers companion object that mixes in the Checkers trait: the self-less trait pattern.
  • Overrode compose on many types that extend function, narrowing the result type to the ScalaTest type. For example, overrode compose[U] on Matcher[T], which extends (T => MatchResult), and narrowed the result type to Matcher[U]. This way when you compose a Matcher and a function, you get a Matcher back. Also did this on BeMatcher, BePropertyMatcher, and HavePropertyMatcher.
  • Made Informer extend (String => Unit), the function type it always felt it was at heart.
  • Added a companion object for Matcher with a factory method that allows you to create a Matcher[T] from a passed function of type (T => MatchResult).
  • Added a companion object for BeMatcher with a factory method that allows you to create a BeMatcher[T] from a passed function of type (T => MatchResult).
  • Added a companion object for BePropertyMatcher with a factory method that allows you to create a BePropertyMatcher[T] from a passed function of type (T => MatchResult).
  • Added a toNoArgTest method to OneArgTest, to make it easier to delegate to an inherited implementation of the form of withFixture that takes a NoArgTest.
  • Created a Suites class that takes a variable length arg of suites to nest. Made SuperSuite extend this, and deprecated SuperSuite. Suites is a much clearer name, and I'd like to use "SuperSuite" and "SuperSpec" as the suggested name of a trait or class that mixes in your favorite things.
  • Created a Specs class that performs the same function as Suites, but sounds more BDDish.
  • Removed org.scalatest.BeforeAndAfter trait, which had been deprecated since 1.0.
  • Removed groups method from Suite, which had been deprecated since 1.0.
  • Removed org.scalatest.Group class, which had been deprecated since 1.0.
  • Removed org.scalatest.Report class, which had been deprecated since 1.0.
  • Removed org.scalatest.SpecReport class, which had been deprecated since 1.0.
  • Removed the failedTestCodeStackDepth and failedTestCodeFileNameAndLineNumberString methods from StackDepth and its descendants. These had both been deprecated since 1.0.
  • Removed trait Rerunnable, which had been deprecated since 1.0.
  • Removed ScalaTestTask, which had been deprecated since 1.0.
  • Started requiring TagAnnotation for tagging methods of Suite. As of 1.5 any annotations that aren't themselves annotated with org.scalatest.TagAnnotation will not be interpretted as tags. Interpretting arbitrary annotations as tags had been deprecated since 1.0.
  • Stop running nested suites in runNestedSuites and tests in runTests after the stopper returns true. The current test or nested suite running when the stop flag is set to true will continue, but will be the last, when running sequentially. When running tests in parallel, more than one may be currently running, which means more than one may continue to completion before the flag is again checked, after which it will stop. A Stopper is a polite request to stop, so some implementations of Suite may not stop right away once a stop is requested, but prior to this release, most traits in ScalaTest kept right on going after a stop was requested.
  • Fixed bug whereby passing in a non-existing test name to run on FunSuite and FixtureFunSuite would result in a failed test rather than an IllegalArgumentException.
  • Insert line breaks where they should be in the GUI reporter's message display. This made PropertyCheckFailedExceptions read more nicely in the GUI Reporter.
  • Enhanced StackDepthException by adding a new constructor. Moved the formerly primary constructor, which took an Option[String] message, Option[Throwable] cause, and Int failedCodeStackDepth, to be an auxiliary constructor. The new primary constructor takes functions that produce a message and the stack depth. This allows the calculation of these to be delayed. One reason is to avoid doing the processing to find the stack depth an build the string if it is never used, which can happen, for example, during a shrink phase of a failed property check. It also enables the message to include the failed code file name and line number. Also added one other auxiliary constructor, and modified TestFailedException, NotAllowedException, DuplicateTestNameException, TestRegistrationClosedException, but left the previous primary constructor as an auxiliary contructor in each case, so existing code will not break. (New exceptions PropertyTestFailedException, TableDrivenPropertyTestFailedException, and GeneratorDrivenPropertyTestFailedException also take the same functions.)
  • Changed the test names in FeatureSpec and FixtureFeatureSpec so they include "Scenario:". Given this FeatureSpec:
    class MySpec extends FeatureSpec {
      feature("Remote control mute button") {
        scenario("Mute button pressed when TV not muted") {}
        scenario("Mute button pressed when TV is muted") {}
      }
    }
    

    The test names prior to 1.5 would have been:

    Feature: Remote control mute button Mute button pressed when TV is muted
    Feature: Remote control mute button Mute button pressed when TV not muted
    

    And in 1.5 they are:

    Feature: Remote control mute button Scenario: Mute button pressed when TV is muted
    Feature: Remote control mute button Scenario: Mute button pressed when TV not muted
    

    This is a breaking change if anyone has written code that depends on those test name strings. It is unlikely anyone has done that, but if so, the fix is to insert "Scenario: " between the feature part and the scenario part of the test name.

  • Removed four overloaded execute methods on Suite and replaced them with one execute method that takes parameters with default argument values. This will not break source code unless you invoked the overloaded form of execute that takes a config map only. I figure it is unlikely anyone has done that, because config maps are used rarely and execute is likely most often called from an interpreter command not source code. If this does strike you, however, my apologies. The fix is to change this:
    mySuite.execute(myConfigMap)
    

    to this:

    mySuite.execute(configMap = myConfigMap)
    

    At the call site.

    In addition to testName and configMap, which you could pass to execute prior to 1.5, in 1.5 you can also configure the run with parameters color, durations, shortstacks, fullstacks, and stats. See the Scaladoc documenation for execute for details on these parameters.

  • Enhanced TDD-style traits so that they send formatted events. Previously only BDD-style traits sent formatted events. This change affects the output of Suite, FunSuite, JUnit3Suite, JUnitSuite, and TestNGSuite.
  • Enhanced style traits that allow nesting so that the output shows up nested as well. Previously all style traits flattened their output to at most one level of indentation. This change affects the output of Spec and WordSpec.
  • Previously you could choose between short and full stack traces in reporters that printed strings (such as the standard out reporter). Added an option in 1.5 to show no stack trace for stack depth exceptions, and made this the default. Added a 'S' configuration option for reporters that selects short stack traces. This option can be specified to Runner, the ant task, and sbt via the ScalaTestFramework implementation.
  • Added two new style traits, PropSpec and FreeSpec, including sister traits in the fixture package: FixturePropSpec, FixtureFreeSpec, MultipleFixturePropSpec, and MultipleFixtureFreeSpec.
  • Severed the inheritance relationship between function types and several of the traits passed to Suite.run. This was done to allow tools vendors to implement these types in Java. This could be a breaking change if a user used any of these types as a function. That is very unlikely, but just in case, also added implicit conversions from each type to the corresponding function type. This affected traits Reporter, Stopper, and Distributor.
  • Added the org.scalatest.Shell and an org.scalatest package object that implements "the ScalaTest shell," its DSL for the Scala interpreter.
  • Enhanced sbt integration so that SuiteStarting/Completed/Aborted events are fired. This makes the output when running with sbt consistent with the output when running via the Runner with the standard out reporter. (Mainly, you get the simple name of the suite class at the top, terminated by a colon.)

Changes in 1.4.1 (for Scala 2.9.x)

The goal of the 1.4.1 release is to provide a smooth transition to Scala 2.9.0 for ScalaTest users. It is identical to 1.3 except for changes required to get it to compile as 1.4.1 under Scala 2.9.0, plus one bug fix to the build to ensure the Java portions are cross-compiled for JDK 1.5.

  • Changed the ScalaTest 1.3 build file to ensure the parts of ScalaTest 1.4.1 written in Java (only two classes) get cross-compiled for JDK 1.5. (This bug fix will also soon be released as ScalaTest 1.3.1. Those two Java classes were accidentally compiled for JDK 1.6 in ScalaTest 1.3.)
  • Renamed 1.3 to 1.4.1.
  • Updated the Scala library and compiler to 2.9.0, and the ScalaCheck JAR file to version 1.8 compiled with Scala 2.9.0.
  • Added the Scala version number to the artifact ID. I.e., the artifact ID for ScalaTest 1.4.1 is scalatest_2.9.0 instead of plain old scalatest.
  • Recompiled 1.3 to run under 1.4.1.

Note: What happened to 1.4 you ask? We were given an early access release of Scala 2.9.0, so that ScalaTest could be recompiled for it early. We quietly deployed 1.4 so that other projects could be built prior to the official announcement that Scala 2.9.0 was ready. We didn't announce it, but nevertheless someone could have run across it and downloaded it. One day later, we got a brand new Scala 2.9.0. They had fixed a couple show-stopper bugs. We needed to redeploy, so we bumped the version number up to 1.4.1 in case someone had found 1.4 in the meantime. That 1.4 release was then removed from scala-tools.org.

Changes in 1.3 (for Scala 2.8.x)

  • Improved the sbt support provided by ScalaTestFramework by allowing the formatted output generated by specification-style traits like Spec or WordSpec to be seen by the user when running through sbt. Also enable color output.
  • Fixed a bug when running suites concurrently that usually manifested itself as an eventual out of memory error. The problem was the executor service being used wasn't being shutdown after each use. So if repeated runs were done in the same JVM, eventually too meany threads would pile up.
  • Added a SeveredStackTraces stack traces trait that violently chops off test-related stack traces just above the offending line of test code. Although ScalaTest's exception types provide sufficient information to do this when presenting a stack trace to the user, some tools (such as IDEs that run ScalaTest via JUnit) don't use this information yet and therefore show the full stack trace. SeveredStackTraces can be helpful until such tools provide better support for ScalaTest.
  • Added traits BeforeAndAfterEachFunctions and BeforeAndAfterAllFunctions, which offer a more concise form of before and after functionality (but which can't be stacked or provide access to a config map).
  • Updated the version of EasyMock to EasyMock 3.0. Unfortuately, EasyMock 3.0 breaks compatibility with previous versions of EasyMock by moving very important types to new packages. So if you're using EasyMock with ScalaTest, you'll need to upgrade to EasyMock 3.0 to use ScalaTest 1.3.
  • Updated the version of ScalaCheck to 1.8. If you're using ScalaTest with ScalaCheck, you'll likely need to upgrade to ScalaCheck 1.8 to use ScalaTest 1.3.
  • Added documentation explaining that because Conductor uses the result of calling thread.getState for control flow, occasionally Conductor can produce incorrect results.
  • Now only use as tags on Suite annotations that are themselves annotated with org.scalatest.TagAnnotation. This is a breaking change that was marked as deprecated since 1.0, and the deprecation cycle has ended.
  • Added withClue to Assertions to provide a way to supply extra information in an exception thrown by intercept or a ScalaTest matcher expression.
  • Made assert(a1 === a2) compare arrays structurally. In matcher expressions, the === operator means equality in terms of invoking ==, except for arrays, for which == checks that the two arrays are the same instance. For arrays, ScalaTest matchers passes the two arrays to java.util.Arrays.deepEquals. Now === has the same behavior for arrays when used in an assertion. As a result, the === operator now always compares arrays structurally no matter how it is used in ScalaTest. Note, this change would break any code that was intentionally using === to ensure two arrays are the same instance in an assert. It is likely no one was actually doing that, but if so, this will show up as a failed test and that code will need to be changed to assert(a1 eq a2). (Sorry, there was no way to deprecate a particular use of ===.)

Changes in 1.1 (for Scala 2.7.x) and 1.2 (for Scala 2.8.x)

Note: ScalaTest 1.1 and 1.2 basically have identical feature sets. The only difference is the Scala version they support. ScalaTest 1.1 works with Scala 2.7, and ScalaTest 1.2 with Scala 2.8. For a while we will support both 2.7 and 2.8 with new enhancements, with corresponding ScalaTest 1.1.x and 1.2.x releases. At some point we'll bump up to 1.3 and only support Scala 2.8 and beyond.

  • The main change from 1.0 to 1.1/1.2 is many changes to support Scala 2.8. Many of these changes went into both 1.1 and 1.2 codebases to simplify supporting both codebases in future releases. Some changes went just to the 1.2 branch. None of these should be visible to users of ScalaTest. In other words, the ScalaTest API did not need to change between Scala 2.7 and Scala 2.8, only its implementation.
  • Added dir attribute to ScalaTest ant task.
  • Renamed XML reporter to junitxml in the ScalaTest ant task.
  • Made Tag a concrete class (in earlier releases it was abstract), and added a Tag companion object with an apply method. The abstract Tag class enabled users to define tags with object SlowTest extends Tag("SlowTest"). This enhancements allows tags to also be created on the fly, for example, with Tag("SlowTest").
  • Added ScalaTestFramework to support sbt.
  • Fixed "smiley-face" bug in XML reporter, an extra few characters that caused the XML to be invalid.
  • When running failed or ignored tests in WordSpec, print the "should" verb in the GUI report, which was missing.

Changes in 1.0 (for Scala 2.7.x)

  • Dropped ImpSuite, which had been deprecated for two releases
  • Changed JUnit3Suite so it delegates to JUnit 4's JUnitCore class to run the tests. Before it was simulating JUnit execution without actually using JUnit.
  • Dropped two forms of intercept, which had been deprecated for two releases
  • Made Distributor into a function (breaking change).
  • Made Stopper into a function (breaking change).
  • Renamed Rerunnable into Rerunner and made it a function. Deprecated Rerunnable (breaking change).
  • Renamed main execute method (the one with many parameters) in Suite to run. Changed its signature (breaking change).
  • Added two more final execute methods to Suite that each take a configMap
  • Added a sealed hierarcy of Event classes.
  • Refactored Reporter so that instead of having one method per event type, it has one apply method, which takes an Event (breaking change).
  • Added ResourcefulReporter, which models a Reporter that holds finite, non-memory resources.
  • Added Ordinal. Every Event class includes an Ordinal so that the events of parallel runs can be sorted in sequential order. Made changes to all Suite traits to ensure they fire events with proper Ordinals.
  • Added Tracker to support the inclusion of good Ordinals in fired Events.
  • Added Filter, which changes how tags to include and exclude are handled. Previously the two include and exclude sets were passed into Suite methods, and it was the responsibility of those methods to filter tests correctly. Now the two include and exclude sets are passed to a Filter, and the Filter is passed to the Suite methods, which can delegate to Filter to do the filtering.
  • Renamed groups to tags (deprecated groups)
  • Greatly improved JUnit integration. Added JUnitSuite, JUnitRunner, JUnitWrapperSuite, AssertionsForJUnit, ShouldMatchersForJUnit, MustMatchersForJUnit, JUnitTestFailedError
  • Added -j option to Runner to run JUnit tests with the JUnitWrapperSuite
  • Changed the configuration parameters for Reporters. The old ones will be accepted but have no effect for a two-release deprecation cycle, after which they will cause Runner to halt with an error message.
  • Added optional color output to standard out, standard error, and file reporters.
  • Added Reporter configuration parameters for turning off color, showing durations, and showing full stack traces.
  • Added a jvmargs option to the Ant task, which makes it possible to pass args to the underlying JVM (important if you need more memory)
  • Added a fork option to the Ant task, which makes it possible to run ScalaTest in a separate JVM from Ant (also useful for dealing with memory problems).
  • Renamed goodies to configMap.
  • Renamed the Ant task from ScalaTestTask to ScalaTestAntTask. Deprecated ScalaTestTask.
  • Added ability to specify the number of threads to use when running tests in parallel, in both Runner and the Ant task.
  • Enhanced Runner so that it treats a backslash followed by a space as a literal space in a runpath element, rather than a space separating two runpath elements. This was necessary to allow people to include pathnames in the runpath that included spaces.
  • Added two new forms to matchers DSL: result should be === 7 and evaluating { "hi".charAt(-1) } should produce [StringIndexOutOfBoundsException].
  • Added support for pending tests.
  • Added pendingUntilFixed method to Suite.
  • Deprecated Report and SpecReport. These are no longer used, as they were made obsolete by the Reporter refactor. Instead of SpecReport, the Formatter and its two subclasses, MotionToSuppress and IndentedText are used to format specification-style output.
  • Added the org.scalatest.concurrent package, which includes Conductor, ConductorFixture, ConductorMultiFixture, and ConductorMethods.
  • Added the org.scalatest.mock package, which includes EasyMockSugar, JMockCycle, JMockCycleFixture, JMockExpectations, and MockitoSugar.
  • Added three new BDD traits: WordSpec, FlatSpec, and FeatureSpec.
  • Added the org.scalatest.fixture package, which includes FixtureSuite, FixtureFunSuite, FixtureSpec, FixtureWordSpec, FixtureFlatSpec, FixtureFeatureSpec, ConfigMapFixture, MultipleFixtureFunSuite, MultipleFixtureSpec, MultipleFixtureWordSpec, MultipleFixtureFlatSpec, and MultipleFixtureFeatureSpec.
  • Added the org.scalatest.verb package to support FlatSpec and WordSpec.
  • Split the BeforeAndAfter trait into BeforeAndAfterEach and BeforeAndAfterAll. Deprecated BeforeAndAfter.
  • Added trait GivenWhenThen.
  • Renamed Group to Tag. Deprecated Group. Also renamed groups method to tags in Suite, and deprecated groups.
  • Added TagAnnotation to serve as base class for tag annotations for Suite, however, won't enforce for a two-release deprecation cycle that tag annotations used on Suite test methods need to extend TagAnnotation.
  • Dropped the apply method that takes a Report (which is no longer used) from Informer, and also dropped the nameForReport method (breaking changes).
  • Added the OneInstancePerTest trait.
  • Added the ParallelTestExecution trait.
  • Added the SequentialNestedSuiteExecution trait.
  • Refactored TestFailedException so that it extends StackDepthException, which mixes in StackDepth. Added other StackDepthExceptions so that other errors in test code beyond just assertion failures will be reported with stack depths: DuplicateTestNameException, NotAllowedException, and TestRegistrationClosedException. Set aside StackDepth as a separate trait to support JUnitTestFailedError.
  • Renamed ExecuteAndRun to AbstractSuite, and added more methods to it, to facilitate the stackable trait pattern (breaking change).

Changes in 0.9.5

  • Added ShouldMatchers and MustMatchers traits, plus many supporting classes and traits.
  • Added support for SpecReport in the GUI, so that SpecReports get displayed in a specification style. This makes org.scalatest.Spec output look nicer in the GUI, and also makes it possible for org.specs.Specification to send SpecReports that will look nice in the ScalaTest GUI.
  • Enhanced the GUI to automatically select the first error, if there is one. This saves users a bit of time each error.
  • Added a TestFailedException, which allows a stack trace depth to be sent along with test failures. This allows reports to highlight the exactly filename and line number at which a test failed. This makes it quicker for users to find the cause of the problem. Enhanced everything to send TestFailedExceptions where possible.
  • Formatted the details area of the GUI so that it is easier to read, including making bold the most important part of the report, the report message and filename and line number, if it exists.
  • Made sure the entire stack trace is printed in the GUI's detail area. Previously it might say things like 37 more...
  • Made the GUI report list scroll during a rerun, to make it easier to find the reports of the rerun.
  • Filled in the name in run completed reports, which was missing.
  • Fixed a bug that prevented suites from being automatically discovered inside JAR files.
  • Added the exception that caused a ScalaCheck failure as the cause in the TestFailedException, which was not being propagated to the reporters before.
  • Display more nicely the information, arguments and labels, provided with a failed ScalaCheck property check.
  • In some cases, FunSuite and Spec were not sending Rerunnables in reports even though they could, which made rerunning not work in some cases in the GUI. They now send a Rerunnable whenever they can.
  • Removed SpecDasher from the API, which also is no longer mixed in by Spec. The only reason this was included in 0.9.4 was that it was demonstrated in Programming in Scala. It still works and can be used, so if you liked it, you can just grab it from 0.9.4 and mix it in yourself as a local mix in.

Changes in 0.9.4

  • Renamed trait ImpSuite to BeforeAndAfter. ImpSuite remains for now but is marked as deprecated. In a future release of ScalaTest ImpSuite will be dropped, so please rename your uses of ImpSuite to BeforeAndAfter as soon as convenient. The reason this was changed is that the "Imp" stood for imperative, because the trait facilitates an imperative style of testing. Also "imp" in English means "little devil," which projected a good attitude, because one goal of ScalaTest is to encourage users to adopt a more functional style for fixture "setup" and "teardown." But it's a bit too cute and its meaning not obvious enough from its name. Plus mixing an ImpSuite into a Spec felt like mixing metaphors.
  • Pulled the triple equals operator mechanism and the expect and intercept constructs from Suite into their own trait, named Assertions. Suite now mixes in this trait, so that it has the same functionality as before. The main reason this was done was so that these constructs could more easily be used outside of ScalaTest Suites, and also so the trait could be extended by ScalaTest's matchers. (ScalaTest's matchers are not yet released, but will probably make their debut in ScalaTest version 0.9.5.) This allows ScalaTest's matchers to assume these constructs exist, and makes it easier to use ScalaTest matchers independently from ScalaTest Suites.
  • Added a version of intercept that takes an implicit Manifest, so that you need not explicitly pass a class instance anymore. Thus, instead of saying this:
    intercept(classOf[StringIndexOutOfBoundsException]) {
      "hi".charAt(-1)
    }
    
    You can write:
    intercept[StringIndexOutOfBoundsException] {
      "hi".charAt(-1)
    }
    
    The old style that requires an explicit class instance are still in the API, but they are deprecated. They will be removed in some future release of ScalaTest, so please start migrating to the new style. There are a few corner cases where the new form can't be used, which are caused by overloading ambiguities with the old style. The two corners are passing an explicit null to intercept and throwing an explicit exception such that the inferred type of the by-name parameter is Nothing. It is unlikely anyone would do that anyway, and these two issues will resolve themselves once the deprecated old-style intercept methods are removed.
  • Added Spec. This trait is inspired by Ruby's RSpec BDD tool.
  • Renamed PropSuite to FunSuite in org.scalatest.prop. This was done because it made sense to add the extra ScalaCheck methods to Spec as well, so it seemed more reasonable to just have traits with the same name in two different packages. So there would be a FunSuite and Spec in both org.scalatest and org.scalatest.prop. The latter batch would already mix in Checkers and offer the extra ScalaCheck property-taking methods, and of course, have a dependency on ScalaCheck. Thus, PropSuite was deprecated, but it is left in the API for the time being to avoid client code breakage. PropSuite will go away in a future release. Later, the traits that had the extra property-taking methods were judged not worth their weight, so ScalaTest 0.9.4 does not include a Spec in package org.scalatest.prop. However because an example using org.scalatest.prop.FunSuite was included in Programming in Scala, it was included in 0.9.4, already deprecated. Both PropSuite and FunSuite will be removed from the org.scalatest.prop package in a future ScalaTest release, so please start migrating to use org.scalatest.FunSuite, mixing in Checkers and passing properties to ScalaCheck with check, as shown in this example.
  • Released SpecDasher trait. This may or may not be a good idea, but an example of the "dashes" style it enables was shown in Programming in Scala, so it was included in 0.9.4, but deprecated. It may stay or go in a future release, depending on user feedback. Currently, Spec already mixes SpecDasher in by default, but if it ends up staying in the API, this connection will almost certainly be severed so that users must explicitly mix it in to get the "dashes" style. So use it with caution, but even if it is removed, if you like it you can always just add the code for it to your own project and use it that way.
  • Added SpecReport, a subclass of Report. Specs emit SpecReports when they run, and the print reporters (standard out, err, file, etc.) recognize them and generate specification-like output. In this release the graphic reporter does not handle SpecReport's specially, so the output looks like regular test output. In a future release the graphic reporter will generate a specification-like output. One use case for SpecReports is integration with Eric Torreborre's specs tool, however one piece still remains for that integration. Specs (and JUnit and TestNG) generate nested test output, whereas ScalaTest's model is flat. In a future version ScalaTest's Report class will be enhanced to support nested reports. This will enable specs tests run through ScalaTest to provide nice nested, specification-like reports.
  • Created an ExecuteAndRun trait. The purpose of this trait is to pull out abstract method signatures for execute and runTest (and later if it turns out to be useful, runNestedSuites and runTests) so they can be called by traits that have Suite as their self type, but extend ExecuteAndRun not Suite, such as BeforeAndAfter. This makes such traits stackable behaviors that can be mixed into a Suite while not being a Suite themselves.
  • Removed testWithInformer and ignoreWithInformer methods from FunSuite, because they were simply too ugly to live. Unfortunately these were not deprecated, so if you used them you'll need to already make an adjustment to upgrade to 0.9.4. What replaced them is an imperative approach, which while more error prone will work fine unless abused, and makes for much nicer client code. There's a new method named info in FunSuite, which returns an Informer. So where before you said:
    testWithInformer("test name") {
      info => {
        info("send info to the reporter")
      }
    }
    
    Now you'll just say:
    test("test name") {
      info("send info to the reporter")
    }
    
    During suite execution, the info method returns an Informer that immediately forwards information passed to its apply methods to the reporter. During construction of the object, the info method returns an Informer that saves the information passed to its apply methods for later sending to the reporter when the suite is executed. At any other time, the info method will return an Informer that throws an exception.

Changes in 0.9.3

  • Moved FunSuite from the org.scalatest.fun to org.scalatest package.
  • Dropped FunSuite1 through FunSuite9, which were also in the org.scalatest.fun package, because the surface area they added to the API didn't seem worth the small savings in code size over creating "with" methods with meaningful names and explicitly calling them.
  • Added an ImpSuite, which can be mixed into another suite to gain methods that will run before and after each suite and test.
  • Updated the portions of Scaladoc comments for Suite and FunSuite that discuss mutable fixtures. Dropped the suggestion to override runTest, which still works but is verbose, and instead recommended defining and explicitly calling "create" methods or "with" methods from tests that need the mutable fixtures, or mixing in ImpSuite and using beforeEach, afterEach, etc.
  • Added an Informer trait, which provides a simpler way to send information to the infoProvided method of a Reporter.
  • Changed Suite such that instead of being able to define test methods that take a Reporter, you can define test methods that take an Informer. Replaced the section of scaladoc for Suite that used to explain how to use a passed-in Reporter to now show how to use an Informer.
  • Changed FunSuite such that instead of being able to register tests whose function values take a Reporter, you can register tests whose function values take an Informer. Changed the names of the registration methods from test/ignoreWithReporter to test/ignoreWithInformer. Replaced the section of scaladoc for FunSuite that used to explain how to use a Reporter passed in to a test function to now show how to use an Informer.
  • Moved org.scalatest.scalacheck.CheckSuite to org.scalatest.prop.Checkers, and renamed its checkProperty methods to plain old check.
  • Added PropSuite, a subtrait of FunSuite, which offers methods that register tests composed of a single property check.
  • Improved the messages generated by the assert(... === ...) and expect(...){...} methods. Strings are now reported surrounded by double quotes, with ends truncated where matching if very long, and with square brackets surrounding the portions that differ. Characters are reported surrounded by single quotes, integer and floating point numbers as is, and any other object's toString result surrounded by angle brackets.
  • Removed dollar signs and non-essential text from suite and test names in reports when running from the interpreter.
  • Improved the look of the output generated by the build-in reporters that print to the standard out, error, or to files.
  • Added a section to Suite's scaladoc that describes using external assertions, such as JUnit's assertions, Hamcrest matchers, or specs matchers, in ScalaTest tests.
  • Improved the failure message when a ScalaCheck property check fails.
  • Renamed user-defined properties to "goodies," because the term "properties" was overloaded to mean both key-value pairs that can be passed to suites and ScalaCheck-like properties. "Goodies" is not a very obvious name, so suggestions for improvement are welcome.

Changes in 0.9.2

  • Moved Runner from org.scalatest to org.scalatest.tools package.
  • Added JUnit integration classes in package org.scalatest.junit.
  • Added TestNG integration classes in package org.scalatest.testng.
  • Added ScalaCheck integration classes in package org.scalatest.scalacheck.
  • Added FunSuite family of traits in package org.scalatest.fun.
  • Added an Ant task.

Changes in 0.9.1

  • Changed testGroups method name to groups.
  • Changed operand names of ===, from expected === actual to left === right.
  • Changed message generated from 2 === 1, from "Expected 1, but got 2" to "2 did not equal 1".
  • Changed expect method name to intercept.
  • Dropped expectNPE and expectIAE methods.
  • Added a different expect method, which takes an expected and actual value. expect(1) {2} would generate the error message "Expected 1, but got 2.
  • Made versions of expect and intercept that take a message.
  • Implemented a Suite discovery mechanism in Runner, accessed via -m or -w command-line arguments.
  • Dropped !== method from the great Equalizer.

Known issues

  1. Reloading classes from JAR files between runs in the GUI doesn't work, because URLClassLoader uses a JAR cache. For now, you'll have to restart the app to pick up new classes in JAR files. But a better alternative is to simply point ScalaTest to the build directory containing .class files produced by the Scala and Java compiler. URLClassLoader will not cache these files, so changes to them will be picked up each time you press the Run or Rerun buttons.

ScalaTest is brought to you by Bill Venners, with contributions from several other folks. It is sponsored by Artima, Inc.
ScalaTest is free, open-source software released under the Apache 2.0 license.

Copyright © 2009-2011 Artima, Inc. All Rights Reserved.

artima