ScalaTest User Guide
Writing your first test
Using assertions
Tagging your tests
Running your tests
Sharing fixtures
Sharing tests
Using matchers
Testing with mock objects
Tests as specifications
Property-based testing
Philosophy and design
Selecting a style
|
ScalaTest Release Notes
Changes in 1.6.1 (for Scala 2.9.x)
1.6.1 includes all the enhancements listed in the ScalaTest 1.5 release notes, plus a few other enhancements relative to 1.5:
- Reworked some of the Scaladoc documentation, particularly the documentation on how to use shared fixtures to make clearer the intent
and tradeoffs of each way of doing shared fixtures
- Added the
BeforeAndAfter trait and deprecated BeforeAndAfterEachFunctions and BeforeAndAfterAllFunctions.
The BeforeAndAfterEach/AllFunctions traits were added in ScalaTest 1.3. They were deprecated for two reasons. One is that the trait names were too long. The other was that BeforeAndAfterEachFunctions has a beforeEach method that takes one parameter, and so does BeforeAndAfterEach. If these two traits were mixed together there was an (extremely unlikely) chance you could accidentally invoke the wrong beforeEach. BeforeAndAfter replaces BeforeAndAfterEachFunctions, with before replacing beforeEach. If you used BeforeAndAfterEachFunctions, you can get rid of the deprecation warning by changing your code to use BeforeAndAfter instead. If you used BeforeAndAfterAllFunctions, you can get rid of the deprecation warning by changing your code to use BeforeAndAfterAll instead.
- Deprecated the
MultipleFixtureXXX traits in the fixture package.
The MultipleFixtureXXX traits were deprecated because because using explicit conversions to implement multiple fixtures yields more readable code than using implicit conversions. You can continue to use the implicit conversion approach if you wish, of course. To get rid of the deprecation warning, change "MultipleFixtureXXX" to "FixtureXXX with ConfigMapFixture".
Changes in 1.5.1 (for Scala 2.8.1)
Version 1.5.1 has the same features as version 1.6.1, but ScalaTest works on Scala 2.8.1+ whereas ScalaTest 1.6.1 works on Scala 2.9.0+.
See the release notes for ScalaTest 1.6.1 for information on the enhancements this release contains on top of ScalaTest 1.5.
Changes in 1.5 (for Scala 2.8.1)
- Changed the method signature of
whenExecuting in EasyMockSugar so that it takes a single var arg of mock objects instead of one mock object followed by a second var arg of optional mock objects. The original goal was to make it a compiler error to not pass no mock objects. But a user who had a list of mock objects (say, named mocks) wanted to say whenExecuting(mocks: _*), and could not. I decided it was better to make that situation easy and just allow the error of passing no mocks to when executing to become a runtime error. Since this method will most likely be called inside test code, that error would simply result in a test failure. This change breaks binary compatibility, but not source code, unless someone was using named parameters in an invocation of whenExecuting, which is unlikely). So a simple recompile should suffice.
- Added
failedCodeFileName and failedCodeLineNumber methods to the StackDepth trait. These are implemented already, so should be source (and even binary) compatible, unless you have a method of that name already in a custom exception you made that extends StackDepth. Likely no one does.
- Added
failureOf method in a FailureOf trait. This method executes a passed block of code. If no exception is thrown it returns a None. Else if it is one of the exceptions that normally causes a test to fail in ScalaTest (all but some Error types), it returns that exception wrapped in a Some.
- Added
TableFor1 through TableFor22, which host tables of data on which property checks can be performed. Added a TableDrivenPropertyChecks trait that provides convenient factory methods for creating tables and checking properties against them. Fleshed out a PropertyCheckFailedException hierarchy, adding PropertyCheckFailedException ,TableDrivenPropertyCheckFailedException, and GeneratorDrivenPropertyCheckFailedException (deprecated PropertyTestFailedException in favor of the name GeneratorDrivenPropertyCheckFailedException).
- Enhanced the standard out/error reporters, and the reporters used with sbt, so they show truncated stacktraces if available even if the exception contains or is a "cause" exception. Previously if a cause existed, full stack traces were always displayed.
- Fixed tiny little display bug which caused "duplicate test name" to show up twice in the error message when duplicate tests were discovered in a
FunSuite or FixtureFunSuite.
- Made
Iterable and Collection into Traversable in ScalaTest matchers. This was the final step in bringing matchers up to date with Scala 2.8. In the process, some classes that implement the matchers DSL were renamed. This will be a breaking change only if someone did something that used those explicit names, which is highly unlikely. For example, the name of implicit conversion convertIterableMatcherToArrayMatcher was changed to convertTraversableMatcherToArrayMatcher. Most people would be using that implicitly, so the name change won't break code. Anyone who just used the ScalaTest matchers DSL, and did not attempt to explicitly use its internals, will only need to recompile. In the process, ScalaTest's matchers became more general. Where before they were only working with Iterables, they now they work with Traversables.
- Dropped the
Suite self type from Checkers, and made a Checkers companion object that mixes in the Checkers trait: the self-less trait pattern.
- Overrode
compose on many types that extend function, narrowing the result type to the ScalaTest type. For example, overrode compose[U] on Matcher[T], which extends (T => MatchResult), and narrowed the result type to Matcher[U]. This way when you compose a Matcher and a function, you get a Matcher back. Also did this on BeMatcher, BePropertyMatcher, and HavePropertyMatcher.
- Made
Informer extend (String => Unit), the function type it always felt it was at heart.
- Added a companion object for
Matcher with a factory method that allows you to create a Matcher[T] from a passed function of type (T => MatchResult).
- Added a companion object for
BeMatcher with a factory method that allows you to create a BeMatcher[T] from a passed function of type (T => MatchResult).
- Added a companion object for
BePropertyMatcher with a factory method that allows you to create a BePropertyMatcher[T] from a passed function of type (T => MatchResult).
- Added a
toNoArgTest method to OneArgTest, to make it easier to delegate to an inherited implementation of the form of withFixture that takes a NoArgTest.
- Created a
Suites class that takes a variable length arg of suites to nest. Made SuperSuite extend this, and deprecated SuperSuite. Suites is a much clearer name, and I'd like to use "SuperSuite" and "SuperSpec" as the suggested name of a trait or class that mixes in your favorite things.
- Created a
Specs class that performs the same function as Suites, but sounds more BDDish.
- Removed
org.scalatest.BeforeAndAfter trait, which had been deprecated since 1.0.
- Removed
groups method from Suite, which had been deprecated since 1.0.
- Removed
org.scalatest.Group class, which had been deprecated since 1.0.
- Removed
org.scalatest.Report class, which had been deprecated since 1.0.
- Removed
org.scalatest.SpecReport class, which had been deprecated since 1.0.
- Removed the
failedTestCodeStackDepth and failedTestCodeFileNameAndLineNumberString methods from StackDepth and its descendants. These had both been deprecated since 1.0.
- Removed trait
Rerunnable, which had been deprecated since 1.0.
- Removed ScalaTestTask, which had been deprecated since 1.0.
- Started requiring
TagAnnotation for tagging methods of Suite. As of 1.5 any annotations that aren't themselves annotated with org.scalatest.TagAnnotation will not be interpretted as tags. Interpretting arbitrary annotations as tags had been deprecated since 1.0.
- Stop running nested suites in
runNestedSuites and tests in runTests after the stopper returns true. The current test or nested suite running when the stop flag is set to true will continue, but will be the last, when running sequentially. When running tests in parallel, more than one may be currently running, which means more than one may continue to completion before the flag is again checked, after which it will stop. A Stopper is a polite request to stop, so some implementations of Suite may not stop right away once a stop is requested, but prior to this release, most traits in ScalaTest kept right on going after a stop was requested.
- Fixed bug whereby passing in a non-existing test name to run on
FunSuite and FixtureFunSuite would result in a failed test rather than an IllegalArgumentException.
- Insert line breaks where they should be in the GUI reporter's message display. This made
PropertyCheckFailedExceptions read more nicely in the GUI Reporter.
- Enhanced
StackDepthException by adding a new constructor. Moved the formerly primary constructor, which took an Option[String] message, Option[Throwable] cause, and Int failedCodeStackDepth, to be an auxiliary constructor. The new primary constructor takes functions that produce a message and the stack depth. This allows the calculation of these to be delayed. One reason is to avoid doing the processing to find the stack depth an build the string if it is never used, which can happen, for example, during a shrink phase of a failed property check. It also enables the message to include the failed code file name and line number. Also added one other auxiliary constructor, and modified TestFailedException, NotAllowedException, DuplicateTestNameException, TestRegistrationClosedException, but left the previous primary constructor as an auxiliary contructor in each case, so existing code will not break. (New exceptions PropertyTestFailedException, TableDrivenPropertyTestFailedException, and GeneratorDrivenPropertyTestFailedException also take the same functions.)
- Changed the test names in
FeatureSpec and FixtureFeatureSpec so they include "Scenario:". Given this FeatureSpec:
class MySpec extends FeatureSpec {
feature("Remote control mute button") {
scenario("Mute button pressed when TV not muted") {}
scenario("Mute button pressed when TV is muted") {}
}
}
The test names prior to 1.5 would have been:
Feature: Remote control mute button Mute button pressed when TV is muted
Feature: Remote control mute button Mute button pressed when TV not muted
And in 1.5 they are:
Feature: Remote control mute button Scenario: Mute button pressed when TV is muted
Feature: Remote control mute button Scenario: Mute button pressed when TV not muted
This is a breaking change if anyone has written code that depends on those test name strings. It is unlikely anyone has done that, but if so, the fix is to insert "Scenario: " between the feature part and the scenario part of the test name.
- Removed four overloaded execute methods on
Suite and replaced them with one execute method that takes parameters with default argument values. This will not break source code unless you invoked the overloaded form of execute that takes a config map only. I figure it is unlikely anyone has done that, because config maps are used rarely and execute is likely most often called from an interpreter command not source code. If this does strike you, however, my apologies. The fix is to change this:
mySuite.execute(myConfigMap)
to this:
mySuite.execute(configMap = myConfigMap)
At the call site.
In addition to testName and configMap, which you could pass to execute prior to 1.5, in 1.5 you can also configure the run with parameters color, durations, shortstacks, fullstacks, and stats. See the Scaladoc documenation for execute for details on these parameters.
- Enhanced TDD-style traits so that they send formatted events. Previously only BDD-style traits sent formatted events. This change affects the output of
Suite, FunSuite, JUnit3Suite, JUnitSuite, and TestNGSuite.
- Enhanced style traits that allow nesting so that the output shows up nested as well. Previously all style traits flattened their output to at most one level of indentation. This change affects the output of
Spec and WordSpec.
- Previously you could choose between short and full stack traces in reporters that printed strings (such as the standard out reporter). Added an option in 1.5 to show no stack trace for stack depth exceptions, and made this the default. Added a
'S' configuration option for reporters that selects short stack traces. This option can be specified to Runner, the ant task, and sbt via the ScalaTestFramework implementation.
- Added two new style traits,
PropSpec and FreeSpec, including sister traits in the fixture package: FixturePropSpec, FixtureFreeSpec, MultipleFixturePropSpec, and MultipleFixtureFreeSpec.
- Severed the inheritance relationship between function types and several of the traits passed to
Suite.run. This was done to allow tools vendors to implement these types in Java. This could be a breaking change if a user used any of these types as a function. That is very unlikely, but just in case, also added implicit conversions from each type to the corresponding function type. This affected traits Reporter, Stopper, and Distributor.
- Added the
org.scalatest.Shell and an org.scalatest package object that implements "the ScalaTest shell," its DSL for the Scala interpreter.
- Enhanced sbt integration so that
SuiteStarting/Completed/Aborted events are fired. This makes the output when running with sbt consistent with the output when running via the Runner with the standard out reporter. (Mainly, you get the simple name of the suite class at the top, terminated by a colon.)
Changes in 1.4.1 (for Scala 2.9.x)
The goal of the 1.4.1 release is to provide a smooth transition to Scala 2.9.0 for ScalaTest users. It is identical
to 1.3 except for changes required to get it to compile as 1.4.1 under Scala 2.9.0, plus one bug fix to the build to ensure
the Java portions are cross-compiled for JDK 1.5.
- Changed the ScalaTest 1.3 build file to ensure the parts of ScalaTest 1.4.1 written in Java (only two classes) get cross-compiled
for JDK 1.5. (This bug fix will also soon be released as ScalaTest 1.3.1. Those two Java classes were accidentally
compiled for JDK 1.6 in ScalaTest 1.3.)
- Renamed 1.3 to 1.4.1.
- Updated the Scala library and compiler to 2.9.0, and the ScalaCheck JAR file to version 1.8 compiled with Scala 2.9.0.
- Added the Scala version number to the artifact ID. I.e., the artifact ID for ScalaTest 1.4.1 is
scalatest_2.9.0
instead of plain old scalatest.
- Recompiled 1.3 to run under 1.4.1.
Note: What happened to 1.4 you ask? We were given an early access release of Scala 2.9.0, so that ScalaTest could be recompiled for it
early. We quietly deployed 1.4 so that other projects could be built prior to the official announcement that Scala 2.9.0 was ready. We didn't
announce it, but nevertheless someone could have run across it and downloaded it. One day later, we got a brand new Scala 2.9.0. They had
fixed a couple show-stopper bugs. We needed to redeploy, so we bumped the version number up to 1.4.1 in case someone had found 1.4 in the
meantime. That 1.4 release was then removed from scala-tools.org.
Changes in 1.3 (for Scala 2.8.x)
- Improved the sbt support provided by
ScalaTestFramework by allowing the formatted
output generated by specification-style traits like Spec or WordSpec to be seen by the
user when running through sbt. Also enable color output.
- Fixed a bug when running suites concurrently that usually manifested itself as an eventual out of memory error. The
problem was the executor service being used wasn't being shutdown after each use. So if repeated runs were done in the same JVM,
eventually too meany threads would pile up.
- Added a
SeveredStackTraces stack traces trait that violently chops off test-related stack traces just above the
offending line of test code. Although ScalaTest's exception types provide sufficient information to do this when presenting a
stack trace to the user, some tools (such as IDEs that run ScalaTest via JUnit) don't use this information yet and therefore show
the full stack trace. SeveredStackTraces can be helpful until such tools provide better support for
ScalaTest.
- Added traits
BeforeAndAfterEachFunctions and BeforeAndAfterAllFunctions, which offer a more concise form
of before and after functionality (but which can't be stacked or provide access to a config map).
- Updated the version of EasyMock to EasyMock 3.0. Unfortuately, EasyMock 3.0 breaks compatibility with previous versions of EasyMock by
moving very important types to new packages. So if you're using EasyMock with ScalaTest, you'll need to upgrade to EasyMock 3.0 to use
ScalaTest 1.3.
- Updated the version of ScalaCheck to 1.8. If you're using ScalaTest with ScalaCheck, you'll likely need to upgrade to ScalaCheck 1.8 to use
ScalaTest 1.3.
- Added documentation explaining that because
Conductor uses the result of calling thread.getState for control flow,
occasionally Conductor can produce incorrect results.
- Now only use as tags on
Suite annotations that are themselves annotated with org.scalatest.TagAnnotation. This
is a breaking change that was marked as deprecated since 1.0, and the deprecation cycle has ended.
- Added
withClue to Assertions to provide a way to supply extra information in an exception thrown by intercept
or a ScalaTest matcher expression.
- Made
assert(a1 === a2) compare arrays structurally. In matcher expressions, the === operator means equality in terms
of invoking ==, except for arrays, for which == checks that the two arrays are the same instance. For arrays, ScalaTest
matchers passes the two arrays to java.util.Arrays.deepEquals. Now === has the same behavior for arrays when used in
an assertion. As a result, the === operator now always compares arrays structurally no matter how it is used in ScalaTest. Note, this
change would break any code that was intentionally using === to ensure two arrays are the same instance in an assert. It is
likely no one was actually doing that, but if so, this will show up as a failed test and that code will need to be
changed to assert(a1 eq a2). (Sorry, there was no way to deprecate a particular use of ===.)
Changes in 1.1 (for Scala 2.7.x) and 1.2 (for Scala 2.8.x)
Note: ScalaTest 1.1 and 1.2 basically have identical feature sets. The only difference is the Scala version they support. ScalaTest 1.1 works with Scala 2.7,
and ScalaTest 1.2 with Scala 2.8. For a while we will support both 2.7 and 2.8 with new enhancements, with corresponding ScalaTest 1.1.x and 1.2.x releases.
At some point we'll bump up to 1.3 and only support Scala 2.8 and beyond.
- The main change from 1.0 to 1.1/1.2 is many changes to support Scala 2.8. Many of these changes went into both 1.1 and 1.2 codebases to
simplify supporting both codebases in future releases. Some changes went just to the 1.2 branch. None of these should be visible
to users of ScalaTest. In other words, the ScalaTest API did not need to change between Scala 2.7 and Scala 2.8, only its implementation.
- Added
dir attribute to ScalaTest ant task.
- Renamed XML reporter to
junitxml in the ScalaTest ant task.
- Made
Tag a concrete class (in earlier releases it was abstract), and added a Tag companion object with an apply method. The
abstract Tag class enabled users to define tags with object SlowTest extends Tag("SlowTest"). This enhancements allows
tags to also be created on the fly, for example, with Tag("SlowTest").
- Added
ScalaTestFramework to support sbt.
- Fixed "smiley-face" bug in XML reporter, an extra few characters that caused the XML to be invalid.
- When running failed or ignored tests in WordSpec, print the "should" verb in the GUI report, which was missing.
Changes in 1.0 (for Scala 2.7.x)
- Dropped
ImpSuite, which had been deprecated for two releases
- Changed
JUnit3Suite so it delegates to JUnit 4's JUnitCore class to run the tests. Before it was simulating JUnit execution without
actually using JUnit.
- Dropped two forms of
intercept, which had been deprecated for two releases
- Made
Distributor into a function (breaking change).
- Made
Stopper into a function (breaking change).
- Renamed
Rerunnable into Rerunner and made it a function. Deprecated Rerunnable (breaking change).
- Renamed main
execute method (the one with many parameters) in Suite to run. Changed its signature (breaking change).
- Added two more final
execute methods to Suite that each take a configMap
- Added a sealed hierarcy of
Event classes.
- Refactored
Reporter so that instead of having one method per event type, it has one apply method, which takes an Event (breaking change).
- Added
ResourcefulReporter, which models a Reporter that holds finite, non-memory resources.
- Added
Ordinal. Every Event class includes an Ordinal so that the events of parallel runs can be sorted
in sequential order. Made changes to all Suite traits to ensure they fire events with proper Ordinals.
- Added
Tracker to support the inclusion of good Ordinals in fired Events.
- Added
Filter, which changes how tags to include and exclude are handled. Previously the two include and exclude sets were passed into
Suite methods, and it was the responsibility of those methods to filter tests correctly. Now the two include and exclude sets are passed to a Filter,
and the Filter is passed to the Suite methods, which can delegate to Filter to do the filtering.
- Renamed
groups to tags (deprecated groups)
- Greatly improved JUnit integration. Added
JUnitSuite, JUnitRunner, JUnitWrapperSuite, AssertionsForJUnit, ShouldMatchersForJUnit, MustMatchersForJUnit, JUnitTestFailedError
- Added -j option to
Runner to run JUnit tests with the JUnitWrapperSuite
- Changed the configuration parameters for
Reporters. The old ones will be accepted but have no effect for a two-release deprecation
cycle, after which they will cause Runner to halt with an error message.
- Added optional color output to standard out, standard error, and file reporters.
- Added
Reporter configuration parameters for turning off color, showing durations, and showing full stack traces.
- Added a
jvmargs option to the Ant task, which makes it possible to pass args to the underlying JVM (important if you need more memory)
- Added a
fork option to the Ant task, which makes it possible to run ScalaTest in a separate JVM from Ant (also useful for dealing with memory problems).
- Renamed
goodies to configMap.
- Renamed the Ant task from
ScalaTestTask to ScalaTestAntTask. Deprecated ScalaTestTask.
- Added ability to specify the number of threads to use when running tests in parallel, in both
Runner and the Ant task.
- Enhanced
Runner so that it treats a backslash followed by a space as a literal space in a runpath element, rather than a space separating two
runpath elements. This was necessary to allow people to include pathnames in the runpath that included spaces.
- Added two new forms to matchers DSL:
result should be === 7 and evaluating { "hi".charAt(-1) } should produce [StringIndexOutOfBoundsException].
- Added support for pending tests.
- Added
pendingUntilFixed method to Suite.
- Deprecated
Report and SpecReport. These are no longer used, as they were made obsolete by the Reporter refactor. Instead
of SpecReport, the Formatter and its two subclasses, MotionToSuppress and IndentedText are used to format
specification-style output.
- Added the
org.scalatest.concurrent package, which includes Conductor, ConductorFixture, ConductorMultiFixture,
and ConductorMethods.
- Added the
org.scalatest.mock package, which includes EasyMockSugar, JMockCycle, JMockCycleFixture, JMockExpectations, and MockitoSugar.
- Added three new BDD traits:
WordSpec, FlatSpec, and FeatureSpec.
- Added the
org.scalatest.fixture package, which includes FixtureSuite, FixtureFunSuite, FixtureSpec, FixtureWordSpec, FixtureFlatSpec, FixtureFeatureSpec, ConfigMapFixture, MultipleFixtureFunSuite, MultipleFixtureSpec, MultipleFixtureWordSpec, MultipleFixtureFlatSpec, and MultipleFixtureFeatureSpec.
- Added the
org.scalatest.verb package to support FlatSpec and WordSpec.
- Split the
BeforeAndAfter trait into BeforeAndAfterEach and BeforeAndAfterAll. Deprecated BeforeAndAfter.
- Added trait
GivenWhenThen.
- Renamed
Group to Tag. Deprecated Group. Also renamed groups method to tags in Suite,
and deprecated groups.
- Added
TagAnnotation to serve as base class for tag annotations for Suite, however, won't enforce for a two-release deprecation cycle
that tag annotations used on Suite test methods need to extend TagAnnotation.
- Dropped the
apply method that takes a Report (which is no longer used) from Informer, and also dropped
the nameForReport method (breaking changes).
- Added the
OneInstancePerTest trait.
- Added the
ParallelTestExecution trait.
- Added the
SequentialNestedSuiteExecution trait.
- Refactored
TestFailedException so that it extends StackDepthException, which mixes in StackDepth. Added other
StackDepthExceptions so that other errors in test code beyond just assertion failures will be reported with stack depths:
DuplicateTestNameException, NotAllowedException, and TestRegistrationClosedException. Set aside StackDepth
as a separate trait to support JUnitTestFailedError.
- Renamed
ExecuteAndRun to AbstractSuite, and added more methods to it, to facilitate the stackable trait pattern (breaking change).
Changes in 0.9.5
- Added
ShouldMatchers and MustMatchers traits, plus many supporting classes
and traits.
- Added support for
SpecReport in the GUI, so that SpecReports get displayed in
a specification style. This makes org.scalatest.Spec output look nicer in the GUI,
and also makes it possible for org.specs.Specification to send SpecReports that will
look nice in the ScalaTest GUI.
- Enhanced the GUI to automatically select the first error, if there is one. This saves users a
bit of time each error.
- Added a
TestFailedException, which allows a stack trace depth to be sent along with
test failures. This allows reports to highlight the exactly filename and line number at which a test
failed. This makes it quicker for users to find the cause of the problem. Enhanced everything to
send TestFailedExceptions where possible.
- Formatted the details area of the GUI so that it is easier to read, including making bold the
most important part of the report, the report message and filename and line number, if it exists.
- Made sure the entire stack trace is printed in the GUI's detail area. Previously it might say things like
37 more...
- Made the GUI report list scroll during a rerun, to make it easier to find the reports of
the rerun.
- Filled in the name in run completed reports, which was missing.
- Fixed a bug that prevented suites from being automatically discovered inside JAR files.
- Added the exception that caused a ScalaCheck failure as the cause in the
TestFailedException,
which was not being propagated to the reporters before.
- Display more nicely the information, arguments and labels, provided with a failed ScalaCheck
property check.
- In some cases,
FunSuite and Spec were not sending Rerunnables in
reports even though they could, which made rerunning not work in some cases in the GUI. They now send
a Rerunnable whenever they can.
- Removed
SpecDasher from the API, which also is no longer mixed in by Spec.
The only reason this was included in 0.9.4 was that it was demonstrated in Programming in
Scala. It still works and can be used, so if you liked it, you can just grab it from
0.9.4 and mix it in yourself as a local mix in.
Changes in 0.9.4
- Renamed trait
ImpSuite to BeforeAndAfter. ImpSuite remains for now but is marked as deprecated. In
a future release of ScalaTest ImpSuite will be dropped, so please rename your uses of ImpSuite to BeforeAndAfter
as soon as convenient. The reason this was changed is that the "Imp" stood for imperative, because the trait facilitates an imperative style
of testing. Also "imp" in English means "little devil," which projected a good attitude, because one goal of ScalaTest is to encourage users to adopt
a more functional style for fixture "setup" and "teardown." But it's a bit too cute and its meaning not obvious enough from its name. Plus mixing
an ImpSuite into a Spec felt like mixing metaphors.
- Pulled the triple equals operator mechanism and the
expect and intercept constructs from Suite
into their own trait, named Assertions. Suite now mixes in this trait, so that it has the same functionality as
before. The main reason this was done was so that these constructs could more easily be used outside of ScalaTest Suites, and also
so the trait could be extended by ScalaTest's matchers. (ScalaTest's matchers are not yet released, but will probably make their debut in
ScalaTest version 0.9.5.) This allows ScalaTest's matchers to assume these constructs exist, and makes it easier to use ScalaTest matchers
independently from ScalaTest Suites.
- Added a version of
intercept that takes an implicit Manifest, so that you need not explicitly pass
a class instance anymore. Thus, instead of saying this:
intercept(classOf[StringIndexOutOfBoundsException]) {
"hi".charAt(-1)
}
You can write:
intercept[StringIndexOutOfBoundsException] {
"hi".charAt(-1)
}
The old style that requires an explicit class instance are still in the API, but they are deprecated. They will be removed in some future
release of ScalaTest, so please start migrating to the new style. There are a few corner cases where the new form can't be used, which
are caused by overloading ambiguities with the old style. The two corners are passing an explicit null to intercept and throwing
an explicit exception such that the inferred type of the by-name parameter is Nothing. It is unlikely anyone would do that
anyway, and these two issues will resolve themselves once the deprecated old-style intercept methods are removed.
- Added
Spec. This trait is inspired by Ruby's RSpec BDD tool.
- Renamed
PropSuite to FunSuite in org.scalatest.prop. This was done because it made sense
to add the extra ScalaCheck methods to Spec as well, so it seemed
more reasonable to just have traits with the same name in two different packages. So there would be a FunSuite and Spec
in both org.scalatest and org.scalatest.prop. The latter batch would already mix
in Checkers and offer the extra ScalaCheck property-taking methods, and of course, have a dependency on ScalaCheck. Thus,
PropSuite was deprecated, but it is left in the API for the time being to avoid client code breakage. PropSuite
will go away in a future release. Later, the traits that had the extra property-taking methods were judged not worth their weight, so
ScalaTest 0.9.4 does not include a Spec in package org.scalatest.prop.
However because an example using org.scalatest.prop.FunSuite was included in Programming in Scala, it was included
in 0.9.4, already deprecated. Both PropSuite and FunSuite will be removed from the org.scalatest.prop
package in a future ScalaTest release, so please start migrating to use org.scalatest.FunSuite, mixing in Checkers
and passing properties to ScalaCheck with check, as shown in this example.
- Released
SpecDasher trait. This may or may not be a good idea, but
an example of the "dashes" style it enables was shown in Programming in Scala, so it was included in 0.9.4, but deprecated. It may stay or go in
a future release, depending on user feedback. Currently, Spec already mixes SpecDasher in by default, but if
it ends up staying in the API, this connection will almost certainly be severed so that users must explicitly mix it in to get the "dashes" style.
So use it with caution, but even if it is removed, if you like it you can always just add the code for it to your own project and
use it that way.
- Added
SpecReport, a subclass of Report. Specs emit SpecReports when they run, and
the print reporters (standard out, err, file, etc.) recognize them and generate specification-like output. In this release the graphic
reporter does not handle SpecReport's specially, so the output looks like regular test output. In a future release the graphic
reporter will generate a specification-like output. One use case for SpecReports is integration with Eric Torreborre's specs
tool, however one piece still remains for that integration. Specs (and JUnit and TestNG) generate nested test output, whereas ScalaTest's
model is flat. In a future version ScalaTest's Report class will be enhanced to support nested reports. This will enable
specs tests run through ScalaTest to provide nice nested, specification-like reports.
- Created an
ExecuteAndRun trait. The purpose of this trait is to pull out abstract method signatures for execute and
runTest (and later if it turns out to be useful, runNestedSuites and runTests) so they can be
called by traits that have Suite as their self type, but extend ExecuteAndRun not Suite, such
as BeforeAndAfter. This makes such traits stackable behaviors that can be mixed into a Suite while not
being a Suite themselves.
- Removed
testWithInformer and ignoreWithInformer methods from FunSuite, because they were
simply too ugly to live. Unfortunately these were not deprecated, so if you used them you'll need to already make an adjustment to upgrade
to 0.9.4. What replaced them is an imperative approach, which while more error prone will work fine unless abused, and makes
for much nicer client code. There's a new method named info in FunSuite, which returns an Informer.
So where before you said:
testWithInformer("test name") {
info => {
info("send info to the reporter")
}
}
Now you'll just say:
test("test name") {
info("send info to the reporter")
}
During suite execution, the info method returns an Informer that immediately forwards information passed to its apply
methods to the reporter. During construction of the object, the info method returns an Informer that saves the information
passed to its apply methods for later sending to the reporter when the suite is executed. At any other time, the info
method will return an Informer that throws an exception.
Changes in 0.9.3
- Moved
FunSuite from the org.scalatest.fun to org.scalatest package.
- Dropped
FunSuite1 through FunSuite9, which were also in the org.scalatest.fun package, because the surface area
they added to the API didn't seem worth the small savings in code size over creating "with" methods with meaningful
names and explicitly calling them.
- Added an
ImpSuite, which can be mixed into another suite to gain methods that will run before and after each suite and test.
- Updated the portions of Scaladoc comments for
Suite and FunSuite that discuss mutable fixtures. Dropped
the suggestion to override runTest, which still works but is verbose, and instead recommended defining and explicitly calling "create" methods or
"with" methods from tests that need the mutable fixtures, or mixing in ImpSuite and using beforeEach, afterEach, etc.
- Added an
Informer trait, which provides a simpler way to send information to the infoProvided
method of a Reporter.
- Changed
Suite such that instead of being able to define test methods that take a Reporter, you can define
test methods that take an Informer. Replaced the section of scaladoc for Suite that used to explain how to
use a passed-in Reporter to now show how to use an Informer.
- Changed
FunSuite such that instead of being able to register tests whose function values take a Reporter, you can register
tests whose function values take an Informer. Changed the names of the registration methods from test/ignoreWithReporter to
test/ignoreWithInformer. Replaced the section of scaladoc for FunSuite that used to explain how to
use a Reporter passed in to a test function to now show how to use an Informer.
- Moved
org.scalatest.scalacheck.CheckSuite to org.scalatest.prop.Checkers, and renamed its checkProperty
methods to plain old check.
- Added
PropSuite, a subtrait of FunSuite, which offers methods that register tests composed of a single property check.
- Improved the messages generated by the
assert(... === ...) and expect(...){...} methods. Strings are now reported surrounded
by double quotes, with ends truncated where matching if very long, and with square brackets surrounding the portions that differ. Characters are reported
surrounded by single quotes, integer and floating point numbers as is, and any other object's toString result surrounded by angle brackets.
- Removed dollar signs and non-essential text from suite and test names in reports when running from the interpreter.
- Improved the look of the output generated by the build-in reporters that print to the standard out, error, or to files.
- Added a section to
Suite's scaladoc that describes using external assertions, such as JUnit's assertions, Hamcrest
matchers, or specs matchers, in ScalaTest tests.
- Improved the failure message when a ScalaCheck property check fails.
- Renamed user-defined properties to "goodies," because the term "properties" was overloaded to mean both key-value pairs that can be passed to suites
and ScalaCheck-like properties. "Goodies" is not a very obvious name, so suggestions for improvement are welcome.
Changes in 0.9.2
- Moved
Runner from org.scalatest to org.scalatest.tools package.
- Added JUnit integration classes in package
org.scalatest.junit.
- Added TestNG integration classes in package
org.scalatest.testng.
- Added ScalaCheck integration classes in package
org.scalatest.scalacheck.
- Added
FunSuite family of traits in package org.scalatest.fun.
- Added an Ant task.
Changes in 0.9.1
- Changed
testGroups method name to groups.
- Changed operand names of
===, from expected === actual to left === right.
- Changed message generated from
2 === 1, from "Expected 1, but got 2" to "2 did not equal 1".
- Changed
expect method name to intercept.
- Dropped
expectNPE and expectIAE methods.
- Added a different
expect method, which takes an expected and actual value. expect(1) {2} would generate
the error message "Expected 1, but got 2.
- Made versions of
expect and intercept that take a message.
- Implemented a
Suite discovery mechanism in Runner, accessed via -m or -w
command-line arguments.
- Dropped
!== method from the great Equalizer.
Known issues
- Reloading classes from JAR files between runs in the GUI doesn't work, because URLClassLoader uses a JAR cache. For now, you'll have to restart the app to pick
up new classes in JAR files. But a better alternative is to simply point ScalaTest to the build directory containing .class files produced by the Scala and
Java compiler. URLClassLoader will not cache these files, so changes to them will be picked up each time you press the Run or Rerun buttons.
|