org.scalatest.tools

Framework

class Framework extends sbt.testing.Framework

This class is ScalaTest's implementation of the new Framework API that is supported in sbt 0.13.

To use ScalaTest in sbt, you should add ScalaTest as dependency in your sbt build file, the following shows an example for using ScalaTest 2.0 with Scala 2.10.x project:

"org.scalatest" % "scalatest_2.10" % "2.0" % "test"

To pass argument to ScalaTest from sbt, you can use testOptions:

testOptions in Test += Tests.Argument("-h", "target/html")  // Use HtmlReporter

If you are using multiple testing frameworks, you can pass arguments specific to ScalaTest only:

testOptions in Test += Tests.Argument(TestFrameworks.ScalaTest, "-h", "target/html") // Use HtmlReporter

Supported arguments

Integration in sbt 0.13 supports same argument format as Runner, except the following arguments:

New Features of New Framework API

New Framework API supports a number of new features that ScalaTest has utilized to support a better testing experience in sbt. The followings are summary of new features supported by the new Framework API:

Specified behavior of single instance of Runner per project run (non-fork), and a new done method

In new Framework API, it is now a specified behavior that Framework's runner method will be called to get a Runner instance once per project run. Arguments will be passed when calling Framework's runner and this gives ScalaTest a good place to perform setup tasks, such as initializing Reporters.

There's also a new done on Runner interface, which in turns provide a good spot for ScalaTest to perform cleanup tasks, such as disposing the Reporters. HtmlReporter depends on this behavior to generate its index.html. In addition, done can return framework-specific summary text for sbt to render at the end of the project run, which allows ScalaTest to return its own summary text.

API to return nested Suites as sbt Tasks

In sbt versions before 0.13, ScalaTest's nested suites were always executed sequentially regardless of the parallelExecution value of the sbt build. In new Framework API, a new concept of Task was introduced. A Task has an execute method that can return more Tasks for execution. When parallelExecution is set to true (sbt's default), sbt will execute returned tasks in parallel.

Each Suite in ScalaTest maps to a Task in sbt. Any nested suites of a Suite executed by sbt are returned to sbt as Tasks so that sbt can execute them using its own thread pool, either in parallel (sbt's default), or on a single thread if sequential execution was requested by the sbt build. The way Framework achieves this behavior is by inserting a Distributor that records any nested suites added to it, passing those Suites back to sbt as Tasks.

Note that the way nested suites are executed sequentially is different when using sbt than when running directly with ScalaTest. The reason is that when ScalaTest executes sequentially, it passes in None for the Option[Distributor] parameter, where as Framework passes in Some[Distributor] to collect the nested suites so they can be returned to sbt as Tasks. As a result, when ScalaTest executes a Suite sequentially, that Suite's nested suites are executed before its tests. When sbt asks ScalaTest through this Framework class to execute a Suite sequentially, the Suite's nested suites will be executed after its tests. To make nested suites run sequentially before the tests when using sbt, mix in trait SequentialNestedSuiteExecution, which overrides runNestedSuites to replace the Some[Distributor] passed in by Framework with None.

API to support test execution in fork mode

Forking was added to sbt since version 0.12, you can find documentation for forking support in sbt at Forking in sbt.

Although forking is already available in sbt since 0.12, there's no support in old Framework API, until it is added in new Framework API that is supported in sbt 0.13. With API provided with new Framework API, ScalaTest creates real Reporters in the main process, and uses SocketReporter in forked process to send events back to the main process, and get processed by real Reporters at the main process. All of this is transparent to any custom Reporter implementation, as only one instance of the custom Reporter will be created to process the events, regardless of whether the tests run in same or forked process.

Selector API to selectively run tests

New Framework API includes a set of comprehensive API to select tests for execution. Though new Framework API supports fine-grained test selection, current sbt's test-only and test-quick supports up to suite level selection only, or SuiteSelector as defined in new Framework API. This Framework implementation already supports SuiteSelector, NestedSuiteSelector, TestSelector and NestedTestSelector, which should work once future sbt version supports them.

Added new Ignored, Canceled and Pending status

Status Ignored, Canceled and Pending are added to new Framework API, and they match perfectly with ScalaTest's ignored tests (now reported as Ignored instead of Skipped), as well as canceled and pending tests newly added in ScalaTest 2.0.

Added sbt Tagging support

Sbt supports task tagging, but has no support in old Framework API for test frameworks to integrate it. New Framework API supports it, and you can now use the following annotations to annotate your suite for sbt built-in resource tags:

They will be mapped to corresponding resource tag CPU, Disk and Network in sbt.

You can also define custom tag, which you'll need to write it as Java annotation:

import java.lang.annotation.Target;
import java.lang.annotation.Retention;
import org.scalatest.TagAnnotation;

@TagAnnotation("custom") @Retention(RetentionPolicy.RUNTIME) @Target({ElementType.TYPE}) public @interface Custom {}

which will be translated to Tags.Tag("custom") in sbt.

Source
Framework.scala
Linear Supertypes
sbt.testing.Framework, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. Framework
  2. Framework
  3. AnyRef
  4. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new Framework()

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  6. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  7. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws()
  8. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  9. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  10. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws()
  11. def fingerprints(): Array[Fingerprint]

    An array of Fingerprints that specify how to identify ScalaTest's test classes during discovery.

    An array of Fingerprints that specify how to identify ScalaTest's test classes during discovery.

    returns

    SubclassFingerprint for org.scalatest.Suite and AnnotatedFingerprint for org.scalatest.WrapWith

    Definition Classes
    Framework → Framework
  12. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  13. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  14. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  15. def name(): String

    Test framework name.

    Test framework name.

    returns

    ScalaTest

    Definition Classes
    Framework → Framework
  16. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  17. final def notify(): Unit

    Definition Classes
    AnyRef
  18. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  19. def runner(args: Array[String], remoteArgs: Array[String], testClassLoader: ClassLoader): Runner

    Initiates a ScalaTest run.

    Initiates a ScalaTest run.

    args

    the ScalaTest arguments for the new run

    remoteArgs

    the ScalaTest remote arguments for the run in a forked JVM

    testClassLoader

    a class loader to use when loading test classes during the run

    returns

    a Runner implementation representing the newly started run to run ScalaTest's tests.

    Definition Classes
    Framework → Framework
    Exceptions thrown
    IllegalArgumentException

    when invalid or unsupported argument is passed

  20. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  21. def toString(): String

    Definition Classes
    AnyRef → Any
  22. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws()
  23. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws()
  24. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws()

Inherited from sbt.testing.Framework

Inherited from AnyRef

Inherited from Any

Ungrouped