Packages

c

org.scalatest.tools

Framework

class Framework extends sbt.testing.Framework

This class is ScalaTest's implementation of the new Framework API that is supported in sbt 0.13.

To use ScalaTest in sbt, you should add ScalaTest as dependency in your sbt build file, the following shows an example for using ScalaTest 2.0 with Scala 2.10.x project:

"org.scalatest" % "scalatest_2.10" % "2.0" % "test"

To pass argument to ScalaTest from sbt, you can use testOptions:

testOptions in Test += Tests.Argument("-h", "target/html")  // Use HtmlReporter

If you are using multiple testing frameworks, you can pass arguments specific to ScalaTest only:

testOptions in Test += Tests.Argument(TestFrameworks.ScalaTest, "-h", "target/html") // Use HtmlReporter

Supported arguments

Integration in sbt 0.13 supports same argument format as Runner, except the following arguments:

  • -R -- runpath is not supported because test path and discovery is handled by sbt
  • -s -- suite is not supported because sbt's test-only serves the similar purpose
  • -A -- again is not supported because sbt's test-quick serves the similar purpose
  • -j -- junit is not supported because in sbt different test framework should be supported by its corresponding Framework implementation
  • -b -- testng is not supported because in sbt different test framework should be supported by its corresponding Framework implementation
  • -P -- concurrent/parallel is not supported because parallel execution is controlled by sbt.
  • -q is not supported because test discovery should be handled by sbt, and sbt's test-only or test filter serves the similar purpose
  • -T is not supported because correct ordering of text output is handled by sbt
  • -g is not supported because current Graphic Reporter implementation works differently than standard reporter
New Features of New Framework API

New Framework API supports a number of new features that ScalaTest has utilized to support a better testing experience in sbt. The followings are summary of new features supported by the new Framework API:

  • Specified behavior of single instance of Runner per project run (non-fork), and a new done method
  • API to return nested tasks
  • API to support test execution in fork mode
  • Selector API to selectively run tests
  • Added new Ignored, Canceled and Pending status
  • Added sbt Tagging support
Specified behavior of single instance of Runner per project run (non-fork), and a new done method

In new Framework API, it is now a specified behavior that Framework's runner method will be called to get a Runner instance once per project run. Arguments will be passed when calling Framework's runner and this gives ScalaTest a good place to perform setup tasks, such as initializing Reporters.

There's also a new done on Runner interface, which in turns provide a good spot for ScalaTest to perform cleanup tasks, such as disposing the Reporters. HtmlReporter depends on this behavior to generate its index.html. In addition, done can return framework-specific summary text for sbt to render at the end of the project run, which allows ScalaTest to return its own summary text.

API to return nested Suites as sbt Tasks

In new Framework API, a new concept of Task was introduced. A Task has an execute method that can return more Tasks for execution. ScalaTest does not utilize this feature, it always return empty array for sub-tasks.

API to support test execution in fork mode

Forking was added to sbt since version 0.12, you can find documentation for forking support in sbt at Forking in sbt.

Although forking is already available in sbt since 0.12, there's no support in old Framework API, until it is added in new Framework API that is supported in sbt 0.13. With API provided with new Framework API, ScalaTest creates real Reporters in the main process, and uses SocketReporter in forked process to send events back to the main process, and get processed by real Reporters at the main process. All of this is transparent to any custom Reporter implementation, as only one instance of the custom Reporter will be created to process the events, regardless of whether the tests run in same or forked process.

Selector API to selectively run tests

New Framework API includes a set of comprehensive API to select tests for execution. Though new Framework API supports fine-grained test selection, current sbt's test-only and test-quick supports up to suite level selection only, or SuiteSelector as defined in new Framework API. This Framework implementation already supports SuiteSelector, NestedSuiteSelector, TestSelector and NestedTestSelector, which should work once future sbt version supports them.

Added new Ignored, Canceled and Pending status

Status Ignored, Canceled and Pending are added to new Framework API, and they match perfectly with ScalaTest's ignored tests (now reported as Ignored instead of Skipped), as well as canceled and pending tests newly added in ScalaTest 2.0.

Added sbt Tagging support

Sbt supports task tagging, but has no support in old Framework API for test frameworks to integrate it. New Framework API supports it, and you can now use the following annotations to annotate your suite for sbt built-in resource tags:

They will be mapped to corresponding resource tag CPU, Disk and Network in sbt.

You can also define custom tag, which you'll need to write it as Java annotation:

import java.lang.annotation.Target;
import java.lang.annotation.Retention;
import org.scalatest.TagAnnotation;

@TagAnnotation("custom") @Retention(RetentionPolicy.RUNTIME) @Target({ElementType.TYPE}) public @interface Custom {}

which will be translated to Tags.Tag("custom") in sbt.

Source
Framework.scala
Linear Supertypes
sbt.testing.Framework, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Framework
  2. Framework
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new Framework()

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. def fingerprints(): Array[Fingerprint]

    An array of Fingerprints that specify how to identify ScalaTest's test classes during discovery.

    An array of Fingerprints that specify how to identify ScalaTest's test classes during discovery.

    returns

    SubclassFingerprint for org.scalatest.Suite and AnnotatedFingerprint for org.scalatest.WrapWith

    Definition Classes
    Framework → Framework
  10. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  12. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  13. def name(): String

    Test framework name.

    Test framework name.

    returns

    ScalaTest

    Definition Classes
    Framework → Framework
  14. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  15. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  16. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  17. def runner(args: Array[String], remoteArgs: Array[String], testClassLoader: ClassLoader): Runner

    Initiates a ScalaTest run.

    Initiates a ScalaTest run.

    args

    the ScalaTest arguments for the new run

    remoteArgs

    the ScalaTest remote arguments for the run in a forked JVM

    testClassLoader

    a class loader to use when loading test classes during the run

    returns

    a Runner implementation representing the newly started run to run ScalaTest's tests.

    Definition Classes
    Framework → Framework
    Exceptions thrown

    IllegalArgumentException when invalid or unsupported argument is passed

  18. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  19. def toString(): String
    Definition Classes
    AnyRef → Any
  20. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )

Inherited from sbt.testing.Framework

Inherited from AnyRef

Inherited from Any

Ungrouped