trait RandomTestOrder extends OneInstancePerTest
Trait that causes tests to be run in pseudo-random order.
Although the tests are run in pseudo-random order, events will be fired in the “normal” order for the Suite
that mixes in this trait, as determined by runTests.
The purpose of this trait is to reduce the likelihood of unintentional order dependencies between tests in the same test class.
- Self Type
- RandomTestOrder with Suite
- Source
- RandomTestOrder.scala
- Alphabetic
- By Inheritance
- RandomTestOrder
- OneInstancePerTest
- SuiteMixin
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def expectedTestCount(filter: Filter): Int
The total number of tests that are expected to run when this
Suite'srunmethod is invoked.The total number of tests that are expected to run when this
Suite'srunmethod is invoked.- filter
a
Filterwith which to filter tests to count based on their tags
- Definition Classes
- SuiteMixin
- abstract def nestedSuites: IndexedSeq[Suite]
An immutable
IndexedSeqof thisSuiteMixinobject's nestedSuites.An immutable
IndexedSeqof thisSuiteMixinobject's nestedSuites. If thisSuiteMixincontains no nestedSuites, this method returns an emptyIndexedSeq.- Definition Classes
- SuiteMixin
- abstract def rerunner: Option[String]
The fully qualified name of the class that can be used to rerun this suite.
The fully qualified name of the class that can be used to rerun this suite.
- Definition Classes
- SuiteMixin
- abstract def runNestedSuites(args: Args): Status
Runs zero to many of this suite's nested suites.
Runs zero to many of this suite's nested suites.
- args
the
Argsfor this run- returns
a
Statusobject that indicates when all nested suites started by this method have completed, and whether or not a failure occurred.
- Attributes
- protected
- Definition Classes
- SuiteMixin
- Exceptions thrown
NullArgumentExceptionifargsisnull.
- abstract def suiteId: String
A string ID for this
Suitethat is intended to be unique among all suites reported during a run.A string ID for this
Suitethat is intended to be unique among all suites reported during a run.The suite ID is intended to be unique, because ScalaTest does not enforce that it is unique. If it is not unique, then you may not be able to uniquely identify a particular test of a particular suite. This ability is used, for example, to dynamically tag tests as having failed in the previous run when rerunning only failed tests.
- returns
this
Suiteobject's ID.
- Definition Classes
- SuiteMixin
- abstract def suiteName: String
A user-friendly suite name for this
Suite.A user-friendly suite name for this
Suite.This trait's implementation of this method returns the simple name of this object's class. This trait's implementation of
runNestedSuitescalls this method to obtain a name forReports to pass to thesuiteStarting,suiteCompleted, andsuiteAbortedmethods of theReporter.- returns
this
Suiteobject's suite name.
- Definition Classes
- SuiteMixin
- abstract def tags: Map[String, Set[String]]
A
Mapwhose keys areStringnames of tagged tests and whose associated values are theSetof tag names for the test.A
Mapwhose keys areStringnames of tagged tests and whose associated values are theSetof tag names for the test. If a test has no associated tags, its name does not appear as a key in the returnedMap. If thisSuitecontains no tests with tags, this method returns an emptyMap.Subclasses may override this method to define and/or discover tags in a custom manner, but overriding method implementations should never return an empty
Setas a value. If a test has no tags, its name should not appear as a key in the returnedMap.- Definition Classes
- SuiteMixin
- abstract def testDataFor(testName: String, theConfigMap: ConfigMap): TestData
Provides a
TestDatainstance for the passed test name, given the passed config map.Provides a
TestDatainstance for the passed test name, given the passed config map.This method is used to obtain a
TestDatainstance to pass towithFixture(NoArgTest)andwithFixture(OneArgTest)and thebeforeEachandafterEachmethods of traitBeforeAndAfterEach.- testName
the name of the test for which to return a
TestDatainstance- theConfigMap
the config map to include in the returned
TestData- returns
a
TestDatainstance for the specified test, which includes the specified config map
- Definition Classes
- SuiteMixin
- abstract def testNames: Set[String]
A
Setof test names.A
Setof test names. If thisSuitecontains no tests, this method returns an emptySet.Although subclass and subtrait implementations of this method may return a
Setwhose iterator producesStringtest names in a well-defined order, the contract of this method does not required a defined order. Subclasses are free to implement this method and return test names in either a defined or undefined order.- Definition Classes
- SuiteMixin
- abstract val styleName: String
The
styleNamelifecycle method has been deprecated and will be removed in a future version of ScalaTest.The
styleNamelifecycle method has been deprecated and will be removed in a future version of ScalaTest.This method was used to support the chosen styles feature, which was deactivated in 3.1.0. The internal modularization of ScalaTest in 3.2.0 will replace chosen styles as the tool to encourage consistency across a project. We do not plan a replacement for
styleName.- Definition Classes
- SuiteMixin
- Annotations
- @deprecated
- Deprecated
(Since version 3.1.0) The styleName lifecycle method has been deprecated and will be removed in a future version of ScalaTest with no replacement.
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def newInstance: Suite with RandomTestOrder
Construct a new instance of this
Suite.Construct a new instance of this
Suite.This trait's implementation of
runTestsinvokes this method to create a new instance of thisSuitefor each test. This trait's implementation of this method uses reflection to callthis.getClass.newInstance. This approach will succeed only if thisSuite's class has a public, no-arg constructor. In most cases this is likely to be true, because to be instantiated by ScalaTest'sRunneraSuiteneeds a public, no-arg constructor. However, this will not be true of anySuitedefined as an inner class of another class or trait, because every constructor of an inner class type takes a reference to the enclosing instance. In such cases, and in cases where aSuiteclass is explicitly defined without a public, no-arg constructor, you will need to override this method to construct a new instance of theSuitein some other way.Here's an example of how you could override
newInstanceto construct a new instance of an inner class:import org.scalatest._
class Outer { class InnerSuite extends Suite with RandomTestOrder { def testOne() {} def testTwo() {} override def newInstance = new InnerSuite } }- Definition Classes
- RandomTestOrder → OneInstancePerTest
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- def run(testName: Option[String], args: Args): Status
Modifies the behavior of
super.runto facilitate pseudo-random order test execution.Modifies the behavior of
super.runto facilitate pseudo-random order test execution.If both
testNameandargs.distributedTestSorterare defined, this trait's implementation of this method will create a "test-specific reporter" whoseapplymethod will invoke theapplymethod of theDistributedTestSorter, which takes a test name as well as the event. It will then invokesuper.runpassing along the sametestNameand anArgsobject that is the same except with the original reporter replaced by the test-specific reporter.If either
testNameorargs.distributedTestSorteris empty, it will createTestSortingReporterand overrideargs'sreporteranddistributedTestSorterwith it. It then callsuper.runto delegate the run to super's implementation, and to collect all children suites insuiteRunQueue. Aftersuper.runcompleted, it then shuffle the order of the suites collected insuiteRunQueueand run them.- testName
an optional name of one test to execute. If
None, all relevant tests should be executed. I.e.,Noneacts like a wildcard that means execute all relevant tests in thisSuite.- args
the
Argsfor this run- returns
a
Statusobject that indicates when all tests and nested suites started by this method have completed, and whether or not a failure occurred.
- Definition Classes
- RandomTestOrder → SuiteMixin
- final def runTest(testName: String, args: Args): Status
Modifies the behavior of
super.runTestto facilitate pseudo-random order test execution.Modifies the behavior of
super.runTestto facilitate pseudo-random order test execution.If
runTestInNewInstanceisfalse, this is the test-specific (distributed) instance, so this trait's implementation of this method simply invokessuper.runTest, passing along the sametestNameandargsobject, delegating responsibility for actually running the test to the super implementation. Aftersuper.runTestreturns (or completes abruptly by throwing an exception), it notifiesargs.distributedTestSorterthat it has completed running the test by invokingcompletedTeston it, passing in thetestName.If
runTestInNewInstanceistrue, it notifiesargs.distributedTestSorterthat it is distributing the test by invokingdistributingTeston it, passing in thetestName. The test execution will be deferred to be run in pseudo-random order later.Note: this trait's implementation of this method is
finalto ensure that any other desiredrunTestbehavior is executed by the same thread that executes the test. For example, if you were to mix inBeforeAndAfterafterRandomTestOrder, thebeforeandaftercode would be executed by the general instance on the main test thread, rather than by the test-specific instance on the distributed thread. Marking this methodfinalensures that traits likeBeforeAndAftercan only be &lquot;super&rquot; toRandomTestOrderand, therefore, that itsbeforeandaftercode will be run by the same distributed thread that runs the test itself.- testName
the name of one test to execute.
- args
the
Argsfor this run- returns
a
Statusobject that indicates when the test started by this method has completed, and whether or not it failed .
- Attributes
- protected
- Definition Classes
- RandomTestOrder → OneInstancePerTest → SuiteMixin
- def runTests(testName: Option[String], args: Args): Status
Modifies the behavior of
super.runTeststo facilitate running each test in its own instance of thisSuite's class.Modifies the behavior of
super.runTeststo facilitate running each test in its own instance of thisSuite's class.This trait's implementation of
runTestuses therunTestInNewInstanceflag of the passedArgsobject to determine whether this instance is the general instance responsible for running all tests in the suite (runTestInNewInstanceisfalse), or a test-specific instance responsible for running just one test (runTestInNewInstanceistrue). Note that theseBooleanvalues are reverse those used byrunTest, becauserunTestsalways inverts theBooleanvalue ofrunTestInNewInstancewhen invokingrunTest.If
runTestInNewInstanceisfalse, this trait's implementation of this method will invokesuper.runTests, passing alongtestNameandargs, but with therunTestInNewInstanceflag set totrue. By settingrunTestInNewInstancetotrue,runTestsis tellingrunTestto create a new instance to run each test.If
runTestInNewInstanceistrue, this trait's implementation of this method will invokerunTestdirectly, passing intestName.getand theargsobject, with therunTestInNewInstanceflag set tofalse. By settingrunTestInNewInstancetofalse,runTestsis tellingrunTestthat this is the test-specific instance, so it should just run the specified test.- testName
an optional name of one test to run. If
None, all relevant tests should be run. I.e.,Noneacts like a wildcard that means run all relevant tests in thisSuite.- args
the
Argsfor this run- returns
a
Statusobject that indicates when all tests started by this method have completed, and whether or not a failure occurred.
- Attributes
- protected
- Definition Classes
- OneInstancePerTest → SuiteMixin
- Exceptions thrown
IllegalArgumentExceptioniftestNameis defined, but no test with the specified test name exists in thisSuite, or ifrunTestInNewInstanceistrue, buttestNameis empty.NullPointerExceptionif any of the passed parameters isnull.
- def sortingTimeout: Span
A maximum amount of time to wait for out-of-order events generated by running the tests of this
Suitein parallel while sorting the events back into a more user-friendly, sequential order.A maximum amount of time to wait for out-of-order events generated by running the tests of this
Suitein parallel while sorting the events back into a more user-friendly, sequential order.The default implementation of this method returns the value specified via
-Tto Suite, or 2 seconds, if no-Twas supplied.- returns
a maximum amount of time to wait for events while resorting them into sequential order
- Attributes
- protected
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()