ScalaTest User Guide
Selecting testing styles
Defining base classes
Writing your first test
Tagging your tests
Running your tests
Testing with mock objects
Philosophy and design
Migrating to 3.0
The Philosophy and design of ScalaTest
by Bill Venners, creator of ScalaTest
My main goal for creating ScalaTest was the same goal I had for promoting Scala in the first place: to help programmers become more productive. Increased productivity means releasing new features more quickly, with better accuracy and quality. Better accuracy means building the right features. Better quality means fewer defects.
Although writing tests can slow you down in the short term compared to just writing the production code, the time spent writing tests can pay back in increased quality and reduced time in the debugger. Over the long term tests can provide a safety net that enables you to make changes to existing code with greater speed and confidence, though during major refactors, fixing the many broken tests can add a significant extra cost. You can consider this cost another investment in testing, weighing the up-front cost of fixing the broken tests against the potential for returns in increased velocity and quality over time.
Accuracy, building the right features, requires fluid communication between developers and other stakeholders such as management, testers, and customers. Tests can potentially facilitate such communication because tests represent a highly accurate view of the state of the system.
My goal with ScalaTest is to improve the return on investment of testing. I've tried to minimize the cost of testing by making test code concise, plainly obvious, quickly understandable. A guiding design principle of ScalaTest is that different people on a team should be able look at each others test code and know immediately what's going on. I've also tried to bring that clarity out of the code and into the artifacts generated as tests run. My goal with the generated artifacts is to facilitate communication between all stakeholders—developers certainly, but also management, testers, and customers.
Lastly, ScalaTest is not a one-size-fits-all testing framework, it is a platform that can host different styles of testing. Why? Because different people with different problems need different tools to solve them. Even the same people need different tools in different situations. ScalaTest is designed to make it easy for you to customize your testing tool to meet your current needs, and for the built-in traits at least, make it easy for anyone who comes along later to read and understand your code.
ScalaTest is the “scalable test framework.” Like the Scala language on which it is built, ScalaTest is designed to grow with the demands of its users.
Scala is a scalable language in two ways. First, Scala can be molded through library and DSL design to fit widely different tasks. Second, Scala
scales both down to small tasks and up to large ones. It feels as natural to use for small tasks like scripting as it does for large tasks like major software projects built
by large teams. ScalaTest is scalable in similar ways. First, it can be easily molded by overriding its lifecycle methods to address
special testing needs when they arise. Second, it is designed facilitate both small tasks in the Scala interpreter (see
The design of ScalaTest can be summed up in one sentence:
ScalaTest allows you to ask a suite of tests to run itself.
The terms suite and test are defined abstractly in ScalaTest to
enable a wide variety of implementations. A test in ScalaTest is anything with a name that
can start and either succeed or fail. A suite is a collection of zero to many tests. ScalaTest's core trait
Scaling to different testing needs
How does this design enable ScalaTest to address widely different testing needs? You need look no further than ScalaTest itself
for examples. First, ScalaTest facilitates several different styles of testing through its style traits such as
ScalaTest also provides many stackable traits that can be mixed into a style trait, such as
There is no magic happening behind the scenes. In Scala you could implement something that looks and behaves like a while loop as a curried function that takes a by-name. Similarly you could implement any of ScalaTest's style or stackable traits yourself outside of ScalaTest. As a result, when you do have a special testing need, you will be able to easily accommodate it by overriding one or more of ScalaTest's lifecycle methods in a trait, and mixing that into your test classes. Need to run each test in their own class loader? Need to treat a directory full of files as one suite of tests per file? Need to define a particular order of test or suite execution? Any of these would be easy to implement by overriding lifecycle methods. And because it is so easy to customize, ScalaTest will grow with you as your project scales up and your needs change.
Enabling team productivity
Another design focus of ScalaTest is to facilitate the productivity of teams. Teams in the real world are made up of many different kinds of people. These people are often very busy, sometimes tired or under stress. Many are not fans of testing, and even those who think testing is a good idea may not have much time to invest in becoming an expert in a test framework.
One way in which ScalaTest is designed for teams is that it is easy to get started with because it offers styles
familiar to many users. Users of JUnit or TestNG, for example, can
continue using those tools with a few productivity enhancements from ScalaTest sprinkled in. Or if they use
Another way ScalaTest is designed for teams is that it is very thoroughly documented in its user guide and Scaladoc, both of which include many examples. The goal is that you should never need to look through ScalaTest source code to figure out what is going on.
Another major design focus of ScalaTest is to use the flexibility of the Scala language to make testing code clear. Because some people working in a team will inevitably be casual, non-expert users of the test framework, ScalaTest's design is focused on making client code “guessable.” The goal is to make client code so plainly obvious that even someone completely unfamiliar with ScalaTest could understand client code written by others without looking anything up in ScalaTest documentation.
In addition to trying to make ScalaTest's syntax as clear and meaningful as possible, the design is also focused on making ScalaTest's behavior as simple as possible. Keeping the test framework simple helps make your test code easier to reason about. The less you need to worry about what ScalaTest is doing behind the scenes, the more you can focus on what your own code is doing.
Lastly, in addition to trying to make code easy to read and understand, ScalaTest is designed to make the code as easy as possible
to write. To write ScalaTest code, you will need to look at the documentation for examples, but
the library is designed to be easy to remember so that you don't have to look again each time you want to write something.
Besides obviousness of code, one other way in which ScalaTest tries to achieve this is through consistency. Once you learn how
The upshot is that ScalaTest is designed to facilitate productivity of teams by being:
Next, find out about Selecting a style.