Testing is a very important part of the development process. It allows us to verify the functionality of our projects as well as judge the quality of our work.
At Mozilla, we have multiple ways of testing our code, including:
- Unit tests and integration tests, which are automated tests that verify that pieces of code work as expected.
- End-to-end tests, automated tests which check the functionality of a project as a whole. For example, simulating clicks in a web browser to test how a site functions.
- Manual testing, which is performed by a human and involves verifying features work as expected and exploratory tests.
Assessing and managing risk¶
The end goal of testing is to manage the risk of something going wrong with your project. To this end, one of the first steps you should take is to assess the risk of each area of your project.
More concretely, some parts of your project are going to be more likely to fail than others. Also, some parts of your project are more important than others, and it may be more harmful for them to fail than less important parts.
A risk assessment lists out the different parts of your project (such as certain webpages or parts of an API) and ranks them based on their importance. For example, a news site rank being able to read existing articles as more important than being able to submit new articles. Ranking these parts allows you to make decisions about which to test more and what kind of tests to run.
Unit and integration tests¶
As a developer, the most common type of tests you will write are unit and integration tests. Unit tests test the smallest possible chunk of functionality and are isolated from each other. Integration tests test the interaction between these chunks.
In practice, any changes you make to a project should be tested in some automated way if it’s reasonable to do so. While each project varies, generally we’re not that picky about having perfect unit tests or perfect test isolation. If you’re unsure, look at existing tests for the project for guidance on the preferred style.
Mozilla runs a Jenkins server for running these tests automatically for several projects. Other projects rely on Travis CI for executing their tests.
For Django projects, these tests live within the
tests module of each
included Django application. For Node-based projects, they normally live in
a directory named
tests at the root of the repository. Refer to
your project’s documentation for more details.
End-to-end tests simulates how your project will be used by users and verifies that it behaves as expected. This is most commonly applied to websites, where we use tools like Selenium to simulate users interacting with the website.
For many sites, these tests are written by WebQA contributors and run against the various server environments.
Manual testing is good old-fashioned human-powered testing, where a living, breathing human uses your project and checks for any errors. Typically this is either for verifying that a new feature works as expected, or for free-form exploratory testing.
In addition to writing automated tests, you almost certainly should be manually testing any changes you make to a project.
The following is a non-exhaustive, possibly-out-of-date list of tools and libraries that may aid you in testing your projects.
nose is a highly recommended testing library for Python.
- django-nose integrates nose into a Django test runner.
- nose-progressive is a nose plugin that makes test output much easier to read.
factory-boy replaces test fixtures with factories that generate test objects easily. It integrates with the Django ORM to generate model instances with a very conveninent syntax.
Mock is one of the most popular libraries for replacing parts of the system you’re testing with mock objects and asserting things about their behavior.