Testing Tools

Django provides a set of tools that come in handy when writing tests.

The Test Client

The test client is a Python class that acts as a dummy Web browser, allowing you to test your views and interact with your Django-powered application programmatically. Some of the things you can do with the test client are:

  • Simulate GET and POST requests on a URL and observe the response – everything from low-level HTTP (result headers and status codes) to page content.
  • See the chain of redirects (if any) and check the URL and status code at each step.
  • Test that a given request is rendered by a given Django template, with a template context that contains certain values.

Note that the test client is not intended to be a replacement for Selenium or other in-browser frameworks. Django’s test client has a different focus.

In short:

  • Use Django’s test client to establish that the correct template is being rendered and that the template is passed the correct context data.
  • Use in-browser frameworks like Selenium to test rendered HTML and the behavior of Web pages, namely JavaScript functionality.

Django also provides special support for those frameworks; see the section on LiveServerTestCase for more details. A comprehensive test suite should use a combination of both test types. For a more detailed look at the Django test client with examples, see the Django Project website.

Provided Test Case Classes

Normal Python unit test classes extend a base class of unittest.TestCase. Django provides a few extensions of this base class:

SimpleTestCase

Extends unittest.TestCase with some basic functionality like:

  • Saving and restoring the Python warning machinery state.
  • Adding a number of useful assertions including:
    • Checking that a callable raises a certain exception.
    • Testing form field rendering and error treatment.
    • Testing HTML responses for the presence/lack of a given fragment.
    • Verifying that a template has/hasn’t been used to generate a given response content.
    • Verifying a HTTP redirect is performed by the app.
    • Robustly testing two HTML fragments for equality/inequality or containment.
    • Robustly testing two XML fragments for equality/inequality.
    • Robustly testing two JSON fragments for equality.
  • The ability to run tests with modified settings.
  • Using the test Client.
  • Custom test-time URL maps.
TransactionTestCase

Django’s TestCase class (described below) makes use of database transaction facilities to speed up the process of resetting the database to a known state at the beginning of each test. A consequence of this, however, is that some database behaviors cannot be tested within a Django TestCase class.

In those cases, you should use TransactionTestCase. TransactionTestCase and TestCase are identical except for the manner in which the database is reset to a known state and the ability for test code to test the effects of commit and rollback:

  • A TransactionTestCase resets the database after the test runs by truncating all tables. A TransactionTestCase may call commit and rollback and observe the effects of these calls on the database.
  • A TestCase, on the other hand, does not truncate tables after a test. Instead, it encloses the test code in a database transaction that is rolled back at the end of the test. This guarantees that the rollback at the end of the test restores the database to its initial state.

TransactionTestCase inherits from SimpleTestCase.

TestCase

This class provides some additional capabilities that can be useful for testing Web sites. Converting a normal unittest.TestCase to a Django TestCase is easy: Just change the base class of your test from unittest.TestCase to django.test.TestCase. All of the standard Python unit test functionality will continue to be available, but it will be augmented with some useful additions, including:

  • Automatic loading of fixtures.
  • Wraps the tests within two nested atomic blocks: one for the whole class and one for each test.
  • Creates a TestClient instance.
  • Django-specific assertions for testing for things like redirection and form errors.

TestCase inherits from TransactionTestCase.

LiveServerTestCase

LiveServerTestCase does basically the same as TransactionTestCase with one extra feature: it launches a live Django server in the background on setup, and shuts it down on teardown. This allows the use of automated test clients other than the Django dummy client such as, for example, the Selenium client,
to execute a series of functional tests inside a browser and simulate a real user’s actions.

Test Cases Features

Default Test Client

Every test case in a *TestCase instance has access to an instance of a Django test client. This client can be accessed as self.client. This client is recreated for each test, so you don’t have to worry about state (such as cookies) carrying over from one test to another. This means, instead of instantiating a Client in each test:

import unittest
from django.test import Client

class SimpleTest(unittest.TestCase):
    def test_details(self):
        client = Client()
        response = client.get('/customer/details/')
        self.assertEqual(response.status_code, 200)

    def test_index(self):
        client = Client()
        response = client.get('/customer/index/')
        self.assertEqual(response.status_code, 200)

… you can just refer to self.client, like so:

from django.test import TestCase

class SimpleTest(TestCase):
    def test_details(self):
        response = self.client.get('/customer/details/')
        self.assertEqual(response.status_code, 200)

    def test_index(self):
        response = self.client.get('/customer/index/')
        self.assertEqual(response.status_code, 200)
Fixture Loading

A test case for a database-backed Web site isn’t much use if there isn’t any data in the database. To make it easy to put test data into the database, Django’s custom TransactionTestCase class provides a way of loading fixtures. A fixture is a collection of data that Django knows how to import into a database. For example, if your site has user accounts, you might set up a fixture of fake user accounts in order to populate your database during tests.

The most straightforward way of creating a fixture is to use the manage.py dumpdata command. This assumes you already have some data in your database. (See the dumpdata documentation for more details).

Once you’ve created a fixture and placed it in a fixtures directory in one of your INSTALLED_APPS, you can use it in your unit tests by specifying a fixtures class attribute on your django.test.TestCase subclass:

from django.test import TestCase
from myapp.models import Animal

class AnimalTestCase(TestCase):
    fixtures = ['mammals.json', 'birds']

    def setUp(self):
        # Test definitions as before.
        call_setup_methods()

    def testFluffyAnimals(self):
        # A test that uses the fixtures.
        call_some_test_code()

Here’s specifically what will happen:

  • At the start of each test case, before setUp() is run, Django will flush the database, returning the database to the state it was in directly after migrate was called.
  • Then, all the named fixtures are installed. In this example, Django will install any JSON fixture named mammals, followed by any fixture named birds.

See the loaddata documentation for more details on defining and installing fixtures.
This flush/load procedure is repeated for each test in the test case, so you can be certain that the outcome of a test will not be affected by another test, or by the order of test execution. By default, fixtures are only loaded into the default database. If you are using multiple databases and set multi_db=True, fixtures will be loaded into all databases.

Overriding Settings
settings()

For testing purposes it’s often useful to change a setting temporarily and revert to the original value after running the testing code. For this use case Django provides a standard Python context manager (see PEP 343) called settings(), which can be used like this:

from django.test import TestCase

class LoginTestCase(TestCase):

    def test_login(self):

        # First check for the default behavior
        response = self.client.get('/sekrit/')
        self.assertRedirects(response, '/accounts/login/?next=/sekrit/')

        # Then override the LOGIN_URL setting
        with self.settings(LOGIN_URL='/other/login/'):
            response = self.client.get('/sekrit/')
            self.assertRedirects(response, '/other/login/?next=/sekrit/')

This example will override the LOGIN_URL setting for the code in the with block and reset its value to the previous state afterwards.

modify_settings()

It can prove unwieldy to redefine settings that contain a list of values. In practice, adding or removing values is often sufficient. The modify_settings() context manager makes it easy:

from django.test import TestCase

class MiddlewareTestCase(TestCase):

    def test_cache_middleware(self):
        with self.modify_settings(MIDDLEWARE_CLASSES={
            'append': 'django.middleware.cache.FetchFromCacheMiddleware',
            'prepend': 'django.middleware.cache.UpdateCacheMiddleware',
            'remove': [
               'django.contrib.sessions.middleware.SessionMiddleware',
               'django.contrib.auth.middleware.AuthenticationMiddleware', 
               'django.contrib.messages.middleware.MessageMiddleware',
            ],
        }):
            response = self.client.get('/')
            # ...

For each action, you can supply either a list of values or a string. When the value already exists in the list, append and prepend have no effect; neither does remove when the value doesn’t exist.

override_settings()

In case you want to override a setting for a test method, Django provides the override_settings() decorator (see PEP 318). It’s used like this:

from django.test import TestCase, override_settings

class LoginTestCase(TestCase):

    @override_settings(LOGIN_URL='/other/login/')
    def test_login(self):
        response = self.client.get('/sekrit/')
        self.assertRedirects(response, '/other/login/?next=/sekrit/')

The decorator can also be applied to TestCase classes:

from django.test import TestCase, override_settings

@override_settings(LOGIN_URL='/other/login/')
class LoginTestCase(TestCase):

    def test_login(self):
        response = self.client.get('/sekrit/')
        self.assertRedirects(response, '/other/login/?next=/sekrit/')
modify_settings()

Likewise, Django provides the modify_settings() decorator:

from django.test import TestCase, modify_settings

class MiddlewareTestCase(TestCase):

    @modify_settings(MIDDLEWARE_CLASSES={
        'append': 'django.middleware.cache.FetchFromCacheMiddleware',
        'prepend': 'django.middleware.cache.UpdateCacheMiddleware',
    })
    def test_cache_middleware(self):
        response = self.client.get('/')
        # ...

The decorator can also be applied to test case classes:

from django.test import TestCase, modify_settings

@modify_settings(MIDDLEWARE_CLASSES={
    'append': 'django.middleware.cache.FetchFromCacheMiddleware',
    'prepend': 'django.middleware.cache.UpdateCacheMiddleware',
})
class MiddlewareTestCase(TestCase):

    def test_cache_middleware(self):
        response = self.client.get('/')
        # ...

When overriding settings, make sure to handle the cases in which your app’s code uses a cache or similar feature that retains state even if the setting is changed. Django provides the django.test.signals.setting_changed signal that lets you register call-backs to clean up and otherwise reset state when settings are changed.

Assertions

As Python’s normal unittest.TestCase class implements assertion methods such as assertTrue() and assertEqual(), Django’s custom TestCase class provides a number of custom assertion methods that are useful for testing Web applications:

  • assertRaisesMessage – Asserts that execution of the callable object raised an exception with an expected_message representation.
  • assertFieldOutput – Asserts that a form field behaves correctly with various inputs.
  • assertFormError – Asserts that a field on a form raises the provided list of errors when rendered on the form.
  • assertFormsetError – Asserts that the formset raises the provided list of errors when rendered.
  • assertContains – Asserts that a Response instance produced the given status_code and that text appears in the content of the response.
  • assertNotContains – Asserts that a Response instance produced the given status_code and that text does not appear in the content of the response.
  • assertTemplateUsed – Asserts that the template with the given name was used in rendering the response. The name is a string such as 'admin/index.html'.
  • assertTemplateNotUsed – Asserts that the template with the given name was not used in rendering the response.
  • assertRedirects – Asserts that the response returned a status_code redirect status, redirected to expected_url (including any GET data), and that the final page was received with target_status_code.
  • assertHTMLEqual – Asserts that the strings html1 and html2 are equal. The comparison is based on HTML semantics. The comparison takes following things into account:
    • Whitespace before and after HTML tags is ignored.
    • All types of whitespace are considered equivalent.
    • All open tags are closed implicitly, e.g. when a surrounding tag is closed or the HTML document ends.
    • Empty tags are equivalent to their self-closing version.
    • The ordering of attributes of an HTML element is not significant.
    • Attributes without an argument are equal to attributes that equal in name and value (see the examples).
  • assertHTMLNotEqual – Asserts that the strings html1 and html2 are not equal. The comparison is based on HTML semantics. See assertHTMLEqual() for details.
  • assertXMLEqual – Asserts that the strings xml1 and xml2 are equal. The comparison is based on XML semantics. Similarly to assertHTMLEqual(), the comparison is made on parsed content, hence only semantic differences are considered, not syntax differences.
  • assertXMLNotEqual – Asserts that the strings xml1 and xml2 are not equal. The comparison is based on XML semantics. See assertXMLEqual() for details.
  • assertInHTML – Asserts that the HTML fragment needle is contained in the haystack one.
  • assertJSONEqual – Asserts that the JSON fragments raw and expected_data are equal.
  • assertJSONNotEqual – Asserts that the JSON fragments raw and expected_data are not equal.
  • assertQuerysetEqual – Asserts that a queryset qs returns a particular list of values values. The comparison of the contents of qs and values is performed using the function transform; by default, this means that the repr() of each value is compared.
  • assertNumQueries – Asserts that when func is called with *args and **kwargs that num database queries are executed.

Email Services

If any of your Django views send email using Django’s email functionality, you probably don’t want to send email each time you run a test using that view. For this reason, Django’s test runner automatically redirects all Django-sent email to a dummy outbox. This lets you test every aspect of sending email – from the number of messages sent to the contents of each message – without actually sending the messages. The test runner accomplishes this by transparently replacing the normal email backend with a testing backend. (Don’t worry – this has no effect on any other email senders outside of Django, such as your machine’s mail server, if you’re running one.)

During test running, each outgoing email is saved in django.core.mail.outbox. This is a simple list of all EmailMessage instances that have been sent. The outbox attribute is a special attribute that is created only when the locmem email backend is used. It doesn’t normally exist as part of the django.core.mail module and you can’t import it directly. The code below shows how to access this attribute correctly. Here’s an example test that examines django.core.mail.outbox for length and contents:

from django.core import mail
from django.test import TestCase

class EmailTest(TestCase):
    def test_send_email(self):
        # Send message.
        mail.send_mail('Subject here', 'Here is the message.',
            'from@example.com', ['to@example.com'],
            fail_silently=False)

        # Test that one message has been sent.
        self.assertEqual(len(mail.outbox), 1)

        # Verify that the subject of the first message is correct.
        self.assertEqual(mail.outbox[0].subject, 'Subject here')

As noted previously, the test outbox is emptied at the start of every test in a Django *TestCase. To empty the outbox manually, assign the empty list to mail.outbox:

from django.core import mail

# Empty the test outbox
mail.outbox = []

Management Commands

Management commands can be tested with the call_command() function. The output can be redirected into a StringIO instance:

from django.core.management import call_command
from django.test import TestCase
from django.utils.six import StringIO

class ClosepollTest(TestCase):
    def test_command_output(self):
        out = StringIO()
        call_command('closepoll', stdout=out)
        self.assertIn('Expected output', out.getvalue())

Skipping Tests

The unittest library provides the @skipIf and @skipUnless decorators to allow you to skip tests if you know ahead of time that those tests are going to fail under certain conditions. For example, if your test requires a particular optional library in order to succeed, you could decorate the test case with @skipIf. Then, the test runner will report that the test wasn’t executed and why, instead of failing the test or omitting the test altogether.

The Test Database

Tests that require a database (namely, model tests) will not use your production database; separate, blank databases are created for the tests. Regardless of whether the tests pass or fail, the test databases are destroyed when all the tests have been executed. You can prevent the test databases from being destroyed by adding the --keepdb flag to the test command. This will preserve the test database between runs.

If the database does not exist, it will first be created. Any migrations will also be applied in order to keep it up to date. By default, the test databases get their names by prepending test_ to the value of the NAME
settings for the databases defined in DATABASES. When using the SQLite database engine, the tests will by default use an in-memory database (i.e., the database will be created in memory, bypassing the filesystem entirely!).

If you want to use a different database name, specify NAME in the TEST dictionary for any given database in DATABASES. On PostgreSQL, USER will also need read access to the built-in postgres database. Aside from using a separate database, the test runner will otherwise use all of the same database settings you have in your settings file: ENGINE, USER, HOST, etc. The test database is created by the user specified by USER, so you’ll need to make sure that the given user account has sufficient privileges to create a new database on the system.

Using Different Testing Frameworks

Clearly, unittest is not the only Python testing framework. While Django doesn’t provide explicit support for alternative frameworks, it does provide a way to invoke tests constructed for an alternative framework as if they were normal Django tests.

When you run ./manage.py test, Django looks at the TEST_RUNNER setting to determine what to do. By default, TEST_RUNNER points to django.test.runner.DiscoverRunner. This class defines the default Django testing behavior. This behavior involves:

  1. Performing global pre-test setup.
  2. Looking for tests in any file below the current directory whose name matches the pattern test*.py.
  3. Creating the test databases.
  4. Running migrate to install models and initial data into the test databases.
  5. Running the tests that were found.
  6. Destroying the test databases.
  7. Performing global post-test teardown.

If you define your own test runner class and point TEST_RUNNER at that class, Django will execute your test runner whenever you run ./manage.py test.

In this way, it’s possible to use any test framework that can be executed from Python code, or to modify the Django test execution process to satisfy whatever testing requirements you may have.

See the Django Project website for more information on using different testing frameworks.

What’s Next?

Now that you know how to write tests for your Django projects, we will be moving on to a very important topic once you are ready to turn your project into a real live website – deploying Django to a webserver.