Test Runner

First the good part, Django, by default, uses Python’s built in unittest library - and as of Python 2.7+ that has a reasonable set of built in assertion types. (And for versions of django before 1.8, django backported the python 2.7 unittest library.) Django has a pretty good test discovery system (apparently from the upgraded Python 2.7 unittest library) and will run any code in files matching test*.py. So to run all your test, you just type ./manage.py test at the top of your project. You can also run the tests in individual modules or classes by doing something like: ./manage.py test animals.tests - without having to put the if name == __main__ stuff in each file. You can even run individual tests - though you have to know the method name (more or less like you have to in Ruby’s minitest framework): ./manage.py test animals.tests.AnimalTestCase.test_animals_can_speak

The best part of Django’s test runner is the setup it does for you - creating a new test database for each run, running your migrations, and if you ask it to, importing any fixtures you request. Then, after collection all the discovered tests, it runs each test inside a transaction to provide isolation.

Test Output

But coming from the Ruby and Ruby on Rails worlds, the testing tools in Python and Django are not as elegant as I am used to. At one point I thought the Ruby community’s emphasis on creating testing tools that display English-like output for the running tests bordered on obsessive. But having spent some time in Python/Django which doesn’t encourage tools like that, I have come to appreciate the Rubyist’s efforts. Both RSpec and Minitest have multiple built in output format options - and lots of people have created their own reporter add ons - so you can see your test output exactly the way you want it with very little effort. The django test command allows 4 verbosity levels but for the most part they only change how much detail you get about the setup process for the tests. The only differences in the test output reporting are that you get dots at verbosity levels 0 and 1 and the test names and file locations at levels 2 and 3:

    $ python ./manage.py test -v 2

    ..... setup info omitted .......

    test_debugging (accounts.tests.test_models.UserModelTests) ... ok
    test_login_link_available_when_not_logged_in (accounts.tests.test_templates.LoginLinkTests) ... ok
    test_logout_link_available_when_logged_in (accounts.tests.test_templates.LoginLinkTests) ... ok
    test_signup_link_available_when_not_logged_in (accounts.tests.test_templates.LoginLinkTests) ... ok
    test_user_account_link_available_when_logged_in (accounts.tests.test_templates.LoginLinkTests) ... ok
    test_profile_url (accounts.tests.test_urls.AccuntsURLTests) ... ok
    test_signup_url (accounts.tests.test_urls.AccuntsURLTests) ... ok
    test_url_for_home_page (mk_web_core.tests.GeneralTests) ... ok

    ----------------------------------------------------------------------
        Ran 8 tests in 0.069s

So increasing the verbosity level is useful for debugging your tests but disappointing if you are trying to use the tests to document your intentions.

Behave and django-behave

This is the main reason why, despite being unenthusiastic about Cucumber in Ruby, I am supporting using Python’s behave with django-behave for our new project. One of the things I don’t like about cucumber is it all to frequently becomes an exercise in writing regular expressions (for the step matching). I don’t like that if you need to pass state between steps, you set instance variables; this is effective, but it looks kind of like magic.

With ‘behave’, you need to do the same things but in more explicit ways. The step matching involves litteral text with placeholders. If you want to do full regular expression matching you can, but you need to set the step matcher for that step to be ‘re’ - regular expression matching isn’t the default. For sharing state, there is a global context variable. When you are running features and scenarios, additional namespaces get added to the root context object - and then removed again as they go out of scope again. Adding information to the context variable seems more explicit - but with the namespace adding and removing - I am not sure that this isn’t more magical than the instance variables in Cucumber.

Django’s TestCase Encourages Integration Tests

The main testing tool that Django encourages using is it’s TestCase class which tests a bunch of concerns - request options, routing, the view’s context dictionary, response status and template rendering.

It’s odd to me that Django’s docs only discuss integration tests and not proper unit tests. With Django’s Pythonic explicitness, it is fairly easy to set up isolated pieces of the Django stack by just importing the pieces you care about into your test. For example, you can test your template’s rendering by creating a dictionary for the context information and then rendering the template with that context. Harry Percival’s book “Test-Driven Development with Python” does a very nice job of showing you how to unit test the various sections of the Django stack - routing, views, templates, etc.

More than just not discussing isolated unit tests, at least some of Django’s built in assertions actively require you to write a functional / integration test. I tried rendering my template to html and then called assertContains to test some specific html info. But I got an error about the status code! In order to use assertContains on the template, I have to make a view request.

Coming from the Rails world, I don’t really want the simple assertContains matching. What I really want is Rails’ built-in html testing method, assert_select. I found a Django library that is somewhat similar, django-with-asserts. But like assertContains, django-with-assert’s test mixin class uses the Django TestCase as it’s base and so also wants you to make a view request so it can test the StatusCode. I really wanted django-with-asserts functionality but I want to use it in isolation when I can, so I forked it and removed the dependency on the request / response cycle.