Start on a Validation FT: Preventing Blank Items

As our first few users start using the site, we’ve noticed they sometimes make mistakes that mess up their lists, like accidentally submitting blank list items, or accidentally inputting two identical items to a list. Computers are meant to help stop us from making silly mistakes, so let’s see if we can get our site to help.

Here’s the outline of an FT:

def test_cannot_add_empty_list_items(self):
    # Edith goes to the home page and accidentally tries to submit
    # an empty list item. She hits Enter on the empty input box

    # The home page refreshes, and there is an error message saying
    # that list items cannot be blank

    # She tries again with some text for the item, which now works

    # Perversely, she now decides to submit a second blank list item

    # She receives a similar warning on the list page

    # And she can correct it by filling some text in
    self.fail('write me!')

That’s all very well, but before we go any further—our functional tests file is beginning to get a little crowded. Let’s split it out into several files, in which each has a single test method.

Remember that functional tests are closely linked to “user stories”. If you were using some sort of project management tool like an issue tracker, you might make it so that each file matched one issue or ticket, and its filename contained the ticket ID. Or, if you prefer to think about things in terms of “features”, where one feature may have several user stories, then you might have one file and class for the feature, and several methods for each of its user stories.

We’ll also have one base test class which they can all inherit from. Here’s how to get there step by step.

from unittest import skip
[...]

    @skip
    def test_cannot_add_empty_list_items(self):

$ python manage.py test functional_tests
[...]
Ran 4 tests in 11.577s
OK

class FunctionalTest(StaticLiveServerTestCase):

    def setUp(self):
        [...]
    def tearDown(self):
        [...]
    def wait_for_row_in_list_table(self, row_text):
        [...]


class NewVisitorTest(FunctionalTest):

    def test_can_start_a_list_for_one_user(self):
        [...]
    def test_multiple_users_can_start_lists_at_different_urls(self):
        [...]


class LayoutAndStylingTest(FunctionalTest):

    def test_layout_and_styling(self):
        [...]



class ItemValidationTest(FunctionalTest):

    @skip
    def test_cannot_add_empty_list_items(self):
        [...]

Ran 4 tests in 11.577s

OK

$ git mv functional_tests/tests.py functional_tests/base.py
$ cp functional_tests/base.py functional_tests/test_simple_list_creation.py
$ cp functional_tests/base.py functional_tests/test_layout_and_styling.py
$ cp functional_tests/base.py functional_tests/test_list_item_validation.py

import os
from django.contrib.staticfiles.testing import StaticLiveServerTestCase
from selenium import webdriver
from selenium.common.exceptions import WebDriverException
import time

MAX_WAIT = 10



class FunctionalTest(StaticLiveServerTestCase):

    def setUp(self):
        [...]
    def tearDown(self):
        [...]
    def wait_for_row_in_list_table(self, row_text):
        [...]

from .base import FunctionalTest
from selenium import webdriver
from selenium.webdriver.common.keys import Keys


class NewVisitorTest(FunctionalTest):

    def test_can_start_a_list_for_one_user(self):
        [...]
    def test_multiple_users_can_start_lists_at_different_urls(self):
        [...]

from selenium.webdriver.common.keys import Keys
from .base import FunctionalTest


class LayoutAndStylingTest(FunctionalTest):
        [...]

from selenium.webdriver.common.keys import Keys
from unittest import skip
from .base import FunctionalTest


class ItemValidationTest(FunctionalTest):

    @skip
    def test_cannot_add_empty_list_items(self):
        [...]

Ran 4 tests in 11.577s

OK

class ItemValidationTest(FunctionalTest):

    def test_cannot_add_empty_list_items(self):
        [...]

$ python manage.py test functional_tests.test_list_item_validation
[...]
AssertionError: write me!

$ git status
$ git add functional_tests
$ git commit -m "Moved Fts into their own individual files"

A New Functional Test Tool: A Generic Explicit Wait Helper

First let’s start implementing the test, or at least the beginning of it:

def test_cannot_add_empty_list_items(self):
    # Edith goes to the home page and accidentally tries to submit
    # an empty list item. She hits Enter on the empty input box
    self.browser.get(self.live_server_url)
    self.browser.find_element_by_id('id_new_item').send_keys(Keys.ENTER)

    # The home page refreshes, and there is an error message saying
    # that list items cannot be blank
    self.assertEqual(
        self.browser.find_element_by_css_selector('.has-error').text,  
        "You can't have an empty list item"  
    )

    # She tries again with some text for the item, which now works
    self.fail('finish this test!')
    [...]

This is how we might write the test naively:

We specify we’re going to use a CSS class called .has-error to mark our error text. We’ll see that Bootstrap has some useful styling for those.

And we can check that our error displays the message we want.

But can you guess what the potential problem is with the test as it’s written now?

OK, I gave it away in the section header, but whenever we do something that causes a page refresh, we need an explicit wait; otherwise, Selenium might go looking for the .has-error element before the page has had a chance to load.

Our first explicit wait was built into a helper method. For this one, we might decide that building a specific helper method is overkill at this stage, but it might be nice to have some generic way of saying, in our tests, “wait until this assertion passes”. Something like this:

[...]
    # The home page refreshes, and there is an error message saying
    # that list items cannot be blank
    self.wait_for(lambda: self.assertEqual(  
        self.browser.find_element_by_css_selector('.has-error').text,
        "You can't have an empty list item"
    ))

Rather than calling the assertion directly, we wrap it in a lambda function, and we pass it to a new helper method we imagine called wait_for.

So how would this magical wait_for method work? Let’s head over to base.py, and make a copy of our existing wait_for_row_in_list_table method, and we’ll adapt it slightly:

    def wait_for(self, fn):  
        start_time = time.time()
        while True:
            try:
                table = self.browser.find_element_by_id('id_list_table')  
                rows = table.find_elements_by_tag_name('tr')
                self.assertIn(row_text, [row.text for row in rows])
                return
            except (AssertionError, WebDriverException) as e:
                if time.time() - start_time > MAX_WAIT:
                    raise e
                time.sleep(0.5)

We make a copy of the method, but we name it wait_for, and we change its argument. It is expecting to be passed a function.

For now we’ve still got the old code that’s checking table rows. How to transform this into something that works for any generic fn that’s been passed in?

Like this:

    def wait_for(self, fn):
        start_time = time.time()
        while True:
            try:
                return fn()  
            except (AssertionError, WebDriverException) as e:
                if time.time() - start_time > MAX_WAIT:
                    raise e
                time.sleep(0.5)

The body of our try/except, instead of being the specific code for examining table rows, just becomes a call to the function we passed in. We also return its return value to be able to exit the loop immediately if no exception is raised.

Let’s see our funky wait_for helper in action:

$ python manage.py test functional_tests.test_list_item_validation
[...]
======================================================================
ERROR: test_cannot_add_empty_list_items
(functional_tests.test_list_item_validation.ItemValidationTest)
 ---------------------------------------------------------------------
Traceback (most recent call last):
  File "...python-tdd-book/functional_tests/test_list_item_validation.py", line
15, in test_cannot_add_empty_list_items
    self.wait_for(lambda: self.assertEqual(  
  File "...python-tdd-book/functional_tests/base.py", line 37, in wait_for
    raise e  
  File "...python-tdd-book/functional_tests/base.py", line 34, in wait_for
    return fn()  
  File "...python-tdd-book/functional_tests/test_list_item_validation.py", line
16, in <lambda>  
    self.browser.find_element_by_css_selector('.has-error').text,  
[...]
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate
element: .has-error
 ---------------------------------------------------------------------
Ran 1 test in 10.575s

FAILED (errors=1)

The order of the traceback is a little confusing, but we can more or less follow through what happened:

At line 15 in our FT, we go into our self.wait_for helper, passing it the lambda-ified version of the assertEqual.

We go into self.wait_for in base.py, where we can see that we’ve called the lambda, enough times that we’ve dropped out to the raise e because our timeout expired.

To explain where the exception has actually come from, the traceback takes us back into test_list_item_validation.py and inside the body of the lambda function, and tells us that it was trying to find the .has-error element that failed.

We’re into the realm of functional programming now, passing functions as arguments to other functions, and it can be a little mind-bending. I know it took me a little while to get used to! Have a couple of read-throughs of this code, and the code back in the FT, to let it sink in; and if you’re still confused, don’t worry about it too much, and let your confidence grow from working with it. We’ll use it a few more times in this book and make it even more functionally fun, you’ll see.

Finishing Off the FT

We’ll finish off the FT like this:

    # The home page refreshes, and there is an error message saying
    # that list items cannot be blank
    self.wait_for(lambda: self.assertEqual(
        self.browser.find_element_by_css_selector('.has-error').text,
        "You can't have an empty list item"
    ))

    # She tries again with some text for the item, which now works
    self.browser.find_element_by_id('id_new_item').send_keys('Buy milk')
    self.browser.find_element_by_id('id_new_item').send_keys(Keys.ENTER)
    self.wait_for_row_in_list_table('1: Buy milk')

    # Perversely, she now decides to submit a second blank list item
    self.browser.find_element_by_id('id_new_item').send_keys(Keys.ENTER)

    # She receives a similar warning on the list page
    self.wait_for(lambda: self.assertEqual(
        self.browser.find_element_by_css_selector('.has-error').text,
        "You can't have an empty list item"
    ))

    # And she can correct it by filling some text in
    self.browser.find_element_by_id('id_new_item').send_keys('Make tea')
    self.browser.find_element_by_id('id_new_item').send_keys(Keys.ENTER)
    self.wait_for_row_in_list_table('1: Buy milk')
    self.wait_for_row_in_list_table('2: Make tea')

I’ll let you do your own “first-cut FT” commit.

Refactoring Unit Tests into Several Files

When we (finally!) start coding our solution, we’re going to want to add another test for our models.py. Before we do so, it’s time to tidy up our unit tests in a similar way to the functional tests.

A difference will be that, because the lists app contains real application code as well as tests, we’ll separate out the tests into their own folder:

$ mkdir lists/tests
$ touch lists/tests/__init__.py
$ git mv lists/tests.py lists/tests/test_all.py
$ git status
$ git add lists/tests
$ python manage.py test lists
[...]
Ran 9 tests in 0.034s

OK
$ git commit -m "Move unit tests into a folder with single file"

If you get a message saying “Ran 0 tests”, you probably forgot to add the dunderinit—it needs to be there or else the tests folder isn’t a valid Python package…¹

Now we turn test_all.py into two files, one called test_views.py, which will only contains view tests, and one called test_models.py. I’ll start by making two copies:

$ git mv lists/tests/test_all.py lists/tests/test_views.py
$ cp lists/tests/test_views.py lists/tests/test_models.py

And strip test_models.py down to being just the one test—it means it needs far fewer imports:

from django.test import TestCase
from lists.models import Item, List


class ListAndItemModelsTest(TestCase):
        [...]

Whereas test_views.py just loses one class:

--- a/lists/tests/test_views.py
+++ b/lists/tests/test_views.py
@@ -103,34 +104,3 @@ class ListViewTest(TestCase):
         self.assertNotContains(response, 'other list item 1')
         self.assertNotContains(response, 'other list item 2')

-
-
-class ListAndItemModelsTest(TestCase):
-
-    def test_saving_and_retrieving_items(self):
[...]

We rerun the tests to check that everything is still there:

$ python manage.py test lists
[...]
Ran 9 tests in 0.040s

OK

Great! That’s another small, working step:

$ git add lists/tests
$ git commit -m "Split out unit tests into two files"

Well, that’s our FTs and unit test nicely reorganised. In the next chapter we’ll get down to some validation proper.