Running Firefox Selenium Sessions with Xvfb

The first thing is that PythonAnywhere is a console-only environment, so it doesn’t have a display in which to pop up Firefox. But we can use a virtual display.

In Chapter 1, when we write our first ever test, you’ll find things don’t work as expected. The first test looks like this, and you can type it in using the PythonAnywhere editor just fine:

from selenium import webdriver
browser = webdriver.Firefox()
browser.get('http://localhost:8000')
assert 'Django' in browser.title

But when you try to run it (in a Bash console), you’ll get an error:

(virtualenv)$ python functional_tests.py
Traceback (most recent call last):
File "tests.py", line 3, in <module>
browser = webdriver.Firefox()
[...]
selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable
needs to be in PATH.

Because PythonAnywhere is pinned to an older version of Firefox, we don’t actually need Geckodriver. But we do need to switch back to Selenium 2 instead of Selenium 3:

(virtualenv) $ pip install "selenium<3"
Collecting selenium<3
Installing collected packages: selenium
  Found existing installation: selenium 3.4.3
    Uninstalling selenium-3.4.3:
      Successfully uninstalled selenium-3.4.3
Successfully installed selenium-2.53.6

Now we run into a second problem:

(virtualenv)$ python functional_tests.py
Traceback (most recent call last):
File "tests.py", line 3, in <module>
browser = webdriver.Firefox()
[...]
selenium.common.exceptions.WebDriverException: Message: The browser appears to
have exited before we could connect. If you specified a log_file in the
FirefoxBinary constructor, check it for details.

Firefox can’t start because there’s no display for it to run on, because PythonAnywhere is a server environment. The workaround is to use Xvfb, which stands for X Virtual Framebuffer. It will start up a “virtual” display, which Firefox can use even though the server doesn’t have a real one (we use the same tool in Chapter 24 to run tests on a CI server).

The command xvfb-run will run the next command in Xvfb. Using that will give us our expected failure:

(virtualenv)$ xvfb-run -a python functional_tests.py
Traceback (most recent call last):
File "tests.py", line 11, in <module>
assert 'Django' in browser.title
AssertionError

So the lesson is to use xvfb-run -a whenever you need to run the functional tests.

Setting Up Django as a PythonAnywhere Web App

Shortly after that, we set up Django, using the django-admin.py startproject command. But, instead of using manage.py runserver to run the local development server, we’ll set up our site as a real PythonAnywhere web app.

Go to the Web tab and hit the button to add a new web app. Choose “Manual configuration” and then “Python 3.4”.

On the next screen, enter your virtualenv path (e.g., /home/yourusername/superlists/virtualenv).

Finally, click through to the link to edit your wsgi file and find and uncomment the section for Django. Hit Save and then Reload to refresh your web app.

From now on, instead of running the test server from a console on localhost:8000, you can use the real URL of your PythonAnywhere web app:

    browser.get('http://my-username.pythonanywhere.com')

That should work better.¹ You’ll need to keep using this pattern of pointing the FTs at the PythonAnywhere version of the site, and hitting Reload before each FT run, until Chapter 7, when we switch to using LiveServerTestCase and self.live_​server_url.

Cleaning Up /tmp

Selenium and Xvfb tend to leave a lot of junk lying around in /tmp, especially when they’re not shut down tidily (that’s why I included a try/finally earlier).

In fact they leave so much stuff lying around that they might max out your storage quota. So do a tidy-up in /tmp every so often:

$ rm -rf /tmp/*

Screenshots

In Chapter 5, I suggest using a time.sleep to pause the FT as it runs, so that we can see what the Selenium browser is showing on screen. We can’t do that on PythonAnywhere, because the browser runs in a virtual display. Instead, you can inspect the live site, or you could “take my word for it” regarding what you should see.

The best way of doing visual inspections of tests that run in a virtual display is to use screenshots. Take a look at Chapter 24 if you’re curious—there’s some example code in there.

The Deployment Chapter

When you hit Chapter 9, you’ll have the choice of continuing to use PythonAnywhere, or of learning how to build a “real” server. I recommend the latter, because you’ll get the most out of it.

If you really want to stick with PythonAnywhere, which is cheating really, you could sign up for a second PythonAnywhere account and use that as your staging site. Or you could add a second domain to your existing account. But most of the instructions in the chapter will be irrelevant (there’s no need for Nginx or Gunicorn or domain sockets on PythonAnywhere).

One way or another, at this point, you’ll probably need a paying account:

• If you want to run your staging site on a non-PythonAnywhere domain

• If you want to be able to run the FTs against a non-PythonAnywhere domain (because it won’t be on our whitelist)

• Once you get to Chapter 11, if you want to run Fabric against a PythonAnywhere account (because you need SSH)

If you want to just “cheat”, you could try running the FTs in “staging” mode against your existing web app, and just skip the Fabric stuff, although that’s a big cop-out if you ask me. Hey, you can always upgrade your account and then cancel again straight away, and claim a refund under the 30-day guarantee. ;)