A test case is nothing more than a list of steps and expected results. For example, assume that you need to test authentication functionality. To make sure that authentication works properly, you need to check that after filling out the login form we will be redirected to the main page if the authentication succeeds. So this test case could be defined as shown in the following table.

With the test case defined this way, you can to execute it easily because everything is clear — you know what action you need to execute and what the expectations are. All test cases must fulfill the requirements, which are defined before each functionality is implemented. Each requirement has its own acceptance criteria. Test cases also define steps to determine whether an application meets the acceptance criteria.

Note that the test case described in the preceding table doesn't really cover the requirement that authentication function properly because if the application accepted just any username and password, the test case would end successfully, but the application obviously wouldn't work properly and thus wouldn't meet the acceptance criteria. Each requirement in most cases needs more than one test case. Therefore, in the example you should also create additional test cases that check the negative scenario as well. An exemplary negative scenario test case is shown in the following table.

Other test cases can be made to check the main page when it's shown after successful login. You are free to invent more test cases to cover the requirements. Each test case can end as successful or failed. If it fails, we can show exactly where it's failing, so searching for errors is easier. That's why test cases are so important when we want to deliver quality software.

Test suites are just collections of test cases. Test suites collect test cases testing the same general issue. Most often, test cases from one test suite are executed together to cover a whole functionality.

For the issue of login authentication, you can call the corresponding example test suite AppLogin, containing test cases such as L001_PositiveLoginAdmin and L002_NegativeLoginAdmin, and also L003_PositiveLoginUser or L004_NegativeLoginUser, because this test suite should cover not only administrators' authentication but also users who are not administrators. This test suite should cover all roles that are supported by this authentication module.

And what if you deliver authentication through web services as well? Should this also be covered by this test suite? In most cases, there are separate test suites because web services authentication is an additional functionality that is not implemented at the same time as web authentication. But this also depends on requirements. The same concerns all other functionalities — test suites are created to group test cases by functionality, implementation time, and application structure.

Test coverage is a term that corresponds to the percentage of source code covered by test cases. It may be applied to the coverage of automated or manual tests of source code: code can be covered 50 percent through automated tests and 30 percent through manual tests. We may also speak of test coverage while testing concrete functionalities. We can say whether this functionality is covered by tests or not. A higher test coverage index usually results in higher software quality, but only if the tests are executed in a timely manner, of course.

Black-box tests are so named because they follow the black-box scheme. In software testing this means that a tester doesn't know the code he is executing. Therefore we need to define the input of the application and then check the application's output (see Figure 15-1).

Figure 15.1. Black-box testing idea

The actual output is compared with the expected output value; if they are equal, the test case or step is checked as successful. The two authentication test cases described previously would be black-box tests if we didn't know the application's code. We don't know exactly what is done within the application's authentication module, but we know that we should be redirected to the main page if the authentication ends successfully.

To show the idea of black-box testing, we have used the Lime test framework that is used in Symfony. As you can see, the following code presents a step-by-step scenario that covers the previously defined test case. Note that we don't check each step because we assume that the sfBrowser() methods are doing this correctly. If login page loading fails, we should get an error within the setField() method because we will not be able to set a value to the username field. At the end, we check the result page. If both values are equal, the test case succeeds. Note that we are using only Lime classes and methods, so we do not interfere with the application's code.

<?php
include(dirname(__FILE__).'/../../bootstrap/functional.php');
$browser = new sfTestFunctional(new sfBrowser());

$browser->get('/security/index');
$browser->with('response')->begin()->isStatusCode(200);
$browser->setField('login', 'admin');
$browser->setField('password', 'secret');
$browser->click('submit');
$browser->end();
$response= $browser->getResponse();
$browser->with('response')->
isStatusCode(200);
$browser->test()->like($response->getContent(), '/Main page/');

White-box tests, unlike black-box tests, are based on knowledge of the code that is being tested. The tester needs to have programming skills because he is testing the application at a lower level than in the case of black-box tests. The most popular tests that use white-box testing are unit tests. Unit tests cover the application at the source code level. The tester invokes methods to complete a scenario that realizes a requirement. We have used PHPUnit here to show the main assumptions of white-box testing. It is one of the most popular test frameworks for PHP. An exemplary scenario that tests the authorization module can look as follows:

<?php
require_once("Auth.class.php");
class AuthTest extends PHPUnit_Framework_TestCase {
    protected $_auth = null;
    protected $_adminLogin = "admin";
    protected $_adminPassword = "secret";
    protected $_wrongLogin = "badAdmin";
    protected $_wrongPassword = "wrongSecret";
    public function setUp() {
        $this->_auth = new Auth();
    }
    public function tearDown() {
        unset($this->_auth);
    }
    public function testAdminLogin() {
        $result = $this->_auth->authenticate($this->_adminLogin,
        $this->_adminPassword);
        $this->assertTrue($result);
    }
    public function testAdminLoginNegative() {
        $result = $this->_auth->authenticate($this->_wrongLogin,
        $this->_wrongPassword);
        $this->assertTrue($result);
    }
}

The preceding code has two test cases: AdminLogin and AdminLoginNegative. Unlike black-box testing, in this case we invoke authorization class methods, so we use the code that was written by developers. After getting the result of authentication, we need to check whether it is a true value. If yes, the test case ends successfully.

Grey-box tests are in fact a combination of black- and white-box tests. Application testing is done at the black-box level, but with knowledge of the application's data structures and algorithms. Note that in black-box testing, the data that is input can be given blindly.

Smoke tests are a group of tests that check the most important functionalities. Most important means that these are vital functionalities that are needed for an application to make sense.

Suppose you are developing a banking application. If the funds-transfer functionality of this application has blocking bugs (bugs that don't allow the application to proceed to the next step), the product doesn't make sense for any bank and it must be repaired as soon as possible. Imagine what would happen if you delivered a product without the main functionalities. To prevent that situation, smoke tests should be executed before delivery of every version of the product. These tests are also used in situations when developers deliver a new module and want to know if the whole program still works as designed. Smoke tests are very short tests that should take only a small part of the execution time of all tests.

Load tests and stress tests are sometimes recognized as subgroups of performance tests, but they really aren't. They are properly separate test groups because their goals are different, even if the tools used to execute them are the same.

With performance tests, you search for bottlenecks in your application. Performance can be measured in throughput, response times, scalability, or resource usage. The goal is to find places in our application where we can improve it to handle more web clients. For example, you can reduce HTML or CSS files when it's possible, so you need less bandwidth for the same effect.

Load tests check applications' behaviors in situations when the load is high. For example, we can start 1,000 threads that log in and log out from our application 24 hours per day. Normally, memory usage should finally fix on one value at some moment when testing begins and stays constant for 24 hours. If it is increasing all the time, you probably forgot to free the application's unused memory and should look for such bugs.

Stress tests are responsible for testing the durability of our application against peak load. As an example, let's take a banking application. Stress tests will test how many transfers your application can serve in a fixed period of time. It may be able to handle 1,000 transfers at one moment. If the application is working properly with 1,000, you should increase the amount to 10,000, and so on. If the application starts to crash or behave in an unstable way, this is probably the application's transfer limit. It depends, of course, on the hardware on which it is executed, but we still can say that on this particular machine the limit is 10,000 transfers. Then this can be scaled to a more powerful hardware configuration to achieve better results.

A popular tool for performance, load, and stress tests is Apache JMeter, which is discussed and applied in Chapter 17.

Regression tests are tests that are executed each time a new functionality is delivered. Scrum methodology states that these tests should be executed after each iteration ends. It depends on the situation, time, and human resources, but in most cases, whole tests are executed. Whole tests means all test cases, including both automated and manual test cases. The main reason for executing these tests is to check that nothing changed in the application since it was tested last time. In other words, we need to check that a new functionality didn't break any functionalities that were previously working correctly (the functionality passes all tests).

Figure 15.8. PHPUnit logo.

PHPUnit (the logo is shown in Figure 15-8) is probably the most popular test framework among the PHP frameworks. It's the default test framework in Symfony 2.0 and Zend Framework.

To install PHPUnit separately, you can use your Linux distribution package manager or PEAR. The latter is like installing Symfony or Zend Framework. You need to run this command first:

# pear channel-discover pear.symfony-project.com

You can also use PHPUnit's channel:

# pear channel-discover pear.phpunit.de

After adding the channel, you can commence installation using this command:

# pear install phpunit/PHPUnit

Note that you can now run the PHPUnit command:

$ phpunit

$ symfony plugin:install sfPhpunit2Plugin

$ symfony plugin:install sfPhpUnitPlugin

phpunit
    :generate-compat
    :generate-functional
    :generate-selenium
    :generate-unit
    :test-all
    :test-functional
    :test-selenium
    :test-unit

Figure 15.9. SimpleTest logo.

SimpleTest (the logo is shown in Figure 15-9) is an alternative solution to PHPUnit. It is also a full-featured test framework, very popular as a stand-alone solution. Modified SimpleTest is the default framework for CakePHP.

CakePHP contributors don't include test framework libraries into CakePHP, which can be a little annoying. Do they assume that nobody tests their applications?! If the libraries are not installed, you will see the following error:

$ cake testsuite
Error: Sorry, Simpletest could not be found. Download it from http://simpletest.org
and install it to your vendors directory.

You need to download SimpleTest from http://simpletest.org and extract the package into the CakePHP projects /app/vendors/ directory. Now you can start testing, and the error message will not be shown.

Apart from the /vendors directory where your SimpleTest libraries are placed, CakePHP also provides also a separate directory for test cases. The directory structure looks like this:

app/tests/
    cases/

behaviors/
    components/
    controllers/
    helpers/
    models/
  fixtures/
  groups/

These directories are dedicated for test cases, fixtures, and groups of tests, respectively. Test cases are divided into those covering behaviors (such as ACL), components (such as Email), controllers, helpers, and models.

ZF does not provide any CLI facilities in this case, but thanks to integrated PHPUnit, ZF gives us a lot of classes that are ready to use. If you installed Zend Framework with PEAR following the installation in Chapter 2, the test library is placed in /usr/share/php/Zend/Test (UNIX) or C:\xampp\php\PEAR\Zend\Test (Windows with XAMPP). These classes are (for example) Zend_Test_PHPUnit_Db_Exception or Zend_Test_PHPUnit_ControllerTestCase. ZF also provides a testing directory structure that looks like this:

application/
    bootstrap.php
library/
    bootstrap.php
phpunit.xml

You can configure your tests by editing phpunit.xml. This configuration file is used for running test cases with PHPUnit as follows:

$ phpunit --configuration phpunit.xml

This framework is used in Symfony 1.x. It is obsolete and is not used in Symfony 2.x as a part of the whole framework. We recommend that you use PHPUnit instead of Lime. We don't want to recommend bad practices, which is why we don't use Lime further in this chapter.

Functional tests in Symfony are better done using Selenium. In Symfony 1.4 and later 1.x versions, the PHPUnit plug-in is delivered as a fake wrapper that in fact uses Lime in the background. PHPUnit is a solution we can recommend, while Lime leaves much to be desired. That's why we advise against using Symfony 1.x functional tests. If possible, please use Symfony 2.0 or the Selenium extension delivered within the PHPUnit2 plug-in for Symfony 1.x.

CakePHP is based on a modified version of the SimpleTest framework. Let's test the registration functionality. (Chapter 5 discusses what you need to do to implement a registration form.) The CakePHP testing approach requires that every test file name should end with .test.php. For this example, the filename will be registration.test.php. We are testing the controller, so you need to save this file into the /app/tests/cases/controllers/ directory. Let's call the first test RegistrationControllerTest and add the testRegistrationPositive() method, which should load our registration page and check whether expected form fields exist. An exemplary test can look like the following:

<?php
class RegistrationFormTests extends WebTestCase {
    function testRegistrationPositive() {
        $this->get('http://localhost/registration');
        $this->assertField('username', '');
        $this->assertField('password', '');
        $this->assertField('repeat_password', '');
        $this->assertField('forename', '');
        $this->assertField('surname', '');
        $this->assertField('email', '');
    }
}
code snippet /black-box/cakephp/app/tests/cases/controllers/registration_controller.test.php

That's all you need for the first step. Why do we check to see whether form fields are blank? Because developers sometimes don't clean the code. By mistake, a username field can be set to something like foo or worse. This is a question for a QA engineer. Next, you need to fill out the registration form with proper data. The setField() method is used to set insert data into form input fields. The following code snippet illustrates how to do this:

$this->setField('username', 'admin');
$this->setField('password', 'secret');
$this->setField('repeat_password', 'secret');
$this->setField('forename', 'John');
$this->setField('surname', 'Smith');
$this->setField('email', 'john@example.com');
code snippet /black-box/cakephp/app/tests/cases/controllers/registration_controller.test.php

The preceding code needs to be placed after the assertField() methods within the testRegistrationPositive() method. We can, of course, believe our testing tools that they are setting form fields in the right way, but a good quality assurance engineer never trusts programs completely. That's why a good practice is to check registration form fields again. This time, we need to check whether values within the input fields are equal to those that we set earlier. We need to use the same assertField() method as previously, when we checked whether form fields are blank.

$this->assertField('username', 'admin');
$this->assertField('password', 'secret');
$this->assertField('repeat_password', 'secret');
$this->assertField('forename', 'John');
$this->assertField('surname', 'Smith');
$this->assertField('email', 'john@example.com');
code snippet /black-box/cakephp/app/tests/cases/controllers/registration_controller.test.php

These lines of code need to be placed after the setField() methods and within the testRegistrationPositive() method.

At this point, you are done with Step 2. In the next step, you will check whether the submit button can be pushed and whether any response is given. To click any field, you need to use the click() method. The assertResponse() method is dedicated to checking the response from the server. Step 3 should look like the following:

$this->click('submit');
$this->assertResponse(200);

This code needs to be added at the end of testRegistrationPositive(). In the last step, you expect to be redirected to a Thank You page. As shown in the following code, you need to get the current URL and compare it with the expected one. After that, you should also search page content for the expected text:

$current=$this->getUrl();
$this->assertEqual('http://localhost/index.php/thankyou',$current);
$this->assertText('Thank You');

The preceding code completes Step 4. Normally, you would be finished at this point, but when you go deeper into this test case you will see a lot of problems. What if the registration form is shown on the Thank You page? What if validation rules throw a validation error? These and more issues need to be coped with in further test cases. Use your imagination and write more test cases that cover the whole registration functionality.

$ cake testsuite app all

Welcome to CakePHP v1.3 Console
---------------------------------------------------------------
App : app
Path: /home/wrox/public_html/cake/app
---------------------------------------------------------------
CakePHP Test Shell
---------------------------------------------------------------
Running app all
All App Tests
1/1 test cases complete: 15 passes.

$ cake testsuite app case controllers/registration

$ cake testsuite help

Web-based Test Execution

CakePHP, unlike Symfony or Zend Framework, delivers a great feature for web-level testing. To see all available test cases — those that you wrote and those delivered with CakePHP — you need to access test.php in the CakePHP /webroot directory. The right URL is http://localhost/test.php if your CakePHP application path is set in Apache to http://localhost/. When you access test.php, you should see something similar to Figure 15-10.

Figure 15.10. CakePHP — all tests from web level

If you choose Test Cases from the App group (refer to Figure 15-10), you should see only one test case, the one that you wrote if you completed the example earlier in this section. After clicking the name of the test case, you should see something similar to Figure 15-11.

You can see all assertions that have passed by clicking the Show Passes option.

Figure 15.11. CakePHP — test execution results in the Web

Zend Framework's functional tests are based on Selenium. We don't say that it's impossible to develop functional tests that work like CakePHP's tests, but just as with Symfony, we recommend using Selenium to accomplish this goal. The next section describes how to do this. Note that CakePHP uses SimpleTest, which doesn't support Selenium out of the box because SimpleTest has its own functional test solution. PHPUnit developers decided to use a known, existing solution and integrate it.

As described earlier, Symfony delivers some command-line tasks, and two are dedicated for Selenium tests. One of them generates a default test case. To do it, you need to type the following command:

$ symfony phpunit:generate-selenium frontend login

You should see output similar to this:

>> dir+ /home/wrox/public_html/symfony/test/phpunit/selenium/frontend
>> file+ /home/wrox/public_html/symfony/test/phpunit/selenium/frontend
           /loginActionsTest.php
>> help run this test with: ./symfony phpunit:test-selenium frontend login

Generated code should be as follows:

<?php
require_once dirname(__FILE__).'/../../bootstrap/selenium.php';
class selenium_frontend_loginActionsTest extends sfPHPUnitBaseSeleniumTestCase {
    protected function setUp() {
        $this->setBrowser('*firefox');
        $this->setBrowserUrl('http://localhost/');
    }
    public function testPositiveAdminLogin() {
    }
}
code snippet /selenium/symfony/test/phpunit/selenium/frontend/loginActionsTest.php

We changed the default test case method name to make it more meaningful. Now we need to fill it out with proper code. The following code accomplishes our earlier defined test case:

public function testPositiveAdminLogin() {
        $this->open('http://localhost/login');
        $this->assertTitle('Login Page');
        $this->type("username", "admin");
        $this->type("password", "secret");
        $this->click("submit");
        $this->waitForPageToLoad("30000");
        $this->verifyTextPresent("Successfully logged in");
}
code snippet /selenium/symfony/test/phpunit/selenium/frontend/loginActionsTest.php

You can run Selenium tests with the following command:

$ symfony phpunit:test-selenium

This should give an output similar to this:

PHPUnit 3.4.15 by Sebastian Bergmann.
.
Time: 6 seconds, Memory: 10.00Mb
OK (1 test, 2 assertions)

We get plain text output, but Selenium also provides the Selenium Remote Control Web console, as shown in Figure 15-16.

Figure 15.16. Selenium Remote Control

Also, a second browser window is opened, in which all steps are executed one by one.

CakePHP doesn't support Selenium by default because its SimpleTest doesn't provide Selenium support, unlike PHPUnit. But this problem can be easily solved using the Testing_Selenium package that is available in the PEAR package database. You can download it from http://pear.php.net/package/Testing_Selenium. You can also install it by using the PEAR command-line installation parameter, but in this case, a better solution is to extract this package to CakePHP's /vendors directory. Because of dependencies within the main Selenium.php file, it's better to extract files from the package to the /Testing subdirectory. After proper extraction, your directory structure should look like the following:

app\vendors\Testing\
    examples\
    Selenium\
    tests\
    ChangeLog
    readme
    Selenium.php
    selenium-server.jar
    todo

Make sure that it's exactly the same in your case. Next, you need to make one easy change in Selenium.php. Find this line:

require_once 'Testing/Selenium/Exception.php';

Change it to this:

require_once 'Selenium/Exception.php';

This is only one word, but without this change you would not be able to run tests using Selenium. This PEAR package is something like a wrapper for PHPUnit Selenium classes. After making these changes, you can start implementing a sample test case. The test case can be saved as selenium.test.php and placed in the /app/tests/cases/behaviors directory.

To begin, include the Selenium wrapper. In CakePHP, you can do so by using the App::import() method. An exemplary test class draft can look like the following:

<?php
App::import('Vendor','Selenium', array('file' => 'Testing'.DS.'Selenium.php'));
class LoginSeleniumTest extends UnitTestCase {
    function setUp() {
    }
    function tearDown() {
    }
}
code snippet /selenium/cakephp/app/tests/cases/behaviors/selenium.test.php

The first parameter in the import() method, Vendor, states from which part of CakePHP we want to import a class. The next parameter is only an identifier; you can call it whatever you want. The last parameter is an array. For the file key, assign the path and filename of Selenium.php. The DS variable is the separator between paths. In Windows, it's "\"; in UNIX it's "/". Now you can define what the setUp() and tearDown() functions will do. These methods are very common in many testing frameworks: setUp() is invoked when a test begins, and tearDown() is invoked when all invoked methods are done (except tearDown()). Before executing any test case, we need to define in which Internet browser the test cases should be executed and the starting URL. This method looks like the following:

function setUp() {
        $this->selenium =
            new Testing_Selenium("*firefox /usr/lib/firefox/firefox-bin",
                "http://localhost/login");
        $this->selenium->start();
}
code snippet /selenium/cakephp/app/tests/cases/behaviors/selenium.test.php

Because we don't need to work with Selenium after test cases are done, the tearDown() method should be as follows:

function tearDown() {
        $this->selenium->stop();
}
code snippet /selenium/cakephp/app/tests/cases/behaviors/selenium.test.php

Let's name our test case as we did in Symfony and Zend Framework: testPositiveAdminLogin():

function testPositiveAdminLogin() {
}
code snippet /selenium/cakephp/app/tests/cases/behaviors/selenium.test.php

In CakePHP, follow the same steps as in Symfony. The only difference is that you do not invoke Selenium dedicated methods through $this, but through the $this->selenium wrapper. Therefore, the testPositiveAdminLogin() method content should be as follows:

$this->selenium->open('http://localhost/login');
$this->assertTrue($this->selenium->getTitle('Login page'));
$this->selenium->type("username", "admin");
$this->selenium->type("password", "secret");
$this->selenium->click("login");
$this->selenium->waitForPageToLoad("30000");
$this->selenium->verifyTextPresent("Successfully logged in");
code snippet /selenium/cakephp/app/tests/cases/behaviors/selenium.test.php

Figure 15.17. CakePHP test case web browser

Now, when you go to http://localhost/test.php, you should be able to see two test cases, as shown in Figure 15-17.

Remember to start Selenium RC before you run any Selenium test cases. Otherwise, you will see something similar to Figure 15-18.

Figure 15.18. CakePHP error when Selenium RC is not found

Zend Framework doesn't support any features that help executing test cases. This is not really a problem, though, because ZF delivers PHPUnit classes, so with the phpunit command, you don't need anything else. To start, you need to create a directory in which you want to keep your Selenium test cases. Let it be /tests/application/selenium. You can save your test drafts as LoginTestCases.php. It should look like the following:

<?php
require_once 'PHPUnit/Extensions/SeleniumTestCase.php';
class LoginSeleniumTest extends PHPUnit_Extensions_SeleniumTestCase {
    protected function setUp() {
        $this->setBrowser("*chrome");
        $this->setBrowserUrl("http://localhost/");
    }
    public function testPositiveAdminLogin() {

}
}
?>
code snippet /selenium/zf/tests/application/selenium/LoginTestCases.php

In fact, all the test drafts do not differ from each other a lot. The only difference is what libraries they import. In the case of ZF, Selenium libraries are stored in C:\xampp\php\PEAR\PHPUnit under Windows and in /usr/share/php/PHPUnit under UNIX systems. Note that you don't need to use Chrome or Firefox as testing browsers; you can use any of the following with $this->setBrowser():

*firefox
*chrome
*iexplore
*googlechrome
*safari
*opera
*custom

firefox, googlechrome, opera, safari, and iexplore are obvious. Note that if you choose chrome, Firefox will be used. When you choose custom, you will be able to use other browsers not listed previously. In this case, you need to give a proper path to the browser. (This is also a good solution if you want to test more versions of a browser.)

The test case should look like this:

$this->open('http://localhost/login');
$this->assertTitle('Login Page');
$this->type("username", "admin");
$this->type("password", "secret");
$this->click("login");
$this->waitForPageToLoad("30000");
$this->verifyTextPresent("Successfully logged in");
code snippet /selenium/zf/tests/application/selenium/LoginTestCases.php

It's the same as with Symfony. The difference is in execution. In Symfony, we did it with tasks delivered out of the box. In ZF, we need to use PHPUnit's executables. In ZF, each test can be executed as follows:

$ phpunit application/selenium/LoginTestCases.php

This needs more effort because you need to automate it further when more tests are available. When you switch to the Selenium RC console, you should be able to see something similar to this:

11:12:43.261 INFO - creating new remote session
11:12:43.262 INFO - Allocated session 1b843f2a2f714a049305340656f72dc7 for
    http://localhost/, launching...
11:12:43.285 INFO - Preparing Firefox profile...
11:12:45.658 INFO - Launching Firefox...

11:12:48.702 INFO - Got result: OK,1b843f2a2f714a049305340656f72dc7 on session
    1b843f2a2f714a049305340656f72dc7
11:12:48.748 INFO - Command request: setTimeout[30000, ] on session
    1b843f2a2f714a049305340656f72dc7
11:12:48.755 INFO - Got result: OK on session 1b843f2a2f714a049305340656f72dc7
11:12:48.802 INFO - Command request: open[http://localhost/login, ] on session
    1b843f2a2f714a049305340656f72dc7
11:12:48.912 INFO - Got result: OK on session 1b843f2a2f714a049305340656f72dc7
11:12:48.959 INFO - Command request: getTitle[Login Page, ] on session
    1b843f2a2f714a049305340656f72dc7
11:12:48.975 INFO - Got result: OK,Login Page on session
    1b843f2a2f714a049305340656f72dc7
11:12:49.023 INFO - Command request: type[username, admin] on session
    1b843f2a2f714a049305340656f72dc7
11:12:49.053 INFO - Got result: OK on session 1b843f2a2f714a049305340656f72dc7
11:12:49.101 INFO - Command request: type[password, secret] on session
    1b843f2a2f714a049305340656f72dc7
11:12:49.128 INFO - Got result: OK on session 1b843f2a2f714a049305340656f72dc7
11:12:49.176 INFO - Command request: click[submit, ] on session
    1b843f2a2f714a049305340656f72dc7
11:12:49.288 INFO - Got result: OK on session 1b843f2a2f714a049305340656f72dc7
11:12:49.335 INFO - Command request: waitForPageToLoad[30000, ] on session
    1b843f2a2f714a049305340656f72dc7
11:12:49.358 INFO - Got result: OK on session 1b843f2a2f714a049305340656f72dc7
11:12:49.406 INFO - Command request: isTextPresent[Successfully logged in, ] on
    session 1b843f2a2f714a049305340656f72dc7
11:12:49.426 INFO - Got result: OK,true on session 1b843f2a2f714a049305340656f72dc7
11:12:49.474 INFO - Command request: testComplete[, ] on session
    1b843f2a2f714a049305340656f72dc7
11:12:49.474 INFO - Killing Firefox...
11:12:49.955 INFO - Got result: OK on session 1b843f2a2f714a049305340656f72dc7

This output shows what exactly was executed by Selenium RC step by step. ZF doesn't deliver any scripts to run test cases, but we still can use PHPUnit's configuration XML file to run more test cases at one time. To make it possible, edit the phpunit.xml file placed in the /tests directory. It should look like the following code:

<phpunit bootstrap="./application/bootstrap.php">
    <testsuite name="Wrox Example">
          <directory>./</directory>
    </testsuite>
</phpunit>
code snippet /selenium/zf/tests/phpunit.xml

You need to define the bootstrap file that will set up all needed variables before the test suites are executed. A test suite name should be defined as well as the test suite's directory in which PHPUnit will start to search for test cases. In the case of Zend Framework, bootstrap.php should be as following:

<?php
define('BASE_PATH', realpath(dirname(__FILE__) . '/../../'));
define('APPLICATION_PATH', BASE_PATH . '/application');
define('APPLICATION_ENV', 'testing');
code snippet /selenium/zf/tests/application/bootstrap.php

Note that you define the application's environment as testing, so all testing parameters, such as database connection settings, will be used. To execute tests, you can use PHPUnit's command-line application:

$ phpunit -configuration phpunit.xml

The Mailing class should be saved as Mailing.class.php in the project's /lib directory to make it available for every application.

In CakePHP, the Mailing class should be named as in Symfony and placed in the application's library path: /app/libs. But there is one difference. Because of CakePHP's specific behavior, it's easier to make a model object an argument of the sendMail() method. After this change, the sendMail() method should be as follows:

public function sendMail($mail) {
    /* method content */
}

ZF's Mailing.class.php should be placed in the project's /library path. It should not include the CakePHP change.

Symfony 2.0 uses PHPUnit, whereas Symfony 1.0 uses Lime (except when the PHPUnit2 plug-in is installed, in which case Symfony1 uses PHPUnit). The following example is written in Symfony 1.0 with the PHPUnit2 plug-in installed. To start working with Symfony tests, you need to generate a default unit test:

$ symfony phpunit:generate-unit mailing
This will generate proper files:
>> dir+ /home/wrox/public_html/symfony//test/phpunit/unit
>> file+ /home/wrox/public_html/symfony//test/phpunit/unit/mailingTest.php
>> help run this test with: ./symfony phpunit:test-unit mailingTest

You need to use the Mailing class, so it should be set up for every test case. To do that, you can use the setUp() and tearDown() methods as follows:

<?php
require_once dirname(__FILE__).'/../bootstrap/unit.php';
class unit_mailingTest extends sfPHPUnitBaseTestCase
{
    public function setUp() {
        $this->_mailing = new Mailing();
    }
    public function tearDown() {
        unset($this->_mailing);
    }
    /* test cases methods */
}
code snippet /unit/symfony/test/phpunit/unit/mailingTest.php

The first two test cases are very simple because the implementation needs only to change the attributes that are sent. Finally, the test cases should be as follows:

public function testSubscriptionPositive() {
    $this->assertTrue($this->_mailing
           ->sendMailSubscription("full","example@wroxexample.com"));
    }
    public function testSubscriptionNegative() {
       $this->assertTrue(!$this->_mailing
          ->sendMailSubscription("foobar","example@wroxexample@com"));
    }
code snippet /unit/symfony/test/phpunit/unit/mailingTest.php

Note that $this->_mailing is the Mailing instance. The last test case doesn't need any data in the Mails table, so it can be easily implemented as follows:

public function testRandomMailNegative() {
        $this->assertTrue(!$this->_mailing->sendMail());
    }
code snippet /unit/symfony/test/phpunit/unit/mailingTest.php

In both negative test cases, we place an exclamation mark to the left of $this->_mailing to check whether the inversed values from the returned values are true.

The last test case is the most complicated of all. You need to define fixtures that would be loaded before sendMail() is invoked. Fixture files should be placed in the project's /data/fixtures directory. For example, you can save the fixture file as mailing.yml in the /data/fixtures/mailing directory. This separates fixture files from the functionalities for which they are needed. mailing.yml should be defined as follows:

mails:
  john_smith:
    email: example@wroxexample.com
    forename: John
    surname: Smith
    subscription: full
code snippet /unit/symfony/data/fixtures/mailing/mailing.yml

To load data, you need information about the project configuration, which provides information about database connections that can be used to load fixture data into proper tables. You should use sfDatabaseManager to establish a connection and Doctrine_Core to load data. After testing, the sendMail() method fixtures need to be deleted, as shown in the following code:

public function testRandomMailPositive() {
    $configuration =
       ProjectConfiguration::getApplicationConfiguration('frontend','test',true);
    $conn = new sfDatabaseManager($configuration);
    Doctrine_Core::loadData(sfConfig::get('sf_data_dir').'/fixtures/mailing/');
        $this->assertTrue($this->_mailing->sendMail());
    Doctrine_Core::getTable('mails')->findAll()->delete();
}
code snippet /unit/symfony/test/phpunit/unit/mailingTest.php

The test string defines which environment configuration is to be loaded. sfConfig::get('sf_data_dir') returns the project's /data path (for example, /home/wrox/public_html/symfony/data). To run the test, you need to execute the following command:

$ symfony phpunit:test-unit

Output is similar to that from the Selenium tests you saw earlier in this chapter.

In CakePHP, you should start with defining fixtures. Fixtures, which are classes that inherit from CakeTestFixture, are placed in the /app/test/fixtures directory. CakePHP presents a slightly different approach to fixtures. Fixtures are not loaded to a database as is done in Symfony or ZF. This is a kind of mock that simulates all behaviors of a model with data that can be accessed with commonly used methods such as find(). But first you should define a fixture as follows:

<?php
class MailTestFixture extends CakeTestFixture {
    var $name = 'Mail';

    var $fields = array(
        'id' => array('type' => 'integer', 'key' => 'primary'),
        'email' => 'text',
        'surname' => 'text',
        'forename' => 'text',
        'subscription' => 'text',
    );
    var $records = array(
        array ('id' => 1, 'email' => 'example@wroxexample.com',
           'surname' => 'Smith', 'forename' => 'John', 'subscription' => 'full')
    );
}
?>
code snippet /unit/cakephp/app/test/fixtures/app/test/fixtures

Because fixtures simulate a model, you don't need to define any connections or get configuration information. The fixture presented in the preceding code should be saved as mail_test_fixture.php. Now you can load this fixture file by assigning its name to the $fixtures variable as follows:

<?php
require_once('libs/Mailing.class.php');
class MailingTest extends CakeTestCase {
    var $fixtures = array( 'mail_test');
    public function setUp() {
      $this->_mailing = new Mailing();
    }
    public function tearDown() {
      unset($this->_mailing);
    }
    /* test cases methods */
}
code snippet /unit/cakephp/app/test/cases/components/mailing.test.php

The test case file should be placed in /app/test/cases/components as mailing.test.php. The Mailing class will be used. Therefore, we need to include it and get an instance of Mailing as shown in the preceding code. The first two test cases are obvious and look like the following:

public function testSubscriptionPositive() {
    $this->assertTrue($this->_mailing->sendMailSubscription(
      "full","example@wroxexample.com"));
  }
  public function testSubscriptionNegative() {
    $this->assertTrue(!$this->_mailing->sendMailSubscription(
      "foobar","example@wroxexample@com"));
  }
code snippet /unit/cakephp/app/test/cases/components/mailing.test.php

To get the Mail object fixtures, you need to get its instance by using ClassRegistry:

public function testRandomMailPositive() {
        $this->Mail =& ClassRegistry::init('Mail');
        $this->assertTrue($this->_mailing->sendMail($this->Mail));
 }
code snippet /unit/cakephp/app/test/cases/components/mailing.test.php

When find() is invoked within sendMail() with the given object as the argument, a list of rows defined in the fixture will be returned as the result. This is an advantage of CakePHP because this approach is very easy to use and you don't need to use the database at all. Of course, you can use the database as well and sometimes that will be necessary.

In the last test case, you need only to delete all data defined in the fixture to get an empty database. Use the delete() method, as shown in the following code, to clear the Mails table:

public function testRandomMailNegative() {
        $this->Mail =& ClassRegistry::init('Mail');
        $this->Mail->delete();
        $this->assertTrue(!$this->_mailing->sendMail($this->Mail));
    }
code snippet /unit/cakephp/app/test/cases/components/mailing.test.php

You can now execute the preceding test using the http://localhost/test.php URL.

The most difficult approach for the QAs is delivered by Zend Framework. However, it is still easy to implement, just as it is in the two frameworks already discussed. To begin, you should include ZF's Application class, and PHPUnit's framework and Mailing classes. The first class is used to get the project's configuration, especially the database connection configuration. The second class is used to make testing with the PHPUnit framework possible. The purpose of the third class is obvious. In setUp(), you need to get an instance of Mailing and additionally get a database connection adapter. The adapter looks like the following:

<?php
require_once 'Zend/Application.php';
require_once 'PHPUnit/Framework.php';
require_once '../library/Mailing.class.php';
class MailingTest extends PHPUnit_Framework_TestCase {
  public function setUp() {
    $application = new Zend_Application(
      APPLICATION_ENV,
      APPLICATION_PATH . '/configs/application.ini'
    );
    $bootstrap = $application->getBootstrap();
    $bootstrap->bootstrap('db');
    $dbAdapter = $bootstrap->getResource('db');
    $this->_mailing = new Mailing();
    }

public function tearDown() {
      unset($this->_mailing);
    }
    /* test cases methods */
}
code snippet /unit/zf/tests/application/unit/MailingTest.php

Unit tests should be placed in the /tests/application/unit directory (for example, /tests/application/unit/MailingTest.php). As the following code shows, the first two test cases don't differ from Symfony and CakePHP test case implementations:

public function testSubscriptionPositive() {
  $this->assertTrue($this->_mailing->sendMailSubscription(
    "full","example@wroxexample.com"));
}
public function testSubscriptionNegative() {
  $this->assertTrue(!$this->_mailing->sendMailSubscription(
    "foobar","example@wroxexample@com"));
}
code snippet /unit/zf/tests/application/unit/MailingTest.php

You can also try to load fixtures manually, as described in Chapter 3, or try to execute this script from the code level, but in cases where the test needs to be automated, a simpler approach is available, as shown in the following code. You need to define an array of data that should be loaded into the database. Next, a model and mapper are used in the same way as they are used in controllers:

public function testRandomMailPositive() {
    $fixtures=array('firstName'=>'John',
             'lastName'=>'John',
             'email'=>'John',
    );
    $entry = new Application_Model_Mails($fixtures);
    $mapper = new Application_Model_MailsMapper();
    $id=$mapper->save($entry);
    $this->assertTrue($this->_mailing->sendMail());
    $mapper->deleteOne($id);
}
code snippet /unit/zf/tests/application/unit/MailingTest.php

At the end, you need to delete the entry saved previously with $id that was returned through the save() method. All methods such as save() or deleteOne() are defined for each model, as described in Chapter 4.

The last test case is the same as in Symfony and CakePHP:

public function testRandomMailNegative(){
        $this->assertTrue(!$this->_mailing->sendMail());
}
code snippet /unit/zf/tests/application/unit/MailingTest.php

To run the unit test, you can use the phpunit.xml configuration file, as shown in the "CMS Tests Automation Using Selenium" section of this chapter, or you can use the following command:

$ phpunit tests/application/unit/MailingTest.php

All tests should pass if your Mailing class methods are done correctly. The output is similar to that obtained with Selenium.