TDD: How to test a search?

Question

My website will have an advanced search. Pleople can go there and search about an entitiy (cars, for example). I've created some tests that check the number of results based on the search parameters. I think about what tests should I write, then i write it, then I add data to the test database. But here comes the issue. When I insert new values to the database my old tests break. That's because I'm checking the number of records...

<?php defined('SYSPATH') or die('No direct access allowed!');

class Search_Test extends PHPUnit_Extensions_Database_TestCase
{
    /**
     * @return PHPUnit_Extensions_Database_DB_IDatabaseConnection
     */
    public function getConnection()
    {
        $pdo = new PDO('mysql:dbname=db_test;host=127.0.0.1', 'root', null);
        return $this->createDefaultDBConnection($pdo, 'db_test');
    }

    /**
     * @return PHPUnit_Extensions_Database_DataSet_IDataSet
     */
    public function getDataSet()
    {
        $fixture = realpath(dirname(__FILE__).'/../data/fixture.xml');
        return $this->createXMLDataSet($fixture);
    }

    public function numberOfResultsDataProvider()
    {
        return array(
            array(1, null, null, 1),
            array(2, null, null, 3),
            array(3, null, null, 0),
            array('abc', null, null, 5),
            array(null, 1996, 2003, 3),
            array(null, 1996, 1999, 2),
            array(null, 2002, 2003, 1),
            array(null, 1500, 1800, 0),
            array(null, 2003, 2003, 1),
            array(null, null, 2005, 4),
            array(null, 1996, null, 4),
            array(null, null, null, 4),
            array(null, 2003, 1996, 0),
            array(null, 'abc', 2003, 4),
            array(null, '1996', '1999', 2),
            array(2, 2003, 2005, 2),
            array(null, null, null, 4),
        );
    }

    /**
     * @dataProvider numberOfResultsDataProvider
     */
    public function testNumberOfResults($brandId, $startYear, 
        $endYear, $numberOfResults
    ) {
        $search = ORM::factory('search');
        $search->setBrand($brandId)
            ->setYearRange($startYear, $endYear);
        $results = $search->results();
        $this->assertEquals($results->count(), $numberOfResults);
    }
}
?>

Is it normal? Should my old tests break when I create new tests?

Should my tests be bounded to data?

My search has too many parameters and they will be used in the same form (view). Should I create tests searching for each parameter or should I test they together? Should I split it up in more test classes?

Thanks.

Good question, but perhaps it would be better suited to http://programmers.stackexchange.com. — Maxpm, Sep 13 '11 at 23:15

score 6 · Answer 1 · answered Sep 13 '11 at 23:19

when doing a unit test that involves a database, the test should provide the actual database that it will test against. It might just be a simple database that contains only values that are relevant to the test, like one row that would match and one row that doesn't. It's also reasonable to build a bunch of tests against a snapshot of the live database as it existed at a particular point in time.

One particularly convenient way of achieving this is to us a Sqlite3 Database specifically for the test, and set your application to use that for the test.

score 1 · Answer 2 · answered Sep 14 '11 at 00:21

If at all possible, you should sever the link between how your code is fetching your data and the database itself. Your unit tests should not depend on the database, and you've stumbled upon a few reasons why: You have to maintain your data across tests, creating new tests causes others to break, and so on. If you could somehow fake the data access point and return data strictly in-memory, that would be the ideal case. I don't know how easy it would be in PHP, but building an interface around your data access code and using a fake/mocked implementation of that interface could go a long way in helping to achieve this goal.

score 1 · Answer 3 · answered Sep 14 '11 at 16:14

In short:
No way of doing this on a live database

You need a separate database for tests. This may be a mock of database actually.

Then, before the tests, you need to prepare the database to be in the fixed state you expect it to be (e.g. load data from fixtures or import sql file).

Then, the best option is start database transaction when you plan to do any modification to the original database state.

Once the transaction is started, you may work on the database.

After you complete the tests, you just rollback the transaction, and have database in the clean state.

Sqlite (already mentioned) will do fine in most cases, but it may differ from the live database. Anyway, having different database adapters will be very helpful.

score 0 · Answer 4 · answered Sep 13 '11 at 23:20

0

Your tests should not break when you create new tests, and your tests should not be data-dependent. A test that requires particular data in the database should place that data into the database. And if it requires that no other data be in the database, it should clear everything else out.

This, of course, will make your tests slow - so mock out the data access layer.

answered Sep 13 '11 at 23:20

Carl Manaster

39,912
17
102
155

It's an advanced search and users can set many different parameters. I guess it's impossible to test every path but maybe with few tests I would have to set lots of data fixtures... Is it normal? Am I doing something wrong? Thanks. – thom Sep 14 '11 at 12:25

score 0 · Answer 5 · answered Sep 14 '11 at 02:54

I would have each test method create its own data, then persearch, assert, and then destroy its own sample data.

I'll typically have a shared addData() method, and then a database cleanup method. Hopefully you'll be able to make your added data identifiable in some way so that your cleanup method can be generic. In your example, maybe you could have all of your brandId's start with "test-", and then your query would search out and delete all records where the brandId like 'test-%'.

TDD: How to test a search?

5 Answers5