2

The setup:

My Rail 3.2.12 app uses Rspec & FactoryGirl for testing most of its various model methods.

I use Postgres for production, development, and test.

Problem:

One model, ZipCodes, has about 40,000 rows including longitude/latitude data, and a LOTS of methods for selecting records that meet certain criteria.

I need to test those methods against the real-world data. Because of the size, re-loading the table each time we run tests is far too slow.

Question:

How can I load that static ZipCodes table, via terminal or console or rake task, once, then leave it alone unless/until the data changes (every few months we might add a few zipcodes). (Also, not erasing it with DatabaseCleaner after each test run, although I got that answered in another question, that I can use the :except => [tablename].)

I have the data in CSV and YAML formats now, but can move it to any other format if necessary.

I also have the data loaded into my development database, if there's some way to copy it from the dev to test databases.

(note: we do not use the primary key for ANY associations, we do all lookups by other fields like zipcode or longitude, so it doesn't matter if the method of loading the data into test brings over primary keys from my development database)

jpw
  • 18,697
  • 25
  • 111
  • 187

1 Answers1

3

I suggest you use database_cleaner gem for your test suite https://github.com/bmabey/database_cleaner

Then you can manually populate your ZipCodes table, so you have the data ready

And the trick is to configure database_cleaner to leave ZipCodes tables untouched. To do so you need to set in your spec_helper.rb DatabaseCleaner.strategy = :truncation, {:except => %w[zip_codes]} (given your strategy is truncation, and your table name is zip_codes)

See database_cleaner doc for more info (link above in my answer)

EDIT

To answer your second question about populating DB, you have multiple choices.

You can apply those techniques outside of your test environment (ex: running seed from the command line while specifying RAILS_ENV=test). Doing so you will populate your test DV, and as you specify database_cleaner to NOT clear this table, it will be untouched.

Note that every time you run a migration you will have to run rake db:test:prepare to update the test database structure, therefore you will have to run the import of data in this table again. This is still convenient because doing so save plenty of time between your test scenario

Community
  • 1
  • 1
Benjamin Bouchet
  • 12,971
  • 2
  • 41
  • 73
  • Thank you, although that was the part I did know how to do (I use database_cleaner now). What I'm not aware of how to do it load the data into the test database and leave it there, from either the dev database, or yaml, or csv. – jpw Jul 19 '13 at 00:51
  • Well, it's basic SQL. As I stated in my answer you can manually fill up your DB (import data through you postgres manager app) // Or if you really need to stay in Rails, you can use seed.rb and run `rake db:seed RAILS_ENV=test` – Benjamin Bouchet Jul 19 '13 at 01:19
  • While it might be basic SQL, it also might be something do with seed.db, or fixtures, or some sort of db dump/load, or a use-once model spec to import via csv. And having no experience with loading static data, nor any idea which approach might work best, hence my question. (For example, I can get fixtures to work, they are terribly slow, so maybe there is a way to load via fixtures once.) – jpw Jul 19 '13 at 06:28