If you lock your gemfile to sqlite3-ruby you will have problems starting your application on recent Rails versions since Rails wants to have the gem named "sqlite3". The real solution to this problem is to ensure that the .bundle file that the gem creates indeed links to the right sqlite3 libraries.
By default, sqlite3 gem will link to a nonexisting library in your /usr/lib. This is NOT right. What you need to do is set the compliation options straight for your system and reinstall the sqlite3 gem, and you can easily do this using
$bundle config
Their manpage specifies what it does, but this is the command that I needed to do (I have a roll-your-own SQLite install from their site, not via brew or macports).
$bundle config build.sqlite3 --with-sqlite3-include=/usr/local/include --with-sqlite3-lib=/usr/local/lib
This will always feed the right options to the gem when you try to "bundle install" it and your gem will build properly.
Note that this problem is kinda sneaky in that it will not occur on Ruby 1.9 - apparently rbconfig there has been updated and does better library lookup. But when you are going for 1.8 compatibility - use this bundle configuration and you will be all set.