47

I'm working on upgrading one of our Rails 2.3.8 apps to Rails 3, and have run into an annoying problem with bundler and deployment. I develop the application on a Windows machine, but the production environment is running Ubuntu Linux. Now, my problem is that bundler is ignoring the mysql gem in the production environment, and Passenger spits out: "!!! Missing the mysql gem. Add it to your Gemfile: gem 'mysql', '2.8.1'"

Here is my Gemfile:

# Edit this Gemfile to bundle your application's dependencies.
# This preamble is the current preamble for Rails 3 apps; edit as needed.
source 'http://rubygems.org'

gem 'rails', '3.0.0'
gem 'net-ldap', :require => 'net/ldap'
gem 'highline', :require => 'highline/import'
gem 'mysql', '2.8.1'
gem 'net-ssh', :require => 'net/ssh'

# Bundle gems for the local environment. Make sure to
# put test-only gems in this group so their generators
# and rake tasks are available in development mode:
group :development, :test do
  gem 'fakeweb', :require => 'fakeweb'
  gem 'flexmock', :require => 'flexmock/test_unit'
end

As you can see, the mysql gem is specified. However, when deploying, bundler ignores it. Why? The reason is that Bundler generates the following Gemfile.lock (only relevant parts included):

....
mime-types (1.16)
mysql (2.8.1-x86-mingw32)
net-ldap (0.1.1)
....

Notice that it includes the platform specific gem. This is obviously NOT what I want it to do, as that gem is not suitable (and appearently ignored) when running under Linux.

So, does Bundler come with any way to deal with these issues? Or do I have to remember to manually change the mysql gem version in the generated Gemfile.lock every time I run bundle install on my development machine?

Thank you in advance!

Update

It seems like the bundler team is aware of this issue.

Daniel Abrahamsson
  • 1,945
  • 2
  • 15
  • 20
  • I have a similar issue, I need to use rmagick 2.12.0 on windows, but the current version on any other ruby platform. Bundler errors out if I try to lost rmagick twice. – Jeff Paquette Nov 01 '10 at 13:05
  • 2
    Please try add the mysql (2.8.1) to Gemfile.lock, manually. In our Gemfile.lock, we have `sqlite3-ruby (1.3.1)` _and_ `sqlite3-ruby (1.3.1-x86-mingw32)` – oma Nov 05 '10 at 15:58
  • Morten, this is in fact what I do today. Unfortunately, Bundler removes the "original" mysql row (and only keeps the Windows version) when bundle install is run. – Daniel Abrahamsson Nov 06 '10 at 07:08

11 Answers11

37

This is a known issue in Bundler. The workarounds are either:

  • Generate a Gemfile.lock on a system similar enough to your production environment that you get results that match your production platform. Effectively, that means you can only generate the Gemfile.lock file on Windows if your production system is Windows.
  • Don't commit a Gemfile.lock file at all, and determine dependencies on the production machine at deploy time (bundle install without --deploy). While not recommended in general, this is an often-used workaround until the bug is fixed. For example, this is the recommended solution offered by Heroku.
  • Switch to JRuby, which would have the same platform string across both Windows and Linux (java). I don't recommend this seriously, but I believe it would fix the problem.
  • Fix the problem in the Bundler source code, i.e., help the Bundler team fix the bug. :)
awendt
  • 13,195
  • 5
  • 48
  • 66
wuputah
  • 11,285
  • 1
  • 43
  • 60
  • 1
    This is the best answer yet, offering several workarounds. Therefore I accept it. For my own part, I manually change Gemfile.lock to what I want before commiting it. By doing so you have to avoid checking in Gemfile.lock by accident, though. – Daniel Abrahamsson Nov 08 '10 at 13:07
  • Give `rvm` a try too, it's worth it. – Fábio Batista Nov 09 '10 at 12:40
  • I don't see it anywhere, so I'll write my solution for a similar issue I had with ruby-oci8 (also a pain in the neck): I installed the gem locally (downloaded the mswin32 gem version, because I'm using cygwin on my 64 windows 7), and then I declared it's version on the Gemfile under a platform :mswin, mingw block. It's bundling and running. – ChuckE Jul 18 '13 at 08:02
  • Perhaps `bundle lock --add-platform` can do the trick now? – IronBlood Apr 26 '23 at 16:16
8

I have a similiar problem. I would like to be able to write something like this in my Gemfile:

platforms :ruby do                      # linux
  gem 'nokogiri', "1.5.0.beta.2" 
end

platforms :mswin do
  gem 'nokogiri', "1.4.4.1" 
end

But, bundler tells me I am not allowed. So, my workaround, that works in this specific case is to point out a range of versions:

gem 'nokogiri', ">= 1.4.4.1", "<=1.5.0.beta.2"

Which - at the moment - give the 1.4.4.1 version on my Windows computer and 1.5.0.beta.2 on my linux computer. Maybe it is possible for you to write a similiar ugly workaround ;-)

Stefan Pettersson
  • 453
  • 5
  • 8
  • Thanks for the tip! I have migrated my development environment to Linux, so I no longer suffer from the problem. However, this is still an issue for developers on Windows, so your tip is welcome. – Daniel Abrahamsson Dec 08 '10 at 06:55
5

Our engineers at Engine Yard have submitted a patch to Bundler to address this issue and unfreeze the gems if on a different platform. We've been having the same problem with many Windows trying to deploy after running through the RailsInstaller Demo Tutorial. The best fix we've found is to perform the following:

  1. bundle install like normal on your development machine
  2. Go through the Gemfile.lock and if there are any lines with -x86-mingw32, remove that part.
    • bcrypt-ruby (3.0.1-x86-mingw32) becomes bcrypt-ruby (3.0.1)
  3. Add ruby under the 'Platforms' section in the Gemfile.lock
  4. Make sure to explicitly specify the needed gem in the Gemfile with the platform flag. (Not sure if this is needed but it doesn't hurt)
    • In Gemfile: `gem 'bcrypt-ruby', '~> 3.0', :platform => 'ruby'
  5. bundle install again which will keep the bcrypt-ruby (3.0.1) line and add in bcrypt-ruby (3.0.1-x86-mingw32) again.

If you're curious about the Bundler patch, you can get notifications at https://github.com/carlhuda/bundler/pull/1451

Hope this helps anyone still looking for answers.

Evan Machnic
  • 637
  • 1
  • 6
  • 8
2

I think the issue is that the mysql gem doesn't properly discover the needed headers. You can fix this by moving over to using the mysql2 gem, you'll just have to update your database adapters in database.yml for ActiveRecord integration.

Additionally, you can pass build flags to C extending gems if absolutely necessary:

bundle config build.mysql --with-mysql-config=/usr/local/mysql/bin/mysql_config.

Daniel Mendel
  • 9,862
  • 1
  • 24
  • 37
2

I've ran into this issue before, and using the mysql2 gem does indeed fix the problem. I know that's not the answer you're looking for, but combine that with Diego's answer and you're golden.

Community
  • 1
  • 1
bouchard
  • 820
  • 10
  • 27
  • As you already mentioned in your answer, changing to mysql2 won't solve the problem, but a symptom. There are other gems, such as nokogiri, exposing the same issues. – Daniel Abrahamsson Nov 05 '10 at 12:11
2

Have you tried using rvm (link here)? It can install isolated Ruby Virtual Machines and Gemsets, so you can work with an environment more like the one you have in production. I honestly don't know if it will solve your problems, but it's worth a shot.

Anyway, I know this is not the answer you want to hear, but IMHO Windows is not the best platform use when developing in Rails. I recently bought a MacBook mainly to develop Rails applications, it saves you from a lot of headaches. You can also install Linux on your development machine and use it, it's way better than using Windows ports or Cygwin.

Fábio Batista
  • 25,002
  • 3
  • 56
  • 68
1

Don't commit Gemfile.lock and your gems to production. You have to run bundler install again in production.

Joshua Partogi
  • 16,167
  • 14
  • 53
  • 75
  • 6
    Well, we ARE recommended to put our ``Gemgile.lock`` into version control. And bundler requires this file when using the --deploy flag. – Daniel Abrahamsson Sep 06 '10 at 13:40
  • Committing `Gemfile.lock` will lock you in to the gems listed in `Gemfile.lock`. `Gemfile` itself is enough to define the gems that are needed for your apps. – Joshua Partogi Sep 07 '10 at 00:14
  • 9
    A Gemfile with no Gemfile.lock is only good enough until you have a conflict down your tree of dependent gems. If you're using more than a handful of gems, then you're likely to have conflicts sooner or later. Also, if you don't check in your Gemfile.lock, you cannot be sure that you're using the exact same gems on every machine (unless you only declare strict dependencies in your Gemfile, which makes it a pain to maintain). Basically, by not checking in Gemfile.lock, you're dumping the two main reasons to use bundler instead of, say, a batch file that installs a bunch of gems. – Paolo Perrotta Oct 01 '10 at 19:47
  • 1
    @Paolo Perrotta: While you're generally correct on your second point, I think you're overstating the problem. Bundler will still do dependency resolution for you, which should avoid conflicts in most cases. If you do have a conflict, you will also see it locally when changing the Gemfile and then doing `bundle install`. Giving this a +1 since it's a common solution to this problem. – wuputah Nov 06 '10 at 16:28
1

You can do something like this:

platforms :ruby do
  gem "sqlite3-ruby", :require => "sqlite3", :group => [:development, :test]
end

platforms :jruby do
  gem 'activerecord-jdbc-adapter', :require => false
  gem "jdbc-sqlite3", :require => false
end

Btw, you should put your Gemfile.lock into the version control because this way all machines will run the application with the same gems versions.

Diego Carrion
  • 513
  • 5
  • 8
  • 3
    I can't. The platform directive won't help, as I will have to specify the same gems (mysql and nokogiri) for both plaftorms. In Gemfile.lock, the gems will be listed under DEPENDENCIES, and will not be required, nor installed, by bundler. – Daniel Abrahamsson Nov 05 '10 at 12:17
  • see [the Gemfile manpage](http://gembundler.com/man/gemfile.5.html#PLATFORMS-platforms-). you can declare platform specific gem like this: `gem 'jdbc-sqlite3', :platforms => :jruby` – nil Mar 31 '11 at 01:42
0

I came across this issue and then ended up writing script for this painful task. http://gouravtiwari.blogspot.com/2011/03/development-on-windows-deploying-to.html

gouravtiwari21
  • 369
  • 3
  • 10
0

I deploy to Linux for our developers staging version, and to Windows for production and our users staging version. Thus I need a .lock file for both of them (production and staging only use the .lock file, and I want to be sure that I am using the gem versions used by the developers which are kept in the .lock file).

So I have three versions of my Gemfile.lock committed to my repository. There is the normal Gemfile.lock. I also create lock files for each different platforms (_win and _linux), which are a copy of the Gemfile.lock created on that platform.

It takes an extra round of commits to get both of the .lock files into the repo. Thus if I have to do a bundle install on my Mac after a windows pull request, I copy the pre-existing .lock file as the windows version, and copy my new .lock file as the linux version. I then do a pull request to update the lock files.

When Deploying, I overwrite the Gemfile.lock with the correct version.

Taylored Web Sites
  • 1,017
  • 1
  • 9
  • 15
0

The best solution I've found for this, where you have different platforms for your local development environment and for production is to always use the ruby platform gems on the Gemfile.lock.

You can accomplish this by setting the Bundler config value with

bundle config set --global force_ruby_platform 'true'

That way the bundle will always default to install the ruby platform gems. But this implies that all gems that need native extensions on a specific platform will have to be compiled, so you need to get sure you have the required tools and libraries installed on both your local and production machines.

Santiago Bosio
  • 1