1

My Rails application is connected to the same MS SQL database in production and development.
In development (on my Mac) I am able to render the /clients/active.json view (via rabl).
However in production (behind Passenger/Apache) I receive this error:

Started GET "/clients/active.json"
Processing by ClientsController#active as JSON
  Rendered clients/active.json.rabl (204.6ms)
Completed 500 Internal Server Error in 206.3ms

ActionView::Template::Error (source sequence is illegal/malformed utf-8):
    1: collection @clients
    2: extends 'clients/index'
  app/views/clients/active.json.rabl:1:in `_app_views_clients_active_json_rabl__626601527_78036080'

The database is read only and out of my control. Is there a way to render the JSON in production like it works on my development machine?

Edit
I switched to Nginx and still encounter the same error.

tbeseda
  • 1,838
  • 3
  • 16
  • 25
  • 1
    Sorry that didn't help your case... I'm at a loss... deleted my suggestion. – omarvelous Feb 19 '14 at 05:44
  • 1
    Are you using accents like é or à ë ê û î ï ü ? Something I learnt from beeing french and commenting my code in french is that Ruby does not like accents at all in the .rb files. I had to add the comment `# encoding: UTF-8` at the top of all my .rb files containing accent. – MrYoshiji Feb 21 '14 at 15:45
  • Can u please tell me which front end language are you using – SemperFi Feb 22 '14 at 17:27
  • Please add the source of `app/views/clients/active.json.rabl`. I guess there is some utf-8 char and you're missing the magic encoding line (as MrYoshiji suggests) – Willian Feb 22 '14 at 18:49
  • The actual view files are valid UTF-8 (the one in question is only 100 characters or so). An attribute from the database contains the invalid character. It even cannot be rendered after using `force_encode`. Very strange. – tbeseda Feb 24 '14 at 22:59

3 Answers3

3

You have ruled out:

  • Passenger (by trying Nginx)
  • Database (because you're using same one)
  • Rabl/Rails (probably, assuming same code is deployed to production)

This error appears to be coming from the ruby JSON generator, I see:

and thus likely ruled in

  • difference in operating system, or configuration
  • or more likely versions of software

The first most important question is: what version of ruby on Mac vs server? There were significant changes to how character-set encodings were handled between ruby 1.8 and 1.9.

But if server is on 1.8 and Mac is on 1.9+, then this is very likely why there's a difference. Update to 1.9 or 2.0 and I'll bet systems will behave the same.

If both are at 1.9.2 or higher, then I would look at not just gem versions (which should be the same if you are deploying your Gemfile.lock) but which libraries were used to build the gems. See this similar bug report suggesting compiler flags. Actually same is likely cause if both are at ruby 1.8.

With 1.9.2 you can use force_encoding which is described in painful detail here by our hero, Yehuda Katz.

If this doesn't work, you could resort to the solution/workaround found for another LogStash post -- trap the exception. This probably needs to be done in the controller unless you want to monkey-path Rabl (which I dont recommend :-). More details on where it is likely this error is coming from, JSON::GeneratorError here.

My money is on the ruby version.

Good luck!!!

Community
  • 1
  • 1
Tom Harrison
  • 13,533
  • 3
  • 49
  • 77
  • Thanks for the in depth answer. Both my dev and production machines are running the latest version of Ruby 2. I've narrowed down the attribute of the model causing the issue, and have even tried using force_encode on it. Still throws the same error, but works when I exclude the attribute altogether. Strange! I'll try trapping the exception next. – tbeseda Feb 24 '14 at 22:54
  • Hmmm. Ok, so we know it's something about the value of the attribute (column) likely of a specific row or rows, but that's probably not new info. I had misread and now see that it's MS SQL not MySQL which makes me think about the OS-specific driver used by rails may handle things differently on osx and Linux. They are different gems built in different environments, so this seems a prime candidate. Maybe delete mssql gem on Linux and rebuild? Linux libs all up to date? Compiler switches needed -- you can pass to bundler if needed. All just ideas for googling. – Tom Harrison Feb 26 '14 at 04:57
0

Check your system environment's language.

It most likely should be LANG=en_US.UTF-8

You an check that by issuing "env" in your system

env |grep LANG

if it isn't set, set it.

export LANG=en_US.UTF-8

Alternatively, you can issue this command "locale"

Here's my output

LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

One more item to consider is the environment settings for your application.

I would ensure there are no settings in config/production.rb that are overriding system defaults.

Perhaps you can either copy config/development.rb to config/production.rb and restart the servers to do this. Otherwise, eye ball it and make sure there's nothing suspicious in production.rb

If none of that works, it could be a bug? https://code.google.com/p/phusion-passenger/issues/detail?id=872

lsaffie
  • 1,764
  • 1
  • 17
  • 22
  • Thanks for the reply, my output of `locale` is the same and LAND is set correctly. I'll try running the app with Webrick or similar in dev mode to see if I get the same issue. – tbeseda Feb 24 '14 at 22:52
0

I found this question while googling a similiar problem with "malformed utf-8" and RABL

The solution was to replace the default JSON Generator (which is recommended in the github wiki of rabl).

A simple

#Gemfile
gem 'oj'

fixed the issue for me.

hobofan
  • 1
  • 1
  • I tried this on my production server. It seems to do what I was doing manually: forcing the encoding on wonky strings and replacing bad characters with �.This definitely helps as I don't have to rewrite methods for specific attributes. Thanks for the help. Still an odd issue. – tbeseda Mar 17 '14 at 20:36