2

I'm working on a rails API project. Here is my code snippets

class PeopleController < ApplicationController
  respond_to :json

  def index
    respond_with Person.all
  end
end

and when I visit the url localhost:3000/people.json

Encoding::UndefinedConversionError at /people.json
"\xE7" from ASCII-8BIT to UTF-8

I'm trying to solve this issue since last week, but still fighting with this. I've found the bunch of similar question over stackoverflow such as this & this but non of the solution worked for me.

Here are the configuration I've.

Rails 4.2.7.1

ruby-2.3.1

Operating system: macOS Sierra

Output of locale

LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=

content on ~/.bash_profile

export LC_CTYPE="utf-8"
export LC_CTYPE=en_US.UTF-8
export LANG=en_US.UTF-8
unset LC_ALL

Output of Encoding.default_external

 #<Encoding:UTF-8>
Community
  • 1
  • 1
Bibek Sharma
  • 3,210
  • 6
  • 42
  • 64
  • Where are you getting your input from? I would usually replace invalid characters (according to UTF-8) before saving it in the DB. If it's already in your DB: Did you save it as `String` or `JSON`? – Severin May 04 '17 at 09:41
  • I'm fetching it from MySQL, let me check the encoding of my database, Actually, I imported database from the old application and trying to use it in API. – Bibek Sharma May 04 '17 at 09:45

1 Answers1

5

I have had this problem a lot of times, so I usually try to get rid of any characters that are invalid to UTF-8 BEFORE saving it in the Database. If you have your record saved as a String you can replace invalid characters like so:

string = "This contains an invalid character \xE7"
string.encode('UTF-8', invalid: :replace, undef: :replace)
#=> "This contains an invalid character �"

This is ofc prior to converting it to a JSON object.

Severin
  • 8,508
  • 14
  • 68
  • 117
  • Is there any way to remove all of this character from the existing database?since I've imported content from another old database, and I do not want to encode every time. – Bibek Sharma May 04 '17 at 09:56
  • You could write a data migration that goes through you data and performs exactly this action on each entry. – Severin May 04 '17 at 09:59
  • I found the problem. The problem was I've uuid field in the people table and it is binary type and has some wrong values. Thank you . You saved my day! – Bibek Sharma May 04 '17 at 10:55
  • Glad I could help :) – Severin May 04 '17 at 11:08