ActiveRecord Query Failing For Name Containing Accented Characters

Question

I have a Rails app using Postgres. I have a Document model that has a name attribute. Some names contain accented characters. A couple of example names:

Condições Para Aplicação Da Lei

Considerações Introdutórias

I am querying for models with a specific name using:

document = Document.where(name: "Example Document Name").first

So long as the name doesn't contain special characters, this works fine, however as soon as I use a name containing any accented characters, the query returns nil.

$ Document.all
$ #<Document id: 1, name: "Foo" ... >
$ #<Document id: 1, name: "Considerações Introdutórias" ... >
$ Document.where(name: "Foo").first
$ #<Document id: 1, name: "Foo" ... > 
$ Document.where(name: "Considerações Introdutórias").first
$ # nil

Why is this query failing when the name contains special characters?

In my config/application.rb:

config.encoding = "utf-8"

In my `config/database.yml':

encoding: utf8

Answer 1

Check your utf configuration. config.encoding = "utf-8"

check in database.yml for encoding: utf8.

See these questions

Add "# coding: utf-8" to all files

Set global default encoding for ruby 1.9

Answer 2

Turns out the problem wasn't with Rails, but with OSX. There are different ways of representing the same characters; either as a single symbol, or decomposed into multiple components. Both versions are displayed identically in the Terminal, but copy the names into a plain text file and you will see the difference:

Composed:

Considerações Introdutórias

Decomposed (I can only get the decomposed version to display in here by adding spaces around the decomposed characters):

Condic ̧ o ̃ es Para Aplicac ̧ a ̃ o Da Lei

In order to fix the mismatch, I used Unicode Utils to compose the filenames before adding them to the database and again before using them to search:

require "unicode_utils/nfc"

...

filename = UnicodeUtils.nfc(filename)