Split Unicode entities by graphemes

Question

"d̪".chars.to_a

gives me

["d"," ̪"]

How do I get Ruby to split it by graphemes?

["d̪"]

Answer 1

Edit: As @michau's answer notes, Ruby 2.5 introduced the grapheme_clusters method, as well as each_grapheme_cluster if you just want to iterate/enumerate without necessarily creating an array.

---

In Ruby 2.0 or above you can use str.scan /\X/

> "d̪".scan /\X/
=> ["d̪"]
> "d̪d̪d̪".scan /\X/
=> ["d̪", "d̪", "d̪"]

# Let's get crazy:


> str = 'Z͑ͫ̓ͪ̂ͫ̽͏̴̙̤̞͉͚̯̞̠͍A̴̵̜̰͔ͫ͗͢L̠ͨͧͩ͘G̴̻͈͍͔̹̑͗̎̅͛́Ǫ̵̹̻̝̳͂̌̌͘!͖̬̰̙̗̿̋ͥͥ̂ͣ̐́́͜͞'


> str.length
=> 75
> str.scan(/\X/).length
=> 6

If you want to match the grapheme boundaries for any reason, you can use (?=\X) in your regex, for instance:

> "d̪".split /(?=\X)/
=> ["d̪"]

ActiveSupport (which is included in Rails) also has a way if you can't use \X for some reason:

ActiveSupport::Multibyte::Unicode.unpack_graphemes("d̪").map { |codes| codes.pack("U*") }

Answer 2

The following code should work in Ruby 2.5:

"d̪".grapheme_clusters # => ["d̪"]

Answer 3

Use Unicode::text_elements from unicode.gem which is documented at http://www.yoshidam.net/unicode.txt.

irb(main):001:0> require 'unicode'
=> true
irb(main):006:0> s = "abčd̪é"
=> "abčd̪é"
irb(main):007:0> s.chars.to_a
=> ["a", "b", "č", "d", "̪", "é"]
irb(main):009:0> Unicode.nfc(s).chars.to_a
=> ["a", "b", "č", "d", "̪", "é"]
irb(main):010:0> Unicode.nfd(s).chars.to_a
=> ["a", "b", "c", "̌", "d", "̪", "e", "́"]
irb(main):017:0> Unicode.text_elements(s)
=> ["a", "b", "č", "d̪", "é"]

Answer 4

Ruby2.0

   str = "d̪"

   char = str[/\p{M}/]

   other = str[/\w/]