I am trying to parse a message that possibly contains emojis in it. An example message that could be received looks like:
{"type":"chat","msg":"UserName:\u00a0\ud83d\ude0b \n"}
What should match is \u00a0 as a single character, and \ud83d\ude0b as a pair.
I have regex that can pull individual codes, but not pairs to match the full emoji:
\\u[a-z0-9]{4}
Is there a clean way to account for any/multiple emojis in a sentence so I can replace the surrogate pair with the function I have? Thanks!
Edit: Here is the function I will be using alongside the regex
string ConvertToUnicode(string SurrogatePair)
{
string returnValue = "";
for (var i = 0; i < SurrogatePair.Length; i += char.IsSurrogatePair(SurrogatePair, i) ? 2 : 1)
{
var codepoint = char.ConvertToUtf32(SurrogatePair, i);
returnValue = String.Format("U+{0:X4}", codepoint);
}
return returnValue;
}
I have a dictionary where the key is unicode and the value is a reference to the emoji image so I can display it in a Unity UI element.
Edit 2:
As this question is a bit too broad, I will get specific as to what I am trying to accomplish. I am receiving a json message from a server through a websocket connection. This message is being displayed in a Unity Panel where each message is a text mesh pro text object. When an emoji is sent, the message displays similarly to the example message above, with the only change being that the surrogate pair changes based on the emoji sent. In order to insert the corresponding emoji image correctly into the text mesh pro object, I need to get the sprite atlas id that points to the correct emoji. As I did not insert the sprites by hand into the atlas but read them in from a spliced sprite sheet, the only way of accessing each image is by their indexed id. To correctly identify the emoji in the atlas by id, I created a dictionary that inserted the Unicode in order as they appeared in the sprite sheet as the key, where the value is the index in the atlas. What I am trying to do now is parse the message received for an emoji using regex, send this parsed data into the function I posted above to convert it to the Unicode value, then retrieve the correct id from the dictionary to finally insert the emoji that was originally entered on the front-end. If there is a better way of going about this please let me know, but from the research I did, the only efficient way to insert images into a Unity text object is how I am going about it. Due to this, I need to get the surrogate pairs from the message.
Edit 3:
If someone else happens to stumble upon this question i'll just leave my solution that I came up with for getting emojis from an html website, through a websocket server into Unity textmeshpro.
Here is a google spreadsheet containing the hex values and the sprite sheet I used to create the dictionary / sprite atlas in Unity: https://docs.google.com/spreadsheets/d/1XQY1n9cA1hx_PnsXoisxjanZRiQG0gd25VYEmk1W7mE/edit?usp=sharing.
I then used a library that can be found here: https://github.com/aaronpk/emoji-detector-php
It can parse a string and find emojis. I replaced the regex used with the regex provided by sln below, then tweaked the main script to just return the message text back but with the emojis replaced with the hex wrapped in a delimeter I could find with regex on the Unity side.
<?php
namespace Emoji;
define('LONGEST_EMOJI', 8);
function detect_emoji($string) {
// Find all the emoji in the input string
$prevencoding = mb_internal_encoding();
mb_internal_encoding('UTF-8');
$data = array();
$test = $string;
static $map;
if(!isset($map))
$map = _load_map();
static $regexp;
if(!isset($regexp))
$regexp = _load_regexp();
if(preg_match_all($regexp, $string, $matches)) {
foreach($matches[0] as $ch) {
$points = array();
for($i=0; $i<mb_strlen($ch); $i++) {
$points[] = strtoupper(dechex(uniord(mb_substr($ch, $i, 1))));
}
$hexstr = implode('-', $points);
$theMatch = $string.exec($ch);
$test = substr_replace($test, "[[[[".$hexstr."]]]]", strpos($test, $ch), strlen($ch));
}
}
if($prevencoding)
mb_internal_encoding($prevencoding);
return $test;
}
function _load_map() {
return json_decode(file_get_contents(dirname(__FILE__).'/map.json'), true);
}
function _load_regexp() {
return '/(?:' . json_decode(file_get_contents(dirname(__FILE__).'/regexp.json')) . ')/u';
}
function uniord($c) {
$ord0 = ord($c[0]); if ($ord0>=0 && $ord0<=127) return $ord0;
$ord1 = ord($c[1]); if ($ord0>=192 && $ord0<=223) return ($ord0-192)*64 + ($ord1-128);
$ord2 = ord($c[2]); if ($ord0>=224 && $ord0<=239) return ($ord0-224)*4096 + ($ord1-128)*64 + ($ord2-128);
$ord3 = ord($c[3]); if ($ord0>=240 && $ord0<=247) return ($ord0-240)*262144 + ($ord1-128)*4096 + ($ord2-128)*64 + ($ord3-128);
return false;
}
To call this in my server script, I just had to add:
include("src/Emoji.php");
at the top of my script, and call the function as follows:
$message = Emoji\detect_emoji($message);
I made sure this was only sent to my Unity client as I was storing their resourceId in the server code.
On the Unity side, to find the emojis that need to be replaced, I used:
string emojiPattern = @"(?<=\[\[\[\[).[^\]\]\]\]]*";
MatchCollection emojiMatch = Regex.Matches(messageString, emojiPattern);
for(int x = 0; x < emojiMatch.Count; x++)
{
messageString = messageString.Replace("[[[[" + emojiMatch[x].Value + "]]]]", "<sprite=" + emojiDictionary[emojiMatch[x].Value.ToLower()].ToString() + ">");
}
As the text element, I was instantiating was a text mesh pro text GUI element, it is able to convert the to an image in the sprite atlas I made using the sprite sheet. Hope this helps someone in the future!
