Clickatell arabic text getting garbled

Question

I ma trying to send arabic text from the clickatell sms provider using php, but all i have been able to send yet is garbled text. The text I am using for testing is "غثس هفس ".

I have tried encoding the message text to Windows-1256,1250 and ISO-8859-6 formats using iconv but everytimeit just sends garbled texts.

Can anyone please give me some pointers to what I am missing?

Thanks in advance

Does the Clickatell documentation specify any encoding? This may also vary by language/carrier and/or receiving phone. — deceze, Dec 12 '12 at 12:12
I didnt see any specific encoding details in the clickatell docs. Right now i have run out of ideas on what to try next :) — pinaki, Dec 12 '12 at 12:20

score 3 · Accepted Answer · answered Dec 12 '12 at 15:18

Ok, this is for whoever comes on this path later. Found the solution here Retaining code from Esailija's answer

<?php
$text = "غثس هفس خن";
$arr = unpack('H*hex', iconv('UTF-8', 'UCS-2BE', $text));
$message = strtoupper($arr['hex']);

$username = '';
$password = '';
$API_ID = '';
$to = '';

$url = "http://api.clickatell.com/http/auth?user=$username&password=$password&api_id=$API_ID";
$ret = file($url);
$sess = explode(":",$ret[0]);
if ($sess[0] == "OK") {
    $sess_id = trim($sess[1]);
    $url = "http://api.clickatell.com/http/sendmsg?session_id=$sess_id&to=$to&text=$message&unicode=1";
    $ret = file($url);
    $send = explode(":",$ret[0]);

    if ($send[0] == "ID") {
        echo "success - message ID: ". $send[1];
    } else {
        echo "send message failed";
    }
} else {
    echo "Authentication failure: ". $ret[0];
}

Pretty nice find that it needs to be ad hoc hex encoded, instead of standard percent encoding.. — Esailija, Dec 12 '12 at 16:50

score 1 · Answer 2 · answered Dec 12 '12 at 12:34

Based on this, the default is GSM but you can also choose UCS2.

So:

<?php
                                 //Make sure the PHP source file is physically
                                //saved in utf-8
$text = rawurlencode(iconv( "UTF-8", "UCS-2", "غثس هفس "));

$url = "http://api.clickatell.com/http/auth?user=username&password=password&api_id=API_ID&encoding=UCS2";
$ret = file($url);
$sess = explode(":",$ret[0]);
if ($sess[0] == "OK") {

    $sess_id = trim($sess[1]);
    $url = "http://api.clickatell.com/http/sendmsg?session_id=$sess_id&to=$to&text=$text&encoding=UCS2";
    $ret = file($url);
    $send = explode(":",$ret[0]);

    if ($send[0] == "ID") {
        echo "successnmessage ID: ". $send[1];
    } else {
        echo "send message failed";
    }
} else {
    echo "Authentication failure: ". $ret[0];
}

thanks for the reply. i tried the same code (copy/pasted), but it still sent garbage text as sms. Any pointers on how I would send in GSM mode? — pinaki, Dec 12 '12 at 14:47
Found the solution. Please check my answer below. Thanks again for the help. — pinaki, Dec 12 '12 at 15:19

score 1 · Answer 3 · answered Feb 12 '14 at 09:28

I rewrote ClickATell code for Vtiger 6 so that it can accommodate UTF-8 Extended characters. I use Turkish. I use the below code for conversion. I hope you can port.

/**
 * Function to handle UTF-8 Check and conversion
 * @author Nuri Unver
 * 
 */
public function smstxtcode($data){ 
$mb_hex = '';
$utf = 0;
for($i = 0 ; $i<mb_strlen($data,'UTF-8') ; $i++){ 
    $c = mb_substr($data,$i,1,'UTF-8');

    $o = unpack('N',mb_convert_encoding($c,'UCS-4BE','UTF-8')); 
    $hx = sprintf('%04X',$o[1]); 
    $utf += intval(substr($hx,0,2));
    $mb_hex .= $hx;
} 
if ($utf>0)
{
    $return=$mb_hex;
    $utf=1;
}
else
{
    $return=utf8_decode($data);
    $utf=0;
}
return array($utf,$return); 

}

The you call this function with your message. The response you get it is an array saying wheter to send unicode or normal text depending on the message and also the text to be sent. If there are no extended characters, it just sends it as plain text with unicode=0 in order to save characters. If message contains extended characters, it converts the message to hexcode and sends it as unicode.

This code just does the calculations. You need to implement your own code to port it to your system. For demonstration this is the code I use for Vtiger to extract data and send the message:

/**
 * Function to handle SMS Send operation
 * @param <String> $message
 * @param <Mixed> $toNumbers One or Array of numbers
 */
public function send($message, $toNumbers) {
    if(!is_array($toNumbers)) {
        $toNumbers = array($toNumbers);
    }
    $params = $this->prepareParameters();
    $smsarray = $this->smstxtcode($message);
    $params['text'] = $smsarray[1];
    $params['unicode'] = $smsarray[0];
    $params['to'] = implode(',', $toNumbers);
    $serviceURL = $this->getServiceURL(self::SERVICE_SEND);

score 1 · Answer 4 · answered Sep 16 '15 at 12:09

Short answer: you need to do two things:

a) Convert your Arabic message:

$data = iconv("UTF-8", "UCS-2BE", $data); $data = bin2hex($data);

Use the resulting string as your message text

b) When sending via the HTTP API, include the unicode parameter:

&unicode=1

Clickatell arabic text getting garbled

4 Answers4