RTL (Arabic) misplace plus sing to the end of phone number in python

Question

I have a Arabic SMS template with placeholder for customer support number, and replacing the placeholder with actual phone number (ex. +987654400). The plus sign (+) misplaced at end of number when replaced like 987654400+

Sample code:

>>> 
>>> s = '{customer_number} فتش'
>>> print s.replace('{customer_number}', '+987654400')
987654400+ فتش
>>> print '+987654400'.join(s.split('{customer_number}'))
987654400+ فتش

>>> 
>>> s = u'{customer_number} فتش'
>>> print s.replace('{customer_number}', '+987654400')
987654400+ فتش
>>> print '+987654400'.join(s.split('{customer_number}'))
987654400+ فتش

Python version 2.7.6, 2.7.10
OS: OSX 10.11.14
locale: English

--
It is not only happening in terminal, first I noticed this issue in API response which having following configuration.

Ubuntu 14.04.2 LTS

python 2.7.6
django 1.7.6
django-tastypie 0.12.1

system locale : English
django project locale: English

@RickyA updated the question with requested information. Yes just changed my name ;) — Jayabal, Mar 31 '16 at 12:16
wow, that is weird. on ubuntu 12.04 py 2.7 I get the right thing: `+987654400 فتش ` but on osx py 2.7 I get: `+987654400 فتش` — RickyA, Mar 31 '16 at 12:25
it is even weirder because I copied the last one that showed `[arabic] [number]+` from the terminal but when I copied it here it got reverted back to what you see one comment above. — RickyA, Mar 31 '16 at 12:28
As I also can't reproduce the error you could try messing around with `"\u202E"` (right-to-left-override) or `"\u202D"` (left-to-right-override). — Wombatz, Mar 31 '16 at 12:29
Looks like osx terminal is do magic. Can you try to print it in a file and see what it then looks like? — RickyA, Mar 31 '16 at 12:29
@RickyA I have exactly the same problem in Windows. But if you do `t = [s] ` and then `t[0].replace('{customer_number}', '+987654400')` you get `'+987654400 \xd9\x81\xd8\xaa\xd8\xb4'` so it is handled differently if it's in a list — roganjosh, Mar 31 '16 at 12:32
Similar issue: http://stackoverflow.com/questions/33700735/print-arabic-or-any-right-to-left-writing-system-string-to-linux-terminal-using — RickyA, Mar 31 '16 at 12:47
@roganjosh it handled differently because there is no arabic unicode chars at the beginning or end of the string (but `\xb4`), not because it is in a list... — RickyA, Mar 31 '16 at 13:07
@RickyA Not sure I understand, can you clarify? `s` is still the same string defined in the question (that contains the Arabic word), all I did was define a list called `t` that contains `s`. If I call `replace` on `s` I get identical output as in the original question (with the Arabic characters maintained but with the undesireable rearrangement) but if I do exactly the same with `t[0]` I get a different output with the correct arrangement. — roganjosh, Mar 31 '16 at 13:14

RickyA · Answer 1 · 2016-03-31T12:58:47.400

2

Somehow Osx terminal is doing magic with the string. If you print to a file it outputs the correct string.

python in terminal:

print to file:

file contents:

and the problem is really in the terminal because if we do this we see that the contents of the string are right:

And if we pad it with latin characters it is not doing funky rtl stuff:

Resume: Osx and Windows terminals have rtl support that garbles your string. The contents of the string in python is ok, but is is mutilated by the os. Ubuntu does not have this problem since rtl support is not build in the terminal there.

edited Mar 31 '16 at 12:58

answered Mar 31 '16 at 12:36

RickyA

15,465
5
71
95

It is not the problem only with terminal, actually it is wired in my API response which having following configuration python 2.7.6, django 1.7.6, django-tastypie 0.12.1 – Jayabal Mar 31 '16 at 12:58
yey, hard to where things mess up then, but maybe you only have a problem at the end where things are displayed, and is the data correct in the db. – RickyA Mar 31 '16 at 13:04
yes it is all correct in template, which maintained in file system as of now. – Jayabal Mar 31 '16 at 13:07
That is nice, then you only have a display problem :) – RickyA Mar 31 '16 at 13:07
yes of course, but it display wired when return in response and also write in logs :( – Jayabal Mar 31 '16 at 13:10
well, the easiest solution would be to replace the `+` with `00` ;) – RickyA Mar 31 '16 at 13:29
that is the last choice, but it will increase a char in SMS ;) Thanks for your time and ideas, Ricky. – Jayabal Mar 31 '16 at 13:56

RTL (Arabic) misplace plus sing to the end of phone number in python

1 Answers1