2

this might look like a similar issues for utf8 and Arabic language with MySQL database but i searched for result and found none..

my database endocing is set to utf8_general_ci , i had my php paging to be encoded as ansi by default the arabic language in database shows as : ãÌÑÈ

but i changed it to utf8 , if i add new input to database , the arabic language in database shows as : زين

i dont care how it show indatabase as long as it shows normally in php page , after changing the php page to utf8 , when adding input than retriving it , if show result as it should .
but the old data which was added before converting the page encoding to uft8 show like this : �����

i tried a lot of methods for fixis this like using iconv in ssh and php , utf8_decode() utf8_encode() .. and more but none worked .

so i was hoping that you have a solution for me here ?

update :: Main goal was solved by retrieving data from php page in old encoding ' windows-1256' than update it from ssh .

but one issue left ::

i have some text that was inserted as 'windows-1256' and other that was inserted as 'utf-8' so now the windows encoding was converted to utf-8 and works fine , but the original utf-8 was converted as well to something unreadable , using iconv in php, with old page encoding ..

so is there a way to check what encoding is original in order to convert or not ?

Community
  • 1
  • 1
Hussein
  • 97
  • 12

1 Answers1

0

Try run query set name utf8 after create a DB connection, before run any other query.

Such as :

$dbh = new PDO('mysql:dbname='.DB_NAME.';host='.DB_HOST, DB_USER, DB_PASSWORD);
$dbh->exec('set names utf8');
Zac
  • 1,062
  • 2
  • 7
  • 12
  • For you old data, please create a backup table in old format, then use your php code to re-input it to new table – Zac Feb 07 '14 at 03:41
  • 1
    If you have your old table as backup, and it can be read by your old php code somehow. Then, even it is shows as : زين, you can try to reinsert them to you new table via your php. – Zac Feb 07 '14 at 03:45
  • the old php page can read only old code as it is ANSI encoded , and new one will read the utf8 correctly , so if i do the update it has to be from the new page in order to be read as utf8 , but when doing so , i will be left with something like this: ظ…ظˆظ‚ط¹ .. – Hussein Feb 07 '14 at 03:48
  • You can use you old php page to read it properly, then convert it to utf8, then export it to a utf-8 sql query text file. Then, you can manually run that query to db, or let your UTF8 php to run it. – Zac Feb 07 '14 at 03:55
  • 1
    this will work if we are talking about a small db .. my db is more than 70mb ..what you are talking about is more than 1 page conversion process and as i see , i cant figure how to be done with such db . do you have a way for that or .. ? – Hussein Feb 07 '14 at 03:59
  • well, first, you need prove concept - make sure it is working. Then, from that point, you can work on how to work with large data, such as: curl on fly (only fix the data, when ppl require it), or move data to your local, put huge ram to php, and fix it once for all. 70mb data to your local is nothing. – Zac Feb 07 '14 at 04:08
  • that will work but there will be the risk or QUERY UPDATE as there will be errors because of html/ ' / " .. etc in articles and cannot insect them one by one .. is there a way faster than usual php select/update process... ? like using ssh .. – Hussein Feb 07 '14 at 04:44
  • dump data to you local, then give your php 2G ram, select all old data in one query, update them by a loop. Give one afternoon, go outside with your friend have some beer, when you back it is done. If you worry about string error, you need take a look at `mbstring`. – Zac Feb 07 '14 at 04:53
  • Errormessage: Got a packet bigger than 'max_allowed_packet' bytes :) – Hussein Feb 08 '14 at 07:36
  • @Hussein please check: http://stackoverflow.com/questions/8062496/how-to-change-max-allowed-packet-size – Zac Feb 09 '14 at 22:18