2

I need to convert a CSV file to UTF-8 and rename it using a PHP script.

The following code worked on my PC but now i need to do this on a server as a CRON task

iconv -f UTF-16LE -t UTF-8 OLD-FILE.csv > NEW-FILE.csv

Anyone know the equivalent in PHP. Thanks a lot.

Arnaud Martinn
  • 95
  • 2
  • 11

3 Answers3

7

A simple method would be to load the CSV file to a string using this command:

Then you can UTF-8 Encode the string using this command:

Finally write the string to a file using file_put_contents.

Finished code could look like this: 

$file_data = file_get_contents('/my/path/to/file.csv');
$utf8_file_data = utf8_encode($file_data);
$new_file_name = '/my/path/to/new_file.csv';
file_put_contents($new_file_name , $utf8_file_data );

Make sure the web server has the correct permissions to both read and write to appropriate locations.

Here is the link to file_put_contents():

Funk Forty Niner
  • 74,450
  • 15
  • 68
  • 141
Tom Goetz
  • 96
  • 2
  • Not enough reputation to post all links here is the link to file_put_contents http://php.net/manual/en/function.file-put-contents.php – Tom Goetz Jul 30 '15 at 00:21
  • OK I get this message: Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 244645189 bytes) in Convert_utf16_to_utf8.php on line 4 – Arnaud Martinn Jul 30 '15 at 01:26
  • you can increase the memory limit using this command, ini_set('memory_limit','300M'); would do the trick, my concern now is that these are large files and this might not be the best method. – Tom Goetz Jul 30 '15 at 19:24
  • Might be better to read the file in chunks instead of trying to put it all in memory... – ThatOneDude Jul 30 '15 at 22:20
  • Hi all, I have increased the memory but i still does not work: here is the code i use: – Arnaud Martinn Aug 03 '15 at 17:15
  • `utf8_encode` does not work properly. Use `iconv` instead. http://stackoverflow.com/questions/9499467/iconv-vs-utf8-encode – MarthyM Apr 13 '17 at 08:41
2

So here is the solution I found thanks to our brainstorming: 

<?php
$file_data = file_get_contents('/home/MYFILE.csv');
//$utf8_file_data = utf8_encode($file_data);

$utf8_file_data = mb_convert_encoding($file_data, "UTF-8", "UTF-16LE");
//$utf8_file_data = iconv("UTF-16LE","UTF-8",$file_data);

$new_file_name = '/home/MYFILE_NEW.csv';
file_put_contents($new_file_name , $utf8_file_data );
?>

The only pb is that the output size is twice as big as the input. If I use ICONV on my PC it is HALF the size...

If anyone knows I'd like to hear why.

Arnaud Martinn
  • 95
  • 2
  • 11
0

if iconv is available you can use the PHP equivalent: http://php.net/manual/en/function.iconv.php note that this takes a string, you will need to read in the file http://php.net/manual/en/function.fread.php and then write it out http://php.net/manual/en/function.file-put-contents.php but this approach may be slower and for big files, it will require to load the file to memory.

Josep Valls
  • 5,483
  • 2
  • 33
  • 67