Changing the default separator in csv created with "gdal_translate -of XYZ"

Question

I have 100s of .tiff files, and using the following command:

gdal_translate -of XYZ MCD13A2.t201624.006.global.1km_NDVI.O0.tif MCD13A2.t201624.006.global.1km_NDVI.O0.csv

This generates a .csv file but it is space separated, instead of comma separated.

Does anybody knows why this is happening, or have any solution for converting this space separated file into a comma separated .csv?

You should provide a bit more informations about your case. For example the command or code you had try, the version of libraries. More details more possibilities that someone can help you quick. — Mario Santini, Jan 10 '17 at 10:58
Can you be more specific about the files? Are these standard CSV files that you downloaded from somewhere or your own files? By the way a simple find and replace should solve your problem. — Piyush Shrivastava, Jan 10 '17 at 10:58
@PiyushShrivastava m downloading tiff from ftp server then converting it into csv using gdal — gaurav singh, Jan 10 '17 at 11:39
@MarioSantini i have return the command " gdal_translate -of XYZ MCD13A2.t201624.006.global.1km_NDVI.O0.tif MCD13A2.t201624.006.global.1km_NDVI.O0.csv" — gaurav singh, Jan 10 '17 at 11:40
@gauravsingh actually a *space separated values* file is exactly the same forma as the *comma separated values*. The fact that is called *csv* has nothing to do with the fact that it has to be separated by a comma. I'm not an expert of *gdal_translate*, so I could not point you in the right direction, just suggest you to check the tool options or your environment. — Mario Santini, Jan 10 '17 at 12:12
"The fact that is called csv has nothing to do with the fact that it has to be separated by a comma. " What does "comma" in "comma separated values" stand for ? — iFlo, Jan 10 '17 at 13:18
From wiki : "In addition, the term "CSV" also denotes some closely related delimiter-separated formats that use different field delimiters. These include tab-separated values and space-separated values. A delimiter that is not present in the field data (such as tab) keeps the format parsing simple. These alternate delimiter-separated files are often even given a .csv extension despite the use of a non-comma field separator." — iFlo, Jan 10 '17 at 13:21

iFlo · Answer 1 · 2017-01-10T11:02:56.210

2

You can simply use the CSV module :

import csv

with open(ur_infile) as fin, open(ur_outfile, 'w') as fout:
    o=csv.writer(fout)
    for line in fin:
        o.writerow(line.split())

It will automatically read your CSV with space separator and write it with comma (by default).

Also, you could specify delimiter like that :

csv.reader(csvfile, delimiter=' ')
csv.writer(csvfile, delimiter=',')

edited Jan 10 '17 at 11:02

answered Jan 10 '17 at 10:59

iFlo

1,442
10
19

file is to big....and have 100s of csv comma separated file each file have 30 milln records .... – gaurav singh Jan 10 '17 at 11:01
1

You should write a complete question then. – iFlo Jan 10 '17 at 11:03

score 2 · Accepted Answer · edited May 23 '17 at 10:29

A google search revealed the following approach:

After you are done with the xyz conversion, you can just use the commandline utility sed (suppose you use linux):

sed 's/ /,/g' in.xyz > out.csv

Since you are creating 100s of csv files, you are probably using gdal_translate inside a for loop, so you can just add that additional line right after gdal_translate and you won't even notice anything!

Still, if you want to use @iFlo's method without worrying about out-of-memory problems, you can have a look at this thread.

Changing the default separator in csv created with "gdal_translate -of XYZ"

2 Answers2