1

I am trying to insert some data from compressed .gz (gzip) files using Python. These files are in .tsv format. I know how to read gzip compressed files, but I cannot resolve the problem how to insert it not converting to csv format.

I have already tried to read compressed file using below code.

gzip_file = gzip.open(os.path.join(path, filename), 'rb')
file_cont = gzip_file.read()
vezunchik
  • 3,669
  • 3
  • 16
  • 25
b36
  • 105
  • 1
  • 8
  • that code looks OK, what do you actually want to do? – Sam Mason May 03 '19 at 21:17
  • I want to read each line of this compressed .tsv file and insert it into PostgreSQL table, but I know how it can be done only working on .csv files, but not on compressed .tsv. How can I read each line with attributes using "file_cont" object? I would be so grateful for any advice. – b36 May 03 '19 at 21:22
  • this just functions as a normal file object, so you'd proceed as normal as you would for a CSV file. Postgres `COPY FROM` allows you to specify a `DELIMITER` so you might be able to pass data without processing in Python – Sam Mason May 03 '19 at 21:27
  • Yes, it's possbile, but in such case I need to unzip gzip file. The case is I am wondering if it's possible without uzipping gzip file. – b36 May 03 '19 at 21:33
  • And I have to process it using Python. – b36 May 03 '19 at 21:39
  • **something** needs to uncompress the data, and it probably shouldn't be postgres (which has enough to worry about). your question has very little detail to go on, why not just use something like https://stackoverflow.com/q/41738829/1358308, it would be easy to include a `DELIMITER` clause and drive from Python if you want – Sam Mason May 03 '19 at 21:39
  • I'd suggest editing your question to include more details, what do you need to process? why does it need to be Python? maybe an example couple of lines of input and relevant parts of the database schema – Sam Mason May 03 '19 at 21:40

0 Answers0